Are distortion models really the same between OpenCV and RealSense ?
Hello !
I am trying to make my own calibration based on the white paper you provide and the OpenCV API. The sensors I'm using are D415.
And I would like to know this: Is the distortion model used by RealSense the same than is computed by OpenCV ?
RealSense documentation
From you whitepaper we can find the following code example (based on OpenCV calibration API) :
- In green the distortion values
- In blue the openCV function to compute the intrinsic values
So from this example it seems a good idea to assume that there is a direct relation between OpenCV distortion model and the one used by RealSense (for D415 sensors).
From this page (RealSense documentation), the implementation of the distortion model within your SDK is described as:
OpenCV implementation of Brown-Conrady model
Regarding the "distortion formula" that we can find on your website, I expected to find the same formula implemented in OpenCV API.
However the formula I found there is this one (link to OpenCV documentation):
(NB: the highlighted formulas are actually added up)
I double checked, and I found that the implementation of the "undistortion" matches this formula: link to implementation.
So the two formulas do not seem to be the same: they are really close but they seem to differ. Specifically in the formula described by RealSense, there some f² terms which do not appear in the OpenCV formula (see below the highlighted f²)
Am I correct ?
Thanks a lot for your help !
-
Hi Thomas Specte The streams of RealSense cameras use Modified Brown-Conrady (a variation of the Brown-Conrady model) or Inverse Brown-Conrady (as the name suggests, an inverse of Brown-Conrady).
https://dev.intelrealsense.com/docs/projection-in-intel-realsense-sdk-20#distortion-models
-
Hi MartyG ,
Thanks for your answer !
So shall I assume that OpenCV can't provide me the right parameters since the RealSense implementation is a modified one ?
Within the white paper about calibration it is not explicit that the implementations are not the same, so I just want to make sure.
Thanks again!
-
You can retrieve the correct distortion parameters for a RealSense camera using its rs-enumerate-devices tool in calibration information mode by launching the tool with the following instruction:
rs-enumerate-devices -c
The distortion parameters are listed in the intrinsics.
The distortion coefficients are listed in the Coefficients part of the information. Note that the five coefficients are all zero. On the D415 camera model, all of the coefficients are artificially zeroed for reasons provided at the link below. Simply for this reason, OpenCV distortion coefficients would not match up to RealSense ones.
-
Yes I know that the images are rectified but I'm trying to make my own custom calibration (computed on unrectified images).
The white paper you provide on that subject tells us to use OpenCV to compute the distortion values.
However the models seem to differ, so the question is the following: is your white paper wrong about it?
-
I am not familiar with the custom calibration app tool unfortunately and so do not have advice that I can provide about it or its paper's contents. If you require assistance with it then I recommend creating a new post on this support forum that specifically states that you require assistance with the custom calibration app creation tool. A member of the Intel RealSense Help Center support team can then assist you (I am not a member of that team).
-
Hi Thomas and Marty, I also require the distortion coefficients to be available within an OpenCV format and was wondering if you've had any luck with this issue after contacting the calibration support team?
The roundabout way would be to calibrate these cameras myself, but I'd really prefer if any conversion of factory calibration to OpenCV distortion model (which indeed seems necessary) would be available instead. Having to manually calibrate each camera would not scale within my application.
Thanks!
-
Hi Dorus Van Dintherly The best solution may be to define and apply a fixed set of manually defined coefficient values i a script instead of using the camera's intrinsics, to provide a 'one size fits all' solution no matter which RealSense camera is attached. So that every time the script is run, the same coefficients will be used.
https://github.com/IntelRealSense/librealsense/issues/4061
Another approach that could be considered is using OpenCV's undistort() function to remove the distortion model from the RealSense image.
-
Hi Marty, thank you for the suggestions, but I don't think they provide a solution to my problem.
Setting the coefficients to some fixed value would mean that I'm implying that the physical lens deformation is the same for all RealSense units, which is not the case (ever) for lenses with slight production differences. Setting them to zero would mean I'm implying there's no deformation, while there is, even though it's slight.
My goal is to indeed be able to use the OpenCV undistort function. However, this thread shows that the OpenCV distortion model is different than the RealSense (inverse) Brown/Conrady model. So the OpenCV function won't error out when I input the librealsense coefficients, but the resulting undistorted image is actually incorrect because the model to do this undistortion is different.
So what I'm looking for is a way to convert the RealSense distortion model(s) to the OpenCV standard, such that I'm then able to use the OpenCV functionalities that require distortion. In other words, RealSense coefficients (which I can read through the API) in -> OpenCV coefficients out. Thanks again!
-
For OpenCV camera calibration questions, the best Python / OpenCV reference that I know of is the tutorial at the link below.
https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html
I do not know of an automated way to convert the RealSense distortion model to the OpenCV standard though, unfortunately.
-
Hi Dorus!
There is actually a direct relation between the model implemented in realsense API and OpenCV "5 coefficients" Brown-Conrady model.
I have been using it for some time now and it works well!
My conclusion about infrared images (they use an other model for color images), is that they use this formula:
-
Hi Thomas.
With direction relation, do you mean they're completely the same/equivalent or that there is some (linear or non-linear) relation k_1 = a * k_2 (if so, what is that relation)? Also, are you referring to the 'Inverse Brown-Conrady' Realsense model being the same in your first sentence, or the 'Brown-Conrady' Realsense distortion model?
From your image I think I understand you mean that the Brown-Conrady model is not equivalent?
Thanks for your input!
-
Hi,
I wanted to say that they are the same, you can directly apply the ones computed by OpenCV
They are the appropriate ones for the model they call "Brown Conrady" (used by infrared sensors), but they will not work for the model "Inverse Brown Conrady" (used by the RGB sensor).
Here is the actual code responsible for the point projection: https://github.com/IntelRealSense/librealsense/blob/master/src/rs.cpp#L3931-L3959
Hope this helps!
Please sign in to leave a comment.
Comments
14 comments