I have two questions regarding the Result of the Hand-Eye Calibration.
In the documentation, the Residual is explained as “The residual of the final solution.”.
- → How is the residual calculated and what unit does it have?
In the documentation, the PoseError is explained as “… the difference between the pattern’s position estimated through the camera and the pattern pose estimated by the calibration over the robot position.”
- → Can you clarify that? I understand, that the PatternPose (which is also a parameter of the Result) is estimated globally (how? maybe by averaging all camera pattern poses?), and then for each image, the pattern pose is estimated from the image and the PoseError is the difference between the two. Is that correct?
Residual is simply the norm of the optimizer; in ancient SDK releases this used to be the reprojection error of all pattern points, in current major sdk releases (at least since 3.x) we switched to optimizing the euclidean distance between the fitted calibration object and each observed calibration object long time ago. The coordinates are in millimeters internally, but I’m not sure if this output is correctly normalized though, maybe you shouldn’t rely on this.
PoseError is an array of transformations between the fitted, final pose of the calibration object, and each observed calibration object pose (by chaining each of the robot poses with the (single) estimated camera/hand transformation, and each camera-to-“observed calibration plate” transformation)
All calibration target poses are anchored in the middle of the dot grid, so the poses in ‘PoseError’ also relate to the center of the calibration plate.
Hope that shed some light on the outputs… otherwise let me know if that still unclear.
So from the PatternPose and the Poses within the Results it should be possible to recalculate all those values and check, if the Residual is properly normalized. I will try that to be sure about the Residual.
if I’m not mistaken and interpret the code correctly, you should be able to reproduce the ‘Residual’ by stacking all n translation residuals from the ‘PoseError’ array into a 3 n long vector v, and then ‘Residual’ should be \frac{1}{\sqrt{ 3 n}} \lvert v \rvert ^2 .
Let me know if that works, otherwise we’ll have to dig a little deeper.
I tried your formula and for my data, if I use the translation vectors from PoseError, I get 0.330776, but the residual from the SDK is 0.356763. So something is a little off. Can you dig deeper?
I checked the code in detail and found that it is a bit more complicated than @rainer.voigt described:
The error vector e we use for optimization has one entry for each of the n observed pattern points in all of the given pattern observations. Every component is the euclidean distance between one observed and calculated 3D point position. So if you observe a 7x7 grid twenty times, we calculate this distance for each of the n=980 observed points. If you are using an Ensenso pattern that supports being seen only partially, you could have any number of points.
The residual is the averaged norm of this vector: r = \frac{||e||}{n}.
Distances are indeed all in millimeters.
To recalculate the residual you would first have to generate the coordinates of pattern points in the pattern coordinate system, for which there is no NxLib command. Then transform the coordinates for each obervation in the pattern buffer, once with the estimated pose from EstimatePatternPose (called with Average set to false) and once with the PatternPose returned from CalibrateHandEye. If you are using an Ensenso pattern and have partial observations, you will also have to figure out which points were actually observed by comparing with the Points in the pattern buffer. Then subtract, take the norm and average.
I hope this works, I must admit that I have not tried this myself.
