Order of data within PointMap

Questions & Answers
1

General Information

  • Product: C57-6-M
  • Serial Number: 244888
  • Ensenso SDK Version: 4.0.1502
  • Operating System: Windows 10 64bit

Problem / Question

I am creating a C++ program that uses the triger command, ComputeDisparityMap, ComputePointMap, and nxLibGetBinary() to obtain a PointMap.

I have a question: is the binary PointMap data obtained this way sorted in descending order of Z? Is there a way (command parameter, etc.) to output the PointMap in descending order of Y?

Best regards,
K.N,

2

Hi @techno,

the point map has the same layout as the disparity map. This means the point map is basically a 2D image where the value at each pixel is 3D vector of floats. And because the disparity map can contain invalid values where no match was possible, indicated by the special bit value 0x8000, the point map contains invalid values at the same locations, indicated by a vector where all three components are NaN values.

If you need the points in any other order, I am afraid you will have to extract the valid points and sort them yourself.

Regards,
Raphael

3

Hi Raphael

First, thanks a lot for your quick response!

I understand the content.

Sorry if this is a basic question, but in what order is the data in disparity map arranged?

Best regards,
K.N,

4

How the different images in the binary pipeline relate to each other is described in our guide on aligning images and 3D data. The disparity map has mostly the same layout as the left rectified image, except, as documented in the disparity map manual entry: it may be smaller and offset in x-direction.

But if you only want to extract 3D points from the point map, the pixel layout does not really matter. You can just process the point map data returned from getBinaryData as a flat array. We have example code that processes the point map in the code example in our guide on getting 3D data, which processes the point map as an array of struct {float x,y,z;}, or the computeAverageZ function in C:\Program Files\Ensenso\development\examples\C++\nxSimple\nxGrap.cpp, which uses just float and computes the indices for each coordinate manually. Please note that both process the point map in two nested loops, although they could use a single loop as they are not using the pixel coordinates.

5

Hi Raphael

Thank you for your reply.
I also looked at the information at the link.

The process I want to implement is to evaluate the Z value only for points within a specific Y range in a PointMap data set.
In this case, I plan to read the PointMap data from the beginning in units of struct {float x,y,z;} and check if it’s within the Y range.
If the PointMap data is sorted in ascending Y order, I think I don’t need to check the data beyond the Y range.
(I want to complete the process as quickly as possible.)

Modifying the sample code you provided to match my desired process results in the following:

camera[itmImages][itmPointMap].getBinaryDataInfo(&width, &height, &channels, &bytesPerElement, &isFloat, &timestamp);
std::vector<XYZ> points;
camera[itmImages][itmPointMap].getBinaryData(points,&timestamp); 
double z = 0;
int numberOfValidPixels = 0;
for (int y = 0; y < height; y++) {
	for (int x = 0; x < width; x++) {
		float currentZ = points[y * width + x].z;
		if (!std::isnan(currentZ)) {
			// this is a valid pixel
			if((rng_minY <= points[y * width + x].y)
             &&(points[y * width + x].y <= rng_maxY)){
				z += currentZ;
				numberOfValidPixels++;
			}
			else if(rng_maxY < points[y * width + x].y){
				return;
			}
		} else {
			//  depth data could not be reconstructed for this pixel; x, y and z are set to NaN
		}
	}
}

I expect the PointMap data is stored in the following order, but is this correct?

Point map data order
Point map data order527×367 503 Bytes

Could you please let me know?

Thank you.

Best regards,
K.N,

6

That is correct.

Your code looks mostly correct, except that the early termination of the loop does not work in general. The reason is that the Y-coordinate does not necessarily increase monotonically as you traverse the point map top to bottom for two reasons:

  1. The transformation linking your camera to your reference coordinate system can rotate the coordinate system, so the Y coordinate in space has very little do do with the y index in the point map. Just imagine standing the camera upright, for example.

But even your camera is not linked to a reference frame, i.e. the point map coordinates are relative to the camera coordinate system:

  1. The Y coordinate in the point map at index (x,y), as computed from the disparity map via the reprojection matrix, has the form Y = \frac{y-A}{B \cdot d+C}, where A, B, C are contants and d is the disparity value for that pixel. So even if some Y was already larger than your threshold, if d increases, Y can become smaller again. Intuitively: if something is visible lower down in the image, but closer to the camera, it can be higher up in space than something that is higher up in the image but further away.

You can probably make it work for your application, but I would recommend just going through the entire point map and checking each point, which should not take that long. If you really want to optimize, you should see the most improvement if you set the disparity map area of interest as small as possible, as computing the disparity map is the most computationally expensive step. You should check out our optimization guide if you want to learn more.

7

Hi Raphael

Thank you for your consistently thorough responses.

I understand that Y does not increase monotonically.

As you pointed out, I will check all values.

I learned a lot.

Thank you very much.

Best regards,
K.N,