Front light turned off when getting color image

[alfonsoa (Alfonso)]

General Information

• Product: C57-5MOC-4-S
• Serial Number: 233245
• Ensenso SDK Version: x.y.z
• Operating System: Windows
  files.zip (10.1 MB)

Hello,

I am attempting to extract the color image from the camera using the code provided below. However, I have encountered an issue with the front light behavior. Specifically:

1. During the first capture, the front light is turned off, resulting in a dark image.
2. During the second capture, the front light is turned on, and the image appears correctly illuminated.

The same code is used for both captures, so I am unsure why this inconsistency occurs.

Code Overview

void loadJson() {
	std::ifstream file("configuration_C57.json");
	if (file.is_open() && file.rdbuf())
	{
		std::stringstream buffer;
		buffer << file.rdbuf();
		std::string const& fileContent = buffer.str();
		NxLibItem()[itmCameras][1].setJson(fileContent, true);
	}
}
static Image getTextureRawImage(NxLibItem camera)
{
	Image image;

	NxLibItem leftTextureImage = camera[itmImages][itmRawTexture][itmLeft];
	leftTextureImage.getBinaryDataInfo(&image.width, &image.height, &image.channels, nullptr, nullptr, nullptr);

	std::vector<unsigned char> imageData(image.width * image.height * image.channels);
	leftTextureImage.getBinaryData(image.data, nullptr);
	return image;
}
int main()
{
	try { 
		nxLibInitialize(true);
		NxLibItem()[itmDebug][itmLevel] = valInfo;

		NxLibItem cameras(itmCameras);

		NxLibItem camera = cameras[1];

		std::string serial = camera[itmSerialNumber].asString();
		NxLibCommand open(cmdOpen); 

		open.parameters()[itmCameras] = serial;
		std::this_thread::sleep_for(std::chrono::milliseconds(1000));

		open.execute();
		
		loadJson();

		NxLibCommand trigCmd(cmdCapture);
		
		camera[itmParameters][itmCapture][itmTexture][itmEnabled] = true;

		trigCmd.parameters()[itmCameras] = serial;

		trigCmd.execute(); 

		NxLibCommand(cmdComputeDisparityMap).execute();

		NxLibCommand(cmdComputePointMap).execute();
		
		auto        image_raw = getTextureRawImage(camera);
		saveBMP(image_raw.data, image_raw.width, image_raw.height, "rawColor.bmp");
		
		NxLibCommand trigCmd1(cmdCapture);

		trigCmd1.parameters()[itmCameras] = serial;

		trigCmd1.execute(); 
		
		NxLibCommand(cmdComputeDisparityMap).execute();

		NxLibCommand(cmdComputePointMap).execute();

		auto        image_raw2 = getTextureRawImage(camera);
		saveBMP(image_raw2.data, image_raw2.width, image_raw2.height, "rawColor2.bmp");

	} catch (...)

Steps I Follow

1. I load a JSON configuration file (configuration_C57.json) using NxLibItem::setJson().
2. I execute cmdCapture to capture the first image.
3. I execute another cmdCapture to capture the second image.
4. I extract the raw texture image using getBinaryData() and save it as a BMP file.

Observations

• The first image (rawColor.bmp) is dark due to the front light being off.
• The second image (rawColor2.bmp) is correctly illuminated as the front light is on.

I have attached the following files to help investigate the issue:

1. The two captured images (rawColor.bmp and rawColor2.bmp).
2. The JSON configuration file (configuration_C57.json).

Could you help me identify why the front light behaves differently between the two captures? Is there something missing in my configuration or initialization process?

Thank you for your assistance.

[benthie (Benjamin Thiemann)]

Hi Alfonso,

the reason why the first texture image is darker is the automatic adjustment of the exposure and gain values of the texture node. They both have a default value of 1, which is too low and will be higher during the following captures.

I wrote the following Python script to debug your problem:

#! /usr/bin/env python3
# -*- coding: utf8 -*-

import argparse

import nxlib.api as api
from nxlib.constants import *
from nxlib import NxLibCommand
from nxlib import NxLibItem


parser = argparse.ArgumentParser()
parser.add_argument("serial", type=str, help="the serial of the stereo camera to open")
args = parser.parse_args()


def get_camera_node(serial):
    # Get the root of the tree.
    root = NxLibItem()
    # From here on we can use the [] operator to walk the tree.
    cameras = root[ITM_CAMERAS][ITM_BY_SERIAL_NO]
    for i in range(cameras.count()):
        found = cameras[i].name() == serial
        if found:
            return cameras[i]


def open():
    with NxLibCommand(CMD_OPEN) as cmd:
        cmd.parameters()[ITM_CAMERAS] = args.serial
        cmd.execute()


def capture():
    with NxLibCommand(CMD_CAPTURE) as cmd:
        cmd.parameters()[ITM_CAMERAS] = args.serial
        cmd.execute()


def compute_disparity_map():
    NxLibCommand(CMD_COMPUTE_DISPARITY_MAP).execute()


def compute_point_map():
    NxLibCommand(CMD_COMPUTE_POINT_MAP).execute()


def save_texture(camera, filename):
    with NxLibCommand(CMD_SAVE_IMAGE) as cmd:
        cmd.parameters()[ITM_NODE] = camera[ITM_IMAGES][ITM_RAW_TEXTURE][ITM_LEFT].path
        cmd.parameters()[ITM_FILENAME] = filename
        cmd.execute()


def print_parameters(camera):
    print()
    print(f"texture exposure: {camera[ITM_PARAMETERS][ITM_CAPTURE][ITM_TEXTURE][ITM_EXPOSURE][ITM_VALUE].as_double()}")
    print(f"texture gain: {camera[ITM_PARAMETERS][ITM_CAPTURE][ITM_TEXTURE][ITM_GAIN][ITM_VALUE].as_double()}")


def main():
    print("Initializing API")
    api.initialize()
    api.open_tcp_port()

    print(f"Opening camera {args.serial}")
    open()

    camera = get_camera_node(args.serial)
    camera[ITM_PARAMETERS][ITM_CAPTURE][ITM_TEXTURE][ITM_ENABLED] = True

    auto_adjustment = True
    if not auto_adjustment:
        camera[ITM_PARAMETERS][ITM_CAPTURE][ITM_TEXTURE][ITM_EXPOSURE][ITM_AUTOMATIC] = False
        camera[ITM_PARAMETERS][ITM_CAPTURE][ITM_TEXTURE][ITM_EXPOSURE][ITM_VALUE] = 5
        camera[ITM_PARAMETERS][ITM_CAPTURE][ITM_TEXTURE][ITM_GAIN][ITM_AUTOMATIC] = False
        camera[ITM_PARAMETERS][ITM_CAPTURE][ITM_TEXTURE][ITM_GAIN][ITM_VALUE] = 9

    print_parameters(camera)
    capture()
    compute_disparity_map()
    compute_point_map()
    save_texture(camera, "texture_1.png")

    print_parameters(camera)
    capture()
    compute_disparity_map()
    compute_point_map()
    save_texture(camera, "texture_2.png")

    print_parameters(camera)
    capture()
    compute_disparity_map()
    compute_point_map()
    save_texture(camera, "texture_3.png")


if __name__ == "__main__":
    main()

Here is the output with a C-Series camera:

Initializing API
Opening camera ...

texture exposure: 1.0
texture gain: 1.0

texture exposure: 5.0
texture gain: 2.3762807066255465

texture exposure: 7.447692118659369
texture gain: 6.0

You can either skip the first frames during which the auto adjustment is still ongoing or you find suitable settings for your scene and load the corresponding JSON settings as you’ve done in your provided code.

Kind Regards
Benny

[alfonsoa (Alfonso)]

Hi Benny,

Thank you very much for the detailed explanation and for taking the time to write the script to help me with this issue. It’s very useful to understand how the automatic adjustment of exposure and gain affects the first captures.

Thanks and have a great day!

Best regards,
Alfonso