Frequently Asked Questions (FAQ)

General Questions

Where can I find the technical specifications of HemiStereo NX?

The specifications for HemiStereo NX can be found on the product page or in the datasheet.

Which software interfaces does HemiStereo NX offer?

Configuration and data access can be done by gRPC or REST APIs which can be used together with any programming language (e.g. C++, Python, Java, C#, …). We also ship a ready-to-use Python SDK that wraps the gRPC-API and provides conversion functions for numpy. Additionally, there is a ROS2 interface that allows you to easily integrate the HemiStereo NX in your robotic application as a ROS2 node.

Which external connections / hardware interfaces are available?

The following interfaces are available on the outside of the camera device:

  • Gigabit Ethernet, RJ-45

  • HDMI and DisplayPort Connector

  • USB 3.1 Type-A

  • USB 2.0 Micro-B

Additionally, there are more optional interfaces which are internally and need disassembly of the camera unit in order to reach them (note: this may damage the camera calibration, re-calibration may be necessary):

  • GPIO / I²C / I²S / SPI / UART

  • M.2 Key-E

  • M.2 Key-M (HS-90/180X only)

Equipping these interfaces with devices of your choice or routing to the external IO is possible in some cases. Please contact our sales team for further information.

How do I connect HemiStereo NX to my host computer / PC?

Communication with HemiStereo NX is generally done over a network connection using the HemiStereo Viewer, web interface or APIs. This connection is established over Gigabit Ethernet using the RJ-45 connector or by using a USB 2.0 Micro-B cable connected to the USB port of your PC.

How to get started with HemiStereo NX?

Please start by reading our Quick Start Guide, which you will find in the box. An in-depth explanation und setup guide is part of the online documentation of HemiStereo NX. Also, you can watch our YouTube series about getting started with AI using HemiStereo NX.

How can HemiStereo NX be combined with an external controller?

An external controller (e.g. you robots main controller) should be connected over Gigabit Ethernet (RJ-45). Alternatively, you can use a USB 2.0 Micro-B to Type-A cable if you don’t have Ethernet available on your main controller. By that you can receive data, control camera functions and receive timestamps over gRPC and REST API. Also, a ROS2 node is available for HemiStereo NX if you main controller runs ROS.

A hardware synchronization method (GPIO / trigger) is currently not supported by default. For information on a customized version of HemiStereo NX or an alternative product please contact our sales team.

What is the range for this camera?

The range of a stereo camera generally depends on the required precision for your application, since the accuracy decreases over distance. Please find detailed information about the accuracy of HemiStereo NX in the datasheet.

For applications like navigation, obstacle avoidance and people tracking the intended range is 0.5 to 10m (HS-NX-180) or 1.0 to 20m (HS-NX-90).

Does this camera work indoors? What is the minimum lux level?

Yes, the sensitivity of the image sensor is usually sufficient for normally lit indoor environments (>50 lux). The minimum lux level depends on the maximum allowable exposure time for your application.

Can I use this camera outdoors?

Yes, HemiStereo NX works great outdoors since it is not negatively affected by sun light as it is the case with actively emitting 3d cameras. Please note that HemiStereo NX only has limited weather proofing and should not be used for permanent outdoor installations or during rain or snowy conditions.

What’s the performance outdoors using an active (pattern) light source and sun light?

HemiStereo NX is a passive stereo camera without an integrated active light source. Hence, by default it includes an IR cut filter (ICF) and has no means of suppressing ambient light. If you want to add your own light source or want us to modify HemiStereo NX with a different optical filter, please contact our sales team for details.

What is the field-of-view of the lenses used?

HemiStereo NX is available in two optical configurations:

  • HS-NX-180(N/X): 180° × 145° (V × H) FOV

  • HS-NX-90(N/X): 85° × 67° (V × H) FOV

The lenses itself provide slightly higher FOV (~200° / 86°) but outer image may lie outside image sensor area.

Do you have any technical reports on how HemiStereo works? Are you using fancy lenses with particularly good depth of field / low distortion? Or is this algorithm magic?

Other than normal stereo cameras it is able to use fish-eye lenses for stereo vision. These lenses provide opening angles up to 200° while remaining very sharp into the corners. We are able to use this lens due to special camera calibration procedure and a geometric camera model which allows stereo correspondence for non-rectilinear images.

Normal cameras are based on rectilinear (perspective) projection. This means the lens has to be designed to project straight lines onto the sensor without distortion. Such lenses become very expensive in order to provide a wide-angle FOV without distortion or sharpness degradations. Also, the practical field-of-view is limited to around 110° as the image center becomes dramatically compressed (perspective distortion).

You can find a detailed report about the basic principle of HemiStereo on our white paper page.

What are the hardware requirements?

An OpenGL capable graphics card is required to run the HemiStereo Viewer application. Otherwise, there are no special requirements for the hardware.

Can multiple HemiStereo cameras be used simultaneously?

Yes, since HemiStereo NX is a passive stereo vision camera without an integrated light source, multiple cameras don’t interfere with each other as it is the case with some actively emitting depth sensing technologies (ToF, Lidar).

What are some of the usage recommendations for this device?

HemiStereo NX is ideally suited if you need ultra-high-resolution accurate depth data in a range of 0.5 to over 20 meters. It can be used in many indoor and outdoor applications. Due to its ultra-wide field-of-view of up to 180° it can show more of you scene than any other depth sensing technology.

As a passive stereo vision camera, it requires sufficient external light source (ambient room light or sun light is sufficient). We’re also working on a new product for high-speed and low-light applications. Pease contact our sales team for details and information about launch date and pricing.

At what ambient temperatures does the camera work?

The acceptable operation conditions are -20 °C – 50 °C. The case provides some level of protection against splashing water and dust. However, the I/Os on the side are not protected and remain sensitive to water and dust.

What are the differences between 180N/X and 90N/X models?

The model number suffix consists of two parts:

  • A number (180 or 90) which indicates the approximate field-of-view. Select this according to your scene or mounting requirements.

  • A letter (N or X) which encodes whether the device is equipped with a Jetson Nano or a more-powerful Jetson Xavier NX. Please note that the Nano model is limited in term of resolution due to the limited RAM available.

If you’re unsure which model to choose please refer to our model selection guide (see datasheet) or ask our sales team.

What is the depth accuracy of the device?

This depends on your application and the HemiStereo model that you choose. As common in stereo vision systems, accuracy decreases over distance. Additionally, it highly depends on image sensor resolution. With HemiStereo NX the maximum image sensor resolution is 4072 × 3064 pixels resulting an impressive accuracy even in long distances. Please refer to the datasheet or the following graph for an overview of the achievable accuracy.

../_images/hs-nx_accuracy.png

Where can I find a 3D CAD model of the camera?

A simplified 3D STEP model can be found here.

What is the latency of the camera?

The latency (including stereo correspondence + depth calculation) highly depends on the image resolution chosen. Please refer to the following table for guidance:

Resolution

Matching Algorithm

Latency

4096 × 3072

Block Matching (BM)

310 ms

2048 × 1536

Block Matching (BM)

89 ms

1024 × 768

Block Matching (BM)

20 ms

512 × 396

Block Matching (BM)

7 ms

4096 × 3072

Semi-Global Matching (SGM)

3187 ms

2048 × 1536

Semi-Global Matching (SGM)

332 ms

1024 × 768

Semi-Global Matching (SGM)

90 ms

512 × 396

Semi-Global Matching (SGM)

22 ms

Test configuration: HS-NX-90X (Jetson Xavier NX), max-disp = 64, no texture filter, point-cloud disabled, without network transmission, without exposure time (30ms)

Can I use the integrated NVIDIA Jetson for my own AI applications?

Yes, absolutely! You can load your application onto the device and let it run on-board of HemiStereo NX. The processing power of the NVIDIA Jetson is shared between your application and the HemiStereo depth sensing core. While it is possible to run your application directly on the host OS, it is preferred to use containers for developing and deploying apps. Find a guide here.

How can the software of HemiStereo NX be updated?

The sensor software can be updated using the APT package manager. Please find detailed instructions here.

For updating the HemiStereo Viewer, just download the latest version from our download page.

Where can I find some sample data of HemiStereo NX?

You can find some sample point clouds under the following links:

If you need some different kind of sample data, including rosbag data, feel free to contact us.

Is there an integration for ROS? Where can I find the driver?

We provide a ready-to-use ROS 2 node. Detailed instructions can be found in our documentation.

For ROS 1, you can use the official ROS 1 bridge (from ROS 2 Github) to interface the device with other ROS 1 nodes.

Is it possible to make the point cloud less dense?

The point cloud has the same size as the target image. If you want to reduce the size of the point cloud, you can reduce the resolution of the target camera. This can be done in the HemiStereo Viewer using the corresponding controls:

../_images/screenshot_viewer_image_size.png

If you want to change the resolution using the API, you can configure it via the properties stereo_target_image_width and stereo_target_image_height.


Product Customization

Is HemiStereo NX available in different colors?

Yes, we can ship HemiStereo in different colors (any powder coating, RAL color is possible). However, the price of the device may be higher, depending on the number of units you order. Please contact our sales team for details.

Does HemiStereo NX provide an integrated IMU?

No, HemiStereo NX does not feature an integrated IMU. An external IMU (e.g. via USB) may work, depending on the model you need. Please reach out to our support team for details.

Can I purchase this camera in module form?

We can provide a customized version without the camera body based on your mechanical constraints. Please contact our sales team for details.

I want to add a IR flood or IR pattern projector. Is there a version without the IR-cut filter (ICF) installed?

Yes, we do ship HemiStereo NX with different optical filters installed, for example without the default IR-cut filter (ICF) or an additional 800-900nm band-pass filter blocking out visible light. Please contact our sales team for a quotation.

Is wireless LAN (WLAN) also available as a network interface?

HemiStereo NX does not include an integrated wireless card. The easiest way is probably to use an external USB WLAN dongle for wireless connectivity. eLinux has a list of devices compatible with Jetson (Nano).

Is PROFINET available as an additional interface?

No, but we’re currently testing an expansion interface. Please contact our sales team if you want to know details.

Is Halcon available as an interface?

No, we currently do not provide a Halcon interface nor is HemiStereo NX a GigEVision compliant camera. Contact our sales team if you need this feature anyway.


Usage & Troubleshooting

How do I set a static IP address for a HemiStereo NX camera?

The network settings of the HemiStereo NX are controlled by the NetworkManager. Therefore, you could use the NetworkManager tools nmcli or nmtui to configure a static ip. You can find configuration instructions using nmtui in our documentation.

There is some noise visible in the depth map. Is this normal?

A noisy depth map can have several reasons:

  1. Objects are too close to the camera: If objects are too close to the camera, the required number of disparities will exceed the maximum number of disparities the algorithm can handle. You can increase the number of disparities in the stereo settings up to 256. Alternatively, reducing the stereo resolution will also reduce the required number of disparities.

  2. Low textured objects: Stereo matching is only reliable for good textured environments. You can remove low textured areas using the textureness filter.

  3. De-calibration of the camera due to mechanical impacts: It can happen that the calibration of the stereo camera gets lost due to slight deformation between the two cameras or between the lens and the image sensor. If the depth map is noisy in good textured areas, please try to-re-calibrate using the free calibration tool or contact us for help.

HemiStereo Viewer does not find HemiStereo NX. What should I do?

By default, the HemiStereo NX tries to get an IP via DHCP. Please make sure that a DHCP server is available in the network. If no DHCP server is present, you can also change the network settings on the HemiStereo NX.

Device discovery works over UDP broadcasts. Some networks are blocking these kinds of broadcasts which will make it impossible to find the device automatically. To find out the IP address of the device, you can connect via Micro-USB or login by attaching a screen and a keyboard.

Detailed information for getting access and configuring the network can be found here.

Power LED does not illuminate. What should I do?

Check whether cable of power supply is plugged in properly. Ensure that you use an adequate power supply (5V DC for the Jetson Nano module, 10V - 19V DC for the Jetson Xavier NX module).

I have no connection to network. What should I do?

Check whether the ethernet cable is properly plugged in. Pull out the ethernet cable and plug it in again.

The camera image is blurry. Is it defective?

Please check first whether the lenses need to be cleaned (see Lens Cleaning). The rectilinear low-distortion lenses of HS-NX-90N/X model may show slight unsharpness in the corners of the image. This is normal with rectilinear wide-angle lenses. If you find the sharpness unsufficient, please contact our support team for assistance.