Recording tool

It may be necessary to record raw image data from the HemiStereo 3D sensor for offline processing (e.g. for AI training scenarios or for SLAM map teaching). A common goal is to record this data fast (at high fps), while aquiring high resolution images. These allow for more options during post-processing (scaling, cropping, stereo quality selection etc. with a given image set). For recording these images, an high performance storage medium is recommended, please contact our support if needed. To start recording, please install the 3dvl-recording app as described below.

Preparation

Per default, the recording app does store the recorded images to /mnt/images. If you plan to store them onto a specific medium (USB stick, internal SSD), please make sure, that this is mounted to /mnt, so it will be used automatically.

Installation

Note

Please ensure that you update the sensor before proceeding with the installation.

The recording app can be installed on the sensor using apt:

sudo apt update
sudo apt install 3dvl-recorder

The recording tool requires exclusive access to the camera devices. Therefore, apps using the camera (e.g. the stereo-backend) must be stopped first:

sudo hemistereo_app_stop stereo-backend

After stopping the apps, the recording app can be started:

sudo hemistereo_app_start recorder

Usage

The recording tool provides a web UI which can be accessed using the following URL:

http://<SENSOR-IP>/recorder

The images shown are downscaled versions of the original camera images, the recorded quality on the sensor equal to the camera mode (see below) and significantly higher than the downscaled version presented.

Selecting a camera mode

The cameras of the sensor can be used in two hardware configurations (modes): (0) full 12 MP 4032x3040 pixels or (1) binned 3 MP 2016x1520 pixels, where a wheigted average for each four pixels is used for the output signal.

The Mode - X button at the bottom right of the browser view is used for switching the camera mode. Please be patient here, the cameras have to be stopped and driver-level changes are performed by the kernel. Afterwards the whole imaging pipeline is restarted.

Capturing images

Please use the record button for starting and stopping a recording session. Each start will create a new output folder in /mnt/images/DATE-TIME/ - thus will not overwrite previous recordings.

The image counter in the lower left corner of the screen shows the number of recorded images during the current session.

Note

If you do not see the live images, it could be that the backend is not fully started yet. Please wait some seconds, refresh the page and try it again.

Getting the images

The images are stored as binary files with a .bin extension. These are uncompressed binary representations of YUV420 uint8_t images. If you want to access these directly, please contact our support for the image format description. The recommended approach is to use the render application, which is contained in the 3dvl-recorder application. Please make sure that the recorder app is still running on your sensor (we need the docker environment in the background). Then on the sensor’s (host) command line run the following script in the current image record directory:

cd /mnt/images/DATE-TIME # select appropriate folder
sudo hemistereo_render.sh # --help lists options

You can specify additional stereo options etc. Please let the image record and target directories untouched, these are set inside the docker context and only /mnt/images is correctly mapped there.

The tool will create (remapped) bmp files and depth images for the recording session. If you need something else (pointclouds, …) let us know.