Application service

Applications expose an gRPC service for remote configuration and data exchange. gRPC is an RPC framework which is based on HTTP/2 and Protocol Buffers (Protobuf). To use the gRPC API, you should get familiar with gRPC first. So we encourage you to have a look at the documentation.

The application service is defined in a .proto file which can be found here. The repository contains also descriptions for messages that are requested by inputs or provided by outputs. A full reference of the protocols can be found here.

Usage

This section describes the usage of the application service. The examples are based on the C++ API of gRPC but the usage is similar in other programming languages. Please see the gRPC docs for reference.

Generating the service stub and message classes

The Protobuf compiler protoc is used to generate service stubs and message classes for your programming language. Please make sure, protoc and the gRPC plugin (grpc_cpp_plugin for C++) for the Protobuf compiler is installed correctly. If everything is set up correctly, please run the following commands to create the files:

# set path to the root directory of the api directory
api_dir=<path_to_the_api_dir>

# generate message classes
protoc -I $api_dir --cpp_out=. $api_dir/hemistereo/**/*.proto

# generate gRPC service stubs for the application service
protoc -I $api_dir --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` $api_dir/hemistereo/network/services/application.proto

The commands will generate a bunch of files in your current working directory:

.
└── hemistereo
    └── network
        ├── messages
        │   ├── basic.pb.cc
        │   ├── basic.pb.h
        │   ├── matrix.pb.cc
        │   ├── matrix.pb.h
        │   ├── message.pb.cc
        │   ├── message.pb.h
        │   ├── point.pb.cc
        │   └── point.pb.h
        └── services
            ├── application.grpc.pb.cc
            ├── application.grpc.pb.h
            ├── application.pb.cc
            └── application.pb.h

Note

For other languages than C++, the arguments given to protoc have to be changed. Please check the gRPC documentation.

Using the service stub

The file hemistereo/network/services/application.grpc.pb.h contains the classes that are necessary for the application service. To use it, first create a new channel by providing the address to the service endpoint on the sensor. It is also necessary to increase the message size limit for receiving the images and point cloud. This can be achieved by changing the default channel arguments:

grpc::ChannelArguments channelArgs;
// -1 set the message size to unlimited
channelArgs.SetMaxReceiveMessageSize(-1);

The channel arguments can now be used to create a new channel and service stub. By default, the API gateway is listening on port 8888:

// please replace <hostname_or_ip> with the hostname or the IP of the device
auto channel = grpc::CreateCustomChannel("<hostname_or_ip>:8888", grpc::InsecureChannelCredentials(), channelArgs);
auto stub = hs::network::proto::ApplicationService::NewStub(channel);

Note

While the following samples use the synchronous API, gRPC offers also an asynchronous API for interacting with the service. Please check the gRPC documentation for reference.

Setup the client context

The stub object provides functions for accessing the service. For each call of a stub function, a client context has to be provided. This context allows us to add some metadata which will be included in the HTTP header of the call. Because HemiStereo devices could run multiple apps at the same time, we have to add a header field app to address a specific application. The app field will be read from the API gateway to route the request to the right application. For example, to address the stereo application, create the client context the following way:

grpc::ClientContext context;
context.AddMetadata("app", "stereo");

Note

The client context should not be reused. Please create a new context object for each call of a service stub method.

Query the application structure

The function GetApplication can be used for querying the application structure. This gives the caller the information which inputs, outputs and properties are provided by the application. In C++, the request can be done the following way:

hs::network::proto::GetApplicationRequest request;
hs::network::proto::Node node;
auto status = stub->GetApplication(&context, request, &node);
if (!status.ok())
    throw std::runtime_error("failed to get application data from remote device: " + status.error_message());

node.PrintDebugString();

In this example, the content of the node object is printed to stdout.

Data exchange

Message format

The hs.network.proto.messages.Message message is used to exchange data. It contains the timestamp and the data itself packed into a google.protobuf.Any message. Use the PackFrom and UnpackTo methods from the Any type to fill or read the payload.

Reading data

Data is provided by the outputs of the application. There are two ways for getting the data: streaming and polling.

Streaming

Streaming can be used by calling the SubscribeTopic method. It returns a grpc::ClientReader object which can be used for reading the messages:

hs::network::proto::SubscribeTopicRequest request;
// set the path of the output
request.set_path(<path_to_the_output>);

auto reader = stub->SubscribeTopic(&context, request);

while (m_run)
{
    hs::network::proto::messages::Message msg;
    if (!reader->Read(&msg))
        break;

    // do something
    ...
}

auto status = reader->Finish();
if (!status.ok())
    std::cerr << "reading topic \"" << m_path << "\" failed:" << status.error_message() << std::endl;

Note

On high images resolutions or slow network connections, streaming can introduce a growing delay over time. In these cases, prefer polling to read data from the device.

Polling

For polling, data has to be cached internally. In latest versions of the application service, caching is done automatically. For older versions, you have to enable the output cache using the SetReadTopicEnabled method:

hs::network::proto::SetReadTopicEnabledRequest request;
// set the path of the output
request.set_path(<path_to_the_output>);
// enable ReadTopic
request.set_value(true);
hs::network::proto::SetReadTopicEnabledResponse response;

auto status = stub->SetReadTopicEnabled(&context, request, &response);
if (!status.ok())
    throw std::runtime_error("failed to set read topic enabled status of output \"" + m_path + "\": " +
                             status.error_message());

Then the ReadTopic method can be used to read the message from the application output:

hs::network::proto::ReadTopicRequest request;
// set the path of the output
request.set_path(<path_to_the_output>);
request.set_prev_message_id(0);
hs::network::proto::messages::Message msg;

auto status = stub->ReadTopic(&context, request, &msg);
if (!status.ok())
    throw std::runtime_error("failed to get read topic enabled status of output \"" + <path_to_the_output> + "\": " + status.error_message());

As you can see, a message id can be provided to the request. In the example above, it is set to zero. This instructs the service to return the data that is currently cached. If you poll in a loop, this can lead to receiving the same data multiple times. To avoid this, the message id of the previous message can be set. ReadTopic will then block until a messages with a newer message id is available.

uint32_t prevMsgId = 0;
while(true)
{
    hs::network::proto::ReadTopicRequest request;
    request.set_path(<path_to_the_output>);
    // set the previous message id here
    request.set_prev_message_id(prevMsgId);
    hs::network::proto::ReadTopicResponse response;

    auto status = stub->ReadTopic(&context, request, &response);
    if (!status.ok())
        throw std::runtime_error("failed to get read topic \"" + <path_to_the_output> + "\": " + status.error_message());

    // access message
    auto msg = response.data();

    // do something
    ...

    // keep the current message id
    prevMsgId = response.message_id();
}

Writing data

Apps can also offer inputs for receiving data from clients. To send a message to the app, WriteTopic can be used:

hs::network::proto::messages::Message msg;
// fill message
...

hs::network::proto::WriteTopicRequest request;
// set the path of the input
request.set_path(<path_to_the_output>);
request.set_data(msg);
hs::network::proto::WriteTopicResponse response;

auto status = stub->WriteTopic(&context, request, &response);
if (!status.ok())
    throw std::runtime_error("failed to get write topic \"" + <path_to_the_output> + "\": " + status.error_message());

Properties

Properties are used for the configuration of an application. At the moment, they are restricted to the following types: Int32, UInt32, Int64, UInt64, Float, Double, Bool, String, Enum.

To get the current value of the property you can use the GetProperty method:

hs::network::proto::GetPropertyRequest request;
// set the path of the input
request.set_path(<path_of_the_property>);
hs::network::proto::Property property;

auto status = stub->GetProperty(&context, request, &property);
if (!status.ok())
    throw std::runtime_error("failed to get property \"" + <path_of_the_property> + "\": " + status.error_message());

The property message has a oneof-field which contains a sub-message depending on the property type. For example, if you want to get the value of an Int32-Property, you have to access the prop_int32 field:

if (property.has_prop_int32())
{
    hs::network::proto::PropertyInt32 propInt32 = property.prop_int32();
    std::cout << propInt32.value() << std::endl;
}

Note

Depending on the property type, the messages contain also additional information (e.g. min, max, step).

To change the value of a property, use the SetProperty method:

hs::network::proto::SetPropertyRequest request;
// set the path of the input
request.set_path(<path_of_the_property>);

// set the property value, use set_value_<value_type> depending on the property type
request.set_value_int32(<int32_value>);
hs::network::proto::Property property;

auto status = stub->SetProperty(&context, request, &property);
if (!status.ok())
    throw std::runtime_error("failed to set property \"" + <path_of_the_property> + "\": " + status.error_message());

Here the property value is set directly in the SetPropertyRequest by setting the value_int32-field. If the property is of a different type, set another field accordingly.

Binning Mode

As described earlier, the custom application running on the sensor may expose several properties for tuning its behavior. One property which might be of special interest for many users is the camera binning mode.

Note

Binning is the (mathematical) combination of several camera pixels into one binned pixel, which therefore has a bigger sensor area and is more sensitive esp. in low-light conditions. The read-out time of the imager chip is decreased, which is beneficial for the rolling shutter effect. The IMX477 chip used in the HemiStereo NX sensor, offers 4:1 binning, meaning a decrease in horizontal and vertical camera resolution of 50%.

The binning mode is controlled by the camera mode, of which currently are only two available:

• mode 0, which is the default 4032x3040 pixel mode, which is the native resolution of the camera chip

• mode 1, which is the binned 2016x1520 pixel mode, where each 4 pixels are used in an averaged manner

The mode switch can be triggered using the application service interface, by setting the camera_mode property of the camera stereo source. The export name of this property is application specific, e.g. for the stereo app it is source_camera_camera_mode, which is a UInt64 value, representing the camera mode.

Protocol Documentation

Table of Contents

• hemistereo/network/messages/matrix.proto

  • Matrix

  • Matrix.Depth

• hemistereo/network/messages/message.proto

  • Message

  • MessageMap

  • MessageMap.MapEntry

• hemistereo/network/services/application.proto

  • CallRequest

  • Event

  • GetApplicationRequest

  • GetNodeRequest

  • GetPropertyRequest

  • GetReadTopicEnabledRequest

  • GetReadTopicEnabledResponse

  • Input

  • Method

  • Node

  • Node.InputsEntry

  • Node.MethodsEntry

  • Node.OutputsEntry

  • Node.PropertiesEntry

  • Output

  • Property

  • PropertyBool

  • PropertyDouble

  • PropertyEnum

  • PropertyEnum.ValueMapEntry

  • PropertyFloat

  • PropertyInt32

  • PropertyInt64

  • PropertyString

  • PropertyUInt32

  • PropertyUInt64

  • ReadTopicRequest

  • SetPropertyRequest

  • SetReadTopicEnabledRequest

  • SetReadTopicEnabledResponse

  • SubscribeEventsRequest

  • SubscribeTopicRequest

  • WriteTopicRequest

  • WriteTopicResponse

  • Event.Type

  • ApplicationService

• Scalar Value Types

Top

hemistereo/network/messages/matrix.proto

A message representing a matrix.

Field	Type	Label	Description
rows	uint32		The number of rows.
cols	uint32		The number of columns.
channels	uint32		The number of channels.
depth	Matrix.Depth		The depth of the element data type.
data	bytes		The matrix data in row-major order.

Name	Number	Description
Unknown	0	Unknown depth
Int8	1	8-bit signed integer
UInt8	2	8-bit unsigned integer
Int16	3	16-bit signed integer
UInt16	4	16-bit unsigned integer
Int32	5	32-bit signed integer
UInt32	6	32-bit unsigned integer
Int64	7	64-bit signed integer
UInt64	8	64-bit unsigned integer
Float	9	32-bit floating point
Double	10	64-bit floating point

Top

hemistereo/network/messages/message.proto

Represents a message.

Field	Type	Label	Description
timestamp	int64		The message timestamp.
payload	google.protobuf.Any		The message payload.
type	string		The data type of the message payload.

Represents a message map.

Field	Type	Label	Description
map	MessageMap.MapEntry	repeated	A map that contains multiple messages accessible by name.

Top

hemistereo/network/services/application.proto

Field	Type	Label	Description
path	string		The path of the method.
data	messages.Message		The input parameter of the method.

Represents an event.

Field	Type	Label	Description
type	Event.Type		The type of the Event.
path	string		The path to the corresponding object.
input	Input		The Input message. Field is only set on an InputAdded event.
output	Output		The Output message. Field is only set on an OutputAdded event.
property	Property		The Property message. Field is only set on PropertyAdded and PropertyChanged events.
method	Method		The Method message. Field is only set on an MethodAdded event.

Represents a request to get the current application.

Represents a request to get a Node.

Field	Type	Label	Description
path	string		Represents the path to a Node to be requested.

Represents a request to get a Property.

Field	Type	Label	Description
path	string		Represents the path to a Property to be requested.

Represents a request to query if topic reads are enabled.

Field	Type	Label	Description
path	string		The path of the topic.

Represents the response to query if topic reads are enabled.

Field	Type	Label	Description
value	bool		Indicates if topic reads are enabled.

Represents the input to a Node.

Field	Type	Label	Description
type	string		Defines the type of input to the Node.

Field	Type	Label	Description
typeIn	string		Defines the type of the method's input parameter.
typeOut	string		Defines the type of the method's output parameter.

Represents a specific application with corresponding properties, inputs and outputs.

Represents the output from a Node.

Field	Type	Label	Description
type	string		Defines the type of output from a Node.

Represents a property containing a single datatype.

Field	Type	Label	Description
prop_int32	PropertyInt32		A 32 bit signed integer property.
prop_int64	PropertyInt64		A 64 bit signed integer property.
prop_uint32	PropertyUInt32		A 32 bit unsigned integer property.
prop_uint64	PropertyUInt64		A 64 bit unsigned integer property.
prop_float	PropertyFloat		A 32 bit floating point property.
prop_double	PropertyDouble		A 64 bit floating point property.
prop_string	PropertyString		A string property.
prop_bool	PropertyBool		A boolean property.
prop_enum	PropertyEnum		An enumeration property.
displayname	string		Name of the property.
description	string		Description of the property.

Represents a Boolean value.

Field	Type	Label	Description
value	bool		Value of the property, can be true or false.

Represents a double-precision floating-point number.

Field	Type	Label	Description
value	double		The property value.
min	double		Lower bound of the property.
max	double		Upper bound of the property.

Represents an enumeration type.

Field	Type	Label	Description
value	int32		The property value.
value_map	PropertyEnum.ValueMapEntry	repeated	The value map containing possible values and their string representation.

Represents a floating-point number.

Field	Type	Label	Description
value	float		The property value.
min	float		Lower bound of the property.
max	float		Upper bound of the property.

Represents a 32-bit signed integer.

Field	Type	Label	Description
value	int32		The property value.
min	int32		Lower bound of the property.
max	int32		Upper bound of the property.

Represents a 64-bit signed integer.

Field	Type	Label	Description
value	int64		The property value.
min	int64		Lower bound of the property.
max	int64		Upper bound of the property.

Represents a string type.

Field	Type	Label	Description
value	string		The property value.

Represents a 32-bit unsigned integer.

Field	Type	Label	Description
value	uint32		The property value.
min	uint32		Lower bound of the property.
max	uint32		Upper bound of the property.

Represents a 64-bit unsigned integer.

Field	Type	Label	Description
value	uint64		The property value.
min	uint64		Lower bound of the property.
max	uint64		Upper bound of the property.

Represents a request to read a topic.

Field	Type	Label	Description
path	string		The path of the topic.
prev_timestamp	int64		The timestamp of the previous message. The call will block until newer message is available.

Represents a request to set a Property.

Field	Type	Label	Description
path	string		Represents the path to the property to be set.
value_int32	int32		The value of a 32 bit signed integer property.
value_int64	int64		The value of a 64 bit signed integer property.
value_uint32	uint32		The value of a 32 bit unsigned integer property.
value_uint64	uint64		The value of a 64 bit unsigned integer property.
value_float	float		The value of a 32 bit floating point property.
value_double	double		The value of a 64 bit floating point property.
value_string	string		The value of a string property.
value_bool	bool		The value of a boolean property.
value_enum	int32		The value of an enumeration property.

Request to enable/disable topic reads.

Field	Type	Label	Description
path	string		The path of the topic.
value	bool		Enables or disables topic reads.

Represents a request to subscribe to events.

Represents a request to subscribe to a topic.

Field	Type	Label	Description
path	string		The path of the topic.

Represents a request to write to a topic.

Field	Type	Label	Description
path	string		The path of the topic.
data	messages.Message		The message that should be written.

Represents the response to write to a topic.

Name	Number	Description
Unknown	0	An unknown Event occured.
InputAdded	1	An Input has been added.
InputRemoved	2	An Input has been removed.
OutputAdded	3	An Output has been added.
OutputRemoved	4	An Output has been removed.
PropertyAdded	5	A Property has been added.
PropertyRemoved	6	A Property has been removed.
PropertyChanged	7	A Property has been changed.
MethodAdded	8	A Method has been added.
MethodRemoved	9	A Method has been removed.

Method Name	Request Type	Response Type	Description
GetApplication	GetApplicationRequest	Node	Used to get the current Application.
GetNode	GetNodeRequest	Node	Used to get the Node at {path}.
GetProperty	GetPropertyRequest	Property	Used to get the property at {path}.
SetProperty	SetPropertyRequest	Property	Used to set a property at {path} to {value}.
SubscribeTopic	SubscribeTopicRequest	messages.Message stream	Used to subscribe to updates to a topic at {path}.
ReadTopic	ReadTopicRequest	messages.Message	Used to read a topic at {path}.
GetReadTopicEnabled	GetReadTopicEnabledRequest	GetReadTopicEnabledResponse	Used to query if topic reads are enabled for topic at {path}.
SetReadTopicEnabled	SetReadTopicEnabledRequest	SetReadTopicEnabledResponse	Used to enable/disable topic reads for topic at {path}.
WriteTopic	WriteTopicRequest	WriteTopicResponse	Used to write data to a topic at {path}.
SubscribeEvents	SubscribeEventsRequest	Event stream	Used to subscribe to events of the current application.
CallMethod	CallRequest	messages.Message	Used to call a method at {path}.

Scalar Value Types

.proto Type	Notes	C++	Java	Python	Go	C#	PHP	Ruby
double		double	double	float	float64	double	float	Float
float		float	float	float	float32	float	float	Float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long	int64	long	integer/string	Bignum
uint32	Uses variable-length encoding.	uint32	int	int/long	uint32	uint	integer	Bignum or Fixnum (as required)
uint64	Uses variable-length encoding.	uint64	long	int/long	uint64	ulong	integer/string	Bignum or Fixnum (as required)
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long	int64	long	integer/string	Bignum
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int	uint32	uint	integer	Bignum or Fixnum (as required)
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long	uint64	ulong	integer/string	Bignum
sfixed32	Always four bytes.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sfixed64	Always eight bytes.	int64	long	int/long	int64	long	integer/string	Bignum
bool		bool	boolean	boolean	bool	bool	boolean	TrueClass/FalseClass
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode	string	string	string	String (UTF-8)
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str	[]byte	ByteString	string	String (ASCII-8BIT)