.. _application_service:

Application service
###################

Applications expose an gRPC service for remote configuration and data exchange.
`gRPC <https://grpc.io>`_ is an RPC framework which is based on HTTP/2 and
`Protocol Buffers <https://developers.google.com/protocol-buffers>`_ (Protobuf).
To use the gRPC API, you should get familiar with gRPC first. So we encourage
you to have a look at the `documentation <https://grpc.io/docs/>`_.

The application service is defined in a :code:`.proto` file which can be found
`here
<https://git.3dvisionlabs.com/3dvisionlabs/software/hemistereo/api/-/blob/master/hemistereo/network/services/application.proto>`_.
The repository contains also descriptions for messages that are requested by
inputs or provided by outputs. A full reference of the protocols can be found
`here <proto_docs.md>`_.

Usage
=====

This section describes the usage of the application service. The examples are
based on the C++ API of gRPC but the usage is similar in other programming
languages. Please see the `gRPC docs <https://grpc.io/docs/languages/>`_ for
reference.

Generating the service stub and message classes
-----------------------------------------------

The Protobuf compiler :code:`protoc` is used to generate service stubs and
message classes for your programming language. Please make sure, :code:`protoc`
and the gRPC plugin (:code:`grpc_cpp_plugin` for C++) for the Protobuf compiler
is installed correctly. If everything is set up correctly, please run the
following commands to create the files:

.. code-block:: bash

    # set path to the root directory of the api directory
    api_dir=<path_to_the_api_dir>

    # generate message classes
    protoc -I $api_dir --cpp_out=. $api_dir/hemistereo/**/*.proto

    # generate gRPC service stubs for the application service
    protoc -I $api_dir --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` $api_dir/hemistereo/network/services/application.proto

The commands will generate a bunch of files in your current working directory:

.. code-block::

    .
    └── hemistereo
        └── network
            ├── messages
            │   ├── basic.pb.cc
            │   ├── basic.pb.h
            │   ├── matrix.pb.cc
            │   ├── matrix.pb.h
            │   ├── message.pb.cc
            │   ├── message.pb.h
            │   ├── point.pb.cc
            │   └── point.pb.h
            └── services
                ├── application.grpc.pb.cc
                ├── application.grpc.pb.h
                ├── application.pb.cc
                └── application.pb.h

.. note::

    For other languages than C++, the arguments given to :code:`protoc` have to
    be changed. Please check the gRPC documentation.

Using the service stub
----------------------

The file :code:`hemistereo/network/services/application.grpc.pb.h` contains the
classes that are necessary for the application service. To use it, first create
a new channel by providing the address to the service endpoint on the sensor. It
is also necessary to increase the message size limit for receiving the images
and point cloud. This can be achieved by changing the default channel arguments:

.. code-block:: cpp

    grpc::ChannelArguments channelArgs;
    // -1 set the message size to unlimited
    channelArgs.SetMaxReceiveMessageSize(-1);

The channel arguments can now be used to create a new channel and service stub. By default, the
API gateway is listening on port :code:`8888`:

.. code-block:: cpp

    // please replace <hostname_or_ip> with the hostname or the IP of the device
    auto channel = grpc::CreateCustomChannel("<hostname_or_ip>:8888", grpc::InsecureChannelCredentials(), channelArgs);
    auto stub = hs::network::proto::ApplicationService::NewStub(channel);

.. note::

    While the following samples use the synchronous API, gRPC offers also an
    asynchronous API for interacting with the service. Please check the
    `gRPC documentation <https://grpc.io/docs/languages/cpp/async/>`_ for
    reference. 


.. _setup_client_context:

Setup the client context
************************

The stub object provides functions for accessing the service. For each call of a
stub function, a client context has to be provided. This context allows us to
add some metadata which will be included in the HTTP header of the call. Because
HemiStereo devices could run multiple apps at the same time, we have to add a
header field :code:`app` to address a specific application. The :code:`app`
field will be read from the API gateway to route the request to the right
application. For example, to address the stereo application, create the client
context the following way:

.. code-block:: cpp

    grpc::ClientContext context;
    context.AddMetadata("app", "stereo");

.. note::

    The client context should not be reused. Please create a new context object
    for each call of a service stub method.


Query the application structure
*******************************

The function :code:`GetApplication` can be used for querying the application
structure. This gives the caller the information which inputs, outputs and
properties are provided by the application. In C++, the request can be done the
following way:

.. code-block:: cpp

    hs::network::proto::GetApplicationRequest request;
    hs::network::proto::Node node;
    auto status = stub->GetApplication(&context, request, &node);
    if (!status.ok())
        throw std::runtime_error("failed to get application data from remote device: " + status.error_message());

    node.PrintDebugString();

In this example, the content of the :code:`node` object is printed to
:code:`stdout`.


Data exchange
*************

Message format
^^^^^^^^^^^^^^

The `hs.network.proto.messages.Message
<https://git.3dvisionlabs.com/3dvisionlabs/software/hemistereo/api/-/blob/master/hemistereo/network/messages/message.proto>`_
message is used to exchange data. It contains the timestamp and the data itself
packed into a `google.protobuf.Any
<https://developers.google.com/protocol-buffers/docs/proto3#any>`_ message. Use
the `PackFrom` and `UnpackTo` methods from the Any type to fill or read the
payload.

Reading data
^^^^^^^^^^^^

Data is provided by the outputs of the application. There are two ways for
getting the data: streaming and polling.

Streaming
`````````

Streaming can be used by calling the :code:`SubscribeTopic` method. It returns a
:code:`grpc::ClientReader` object which can be used for reading the messages:

.. code-block:: cpp

    hs::network::proto::SubscribeTopicRequest request;
    // set the path of the output
    request.set_path(<path_to_the_output>);

    auto reader = stub->SubscribeTopic(&context, request);

    while (m_run)
    {
        hs::network::proto::messages::Message msg;
        if (!reader->Read(&msg))
            break;

        // do something
        ...
    }

    auto status = reader->Finish();
    if (!status.ok())
        std::cerr << "reading topic \"" << m_path << "\" failed:" << status.error_message() << std::endl;

.. note::

    On high images resolutions or slow network connections, streaming can
    introduce a growing delay over time. In these cases, prefer polling to read
    data from the device.

Polling
```````

For polling, data has to be cached internally. In latest versions of the application service,
caching is done automatically. For older versions, you have to enable the output cache using the
:code:`SetReadTopicEnabled` method:

.. code-block:: cpp

    hs::network::proto::SetReadTopicEnabledRequest request;
    // set the path of the output
    request.set_path(<path_to_the_output>);
    // enable ReadTopic 
    request.set_value(true);
    hs::network::proto::SetReadTopicEnabledResponse response;

    auto status = stub->SetReadTopicEnabled(&context, request, &response);
    if (!status.ok())
        throw std::runtime_error("failed to set read topic enabled status of output \"" + m_path + "\": " +
                                 status.error_message());

Then the :code:`ReadTopic` method can be used to read the message from the
application output:

.. code-block:: cpp

    hs::network::proto::ReadTopicRequest request;
    // set the path of the output
    request.set_path(<path_to_the_output>);
    request.set_prev_message_id(0);
    hs::network::proto::messages::Message msg;

    auto status = stub->ReadTopic(&context, request, &msg);
    if (!status.ok())
        throw std::runtime_error("failed to get read topic enabled status of output \"" + <path_to_the_output> + "\": " + status.error_message());

As you can see, a message id can be provided to the request. In the example
above, it is set to zero. This instructs the service to return the data that is
currently cached. If you poll in a loop, this can lead to receiving the same
data multiple times. To avoid this, the message id of the previous message can
be set. :code:`ReadTopic` will then block until a messages with a newer message
id is available.

.. code-block:: cpp

    uint32_t prevMsgId = 0;
    while(true)
    {
        hs::network::proto::ReadTopicRequest request;
        request.set_path(<path_to_the_output>);
        // set the previous message id here
        request.set_prev_message_id(prevMsgId);
        hs::network::proto::ReadTopicResponse response;

        auto status = stub->ReadTopic(&context, request, &response);
        if (!status.ok())
            throw std::runtime_error("failed to get read topic \"" + <path_to_the_output> + "\": " + status.error_message());

        // access message
        auto msg = response.data();

        // do something
        ...

        // keep the current message id
        prevMsgId = response.message_id();
    }


Writing data
^^^^^^^^^^^^

Apps can also offer inputs for receiving data from clients. To send a message to
the app, :code:`WriteTopic` can be used:

.. code-block:: cpp

    hs::network::proto::messages::Message msg;
    // fill message
    ...

    hs::network::proto::WriteTopicRequest request;
    // set the path of the input
    request.set_path(<path_to_the_output>);
    request.set_data(msg);
    hs::network::proto::WriteTopicResponse response;

    auto status = stub->WriteTopic(&context, request, &response);
    if (!status.ok())
        throw std::runtime_error("failed to get write topic \"" + <path_to_the_output> + "\": " + status.error_message());

.. _application_service_properties:

Properties
^^^^^^^^^^

Properties are used for the configuration of an application. At the moment, they
are restricted to the following types: :code:`Int32`, :code:`UInt32`,
:code:`Int64`, :code:`UInt64`, :code:`Float`, :code:`Double`, :code:`Bool`,
:code:`String`, :code:`Enum`.

To get the current value of the property you can use the :code:`GetProperty`
method:

.. code-block:: cpp

    hs::network::proto::GetPropertyRequest request;
    // set the path of the input
    request.set_path(<path_of_the_property>);
    hs::network::proto::Property property;

    auto status = stub->GetProperty(&context, request, &property);
    if (!status.ok())
        throw std::runtime_error("failed to get property \"" + <path_of_the_property> + "\": " + status.error_message());

The property message has a `oneof
<https://developers.google.com/protocol-buffers/docs/proto3#oneof>`_-field which
contains a sub-message depending on the property type. For example, if you want
to get the value of an Int32-Property, you have to access the :code:`prop_int32`
field:

.. code-block:: cpp

    if (property.has_prop_int32())
    {
        hs::network::proto::PropertyInt32 propInt32 = property.prop_int32();
        std::cout << propInt32.value() << std::endl;
    }

.. note::

    Depending on the property type, the messages contain also additional
    information (e.g. min, max, step).

To change the value of a property, use the :code:`SetProperty` method:

.. code-block:: cpp

    hs::network::proto::SetPropertyRequest request;
    // set the path of the input
    request.set_path(<path_of_the_property>);

    // set the property value, use set_value_<value_type> depending on the property type
    request.set_value_int32(<int32_value>);
    hs::network::proto::Property property;

    auto status = stub->SetProperty(&context, request, &property);
    if (!status.ok())
        throw std::runtime_error("failed to set property \"" + <path_of_the_property> + "\": " + status.error_message());

Here the property value is set directly in the :code:`SetPropertyRequest` by
setting the :code:`value_int32`-field. If the property is of a different type, set
another field accordingly.

.. _binning_mode:

Binning Mode
------------

As described :ref:`earlier <application_service_properties>`, the custom application running on the sensor
may expose several properties for tuning its behavior. One property which might be of special interest for
many users is the camera binning mode.

.. note::
    Binning is the (mathematical) combination of several camera pixels into one binned pixel, which therefore
    has a bigger sensor area and is more sensitive esp. in low-light conditions. The read-out time of the 
    imager chip is decreased, which is beneficial for the rolling shutter effect. The IMX477 chip used in the 
    HemiStereo NX sensor, offers 4:1 binning, meaning a decrease in horizontal and vertical camera resolution
    of 50%.

The binning mode is controlled by the camera mode, of which currently are only two available:

- *mode 0*, which is the default 4032x3040 pixel mode, which is the native resolution of the camera chip
- *mode 1*, which is the binned 2016x1520 pixel mode, where each 4 pixels are used in an averaged manner

The mode switch can be triggered using the :ref:`application service <application_service>` interface, by setting
the *camera_mode* property of the camera stereo source. The export name of this property is application
specific, e.g. for the :ref:`stereo app <stereo_app>` it is *source_camera_camera_mode*, which is a UInt64 value,
representing the camera mode.


.. mdinclude:: proto_docs.md