Tutorial: Write your own nodes

Within MARV, nodes are responsible to extract and process data from your log files as base for filtering and visualization. MARV Robotics already ships with a set of nodes (marv_robotics). Here, you find a quick tutorial on writing your own.

All code and configuration of this tutorial is included with your release of MARV Robotics EE in the top-level tutorial folder.

Prerequisites

Create python package

First, you need a python package to hold the code of your nodes. A good name for that package might be your company’s name suffixed with _marv:

$ mkdir code/company_marv  # directory holding python distribution
$ cp tutorial/code/setup.py code/company_marv/

$ mkdir code/company_marv/company_marv  # directory holding python package
$ touch code/company_marv/company_marv/__init__.py

It might make sense that the distribution directory name matches the name provided in setup.py (see below). There, also the python package directory is listed as packages – it must not contain dashes but may contain underscores. One python distribution can contain many packages. At some point you might want to dive into Python Packaging

We placed the Python code of this tutorial into the public domain, so you can freely pick from it. Beware, not to copy the license file and headers and adjust setup.py accordingly, except if you intend to release your code into the public domain as well:

setup.py

# Copyright 2016 - 2018  Ternaris.
# SPDX-License-Identifier: CC0-1.0

from setuptools import setup

setup(
    name='marv-tutorial-code',
    version='1.0',
    description='MARV Tutorial Code',
    url='',
    author='Ternaris',
    author_email='team@ternaris.com',
    license='CC0-1.0',
    packages=['marv_tutorial'],
    install_requires=[
        'marv',
        'marv-robotics',
        'matplotlib',
        'mpld3',
        'plotly',
    ],
    include_package_data=True,
    zip_safe=False,
)

Next, it’s a good idea to place this code under version control:

$ cp tutorial/code/.gitignore code/company_marv/
$ git checkout -b custom
$ git add code/company_marv
$ git commit -m 'Add empty company_marv package'

Finally, for marv to make use of your nodes, you need to install the package into the virtual python enviroment. Install it in development mode (-e) for changes to be picked up without the need to reinstall. Activate the virtualenv first, if it is not already activated. Most of the time we use just $ as prompt you can run the commands also with an activated virtualenv, creating a virtualenv being a notable exception. In case we use (venv) $ as prompt, it has to be activated:

$ source venv/bin/activate
(venv) $ pip install -e code/company_marv

docker:

Tell container to run marv init and install all code in development mode.

$ MARV_INIT=1 DEVELOP=1 ./scripts/run-container site site/scanroot

First node: Extract an image

For sake of simplicity, we are placing all code directly into the package’s __init__.py. Later you might want to split this up into individual modules or packages. Following is the full code of the image extraction node. We’ll be dissecting it shortly.

marv_tutorial/__init__.py

import json
from pathlib import Path

import cv2
import matplotlib.pyplot as plt
import mpld3
import plotly.graph_objects as go

import marv_api as marv
from marv_detail.types_capnp import Section, Widget  # pylint: disable=no-name-in-module
from marv_nodes.types_capnp import File  # pylint: disable=no-name-in-module
from marv_robotics.bag import make_deserialize, raw_messages
from marv_ros.img_tools import imgmsg_to_cv2
@marv.node(File)
@marv.input('cam', marv.select(raw_messages, TOPIC))
def image(cam):
    """Extract first image of input stream to jpg file.

    Args:
        cam: Input stream of raw rosbag messages.

    Yields:
        File instance for first image of input stream.

    """
    # Set output stream title and pull first message
    yield marv.set_header(title=cam.topic)
    msg = yield marv.pull(cam)
    if msg is None:
        return

    # Deserialize raw ros message
    deserialize = make_deserialize(cam)
    rosmsg = deserialize(msg.data)

    # Write image to jpeg and push it to output stream
    name = f"{cam.topic.replace('/', ':')[1:]}.jpg"
    imgfile = yield marv.make_file(name)
    img = imgmsg_to_cv2(rosmsg, 'rgb8')
    cv2.imwrite(imgfile.path, img, (cv2.IMWRITE_JPEG_QUALITY, 60))
    yield marv.push(imgfile)

At first glance, there are three blocks of imports: python standard library, external libraries, and own project. Further, there we define a topic used during the tutorial and a node seems to be based on a Python generator function that uses Yield expressions.

Let’s look at this, piece-by-piece.

Declare image node

@marv.node(File)
@marv.input('cam', marv.select(raw_messages, TOPIC))
def image(cam):
    """Extract first image of input stream to jpg file.

    Args:
        cam: Input stream of raw rosbag messages.

    Yields:
        File instance for first image of input stream.

    """

We are declaring a marv.node using decorator syntax based on a function named image, which becomes also the name of the node. The node will output File messages and consume a selected topic of raw messages as input stream cam. According to the docstring it will return the first image of this stream. The docstring is following the Google Python Style Guide which is understood by Sphinx using Napoleon to generate documentation.

Yield to interact with marv

    # Set output stream title and pull first message
    yield marv.set_header(title=cam.topic)
    msg = yield marv.pull(cam)
    if msg is None:
        return

The input stream’s topic is set as title for the image node’s output stream and we are pulling the first message from the input stream. In case there is none, we simply return without publishing anything.

Yield expressions turn Python functions into generator functions. In short: yield works like return, but preserves the function state to enable the calling context – the marv framework – to reactivate the generator function and resume operation where it left as if it were a function call with optional return value. In case of the second line marv sends the first message of the cam input stream as response to the marv.pull, which will be assigned to the msg variable and operation continues within the image node until the next yield statement or the end of the function.

Deserialize raw message

    # Deserialize raw ros message
    deserialize = make_deserialize(cam)
    rosmsg = deserialize(msg.data)

The raw_messages node pushes raw ROS1 or ROS2 messages, which have to be deserialized. Use make_deserialize to create a deserialize function for all messages pulled from a stream.

Write image to file

    # Write image to jpeg and push it to output stream
    name = f"{cam.topic.replace('/', ':')[1:]}.jpg"
    imgfile = yield marv.make_file(name)
    img = imgmsg_to_cv2(rosmsg, 'rgb8')
    cv2.imwrite(imgfile.path, img, (cv2.IMWRITE_JPEG_QUALITY, 60))
    yield marv.push(imgfile)

Define name for the image file and instruct marv to create a file in its store. Then transform the ros image message into an opencv image and save it to the file. Finally, push the file to the output stream for consumers of our image node to pull it.

Next, we’ll create a detail section that pulls and displays this image.

Show image in detail section

In order to show an image in a detail section, the section needs to be coded and added to the configuration along with the image node created in the previous section.

Code

@marv.node(Section)
@marv.input('title', default='Image')
@marv.input('image', default=image)
def image_section(image, title):
    """Create detail section with one image.

    Args:
        title (str): Title to be displayed for detail section.
        image: marv image file.

    Yields:
        One detail section.

    """
    # pull first image
    img = yield marv.pull(image)
    if img is None:
        return

    # create image widget and section containing it
    widget = {'title': image.title, 'image': {'src': img.relpath}}
    section = {'title': title, 'widgets': [widget]}
    yield marv.push(section)

The image_section node’s output stream contains messages of type Section. It consumes one input parameter title with a default value of Image as well as the output stream of the image node declared previously. In case the image node did not push any message to its output stream, we simply return, without creating a section.

Otherwise, a widget of type image is created and finally a section containing this image is pushed to the output stream.

Next, we are adding our nodes to the configuration.

Config

marv.conf

[marv]
collections = bags

[collection bags]
scanner = marv_robotics.bag:scan
scanroots =
    ./scanroot

nodes =
    marv_nodes:dataset
    marv_robotics.bag:bagmeta
    marv_robotics.detail:bagmeta_table
    marv_robotics.detail:connections_section
    marv_tutorial:image
    marv_tutorial:image_section

detail_summary_widgets =
    bagmeta_table

detail_sections =
    connections_section
    image_section

Note

Remember to stop marv serve, run marv init, and start marv serve again.

Run nodes

(venv:~/site) $ marv run --collection=bags
INFO marv.run qmflhjcp6j.image_section.io4thnkdxx.default (image_section) started
INFO marv.run qmflhjcp6j.image.og54how3rb.default (image) started
INFO marv.run qmflhjcp6j.image.og54how3rb.default finished
INFO marv.run qmflhjcp6j.image_section.io4thnkdxx.default finished
INFO marv.run vmgpndaq6f.image_section.io4thnkdxx.default (image_section) started
INFO marv.run vmgpndaq6f.image.og54how3rb.default (image) started
INFO marv.run vmgpndaq6f.image.og54how3rb.default finished
INFO marv.run vmgpndaq6f.image_section.io4thnkdxx.default finished

Et voilà. Reload your browser (http://localhost:8000) and you should see the detail section with an image. Let’s extract multiple images!

docker: Run commands inside container, after entering it with ./scripts/enter-container.

Persistent nodes and custom output types

If nodes do not declare an output message type @marv.node() they are volatile, will run each time somebody needs them, and they can output arbitrary python objects. In order to use node output in listing_columns or filters, the node needs to be persistent. In order to persist a node in the store it needs to declare an output type @marv.node(TYPE) and be listed in nodes. MARV uses capnp to serialize and persist messages and ships with a couple of pre-defined types, which are available via marv.types. Please take a look at that module and the capnp files it is importing from.

In order to create your own capnp message types, place a module.capnp next to your module.py and take a look at the capnp files shipping with marv as well as the capnp schema language.

Summary

You learned to create a python package and wrote your first nodes to extract images, create a plot and table, and display these in detail sections.

Happy coding!