The sample components came with OpenRTM-aist

OpenRTM-aist comes with some sample components that are easy to use. (OpenCV is required for CameraViewer / OpenCVCamera components.) Also, the source code of the sample component is included, so please refer to it for development.

This chapter describes the samples provided with OpenRTM-aist. If you want to use the sample component, start the name service before starting the sample component. The following programs are provided for the name service.

  • Start Naming Service.bat (C ++)
  • rtm-naming.py (Python)

To run the name server on Windows, locate the file in Explorer and double-click it. If you installed using the installer, you can also start the name service by selecting [Start Menu]> [OpenRTM-aist]> [Start Naming Service]. Note that this chapter assumes that RTSystemEditor is installed in your environment. For details on installing RTSystemEditor, see Installing OpenRTP.

SimpleIO

This sample is included with C ++, Python and Java versions of OpenRTM-aist.

Overview

Sample of how to use InPort and OutPort. Start the ConsoleIn and ConsoleOut components. When the ports are connected, the numbers entered on the ConsoleIn side are displayed on the ConsoleOut side. Ports can be connected by executing rtshell commands in addition to using RTSystemEditor.

Start screen

SimpleIO_example_rtse_ja.png
SimpleIO execution example (RTSystemEditor connection screen)

open_consolein_out.png
Execution example of ConsoleIn and ConsoleOut components

How to use

The sample of SimpleIO is a sample that sends the number input by ConsoleIn from the data port to ConsoleOut and displays the same number on ConsoleOut side. Enter a number from the ConsoleIn screen. Then the number entered in ConsoleOut will be output.

  • Start OpenRTP and open RTSystemEditor. See RTSystemEditor for details on how to use RTSystemEditor
  • Start both ConsoleIn and ConsoleOut components. Component startup differs depending on the OS and OpenRTM-aist language. Start by referring to the table below.
For Windows For Linux
ConsoleIn component ConsoleOut component ConsoleIn component ConsoleOut component
C++ version ConsoleIn.bat ConsoleOut.bat ConsoleInComp ConsoleOutComp
Python version ConsoleIn.bat ConsoleOut.bat ConsoleIn.py ConsoleOut.py
Java version ConsoleIn.bat ConsoleOut.bat ConsoleIn.sh ConsoleOut.sh
  • Drag both components onto the SystemEditor as both components appear in the Name Service View of RTSystemEditor.
  • Connect the ports of both components. (Refer to SimpleIO execution example above)
  • Right-click either component and select [Activate Systems].
  • The "Please input number:" prompt appears on the ConsoleIn screen. Enter the number.
  • The number is displayed on the ConsoleOut screen.

SeqIO

This sample is included with the C++, Python and the Java editions of OpenRTM-aist.

Introduction

This sample shows how to use InPort and OutPort. When you start the SeqIn and SeqOut components, a GUI or console screen will be displayed. Both SeqIn and SeqOut have the following types of DataPort. TimedShort, TimedLong, TimedFLoat, TimedDouble, TimedShortSeq, TimedLongSeq, TimedFLoatSeq, TimedDoubleSeq, Output of each port are determined by random numbers, and when each port is connected, the output value of SeqOut and the input value of SeqIn are shown on the GUI or console screens. (Use RTSystemEditor for connection between ports.)

Screenshots

java_seqio_en.png
Running the SeqIO sample.


SeqIO_example_rtse_en.png
The SeqIO sample viewed in RTSystemEditor.


How to use

The SeqIO sample is a sample that sends numerical data continuously from the output of SeqOut. And the data is sent to the data port of SeqIn. Both output data and input data are displayed on each GUI / console screen. Connection of the corresponding ports of SeqOut and SeqIn are done on RTSystemEditor. When both components are activated, not only SeqOut but also the numerical value on the SeqIn GUI / console changes continuously, and you can observe the input and output of the data port.


You can execute this sample using the following procedure.
    • Start RTSystemEditor.
    • Run SeqIn and SeqOut components. Starting the components varies Depend on the OS and OpenRTM-aist version. Please refer to the following table.
Windows Linux
SeqIn component SeqOut component SeqIn component SeqOut component
C++ SeqIn.bat SeqOut.bat SeqInComp SeqOutComp
Python SeqIn.bat SeqOut.bat SeqIn.py SeqOut.py
Java SeqIn.bat SeqOut.bat SeqIn.sh SeqOut.sh
    • Both components are shown in the Name Service View of RTSystemEditor. Add the RT-Components to the System Editor.
    • Drag and drop between each ports to connect them.
    • Right-click one of the components and activate the components by selecting [Activate Systems]

Autocontrol

This sample comes only with the Python version.

Overview

It is a component that determines data received from InPort based on a certain standard and outputs it to OutPort in another form. It can be used with SeqIn and SeqOut.

When the ports are connected, the output value on the SeqOut side and the input value on the SeqIn side are displayed on their respective console screens. (Use RTSystemEditor for connection between ports.)

autocontrol_example_rtse_en.png
Autocontrol execution example (RTSystemEditor)

seqinout_console.png
SeqIn / SeqOut Console

How to use

Autocontrol compares the fourth element of the data input from Sens port with the parameter "distance_to_env" that can be set in Configuration, and if it is less than or equal to the value, (turn_vel, -turn_vel) is output to vel port. If it is larger than it, (velocity, velocity ) is output to the port. Connect the corresponding ports of Autocntrol, SeqOut, and SeqIn on RTSystemEditor. When all the components are activated, the output value of SeqOut and the input value to SeqIn are displayed on each console screen.

  • Procedure
    • Start RTSystemEditor and prepare SystemEditor. See RTSystemEditor for details on how to use RTSystemEditor.
    • Launch Autocontrol, SeqOut, and SeqIn components by Double-clicking “Autocontrol.bat”, “SeqIn.bat”, “SeqOut.bat” from the directory “\Program Files\OpenRTM-asit\1.2.x\Components\Python” in Explorer.
    • 3 components appear in NameServiceView of RTSystemEditor. Drag and drop them to [System Diagram].
    • Connect the corresponding ports of three components. (Refer to the above example)
    • Right-click either component and select [Activate Systems].
    • Select the Autocontrol0 component from the [System Diagram] and click the [Edit] button in the [Configuration View] at the bottom of the screen. If [Configuration View] is not displayed, [Configuration View] tab should be displayed. Click it.
    • Change the parameter value. In this example, setting [velocity] to 10.0, [turn_velocity] to 1.0, and [distance_to_env] to 5.0 is an easy-to-understand setting example.
    • Notice the fourth data value of "float" displayed on the console screen of SeqOut and the two values ​​of Sequence Data on the console screen of SeqIn. Compare the value of SeqOut with the value of [distance_to_env] and confirm that it is output as [turn_vel],-[turn_vel] or [velocity], [velocity] value on the SeqIn console screen. It is also a good practice to change the configuration parameters to see how the input/output relationship looks.

Composite

This sample is included with C ++, Python and Java versions of OpenRTM-aist.

Overview

This is a sample that shows how to use the composite component. Before activating a composite component, you must connect the child components that make up the composite component.

  • The Composite component has three components, Sensor, Controller, and Motor.
  • You can check the operation of the Composite component by connecting it to the ConsoleIn/ConsoleOut component.

Start screen

  • When the Composite component is executed, 4 component names are displayed in the Name Service View.
  • PeriodicECShareComposite is a component that combines the three components of Controller, Motor, and Sensor. Drag and drop it into the System Editor.
    composite_001.jpg
    Composite execution example(drag&drop composite component)

Double-click PeriodicECShareComposite to display the inside of the composite component. It will show another editor view. Connect the components.

composite_002.jpg
Composite execution example (connect between child components of composite component)

Here is an example of an operation when connecting with the ConsoleIn/ConsoleOut component.

composite_003.jpg
Composite execution example (RTSystemEditor connection screen)

If "111" is entered in ConsoleIn and "888" is displayed in ConsoleOut, the operation is correct. The child components that make up the composite component output twice the input value (the values displayed on the composite component screen).

composite_004.jpg composite_005.jpg
Composite execution example (left: composite component screen, right: ConsoleOut screen)

How to use

In the sample of Composite, the value received from the input data port is doubled and output from output data port of each of the three child components, so the sample finally outputs the 8 times value from the output data port of Composite

Procedure

  • Start RTSystemEditor and open a new SystemEditor. See RTSystemEditor for details on how to use RTSystemEditor
  • Start the Composite component. The startup of the component differs depending on the OS and the language of OpenRTM-aist, so start regarding the table below.
Windows case Linux case
C++ version Composite.bat Composite
Python version Composite.bat Composite.py
Java version Composite.bat Composite.sh
  • Since PeriodicECShareComposite appears in Name Service View of RTSystemEditor, drag them onto SystemEditor.
  • Double click PeriodicECShareComposite and tie the port between the child components
  • Launch both ConsoleIn and ConsoleOut components and connect PeriodicECShareComposite to the corresponding port. (See Composite execution example above)
  • Right-click on one of the components and select [Activate Systems].

SimpleService

This sample is included with the C++, Python and the Java editions of OpenRTM-aist.

Overview

This is a sample that shows how to use ServicePort. The service is defined using the IDL shown below.

 typedef sequence<string> EchoList;
 typedef sequence<float> ValueList;

 interface MyService
 {
   string echo(in string msg);
   EchoList get_echo_history();
   void set_value(in float value);
   float get_value();
   ValueList get_value_history();
 };

Start the MyServiceConsumer and MyServiceProvider components. Activate the Consumer component and a list of available commands will be displayed on the console. Try each command. (Use RTSystemEditor for connection between ports.)

Screenshots

MyService_example_en.png
Running the SimpleService sample.

SimpleService_example_rtse_en.png
The SimpleService sample viewed in RTSystemEditor.

How to use

SimpleService is a sample that sends a command from MyServiceConsumer and causes it to be processed by MyServiceProvider. Connect the corresponding ports of MyServiceConsumer and MyServiceProvider on RTSystemEditor. Activate both components (If the Consumer component is activated, it may activate the Provider too). A list of commands will be displayed at the Consumer prompt, so try to use the commands. The response of the Provider can be observed on the prompt.

  • Procedure
    • Start RTSystemEditor and open a new SystemEditor. See RTSystemEditor for details on how to use RTSystemEditor
    • Start both MyServiceConsumer and MyServiceProvider components.

The startup of the component differs depending on the OS and the language of OpenRTM-aist, so start regarding the table below.

Windows Linux
MyServiceConsumer component MyServiceProvider component MyServiceConsumer component MyServiceProvider component
C++ MyServiceConsumer.bat MyServiceProvider.bat MyServiceConsumerComp MyServiceProviderComp
Python MyServiceConsumer.bat MyServiceProvider.bat MyServiceConsumer.py MyServiceProvider.py
Java MyServiceConsumer.bat MyServiceProvider.bat MyServiceConsumer.sh MyServiceProvider.sh
    • Both components are displayed on Name Service View of RTSystemEditor. Drag and drop the components from the Name Service View into the System Editor.
    • Drag and drop between two ports to connect them.
    • Activate both components by right-clinking one of components and selecting [Activate Systems] (in this case, if the Consumer is activated, the provider is activated too.)
    • Input commands at the Consumer console.
  • Commands
    • echo <message>: Displays a message.
    • set_value <value>: Sets a value to the Provider.
    • get_value: Displays the value set on the Provider.
    • get_echo_history: Displays messages that have been set on the Provider.
    • get_value_history: Displays values that have been set on the Provider.

ExtTrigger

Overview

It is a sample of ExecutionContext triggered by external input (event).
  • Run ExtConsoleIn.bat and ExtConsoleOut.bat to launch the two sample components.
  • After starting both components, execute ExtConnector.bat to connect between the ports of the two components.

Start screen

java_exttrigsample0.png
ExtTrigger execution example(ExtConsoleIn)

java_exttrigsample1.png
ExtTrigger execution example(ExtConsoleOut)

java_exttrigsample2.png
ExtTrigger execution example(ExtConnector)

If the connection between Ports is successful, a menu is displayed on the console that executed ExtConnector to select which component to be triggered. With this input value, each component executes its processing one cycle at a time.

ConfigSample

This sample is included with the C++, Python and the Java editions of OpenRTM-aist.

Introduction

This example shows how to use the configuration-set feature of the RT-Component. When you launch the ConfigSample component, its configuration sets are set up automatically. You can check and alter configuration parameters using RTSystemEditor.

Note: If an error "Specified path cannot be found" occurs, modify the value of "example.ConfigSample.config_file" in rtc.conf file under the RTMExamples\ConfigSample directory. Change it to:

 .\\RTMExamples\\ConfigSample\\configsample.conf
 (Use '\\' instead of '\' for the separator between file name and directory names. )

or rewrite it to the full path of "configsample.conf". In this case, you need to specify '\\' as a directory separator, as well.

Screenshots

ConfigSample_example_rtse_en.png
The ConfigSample sample viewed in RTSystemEditor.

Running the sample

This sample keeps displaying the configuration set data of the component on the console screen. It updates the display when the configuration data is changed in RTSystemEditor.

  • Procedure
    • Start RTSystemEditor.
    • Run ConfigSample. Starting the components varies by operating system and OpenRTM-aist version. Please refer to the following table.
Windows Linux
C++ ConfigSample.bat ConfigSampleComp
Python ConfigSample.bat ConfigSample.py
Java ConfigSample.bat ConfigSample.sh
    • Select an appropriate ConfigurationSet (default, mode0, mode1) on the Configuration View of RTSystemEditor.
    • Change the value as needed.
    • Click [Apply] button.

GUIIn

This sample is included only with Java version of OpenRTM-aist. Please note that it is not included with C ++ version and Python version.

Overview

RT component sample with GUI screen. Execute the GUIIn.bat for Windows and GUIIn.sh for Linux to start the sample component.

Screenshots

GUIIn_example_en.png
Running the GUIIn sample component.

GUIIn_example_rtse_en.png
The GUIIn sample component viewed in RTSystemEditor.

The relationship between each GUI element and Port is as follows.
  • Slider (top): TimedLong type OutPort
  • Text box (medium): TimedString type OutPort
  • Spinner (bottom): TimedDouble type OutPort

Each port transmits input data by connecting the corresponding ports and turning on the check box next to the GUI element. (Use RTSystemEditor for connection between ports. Use other samples such as SeqIn to check output data.)

How to use

If you operate the checked controls (sliders, spinners) on the GUI screen of the GUIIn component, the values ​​on the SeqIn console screen will change as the values ​​change.

  • Procedure
    • Start RTSystemEditor and open a new System Editor. See RTSystemEditor for details on how to use RTSystemEditor
    • Start both GUIIn and SeqIn components. Component startup depends on the operating system. Start GUIIn.bat for Windows and GUIIn.sh for Linux.
    • Since these components appear in the Name Service View of RTSystemEditor, drag both of them to the System Editor.
    • Connect the corresponding ports of both components. (See GUIIn execution example above)
    • Right-click either component and select [Activate Systems].

tkLRFViewer

This sample comes only with the Python version.

Overview

tkLRFViewer is an example of an RTC that displays the output from a Laser Range Finder sensor. Connect and use the RTC that handles the input from the laser range finder. Enter the RTC to be connected according to the device to be connected. For example, see Hokuyo Electric Co., Ltd. URG series. The LRF sensor is a sensor that scans the space while rotating the Laser rangefinder and sends out the measured distance data sequentially. Data is output in the form of the initial value and end value of the rotation angle, the angle interval at which each distance data is measured, and a series of measured distance data. This component is to show the scanned data by using a graphic format.

Start screen

When this component is started, the following GUI screen is displayed.

tklrfviewGui.png
tkLRFViewer GUI screen

How to use

To use this RTC, as described above, read the sensor output from the externally connected Laser Range Finder sensor, convert it, and convert it to a RangeData type and also refer to the above link about the laser range finder sensor.

  • In the Windows environment, this tkLRFViewer component can be started by double-clicking tkLRFViewer.bat in the "Program Files \ OpenRTM-aist \ 1.2.x \ Components \ Python" directory in Explorer. And start the RTC for the sensor and connect to this component by using RTSystemEditor or RTShell.
  • The tkLRFViewer GUI has four sliders that determine the parameters of [Scale Factor], [Threshold], [Filter (Time)], and [Filter (Spacial)]. There are checkboxes for [Axis], [Grid], [Line], [Fill], [Threshold], [Filter (Time)], and [Fileter (Spacial)]. And there is [Reset Scale] button. The functions of each one are shown in the table below.
Name function
Scale Factor When drawing, the distance is based on a space of 480m x 480m, and scale is used to scale the drawing by a scale value. For detecting a general range of about several m x several m, 0.01 It is better to set the value to
Reset Scale button Button to reset to Scale Factor 1.0
Axis checkbox If this checkbox is checked, the X and Y axes will be displayed
Grid checkbox Tick marks are displayed when this checkbox is checked
Line checkbox Distance data is drawn as a line when this checkbox checked
Fill checkbox Distance data is drawn as a filled object when this checkbox checked
Threshold checkbox and slider If checked, the parameter is enabled and the lower limit (*) is processed. If the input distance is smaller than that value, it is considered invalid and the detection distance is assumed to be 1000m
Fileter (Time) checkbox and slider If checked, filters for time-domain changes is enabled and the degree of the effect is adjusted with the slider
Fileter (Spacial) checkbox and slider If checked, filters for changes in scan data in the rotation direction are enabled, and the degree of the effect is adjusted with the slider

Note) The current display scale and the way Threshold works are not effective settings in the real world. It is recommended to rewrite and adjust the source code according to the user environment.

GUI output example

The GUI will output something like the following screen.

tkLRFViewGUIinUse.png
GUI screen in use

TkJoyStick / TkMobileRobotSimulator

TkJoyStick

This sample comes with the Python version of OpenRTM-aist. Please note that it is not included in C ++ version and Java version.

Overview

RT component sample with GUI screen. The sample component is started by executing TkJoyStickComp.bat. The value (x, y) corresponding to the position where the joystick on the GUI is moved by dragging is output.

Screenshots

TkJoystick_en.png
TkJoyStick execution example

How to use

  • Drag the small circle displayed in the center of the screen to output the (x, y) value according to the position of the circle. You can adjust the scale of the output coordinate values ​​by adjusting the [Scale Factor] slider at the top right. At the bottom right of the screen, the value of (x, y) in the rectangular coordinate system and (r, theta) in the polar coordinate system are displayed.
  • There are two OutPorts, from the "pos" value according to the position, from the "vel" value in which the coordinates are turned 45 degrees counterclockwise, for example, if the "pos" output value is (10,10), then the "vel" output value will be (14.142, 0).

TkMobileRobotSimulator

This sample comes with the Python version of OpenRTM-aist. Please note that it is not included in C ++ version and Java version.

Overview

RT component sample with GUI screen. The sample component starts by executing TkMobileRobotSimulator.bat.

Start screen

TkMobileRobotSimulator_en.png
TkMobileRobotSimulator execution example

How to use

For the system to recognize it as an RTC and make the RTC visible in the Name Service, press the [Create] button in the center-right of the GUI. A pentagonal object imitating an object that moves back and forth and left and right with a motor is displayed on the GUI. The pentagonal object on the screen moves according to the input value to InPort. The input to InPort is two value vector values and one is the rotation speed of the left wheel motor and the other is the rotation speed of the right wheel motor. The speed of both wheels must be the same to go straight apex.

System configuration

SysEdit_en.png
RTSystemEditor execution example

How to use

Connect TkJoystick (input device) and TkMobileRobotSimulator, and simulate a robot that the joystick controls the moving speed on the GUI.

  • Procedure
    • Start RTSystemEditor and open a new SystemEditor. See RTSystemEditor for details on how to use RTSystemEditor
    • Start both components by double-clicking TkJoyStickComp.bat and TkMobileRobotSimulator.bat.
    • Click once on the [Create] button in the GUI of TkMobileRobotSimulator.
    • Since both components are displayed in the Name Service View of RTSystemEditor, drag them onto SystemEditor.
    • Connect the corresponding ports of both components. (Refer to RTSystemEditor execution example above)
    • Right-click either component and select [Activate Systems].
    • By dragging the center point on the screen of TkJoystickComp, the pentagonal object (representing the robot) on TkMobileRobotSimulator moves. Moving the TkJoystick component in the Y direction (vertical direction) determines the speed of movement in the direction of the pentagonal object, and moving the component in the X direction (left and right) determines the speed and direction of rotation. (When using TkJoyStick's "vel" OutPort, moving the joystick up and down will make the two-element values of the vector the same. Moving it to the left and right will reverse the sign of them. When this is sent to TkMobileRobotSimulator, it operates as a speed control input to each of the left and right motors, so moving up and down is made, straight-line speed control in the vertex direction in such a way that two motors move in the same direction and at the same speed. Also, the movement of the joystick in the left and right direction causes an output vector in which two elements values are inverted values of the positive and negative and they are the control values of the left and right motor speeds for TkMobileRobotSimulator. In this case, since the motor rotates in the opposite direction at the same speed, the robot rotates on the spot.)

TkMotorComp/SliderComp

TkMotorComp

This sample comes with the Python version of OpenRTM-aist. Please note that it is not included in C ++ version and Java version.

Overview

RT component sample with GUI screen. The sample component is started by executing TkMotorComp.bat.

Screenshots

TkMotorComp.png
The TkMotorComp component's GUI.

SliderComp

This sample comes with the Python version of OpenRTM-aist. Please note that it is not included in C ++ version and Java version.

Overview

RT component sample with GUI screen. Executing SliderComp.bat starts the sample component. (The following screen is for Windows.)

Screenshots

SliderComp.png
The SliderComp component's GUI.

System Construction

RTSE_Slider_Motor_en.png
SliderComp and TkMotorComp on RTSystemEditor

How to use

Simulate the environment of controlling the rotation of the motor with the slide knob on the GUI with SliderComp and TkMotorComp.

  • Procedure
    • Start RTSystemEditor and open a new SystemEditor. See RTSystemEditor for details on how to use RTSystemEditor
    • Start SliderComp.bat and TkMotorComp.bat, both components.
    • Since both components are displayed in the Name Service View of RTSystemEditor, drag them onto SystemEditor.
    • Connect the corresponding ports of both components. (Refer to "SlideComp and TkMotorComp on RTSystemEditor" above)
    • Right-click either component and select [Activate Systems].
    • On the GUI of TkMotorComp, six models simulating a rotating disk driven by a motor are displayed and confirm that the rotations can be controlled by the up / down slide knobs on the GUI of SliderComp. Each GUI shows the six controlling knobs and the rotation of six motors.

TkMotorPosComp/SliderComp

TkMotorPosComp

This sample comes with the Python version of OpenRTM-aist. Please note that it is not included in C ++ version and Java version.

Overview

RT component sample with GUI screen. The sample component is started by executing TkMotorComp.bat.

Screenshots

TkMotorPosComp_en.png
The TkMotorPosComp component's GUI.

SliderComp

This sample comes with the Python version of OpenRTM-aist. Please note that it is not included in C ++ version and Java version.

Overview

RT component sample with GUI screen. Executing SliderComp.bat starts the sample component. (The following screen is for Windows.)

Screenshots

SliderCompN.png
The SliderComp component's GUI.

System Construction

RTSE_Slider_MotorPos_en.png
SliderComp and TkMotorPosComp on RTSystemEditor

How to use

Simulate the environment of controlling the rotation of the motor with the slide knob on the GUI with SliderComp and TkMotorPosComp.

  • Procedure
    • Start RTSystemEditor and open a new SystemEditor. See RTSystemEditor for details on how to use RTSystemEditor
    • Start SliderComp.bat and TkMotorComp.bat, both components.
    • Since both components are displayed in the Name Service View of RTSystemEditor, drag them onto SystemEditor.
    • Connect the corresponding ports of both components. (Refer to "SlideComp and TkMotorPosComp on RTSystemEditor" above)
    • Right-click either component and select [Activate Systems].
    • On the GUI of TkMotorComp, six models simulating a rotating disk driven by a motor are displayed and confirm that the rotation angles can be controlled by the up / down slide knobs on the GUI of SliderComp. Each GUI shows the six controlling knobs and the rotation algle of six motors.

OpenCVCamera/CameraViewer and Simple OpenCV Sample RTCs

This sample is included with OpenRTM-aist C ++ Windows version. Note that OpenRTM-aist is not included in the C ++ version and you need to build and install them on Linux by using OpenCV Sample Code Build Procedure. It is not included in the Python version or the Java version.

Overview

By starting OpenCVCamera and CameraViewer, the image of the USB camera is displayed on the monitor. You can connect the OpenCV image processing sample RTC component and apply effects.

Start Screen

new_00.png
Example of executing OpenCVCamera component and CameraViewer component

new_01.png
OpenCVCamera execution example (RTSystemEditor)

new_02.png
CameraViewer execution example (monitor)

How to use

OpenCVCamera is a sample that acquires image data of a USB camera and displays it on a monitor with the CameraViewer component. You can also connect an Image processing OpenCV sample RT component between them to apply effects to image data.

  • Procedure
    • Start RTSystemEditor and open a new SystemEditor. See RTSystemEditor for details on how to use RTSystemEditor.
    • Start both OpenCVCamera (openCVCamera.bat) and CameraViewer (CameraViewer.bat) components.
      • To use the sample OpenCV image processing RT component, after installation, from Start> OpenRTM-aist 1.2.1 x86_64> C ++ _ OpenCV-Examples (Start in a 32bit environment> OpenRTM-aist 1.2.1 x86> Start from C ++ _ OpenCV-Examples).
    • Since these components appear in the Name Service View of RTSystemEditor, drag both of them to SystemEditor.
    • Connect the corresponding ports of both components. (Refer to RTSystemEditor execution example above)
    • Right-click either component and select [Activate Systems].
  • Use OpenCV's Flip
    • Start the Flip component from Start> OpenRTM-aist 1.2.1 x86_64> C ++ _ OpenCV-Examples (in a 32-bit environment, start from Start> OpenRTM-aist 1.2.1 x86> C ++ _ OpenCV-Examples).
    • Drag it onto the SystemEditor to display it, connect to both the OpenCVCamera and CameraViewer components and activate it. (See "Example of Flip execution (RTSystemEditor)" below)
    • Flip can change the output data by changing the value of Configure “flip_mode”. (Refer to "Change flip_mode (RTSystemEditor and monitor)" below.)
      • Click here for the detailed usage and the explanation of Flip RT component.
        new_05.png
        Example of Flip execution (RTSystemEditor)

new_03_04.png
Change flip_mode (RTSystemEditor and monitor)

Other OpenCV Samples

Startup command Function Configuration parameter
Affine.bat Affine transforms the input image Affine matrix
BackgroundSubtractionSimple.bat Outputs the change from the image when the key was input in the input image Parameter indicating the method of detecting image change
Binarization.bat Converts an input image to a binary black and white image Binarization threshold
DialationErosion.bat Perform dialing / erosion processing Binarization threshold
Edge.bat Outputs the first-order derivative image in the X direction, the first-order derivative image in the Y direction, and the Laplacian image (second-order derivative image) Aperture size
Findcontour.bat Extract the outline and display the outline in the image. Threshold for binarization before processing, level of layering, size of outline at the display, outline approximation method
Histgram.bat Displays changes in histogram while changing brightness / contrast of black and white image Brightness, contrast
Hough.bat Line extraction by Hough transform Hough transform parameters and drawing parameters of detected straight lines
Perspective.bat Perspective conversion of image (Convert as viewed from below. No item
RockPaperScissors.bat Judgment of Goocho kipper in image. Janken judgment threshold for solidity and missing interpolation parameter by dilation / reduction processing
Rotate.bat Rotates and scales images Counterclockwise rotation angle and reduction / enlargement ratio
Scale.bat Processes images for scaling Scale in the X and Y directions
Sepia.bat Separates images Sepia color satulation / hue
Translate.bat Performs two-dimensional translation of images Moving direction

OpenCV Sample Code Build Procedure

On Linux, the bulk installation script can not install OpenCV sample code. To install the OpenCV sample code, you need to build executable code by using the source code. The source code can be gotten from OpenRTM.org github repository. The procedure is following.
  • Install OpenRTM-aist by the bulk installation script. The command option is depends on what language you want to use with OpenRTM-aist. (In case of the following steps, you must install c++ version and Python version at least. Also to execute the components might require OpenRTP. Please specify the options based on your requirements.) For example, the following commands can be used.
     $ sudo sh pkg_install_ubuntu.sh -l all --yes
  • Install OpenCV3.4 or later version (Currently, only 3.4.5 is tested. In some case, build from source code might be required.)
  • By using following steps, getting the source code of OpenCV sample and build it.
     $ sudo apt-get install git
     $ git clone https://github.com/OpenRTM/ImageProcessing.git
     $ cd ImageProcessing/opencv
     $ mkdir build
     $ cd build
     $ cmake ..
     $ make
     $ ./build_linux_package.sh
     $ sudo dpkg -i imageprocessing_1.2.1_amd64.deb

Chromakey

Please note that it is not included in Python and Java versions of OpenRTM-aist. On Linux, build and install it according to OpenCV Sample Code Build Procedure on Linux.

Overview

Chromakey RTC is an OpenCV component example that synthesizes two screens with chroma key. It is used with OpenCVCamera and CameraViewer component.

How to use

Chromakey is an RT component that realizes a technique called chroma key synthesis that makes an image taken with a specific color as the background (for example, green) and make it transparent then synthesize it with another screen. In this execution, it uses two OpenCVCamera components to capture two image data. The output uses CameraViewer component.

  • Synthesizing tow screen images
    foreground.png
    FrontImage

background.png
Background Image

  • Procedure (on Windows environment)
    • Start OpenRTP according to Start procedure of OpenRTP (1.2 series, Windows), start RTSystemEditor, and display RTC in Name Service View. See RTSystemEditor for details on how to use RTSystemEditor.
    • Open a command prompt with administrator privileges to execute two OpenCVCamera components instance.
      • Edit the copied rtc.conf. For example
         > cd "\ Program Files \ OpenRTM-aist \ 1.2.1 \ Components \ C++\vc14"
         > notepad rtc.conf
        Add the following line.
         manager.components.naming_policy: ns_unique 
      • Close the command prompt.
    • Navigate to "\Program Files\OpenRTM-aist\1.2.1\Components\C ++\OpenCV" in Explorer.
    • Double-click CameraViewer.bat.
    • Double-click OpenCVCamera.bat.
    • Double-click OpenCVCamera.bat again.
    • Double-click Chromakey.bat.
    • Click [>] in the Name Service view on the RTSystemEditor screen, and confirm that the components of the launched components CameraViewer0, Chromakey0, OpenCVCamera0, and OpenCVCamera1 are displayed.
    • Click [Open New System Editor] button &ref (icon_open_editor_en.png,150%); at the top of RTSystemEditor to open a new System Editor and display [System Dialgram] newly.
    • Drag&drop the above four components onto the System Diagram.
    • Connect the ports of each component as shown below.
      RTSE_chromakey_en.png
      Chromakey component connection
    • Right-click OpenCVCamera0 and select [Activate] and check if it can be turned to green color. If not, select it and change the "device_num" parameter on Configuration View to appropriate to connected USB camera for front Image. To change it, open the configuration view and click [Edit] button and change the value and click [Apply] button.
    • Right-click OpenCVCamera1 and select [Activate] and check if it can be turned to green color. If not, select it and change the "device_num" parameter on Configuration View to appropriate to connected USB camera for background Image. To change it, open the configuration view and click [Edit] button and change the value and click [Apply] button.
    • Right-click on any component and select [Activate Systems].
    • Display the CameraViewer screen while moving the screen windows.
    • Select the Chromakey0 component on the [System Dialog] (System Editor View). Then the Configuration View is displayed at the bottom. If not, click the [Configuration] tab.
    • Click the [Edit] button to set the chroma key color. Set the blue, green, and red component range values ​​of the RGB values ​​of the color (green in this example) that makes the value of lower_blue / uppder_blue / lower_green / upper_green / lower_red / upper_red to make it transparent as the background in the front image.
    • Check that the two images are synthesized as shown below.
      ChromakeyCameraViewer.png
      Chroma key composite output image

ObjectTracking

Please note that it is not included in Python and Java versions of OpenRTM-aist. On Linux, build and install it according to OpenCV Sample Code Build Procedure.

Overview

ObjectTracking is a sample OpenCV component that tracks an object selected on the screen and show the position by a red oval. It is Used with OpenCVCamera and CameraViewer.

Start screen

ObjectTrackingConsole.png
ObjectTracking Console Screen

How to use

ObjectTracking is a component that tracks the object selected on the screen and indicates its position with a red oval. It is used here with the OpenCVCamera component to capture images from a USB Camera, the CameraViewer component used to display processed images and to select objects with the mouse.

  • Procedure
    • Start OpenRTP according to OpenRTP startup procedure and start RTSystemEditor so that RTC is displayed in Name Service View. See RTSystemEditor for details on how to use RTSystemEditor.
    • Navigate to \Program Files\OpenRTM-aist\1.2.1\Components\C ++\OpenCV in Explorer.
    • Double click CameraViewer.bat.
    • Double-click OpenCVCamera.bat.
    • Double-click ObjectTracker.bat.
    • Click [>] in the Name Service view on the RTSystemEditor screen and confirm that the launched components CameraViewer, ObjectTracking, and OpenCVCamera are displayed.
    • Click [Open New System Editor] button icon_open_editor_ja.png at the top of RTSystemEditor to open a new System Editor and display [System Dialgram] newly.
    • Drag&drop the above three components onto the System Diagram.
    • Connect the ports of each component as shown below.
      RTSE_ObjectTracking.png
      Connection of ObjectTracking component
    • Right-click on any component and select [Activate Systems].
    • Display the CameraViewer screen by adjusting the positions of the windows.
      ObjectTrackingCameraViewer.png
      ObjectTracking output image
    • By Hold down the left mouse button and select the object you want to select. At this time, make sure that the rectangle selection screen appears. (Depending on the case, it may take some time until it comes out.)
    • If the selection is successful, a red oval will appear as shown above. Notice that when you physically move this object, the ellipse moves with it on the screen.
    • When the above-mentioned rectangle is selected, from OutPort of "img_hist", the histogram data of the image of the selected part is displayed for a moment.

ImageSubtraction

Please note that it is not included in Python and Java versions of OpenRTM-aist. On Linux, build and install it according toOpenCV Sample Code Build Procedure

Overview

By starting ImageSubtraction component, the background image is extracted from the input image, the foreground image part is output as a mask image and the background image is output. Used with OpenCVCamera and CameraViewer component. (Note that this component currently does not work properly in a Windows environment, so please use it in Linux (Ubuntu 18.04) environment.)

Start screen

ImageSubtract_exe.png
Execution screen of ImageSubtraction component

How to use

ImageSubtraction is a component for extracting the background image from the input image. Here, it is used together with OpenCVCamera for capturing images from USB Camera and CameraViewer component for displaying processed images.

  • Procedure (The following is the procedure on Ubuntu 18.04.)
    • Open a new terminal.
    • Install the sample code according to the above [[OpenCV sample code build procedure for Linux>//node/6974].
    • Start OpenRTP according to OpenRTP and start RTSystemEditor, and display RTC in Name Service View. Open a new SystemEditor. See RTSystemEditor details on how to use RTSystemEditor
    • Open a new terminal.
    • Create rtc1.conf, rtc2.conf, rtc3.conf by the following steps.  $ cp rtc.conf rtc1.conf  $ cp rtc.conf rtc2.conf  $ cp rtc.conf rtc3.conf
    • Open an editor software to edit each "rtc?.conf" and add the following line to the end of the each files.  naming.formats:% h.host_cxt/CameraViewer?.rtc  (Where "?" Is the number "1", "2", "3")
    • Start a new terminal and input the following commands
       $ cd ImageProcessing/opencv/bin
       $ ./CameraViewerComp
    • Repeat to start a new terminal and execute CameraViewerComp with adding "-f rtc1.conf", "-f rtc2.conf", and "-f rtc3.conf" option. Then, you will see 4 CameraViewer screen.
    • Start another terminal and start OpenCVCamera component by inputing following commands.
       $ cd ImageProcessing/opencv/bin
       $ ./OpenCVCameraComp
    • Start Terminal again to launch ImageSubtraction component.
       $ cd ImageProcessing/opencv/bin
       $ ./ImageSubtractionComp
    • Click [>] in the Name Service view on the RTSystemEditor screen and confirm that the components "CameraView0", "CameraViewer1", "CameraViewer2", "CameraViewe3", "OpenCVCamera", and "ImageSubtraction" are displayed.
    • Click [Open New System Editor] button icon_open_editor_ja.png at the top of RTSystemEditor to open a new System Editor and display [System Dialgram] newly.
    • Drag and drop the above five components onto the System Diagram.
    • Connect the ports of each component as shown below.
      RTSystemEditor_ImageSubtraction_en.png
      Execution example of ImageSubtraction component
    • Right-click on any component and select [Activate Systems].
    • Display the CameraViewer screens by adjusting the positions of the windows like the following.
      compexec.png
      ImageSubtraction Outputs
    • When you move the mouse out of the "capture_image" window, the background image is captured at that timing and displayed it on the back_image. Check what happens to the other screens as well.   The following can be set as configuration parameters.
Parameter name meaning
control_mode b and m can be selected. In the case of b, the background image is captured according to the key event. In the case of m, the mode switches between DYNAMIC_MODE, which determines the threshold a pixel by a pixel, and CONSTANT_MODE, which uses one threshold for the entire screen.
image_height Specifies the number of pixels in the vertical direction, but does not work in this sample.
image_width Specifies the number of pixels in the horizontal direction, but does not work in this sample.
threshold_coefficient Coefficient used in DYNAMIC_MODE
constant_threshold Threshold used in CONSTANT_MODE

SubtractCaptureImage

Please note that it is not included in Python and Java versions of OpenRTM-aist. On Linux, build and install it according to OpenCV Sample Code Build Procedure on Linux.

Overview

SubtractCaptureImage is a component that outputs a mask image that extracts the foreground (moving object) from the input image by determining the unchanged part as the background, and is a sample component that updates the background in real-time using the dynamic background subtraction method. You can use it with OpenCVCamera and CameraViewer components.

How to use

SubtractCaptureImage considers the input image that does not move as the background and sets it as the background and if a newly added thing on the screen does not move and is kept for a while, it is also judged as a part of the background and incorporate it into the background image. The output includes an input image, a mask image of the foreground (indicating a moving object) a mask image indicating neither a moving object nor a background, a background image, and an image showing the background of a still object. To use this component, start multiple CameraViewers and view multiple output images at the same time, or look at each output image while changing the connection with CameraViewer. You may also want to use the OpenCVCamera component for the input image.

  • Procedure (Windows environment)
    • Start OpenRTP according to Start procedure of OpenRTP (1.2 series, Windows), start RTSystemEditor, and display RTC in Name Service View. See RTSystemEditor for details on how to use RTSystemEditor.
    • Open a command prompt with administrator privileges so that multiple CameraViewer components can be executed.
      • Edit rtc.conf. For example, enter the following command to edit.
         > cd "\Program Files\OpenRTM-aist\1.2.1\ Components\C++\OpenCV\vc14"
         > notepad rtc.conf
        And add the following line:
         manager.components.naming_policy: ns_unique
      • Close the command prompt.
    • Navigate to "\Program Files\OpenRTM-aist\1.2.1\Components\C++\OpenCV" in Explorer.
    • Double click CameraViewer.bat.
    • Double-click CameraViewer.bat to launch all five CameraVieweer components.
    • Double-click OpenCVCamera.bat.
    • Double-click SubtractCaptureImage.bat.
    • Click [>] in the Name Service view on the RTSystemEditor screen, and confirm that the components of the started components CameraViewer0, CameraViewer1, CameraViewer2, CameraViewer3, CameraViewer4, SubtractCaputreImage0, OpenCVCamera0, OpenCVCamera1 are displayed.
    • Click [Open New System Editor] button icon_open_editor_ja.png at the top of RTSystemEditor to open a new System Editor and display [System Dialgram] newly.
    • Drag&Drop the above seven components onto the System Diagram.
    • Connect the ports of each component as shown below.
      RTSE_SubtractCaptureImage_en.png
      Connection of SubtractCaptureImage component
    • Right-click on any component and select [Activate Systems].
    • Display the CameraViewer screens while moving the screen windows.
    • First, let's look at the CameraViewer3 screen connected to backGraoundImg OutPort and the CameraViewer0 screen connected to output_image OutPort which is the same screen as the input screen. Check that some images are displayed on both.
  • If you look at the CameraViewer1 screen to which foreMaskImg OutPort is connected, you can see some white color parts at first, but it gradually turned black. This screen shows that the black part is considered the background and the white part is considered the foreground. The part considered as a foreground part is a changing part (a moving thing). Then, after the screen turns black, capture some moving object and you'll see that the moving object appears as a white color object on the screen. Also, in the output of CameraView3, which is StillImg, you can see that when the object moving is stooped, the view of the object gradually becomes to be clear. The below shows the example of the output images from the component.
    SubtractCaptureImageOut1.png
    Image of each output port - Moving object detection
  • The image above is an example of backGroundMaskImg turning black after a while and then slowly moving your finger from the side to the screen. When you look at backGroundMaskImg, you can see that the part of the finger that is moving is regarded as the foreground (moving object) and that the finger is partially regarded as a stationary object at the stillImg OutPort because the movement is slow.
  • The following image is a picture of a small robot controlled by a Raspberry Pi called "Rasp-pi mouse" for a while, and at a certain point, a cat figure is placed over the Rasp-pi mouse. It is an image of the process of being considered as an object and gradually being judged as a background.
    SubtractCaptureImageOut2.png
    Image of each output port-Still object detection, background update
  • StillImg's image shows that the cat figure is considered almost stationary. The algorithm requires more time to consider a stationary object as the background and gradually considers that part as the background. The backGroundImg image shows a part of the cat's figure, and you can see that part is considered the background. After that, backGroundImg changes to be almost the same image as StillImg. ForeMaskImg, on the other hand, has a white area, which is still considered the foreground (moving object image).
  • Since it is difficult to express stillMaskImg as a still image, here only the sample image is shown in whole white. To understand this image, I recommend you to use the component and display the image and check it. Note that this output is normally a white image as a whole, and only the part where the background and foreground (moving object) are not considered, that is, the part where the stationary object is removed immediately before it is determined to be the background An image with black is output in short time.

As of February 7, 2020, this sample is considered to need modification and is being modified. We will update the source code on GitHub soon. Please wait for the update and build and use it. For details on building the source code, refer Development of Image Processing Component.

TkCalibGUI

TkCalibGUI

Please note that it is not included in Python and Java versions of OpenRTM-aist. On Linux, build and install it according to OpenCV sample code for Linux.

Overview

RT component sample with GUI screen. The sample component starts by executing TkCalibGUI.bat. GUI for ImageCalibration component that performs camera calibration.

When this component is executed and a camera component is specified from the selectable list, it is automatically started together with ImageCalibrationComp, and the port connection between the components is also performed automatically. You can save camera images required for calibration and calculate camera parameters by operating the buttons on the GUI.

Component execution conditions

If installed using the OpenRTM-aist C ++ installer for Windows, it only needs to be executed by running TkCalibGUI.bat.

If you build from source, you need to arrange the execution conditions. This component checks the environment required for execution. If the conditions are not met, the following message dialog will be displayed. Follow the instructions to install.

calib9.jpg
Message displayed when execution conditions are not met

For Windows environment

Message contents Correspondence processing
OpenRTM-aist Python is not installed. OpenRTM-aist Python version You can download it.
Please match the OS version (32bit or 64bit) of the installer with the OpenRTM-aist C ++ version.
Ttk is not installed. Please install pyttk. It is included in python2.7, but needs to be installed separately in python2.6.
PIL is not installed. Please install PIL.
NumPy is not installed. Please install numpy.
rtctree is not installed. Requires rtctree-3.0.1 or higher
The installer will be released soon, but until then, github page Click "Download Zip", download and extract the rtc-master.zip file, and execute setup.py to install. (> Python setup.py install) Add the path to site-packages\rtctree\rtmidl to the system environment variable PYTHONPATH.
eg) C:\Python27\Lib\site-packages\rtctree\rtmidl

For Linux (Ubuntu) environment

Message contents Required processing
OpenRTM-aist Python is not installed. Execute Bulk installation script with -l python option.
Ttk is not installed. $ Sudo apt-get install python-tk
PIL is not installed. $ Sudo apt-get install python-pil.imagetk
NumPy is not installed. $ Sudo apt-get install python-numpy
rtctree is not installed. Requires rtctree-3.0.1 or higher
Download from github and run setup.py to install. Add PYTHONPATH to ~/.bashrc or other way.
export PYTHONPATH = "/usr/local/lib/python2.7/dist-packages/rtctree/rtmidl"

Start screen

calib1.jpg

At first, select the camera component to use from the list. This list shows the camera component names that can be used in the current environment. The selection can be done by the input-box at the right top side of the dialog. OpenCVCameraComp and MFCameraComp can be selected in the environment of VC2019 of Windows 10. Both of them work in the same way, but OpenCVCameraCompp seems to be newer and less dependent on the environment. The ImageCalibration component is started by the execution file started by tkCalibGUI.bat. This component can be started alone by ImageCalibration.bat, but we did not provide stand-alone components to connect to the ImageCalibration component in the release. If you want to use the ImageCalibration component without using tkCalibGUI you may need to create the component which can be connected to ImageCalibration component by yourself.

calib2.jpg

Click the [All Activate] button to display the camera image. The connection status of all components at this time is as shown below. Components other than TkCalibGUI are started automatically, and components are also connected automatically.

calib10_en.jpg

How to use

Prepare a checker pattern (calibration pattern). For your reference, we use checkerboard.pdf here. The number of corners of the checker pattern must be specified in the ImageCalibration Configuration. The number of shared vertices of the vertical and horizontal white and black squares of the pattern (checker_h, checker_w) - this is the number of black and white squares-1, and the number of shots (image_num) can also be changed in Configuration.

To capture checker patterns, save (by click [保存] button) the specified number of patterns at various angles to cover the entire screen as much as possible. Click the [確認] button to check the saved image. Clicking the [取り消し] button deletes the image, so please save it again.

calib3.jpg calib4.jpg
calib5.jpg calib6.jpg

After saving the specified number of sheets, the [結果表示] button is enabled. Click this to display the camera parameter values.

calib8.jpg

The camera parameters are output to the same directory as the component executable file with the file name camera.yml.

  %YAML:1.0
  calibration_time: "Thu May 22 16:38:06 2014\n"
  image_width: 640
  image_height: 480
  board_width: 13
  board_height: 9
  cameraMatrix: !!opencv-matrix
     rows: 3
     cols: 3
     dt: d
     data: [ 5.8272934483011682e+002, 0., 3.3703801084645710e+002, 0.,
         5.8023846162074653e+002, 2.2824562602176763e+002, 0., 0., 1. ]
  distCoeffs: !!opencv-matrix
     rows: 1
     cols: 5
     dt: d
     data: [ -1.4659954975042236e-001, 5.7825601645508595e-001,
         -3.3745642103035984e-003, 1.2569676956708463e-003,
         -9.8011775330916773e-001 ]

Image output (reference material)

For reference, when the [保存] button is clicked, the captured image file is output to the directory which is specified by the system environment variable TEMP or TMP.
  • Filename: capture0.jpg ~ capture4.jpg (5 images saved: grayscale image)
    capture0.jpg
The above-captured images's distortion is corrected by using the camera parameter made by this calibration process and the corrected image is output to the same directory.
  • Filename: undistorted0.jpg ~ undistorted4.jpg (File numbers correspond to capture*.jpg)
    undistorted0.jpg