- 1. Introduction
- 2. Getting started
- 3. Tools overview
- 4. Use cases
- 5. Scenario features
- 6. Positioning
- 7. Controllers
- 8. OpenSceneGraph and 3D models
- 9. Support / Q&A
- 10. Command reference
- 11. Build guide
- 11.1. Build configurations in Visual Studio and Visual Studio Code
- 11.2. Build configurations from command line
- 11.3. External dependencies
- 11.4. Additional platform dependencies
- 11.5. Run and Debug with Linux and visual studio code
- 11.6. Dynamic protobuf linking
- 11.7. Slim esmini - customize configration
- 11.8. MSYS2 / MinGW-w64 support
- 11.9. Build esmini project
- 11.10. CentOS 7 (Linux)
- 12. For esmini developers and contributors
- 13. esmini lib programming
- 14. Hello World programming tutorial
- 15. Run ASAM OpenSCENARIO examples
- 16. About this document
- 17. Version
esmini is a software tool to play OpenSCENARIO files. It’s provided both as a stand alone application and as a shared library for linking with custom applications. In addition some tools have been developed to support design and analysis of traffic scenarios.
The scope of this user guide is to provide examples on how to use the tools. It also includes a programming tutorial on how to use esmini in custom applications.
Download latest release from here: https://github.com/esmini/esmini/releases/latest
First time make sure to pick the demo package for your platform (Windows, Linux or Mac). In addition to application binaries it also includes some content like example scenarios and basic 3D models. The binary (bin) packages includes only executables and libraries.
On Windows, make sure the zip file is not blocked: Right click, click Properties, at bottom right check Unblock. For more info, see Blocked by Windows Defender SmartScreen.
To install the package, just unzip it anywhere. A single subfolder named esmini is created. This is the root folder for esmini. No files are stored outside this folder structure and no system files or registry is modified in any way.
Try to run one of the examples:
-
go to folder esmini/run/esmini
-
double click on a .bat file, e.g.
run_cut-in.bator run it from a command line.
These scripts should work on all platforms (in spite extension ".bat").
You can also run the examples explicitly from a command line:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc
For common Mac issues, please see Mac issues and limitations.
The demo package contains only a subset of esmini tools and content (e.g. scenario examples, scripts and models).
To get the complete content, first download or clone the project from GitHub: https://github.com/esmini/esmini. E.g:
git clone https://github.com/esmini/esmini.git
Then either build yourself, see step 2 in next chapter Build esmini - quick guide, or:
-
Download pre-built binary package for your system (e.g. esmini-bin_win_x64.zip) from latest release: https://github.com/esmini/esmini/releases. Copy the content into the esmini-demo folder.
-
Download the complete 3D model package from here. Unpack into esmini/resources (it should end up in a
modelssubfolder). These assets work on all platforms. The environment models (roads, landscape, buildings…) have been created using VIRES Road Network Editor.
Supported systems: Windows, Linux and Mac.
Make sure you have a C++ compiler and CMake installed.
On Windows Visual Studio is recommended (Community/free version is good enough for building esmini). Make sure to check the "Desktop development with C++" package for installation, no more is needed.
On Linux, e.g. Ubuntu, some additional system tools and libraries are needed. Run the following command:
sudo apt install build-essential gdb ninja-build git pkg-config libgl1-mesa-dev libpthread-stubs0-dev libjpeg-dev libxml2-dev libpng-dev libtiff5-dev libgdal-dev libpoppler-dev libdcmtk-dev libgstreamer1.0-dev libgtk2.0-dev libcairo2-dev libpoppler-glib-dev libxrandr-dev libxinerama-dev curl cmakeNow we’re ready to build esmini. From esmini root folder:
mkdir build
cd build
cmake ..
cmake --build . --config Release --target installThe build process automatically downloads 3rd party library binaries and the complete 3D model package.
After successful build, the binaries will be copied into esmini/bin folder. Try from command line:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc
That’s all. Details and variants, see Build guide.
Applications:
| esmini |
Play OpenSCENARIO file. Many options, e.g. view on screen, save images to file or just save log data for post processing. |
| replayer |
Replay recorded esmini scenarios. Main difference from esmini is the ability to freely move forward and backward in the scenario at various speeds. See Use cases how to create recordings ( |
| odrviewer |
Visualize and verify OpenDRIVE road networks. Draw road features, like reference line and lanes, on top of a 3D model (provided or generated). Optionally populate with random traffic that will randomly find its way through the road network. |
| odrplot |
Simple 2D plot of OpenDRIVE road lanes with indicated road IDs, in a two step process. |
| plot_dat |
Simple 2D plot of scenario data |
| dat2csv |
Convert esmini recording (.dat) file to standard .csv format |
| osi2csv.py |
Convert OSI trace file (from esmini) to .csv format See Save OSI data how to create OSI groundtruth trace ( |
| plot_csv |
Simple 2D plot of scenario data, similar to plot_dat |
Shared libraries:
| esminiLib |
High level API for running, controlling and monitoring scenarios See headerfile esminiLib.hpp and Hello World programming tutorial |
| esminiRMLib |
High level API for parsing and query road networks (only road manager) See headerfile esminiRMLib.hpp and code example rm-basic |
Here follows basic examples showing some, but not all, features in esmini and companion tools. It should give an idea of the possibilities and limitations. For a full list of features and functions, see Command reference.
To quickly see available launch options, simply run the corresponding application with no arguments, for example:
./bin/odrviewer
Specify a window and scenario file. Example:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc
Visualize trails from moving entities:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --trail_mode 3
Show entities as bounding boxes:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --bounding_boxes
In addition to several OSG standard camera models esmini adds a set of special purpose camera modes that follows following scenario entities in various ways. Further, the user can add custom cameras via API or launch arguments.
Top level camera mode is selected on keys '1' to '8'. Mode 2-8 are OpenSceneGraph standard cameras. Mode 1 hands over camera control to esmini. In this mode, switch camera model by pressing 'k' which will toggle between all available esmini camera model.
The default camera model is esmini "orbit" which allows for rotating (left-mouse button), zooming (right-mouse button) and panning (scroll wheel / middle mouse button) while following the first entity.
To follow another vehicle press Tab (next vehicle) or Shift TAB (previous vehicle). Also Backspace is mapped to previous vehicle, since shift-TAB do not work on all platforms.
For a complete list of cameras and functions, see Command reference.
esmini camera mode can be selected by launch argument. Here are a few common use cases:
Follow exact behind vehicle:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --camera_mode fixed
Orthogonal (no perspective) top view follow vehicle:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --camera_mode top
Follow vehicle from inside "driver" point of view:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --camera_mode driver
Follow vehicle with some flex and allow rotate, pan and zoom:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --camera_mode flex-orbit
In addition to pre-defined camera modes, the user can add custom cameras from API or via launch argument. Custom cameras can be fixed, semi-fixed (fixed position but unconstrained orientation) or relative current vehicle. Here is a few examples:
Custom camera in front of vehicle, e.g. sensor mount position:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --custom_camera 3,0,0.6,0,0
Custom camera with fixed position but dynamic orientation, always pointing at current vehicle:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --custom_fixed_camera 170,10,5
Custom fixed camera:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --custom_fixed_camera 240,15,10,3.5,0.2
Custom fixed top view camera:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --custom_fixed_top_camera 180,-1.54,2301,4.71239
| Space |
Toggle pause/play simulation |
| Enter |
Step simulation one frame forward |
| Tab |
Move camera to next vehicle |
| Shift-Tab or Backspace |
Move camera to previous vehicle |
| 'j' |
Toggle show entity trail points and lines (press mulitpe times to switch modes) |
| ',' |
Toggle bounding box modes |
| Esc |
Quit |
See Command reference for a complete list of key shortcut commands.
Visualize OpenDRIVE road features:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --road_features on
If a 3D model representation of the road network i 10BC8 s missing (i.e. empty or missing SceneGraphFile element of the OpenSCENARIO RoadNetwork class), then esmini will generate a very simple model.
Example:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc
Add optional flat ground plane:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --ground_plane
The generated model can be saved:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --save_generated_model
Then look for generated_road.osgb in the current directory.
esmini default background color is skyish, light blue. Change by launch argument --clear-color <r,g,b>, where r, g, b are the red, green, blue components as floating numbers in the range (0:1). Some examples:
Black background:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --clear-color 0,0,0
White background:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --clear-color 1,1,1
Gray background:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --clear-color 0.3,0.3,0.3
esmini make use of OpenSceneGraph sub-sampling method for anti-alias. Default setting is 4 (samples). This can be controlled by the launch argument --aa_mode <samples>. To disable anti-alias:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --aa_mode 0
esmini can save screenshots in uncompressed TGA format. These images can be converted into other image formats or even video clips in a post process.
esminiLib also supports grabbing screenshots via API, also from a pure off-screen rendering setup (requiring neither monitor nor graphics hardware). See image-capture code example.
Here follows a few examples using launch arguments and runtime key commands.
Press 'c' at any time to store a single screenshot.
Press 'C' at any time to start storing screen shot for every frame onward. Press 'C' again to stop.
First, run the scenario and automatically save screenshots for all frames:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --fixed_timestep 0.033 --capture_screen
Note We added the --fixed_timestep argument to make sure we get a smooth video with constant delta-time for a smooth and continuous video clip. If --fixed_timestep is skipped esmini will progress time between frames according to actual system time passed since last frame, which can vary over time for many reasons (e.g. due to esmini or other processes CPU load).
Then, create a video clip using ffmpeg:
ffmpeg -f image2 -framerate 30 -i screen_shot_%5d.tga -c:v libx264 -vf format=yuv420p -crf 20 out.mp4
The above ffmpeg command will create a MPEG4 video with reasonable compression (crf 20). To create a lossless, but still compressed, video e.g. for further post processing:
ffmpeg -f image2 -framerate 30 -i screen_shot_%5d.tga -c:v libx264 -qp 0 out.mp4
To change framerate of the output video, use the fps filter. E.g. from input 30 fps to output 15 fps:
ffmpeg -f image2 -framerate 30 -i screen_shot_%5d.tga -c:v libx264 -vf format=yuv420p,fps=15 -crf 20 out.mp4
Get ffmpeg: http://ffmpeg.org/download.html
Some details regarding H.264 encoding can be found here.
Whenever esmini is launched with the --window <x-pos> <y-pos> <width> <height> argument a viewer will be created with specified window. The --headless flag controls whether any window will be "on screen" or only off-screen. Of course, --headless in combination with no window (no --window argument) will skip rendering altogether, which is a very performance-saving mode to use when possible.
Example:
./bin/esmini.exe --window 60 60 4000 2000 --headless --capture_screen --osc .\resources\xosc\cut-in.xosc
will render images into a virtual frame buffer of size 4k x 2k pixels and then store to file. Note: The size of the virtual frame buffer is not limited by size of any connected display.
To run esmini completely without rendering, just omit the --window argument:
./bin/esmini.exe --osc .\resources\xosc\cut-in.xosc --fixed_timestep 0.01 --record sim.dat
will run the specified scenario quickly and store a .dat file for later analysis or viewing (with replayer).
Note: If --fixed_timestep is omitted esmini will adapt timesteps to actual frametime, so a 30 seconds scenario will take 30 seconds - i.e. no performance gain.
By default any available 3D graphics hardware will be utilized. As fallback esmini (via OSG) will utilize Mesa3D which is a software implementation of the OpenGL graphics stack including parts normally hosted in the graphics hardware system. The Mesa3D approach is useful when running on cloud/cluster machines lacking graphics hardware, perhaps even lacking a window system (like slimmed and headless Ubuntu without X11 support).
Mesa3D on Linux
Mesa3D is normally installed with Linux distributions. Check OpenGL support with the following command:
glxinfo -B
If the command is not available, then Mesa3D utility package is probably missing. Install as:
sudo apt install mesa-utils
To run headless with a virtual frame buffer:
-
Activate xvfb:
Xvfb :99 -screen 0 1920x1080x24+32 & export DISPLAY=:99 -
Then run esmini as usual
For troubleshooting, the following command might give information about the graphics system:
lspci -vnn | grep VGA -A 12
Mesa3D on Windows
-
Grab the Mesa3D binary from here.
-
Unpack and place opengl32.dll in the same folder as esmini executable.
-
Then run esmini as usual
esmini can produce log files in different formats and for different purposes, explained next.
By default esmini is creating a log.txt in the folder from which esmini is launched. In case of esminiLib it will end up in the folder that the application linking esminiLib was launched from.
The log.txt includes the same information normally seen in the terminal window (stdout).
To disable output to terminal:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --disable_stdout
To disable creation of the logfile (log.txt):
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --disable_log
Developer info: Define the symbol DEBUG_TRACE (either as compile flag or by uncomment this code line) to log more details, like what code module and line number is the origin of the log entry.
In addition esmini can save a .dat file which captures the state of all entities. This file can later be used either to replay (see Replay scenario) the scenario or converted to .csv for further analysis, e.g. in Excel.
To create a recording with regular timesteps:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --fixed_timestep 0.05 --record sim.dat
To convert the .dat file into .csv, do either of:
./bin/dat2csv sim.dat
or
python ./scripts/dat2csv.py sim.dat
Only a subset of the .dat file information is extracted. To extract some more info, e.g. road coordinates, run:
./scripts/dat2csv --extended sim.dat
You can create .dat files from multiple scenarios in a single command, like this on Linux (bash):
for f in ./my_scenarios/*.xosc; do ./bin/esmini --headless --fixed_timestep 0.05 --osc $f --record `basename $f .xosc`.dat; doneIt also works in Git bash on Windows.
To create a more complete csv logfile, compared to the content of the .dat file, activate the CSV_Logger:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --fixed_timestep 0.05 --csv_logger full_log.csv
full_log.csv will contain more detailed states for all scenario entities. To also include collision detection:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --fixed_timestep 0.05 --csv_logger full_log.csv --collision
All collisions (overlap) between entity bounding boxes will be registered in the collision_ids column of each entity. It will contain the IDs of any entities overlapping at given frame.
Replay a scenario recording (.dat file):
./bin/replayer --window 60 60 800 400 --res_path ./resources --file sim.dat
The --res_path is a special path directive should normally point to esmini/resources folder, where it will look for the OpenDRIVE file, 3D models and model_ids table (model_ids.txt). It also works to add one or multiple --path <path> directives, like for esmini.
Once loaded the scenario recording will start playing in normal speed. Here are a few key commands:
| Space |
Toggle pause/play simulation |
| Tab |
Move camera to next vehicle |
| Shift-Tab or Backspace |
Move camera to previous vehicle |
| 'n' |
Toggle show active trajectories |
| ',' |
Toggle bounding box modes |
| Esc |
Quit |
Arrow keys:
| Left |
Pause and move to previous frame |
| Right |
Pause and move to next frame |
| Shift + Left |
Pause and jump 10 frames back |
| Shift + Right |
Pause and jump 10 frames forward |
| Ctrl + Left |
Jump to beginning |
| Ctrl + Right |
Jump to end |
See Command reference for a complete list of key shortcut commands.
Some launch arguments:
| --collision |
Pause and move to previous frame |
| --quit_at_end |
Quit application when reaching end of scenario |
| --repeat |
Loop scenario |
| --dir <path> |
Directory containing replays to overlay, pair with "file" argument, where "file" is .dat filename match substring |
Examples:
Enable collision detection and pause playing at every occasion of collision:
./bin/replayer --window 60 60 800 400 --file sim.dat --res_path ./resources --collision
The .dat file in the clip above was created with the command:
./bin/esmini --headless --osc ./resources/xosc/ltap-od.xosc --disable_controllers --fixed_timestep 0.033 --record sim.dat
and then launched in replayer:
./bin/replayer --window 60 60 800 400 --file sim.dat --res_path ./resources --collision
Note: The --disable_controllers flag is just specified to disable the interactive controller that normally is activated in that specific scenario. For more information about controllers, see Controllers in esmini.
Combine esmini and replayer in a one-liner command:
./bin/esmini --headless --fixed_timestep 0.05 --record sim.dat --osc ./resources/xosc/cut-in.xosc;./bin/replayer --window 60 60 800 400 --res_path ./resources --file sim.dat
With bash (use Git Bash on Windows) preview all scenarios (.xosc files) in a folder, at double speed:
for f in my_scenarios/*.xosc; do ./bin/esmini --headless --fixed_timestep 0.05 --osc $f --record basename $f .xosc.dat;./bin/replayer --window 60 60 1200 600 --res_path ./resources --file basename $f .xosc.dat --time_scale 2 --quit_at_end; done
just replace my_scenarios with your folder path
Next example, play multiple variants of a scenario in parallel:
./bin/replayer --window 60 60 800 400 --file sim --res_path ./resources --capture_screen --dir tmp
The --dir argument points to a folder containing sim1.dat, sim2.dat and sim3.dat.
The --capture_screen argument will save each frame as an image file on disk.
Merged data from multiple .dat files can be saved into one single .dat file, e.g. for plotting:
./bin/replayer --file sim_ --dir . --save_merged sim_merged.dat
replayer will look in current folder (".") for any .dat file starting with "sim_", i.e. "sim_*.dat". The files will be merged and saved as sim_merged.dat.
By default odrviewer will create a simple 3D model of the OpenDRIVE description and populate it with sparse traffic:
./bin/odrviewer --window 60 60 800 400 --odr ./resources/xodr/fabriksgatan.xodr
Add OpenDRIVE features, like reference line and lanes, on top (toggle on 'o'):
./bin/odrviewer --window 60 60 800 400 --odr ./resources/xodr/fabriksgatan.xodr --road_features
Remove all traffic, just visualize the road geometry:
./bin/odrviewer --window 60 60 800 400 --odr ./resources/xodr/fabriksgatan.xodr --density 0
Increase traffic to exercise all junctions and lanes of the road network:
./bin/odrviewer --window 60 60 800 400 --odr ./resources/xodr/fabriksgatan.xodr --density 7 --road_features
Use custom 3D model:
./bin/odrviewer --window 60 60 800 400 --odr ./resources/xodr/fabriksgatan.xodr --model ./resources/models/fabriksgatan.osgb --density 5
Slow down traffic:
./bin/odrviewer --window 60 60 800 400 --odr ./resources/xodr/fabriksgatan.xodr --model ./resources/models/fabriksgatan.osgb --density 3 --speed_factor 0.6
esmini odrplot is a small application that creates a track.csv file that can be plotted with another small Python script xodr.py:
./bin/odrplot ./resources/xodr/fabriksgatan.xodr
./EnvironmentSimulator/Applications/odrplot/xodr.py track.csv
Combine in a one-liner:
./bin/odrplot ./resources/xodr/fabriksgatan.xodr;./EnvironmentSimulator/Applications/odrplot/xodr.py track.csv
To navigate in the plot, click the four-arrow icon (see image below) and then use mouse according to:
| Left button |
Pan (move left, right, up, down) |
| Right button |
Zoom |
esmini can plot data in two ways: Off-line (plot content of .dat files) and runtime (a set of pre-defined values).
esmini provides a Python based script, plot_dat.py, to plot information in .dat files.
First, create a .dat file. For that we don’t need a window. Run esmini headless:
./bin/esmini --headless --fixed_timestep 0.05 --osc ./resources/xosc/acc-test.xosc --record sim.dat
The scenario looks like this:
Next, plot speed over time:
./scripts/plot_dat.py sim.dat --param speed
To see what parameters are available for plot:
./scripts/plot_dat.py sim.dat --list_params
will output a list similar to:
Plottable parameters:
id, model_id, obj_type, obj_category, ctrl_type, time, speed, wheel_angle, wheel_rot, centerOffsetX, centerOffsetY, centerOffsetZ, width, length, height, scaleMode, visibilityMask, x, y, z, h, p, r, roadId, laneId, offset, t, s
Most parameters does normally not make sense to plot, but it depends on the test case. Maybe it’s interesting to see when a car makes a lane change. Plot laneId over time:
./scripts/plot_dat.py sim.dat --param laneId
A plot can show multiple parameters:
./scripts/plot_dat.py sim.dat --param laneId --param speed
To plot trajectories, change X-axis parameter from time to x and plot y parameter (over x):
./scripts/plot_dat.py sim.dat --x_axis x --param y
Lock axis aspect ratio, i.e. make axis scale equal:
./scripts/plot_dat.py sim.dat --x_axis x --param y --equal_axis_aspect
Plot merged .dat files (created as described in Save merged .dat files)
./scripts/plot_dat.py --param speed sim_merged.dat
Entity IDs will get an offset multiplier of 100, corresponding to the order of which the dat files were read by replayer. The mapping between ID range and dat file is printed to the console by replayer when the merged dat file is created, as the following example of four input files:
Scenarios corresponding to IDs (0:99): sim_fsm.dat Scenarios corresponding to IDs (100:199): sim_ref.dat Scenarios corresponding to IDs (200:299): sim_reg.dat Scenarios corresponding to IDs (300:399): sim_rss.dat
From version 2.32.0 esmini can plot some pre-configured data values during simulation. Simply add launch flag --plot.
Features so far:
-
One or multiple entities can be selected by checkboxes on top
-
Selection of variables can be reduced by checkboxes to the right
By default the plotting is executing in a separate thread, having minimal impact on esmini performance. It is also possible to execute in synchronous mode, where plotting is part of esmini main thread and stepping is done in synch. The mode can be specified explicitly. Examples:
Simply enable plotting, default mode:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --fixed_timestep 0.05 --plot
Enforce synchronous mode:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --fixed_timestep 0.05 --plot synchronous
Asynchronous mode can be specified as well, although it’s default hence not needed:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --fixed_timestep 0.05 --plot asynchronous
Note: On macos OpenGL window must be part of main thread. Hence synchronous mode is enforced on macos.
esmini can provide OSI groundtruth data in three ways:
-
Send over UDP
-
API to fetch via function call from a custom application
-
Save to OSI trace file
Only trace file way is described here. To save OSI data, add argument --osi_file [filename]. The filename is optional. If omitted it will be named ground_truth.osi. Example:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --osi_file
will create ground_truth.osi in the current folder. The format is according to the OSI standard "Binary trace file", see OSI documentation 2.2.6 OSI trace files.
esmini provides a script, osi2csv.py, that converts an OSI trace file into .csv format. The script can also serve as example how to parse and extract data from an OSI trace file:
./scripts/osi2csv.py ground_truth.osi
will create ground_truth.csv.
(if file type association is not setup, try: python ./scripts/osi2csv.py ground_truth.osi)
However the .csv file created with osi2csv will not contain all OSI information. To get a readable text file including the complete content of a OSI trace file you can make use of the Python script format/osi2read.py in the OSI project.
Use as following example:
../open-simulation-interface/format/osi2read.py --data ./ground_truth.osi --output ground_truth --type GroundTruth
it should create ground_truth.txth which is readable in a text editor.
Note:
See OSI for Python on Windows for hints how to install dependencies (OSI and Protobuf).
The above script link points to OSI tag v3.5.0. In more recent OSI versions the script has moved to another folder: osi3trace/osi2read.py.
(introduced in esmini v2.23.0)
The OpenSCENARIO SpeedAction will reach a specified speed in a way described by additional attributes, e.g. shape and duration. To achieve multiple speed target over time you would need to add multiple SpeedAction events to the scenario. Moreover, there is no way to explicitly control the jerk (acceleration/deceleration rates).
However with the SpeedProfileAction (introduced in OpenSCENARIO v1.2) you can specify a series of speed targets over time, in one action. Optionally you can also specify dynamic constraints for jerk (gradually changing acceleration and deceleration), acceleration and speed.
If you run the example scenario speed-profile.xosc as follows:
./bin/esmini.exe --window 60 60 800 400 --osc ./resources/xosc/speed-profile.xosc --fixed_timestep 0.01 --record sim.dat
./scripts/plot_dat.py sim.dat --param speedyou should get the following plotted speed curves:
Note: You might need to add python or python2 in front of the python command. And matplotlib is needed, see odrplot in Tools overview.
Tip: For quick experiments, skip the visualization and bring the plot asap by replacing the two commands with:
./bin/esmini.exe --headless --osc ./resources/xosc/speed-profile.xosc --fixed_timestep 0.01 --record sim.dat;./scripts/plot_dat.py sim.dat --param speedThe scenario includes two cars with identical speed profiles with one exception: The white Car1 use activate dynamic constraints by setting FollowingMode="follow" while the red Car2 will apply constant acceleration (linear interpolation) between speed targets by setting FollowingMode="position".
The action for white car:
<SpeedProfileAction followingMode="follow">
<DynamicConstraints
maxAcceleration = "5.0"
maxDeceleration = "10.0"
maxAccelerationRate = "4.0"
maxDecelerationRate = "3.0"
maxSpeed = "50"
/>
<SpeedProfileEntry time="0.0" speed="0.0"/>
<SpeedProfileEntry time="4.0" speed="10.0"/>
<SpeedProfileEntry time="4.0" speed="4.0"/>
<SpeedProfileEntry time="2.0" speed="8.0"/>
</SpeedProfileAction>The time values are relative each other and start of the action. In this case the action is triggered at simulation time = 2 seconds. Initial speed for both cars is 0. First entry has therefor no effect since it applies speed = 0 at time = 2 (2 + 0). After additional 4 seconds (sim time = 6s) the speed target is 10 m/s. At sim time 10s the speed target is 4 m/s and finally after 2 more seconds the final speed target value is 8.0 m/s.
The "follow" mode deserves some additional explanation. As shown in the figure, it start and ends with zero acceleration. Then is basically will try to match the acceleration "lines" but cutting the corners according to acceleration and deceleration rate constraints. This way the intermediate speed values will not always be reached. However, the final speed value will be reached.
If the target speed or accelerations can’t be reached with given constraints the action will revert to linear mode (FollowingMode="position") for the remainder of the profile. This "failure" is logged.
Another approach would be to try to perform a best effort, but that would require additional input to decide whether to prioritize reaching specified speed targets or respect time stamps…
Note: The implementation of this feature is preliminary and experimental. Behavior and details might change.
Let’s manipulate the scenario in different ways to illustrate some special cases of the speed-profile feature.
(Special case implementation introduced in esmini v2.23.1)
Compared to SpeedAction, the SpeedProfileAction offers more tools in terms of dynamic constraints. Hence it can be actually be useful also for single entry, i.e. reach a single target speed.
The implementation differs for the single entry case. Target speed will be reached if constraints allows for it. If not, the speed will still be reached, but later than specified.
There are three sub cases:
1. Speed can be reached within time
The speed profile will contain three phases: Jerk, constant acceleration, jerk.
Example:
Replace the four entries in speed-profile.xosc with the following ones:
<SpeedProfileEntry time="0.0" speed="0.0"/>
<SpeedProfileEntry time="4.0" speed="10.0"/>Initial positive jerk will be applied until necessary acceleration is reached. Keep constant acceleration (linear segment in the speed profile) until negative jerk needs to be applied in order to reach target speed on time and at zero acceleration.
2. Speed can’t be reached in time due to acceleration constraints
This speed profile will also contain three phases: Jerk, constant acceleration, jerk.
Example:
Replace the four entries in speed-profile.xosc with the following ones:
<SpeedProfileEntry time="0.0" speed="0.0"/>
<SpeedProfileEntry time="3.0" speed=" 10.0"/>Initial positive jerk will be applied until maximum acceleration is reached (or maximum deceleration). Keep constant acceleration (linear segment in the speed profile) until negative jerk needs to be applied in order to reach target speed at zero acceleration. Due to the acceleration limitation there will be a delay as well. The log file will include something like:
SpeedProfile: Constraining acceleration from 5.86 to 5.00
SpeedProfile: Extend 0.46 s3. Speed can’t be reached in time due to jerk constraints
This speed profile will contain only two jerk phases.
Example:
Keep entries from last case, but change jerk settings as follows:
maxAccelerationRate="3.0"
maxDecelerationRate="2.0"In this case the jerk settings are too weak to reach target speed in time. Not enough acceleration can be achieved in the given time window.
Positive jerk will be applied until negative jerk has to be applied in order to reach target speed at zero acceleration. Hence there is no room for a phase of constant acceleration. Due to the jerk limitation there will be a delay. The log file will include something like:
SpeedProfile: Can't reach target speed 10.00 on target time 3.00s with given jerk constraints, extend to 4.08sReplace the four entries with the following ones:
<SpeedProfileEntry time="0.0" speed="3.0"/>
<SpeedProfileEntry time="4.0" speed="10.0"/>What we see here is that for linear mode (FollowingMode="position") the speed of the first entry will apply immediately regardless of the current speed at the time of the action being triggered. For constrained mode (FollowingMode="follow") we see that the initial speed value (3.0) is overridden by the current speed (0.0). From there it will strive for the second entry, obeying the constraints.
The overall idea with the "follow" mode is to maintain continuity in the speed profile, up to jerk degree.
Replace any entries with the following ones:
<SpeedProfileEntry speed="10.0"/>Specified max acceleration will be applied until target speed is reached. Note: In the non-linear case and with multiple entries, the function will fail if the specified acceleration can’t be reached with given jerk constraints (maxAcceleration and maxDeceleration). Try to lower the maxAcceleration/deceleration in this case.
You can also combine entries with and without time constraint, like in following example:
<SpeedProfileEntry speed="10.0"/>
<SpeedProfileEntry time="3.0" speed="15.0"/>Car will accelerate until speed 10 m/s is reached, then spend 3 seconds to reach 15 m/s.
(from release v2.23.2)
What if the acceleration is not zero when the SpeedProfileAction is started, for example interrupting an ongoing SpeedAction in Follow mode?
To maintain a continuous acceleration profile the action will use current acceleration as initial value. The standard states that the acceleration is expected to be zero at start and end of the action. The esmini interpretation is that the CHANGE is zero at start while the ACTUAL value is zero at end (since the action can only control acceleration while being active, not before).
Example: Once again starting from speed-profile.xosc, instead of setting an instant initial speed make it ramp up from 0 to 5 m/s over a duration of 4 seconds by tweaking the initial speed actions, for both entities, as below:
<SpeedActionDynamics dynamicsShape="linear" value="5.0" dynamicsDimension="time"/>
<SpeedActionTarget>
<AbsoluteTargetSpeed value="4.0"/>
</SpeedActionTarget>The initial speed action will apply constant acceleration until time = 2.0 seconds, when the SpeedProfileAction is triggered. While the linear profile will jump to 0 m/s (as specified in first entry) the follow mode profile will just apply necessary jerk to reach target acceleration with respect to following entries.
Initial acceleration is also respected for the special case of a single entry, for example:
<SpeedProfileEntry time="3.0" speed="10.0"/>the result becomes:
To skip jerk phase, e.g. as in emergency brake with an abrupt stop, just set AccelerationRate to a large number.
Example: Once again starting from speed-profile.xosc, change speed in the Init speed actions to 10.0 and replace the speed profile actions as below:
<SpeedProfileAction followingMode="follow">
<DynamicConstraints
maxAcceleration = "5.0"
maxDeceleration = "10.0"
maxAccelerationRate = "1E10"
maxDecelerationRate = "3.0"
maxSpeed = "50"
/>
<SpeedProfileEntry time="0.0" speed="10.0"/>
<SpeedProfileEntry time="4.0" speed="0.0"/>
</SpeedProfileAction>Note: A drawback is that the setting will affect any acceleration phase in the complete profile. In other words, a limitation is that entries can’t have individual settings. In the example above it’s not problem since the first jerk phase is a deceleration while the second is an acceleration. If different rate values of same type are needed, it can be achieved by defining multiple SpeedProfile actions in a sequence, with individual performance settings.
To get more understanding of the implementation, see a few slides here.
(framework updated in esmini v2.25.0)
Road signs are specified in the OpenDRIVE road network description file. A road sign is identified by up to four parameters:
-
country code
-
type
-
subtype
-
value
Country code is a two letter word according to the ISO 3166-1 alpha-2 system. Examples: Sweden = se, Germany = de.
Type and subtype defines the semantic meaning of the sign. The value is used when applicable to further specify information, e.g. speed or weight. Data type and meaning of the parameters may differ between countries. However a mapping to each country’s system is often feasible.
Examples: In Sweden a speed sign belongs to the category prohibition signs named "c". Within that category speed signs is a sub group with index 31. Within that group each sign has a value of speed/10, e.g. 5=50km/h, 10=100km/h. So the parameters for a Swedish speed limit 90 km/h sign becomes: se, c, 31, 9.
In Germany there are no similar categories. Instead all signs have a unique group or type number, e.g. speed signs make up group 274. Then the value defines the speed. No need for subtype. So the parameters for a German speed limit 90 km/h sign becomes: de, 274, -, 90.
Instead of hard-coding road sign support esmini will lookup the OSI code from a separate sign definition file, which is unique for each country. Name convention for the file is: country code + "_traffic_signals.txt". Example: Swedish catalog is named "se_traffic_signals.txt".
The file is a simple text format each line defining a sign ID and corresponding OSI type.
Swedish example: c.31-7=TYPE_SPEED_LIMIT_BEGIN
German example: 274=TYPE_SPEED_LIMIT_BEGIN
Type is mandatory. For esmini to distinguish between subtype and value different separators are used: "." before subtype and "-" before value. For some signs the value is optional, e.g. for speed signs it simply specify the speed limit. But for some signs it is used as a subtype to identify a variant of a sign, as in the German 101 type category.
Example files can be found in esmini/resources/traffic_signals.
Currently (v2.25.0) the esmini sign support is very limited. However, it should be fairly straight forward to add signs into existing catalogs and even catalogs (for more countries).
esmini road signs are created as follows:
-
A basic 3D geometry is modeled in Blender, e.g. round face for speed signs
-
Assign a material with any texture image mapped on the sign face
-
The 3D model is exported in
.daeformat (example here) -
Find textures of the needed sign(s) and convert them into some common format, e.g.
.png. Recommended size is 256, but larger is OK. If possible make it size 2^n x 2^m to avoid performance penalty on low spec/old systems. -
Rename image to the same name as the texture image in the dae file (sign_image.png in the example dae)
-
Convert the .dae file into .osgb using osgconv. The texture will be embedded in the.osgb file by default.
A script is available as a starting point for automating the two last steps in the process above.
The OpenDRIVE file straight_500m_signs.xodr includes a bunch of speed signs, some Swedish and some German ones. It shows how the signs are specified in terms of the country code, type, subtype and value attributes.
It also shows how to specify the 3D model using the name attribute: If filename composed by country, type, subtype and value is not found, the name attribute + ".osgb" will be used for a second and final attempt.
From OpenSCENARIO v1.1 expressions including mathematical operations are supported in assignment of value to OpenSCENARIO parameters and XML element attributes. Examples:
<ParameterDeclaration name="PI" parameterType="double" value="3.14159"/>
<ParameterDeclaration name="KPH2MPS" parameterType="double" value="1/3.6"/>
...
<Position>
<LanePosition roadId="1" laneId="-1" s="40" offset="0">
<Orientation type="absolute" h="${sin(0.25*$PI)}"/>
</LanePosition>
</Position>
...
<SpeedActionTarget>
<AbsoluteTargetSpeed value="${110 * $KPH2MPS}"/>
</SpeedActionTarget>h and (speed value) evaluates to 0.707106 and 30.55558 respectively
|
|
|
|
|
multiply |
|
|
|
IEEE 754 remainder, see C++ std::remainder |
|
power |
|
|
|
|
|
|
|
|
|
equal |
|
not equal |
|
logical AND |
|
logical OR |
See expr.h for complete and updated list.
|
see C++ rint (Round to nearest, halfway cases to even. Aligned with IEEE 754 default) |
|
see C++ floor |
|
see C++ ceil |
|
square root |
|
see C++ pow |
Following functions are supported by esmini, but not specified in OpenSCENARIO <= 1.2, hence use with care since they may not work in other tools
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See simple_expr.c for complete and updated list.
From v2.35.0 string concatenation is supported. Whenever a string is detected that is netiher an opeator nor a function, the complete expression is treated as a string and '+' (plus) operator will merge surrounding evaluated strings which also can be evaluated parameters.
This can be useful when running permutations of scenarios, where names of actors or storyboard elements can be made unique by including parameters values.
Examples:
${Event_overtake_ + $ActorName} → Event_overtake_Target4
${Event_change_speed_to_ + $Speed} → Event_change_speed_to_110
Parameter values and distributions can be specified in a separate Parameter value distribution file. This is a convenient way of running multiple variations, or parameter value permutations, of a single scenario.
Here’s an example: cut-in_parameter_set.xosc.
To run all permutations with esmini:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc
How to abort:
"ESC" (in window) will terminate current run
"Ctrl-C" (in terminal) will terminate all runs
The log files are now named according to:
log_<permutation>_of_<total#permutations>.txt
For example:
log_1_of_6.txt relates to the 1:th run of 6, permutation index 0
log_4_of_6.txt relates to the 4:th run of 6, permutation index 3
Of course this will apply also to custom log filenames. Example:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --logfile_path my_scenario.txt
will result in my_scenario_1_of_6.txt…
Also any recording (.dat), osi and csv files will be named in similar way. Example:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --record sim.dat --osi_file gt.osi --csv_logger data_log.csv
will result in complete sets of files:
log_1_of_6.txt …
sim_1_of_6.dat …
gt_1_of_6.osi …
data_log_1_of_6.csv …
To run a specific permutation only, specify index like this:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --param_permutation 2
will run 3:rd permutation (of 6 available in this case)
Save the resulting OpenSCENARIO file with populated parameter values:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --save_xosc
will give cut-in_1_of_6.xosc…
To view all results in parallel, make use of replayer like this:
./bin/esmini --headless --fixed_timestep 0.05 --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --record sim.dat ; ./bin/replayer.exe --window 60 60 800 400 --res_path ./resources/ --file sim_ --dir .
Making use of Python threading pool framework we can utilize any multiple CPU kernels and run scenario variants in parallel. This is handled by the script scripts/run_distribution.py.
Usage: run_distribution.py <esmini args>
Make sure to add at least the following esmini arguments:
--osc <scenario file>
--param_dist <parameter distribution file>
--fixed_timestep <timestep>
--headless
Example:
python ./scripts/run_distribution.py --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --fixed_timestep 0.05 --headless
Of course, the --headless (and --fixed_timestep) is not required. It is possible to run with viewer as well:
python ./scripts/run_distribution.py --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --window 60 60 800 400
but the windows will open on top of each other at the same location on the screen, so you need to manually move them apart to see the parallel runs. Anyway, this use case is probably not relevant so the recommendation is to run headless with fixed time step.
Putting it all together, execute the scenario permutations in parallel and then view the result in replayer:
python ./scripts/run_distribution.py --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --fixed_timestep 0.05 --headless --record sim.dat ; ./bin/replayer.exe --window 60 60 800 400 --res_path ./resources/ --file sim_ --dir .
To find out the number of permutations of a specific scenario and parameter distribution, use the --return_nr_permutations launch argument. Example:
./bin/esmini --osc ./resources/xosc/cut-in.xosc --param_dist ./resources/xosc/cut-in_parameter_set.xosc --return_nr_permutations
Output:
Nr permutations: 6
For scripters: esmini will also return the number of permutations as exit code, for simple use in scripts. After executing the above the exit code can be inspected/verified from shell as well:
PowerShell:
echo $LastExitCode
6
bash:
echo $?
6
Using OpenSCENARIO FollowTrajectoryAction you can define various types of trajectories for entities to follow.
esmini supports all three (as of OpenSCENARIO 1.2) shapes:
-
polyline
-
clothoid
-
nurbs
Here follows a few design choices for the polyline type that affect the behavior.
When FollowMode is follow esmini will:
-
Calculate heading based on direction of the line segments. However corners are interpolated in order to look better and avoid discontinuities.
-
Calculate speed based on current speed (when action is triggered) and then with constant acceleration per segment reach destination vertex on time (or earlier if distance is too short vs time). This approach results in a continuous speed profile.
When FollowMode is position esmini will:
-
For world coordinates, apply whatever heading is specified in the vertices (note that 0.0 is default). Lane and road coordinates will align to road direction as default.
-
For each segment, calculate constant speed needed to reach next vertex on time. This approach results in a discontinuous speed profile.
From v2.37.0 esmini supports road friction defined in OpenDRIVE lane material.
Each lane within a lane section can have one or multiple segments of individual friction values, specified by s-value and friction coefficient.
Default friction coefficient is 1.0 as of v2.37.0, but might change in later versions. Lower friction (<1.0, slippery) will be visualized as bluish color. The lower friction, the more blue color. High friction values (>1.0, "grippy") is indicated by redish color.
The friction is calculated per wheel. The values are reported in OSI ground-truth.
Example road: straight_highway_500m.xodr (used by test scenario friction_and_lane_change_edge_case.xosc)
SUMO is a great open source traffic simulation tool. It has been integrated with esmini, making it possible to run SUMO simulations in esmini, and even mix or co-simulate SUMO vehicles with OpenSCENARIO vehicles.
NOTE: Even though SUMO has been linked with esmini for quite long, it’s on a experimental level and has not been used a lot. Hence, expect shortcomings and issues. The currently integrated SUMO version, as of esmini v2.37.1, is 1.6.0 while the current SUMO version is 1.19.0. There is plans to update SUMO version in esmini in a soon future.
This chapter explains how to create a SUMO simulation and then how to use it in esmini. For just using SUMO simulations there is no need to install SUMO, it is already embedded with esmini (as long as it has not been explicitly excluded, see Slim esmini - customize configration). Only to create SUMO content it has to be installed.
-
Get SUMO from here: https://eclipse.dev/sumo/
-
Install normally, check any option to add sumo to environment path variable
-
Launch
sumo-guiapplication -
File → Open Simulation
-
Navigate to
esmini/resources/sumo_inputsand selectmulti_intersections.sumocfg -
Set Delay (ms) to 100, otherwise simulation will run super-quick
-
Run (Play button or Simulation → Run)
-
In a terminal, from esmini root folder, run:
-
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/sumo-test.xosc
First, there is a great complete guide at https://sumo.dlr.de/docs/
The main flow of our example is:
-
Convert an OpenDRIVE file into a SUMO road network
-
Identify some trips (start and end points) within the road network
-
From the trips, resolve complete routes wthich are assigned to vehicles and distribute over time
-
Create a configuration referring to the road network and route definition
Step-by-step:
-
Create a new folder, e.g. "my_sumo", for this example
-
Copy esmini/EnvironmentSimulator/Unittest/xodr/simple_3way_motorway.xodr into the new folder
-
Open a terminal in the new folder
-
Convert the xodr into a SUMO road network:
netconvert --opendrive fabriksgatan.xodr -o my_sumo.net.xml -
Create some traffic using randomTrips.py (and duarouter):
randomTrips.py -n my_sumo.net.xml -e 60 -o my_sumo.trips.xml --route-file my_sumo.rou.xmlIf this fails, it may be because of lacking .py file association. Try:
python3 randomTrips.py ... python randomTrips.py ... python3 tools/randomTrips.py ... python tools/randomTrips.py ...
-
Create the configuration file
Save the following lines into a new file named "my_sumo.sumocfg":
<?xml version="1.0" encoding="iso-8859-1"?> <configuration> <input> <net-file value="my_sumo.net.xml"/> <route-files value="my_sumo.rou.xml"/> </input> <time> <begin value="0"/> <end value="60"/> <step-length value="0.01"/> </time> </configuration>
-
Open Sumo, load the created config file "my_sumo.sumocfg" and run as in first example, Try example from esmini.
Currently, as of esmini v2.37.1, SUMO is integrated in terms of a controller. To add a SUMO simulation, define an entity as usual, assign the SumoController and set its "filepath" property to the sumocfg file.
Now when we have a complete SUMO configuration, do the following steps in order to use it in esmini:
-
Copy the three files: my_sumo.net.xml, my_sumo.rou.xml and my_sumo.sumocfg (roadnetwork, routes, config) to
esmini/resources/sumo_inputs -
Create a file named "my_sumo.xosc", in
esmini/resources/xoscwith the following content:<?xml version="1.0" encoding="UTF-8"?> <OpenSCENARIO> <FileHeader author="esmini team" date="2024-03-06T10:00:00" description="sumo integration example" revMajor="1" revMinor="2"/> <ParameterDeclarations/> <CatalogLocations> <VehicleCatalog> <Directory path="../xosc/Catalogs/Vehicles"/> </VehicleCatalog> <ControllerCatalog> <Directory path="../xosc/Catalogs/Controllers"/> </ControllerCatalog> </CatalogLocations> <RoadNetwork> <LogicFile filepath="../xodr/fabriksgatan.xodr"/> </RoadNetwork> <Entities> <ScenarioObject name="SumoVehicles"> <CatalogReference catalogName="VehicleCatalog" entryName="car_red"/> <ObjectController> <Controller name="SumoController"> <Properties> <File filepath="../sumo_inputs/my_sumo.sumocfg"/> </Properties> </Controller> </ObjectController> </ScenarioObject> </Entities> <Storyboard> <Init> <Actions/> </Init> <StopTrigger/> </Storyboard> </OpenSCENARIO>
-
Run from esmini root folder:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/my_sumo.xosc
esmini supports all OpenSCENARIO position types. To support various use cases esmini add some mechanisms and interpretations:
-
Align by omit: Lacking position component values (x, y, z, heading, pitch, roll) in the scenario file is interpreted as "please align this component to the road surface". E.g. missing z value will project the position on the road surface, missing pitch value will ensure object is aligned with road pitch (e.g. up- and downhill).
-
PositionMode: Any position/object can be set and configured with respect to alignment via API calls. For example: x, y and heading can be set as absolute values while pitch and roll can be set as relative to the road surface.
See more details below in Position modes
-
Trajectory interpolation: Based omitted position components and followingMode setting esmini will apply different mechanisms to follow and interpolate trajectory vertices/control points.
See details below in Trajectory interpolation and alignment
The purpose of position modes is to allow flexible and detailed control whether to interpret Z (elevation), heading, pitch and roll as absolute values or relative to the road. X and Y will always be interpreted as absolute values.
For some use cases orientation should be aligned to road surface, e.g. a vehicle simply driving along a hilly road. In some cases pitch and roll should be zero, e.g. signs and buildings along the road. External vehicle models can choose whether to report relative or absolute values. It can also be combination, e.g. explicit/absolute heading while z, roll and pitch are realative (to the road) - or any other combination.
The mode can be specified explicitly when calling a set position function. Example:
pos.SetInertiaPosMode(x, y, z, h, p, r, PosMode::Z_REL | PosMode::H_ABS | PosMode::P_REL | PosMode::R_REL);
In above example x, y, and h will be interpretated as absolute values while z, p, and r as relative to the road.
In most cases the same mode setting will be used throughout the simulation for each object. For this reason, as conveience, the mode can be set once per object. Example:
pos.SetMode(Position::PosModeType::SET, PosMode::Z_REL | PosMode::H_ABS | PosMode::P_REL | PosMode::R_REL);
Then, when calling set functions, mode can be set to zero. Example:
pos.SetInertiaPosMode(x, y, z, h, p, r, 0);
or use the simple variants skipping the mode argument. Example:
pos.SetInertiaPos(x, y, z, h, p, r);
These cases will utilize the current setting.
Note the type=SET argument of the SetMode() example above. It specifies as to which type of operations the settings applies to. There are two types of modes: SET and UPDATE. SET applies to API calls setting Cartesian coordinates. UPDATE applies to position updates by default controller and API calls setting Lane and Road coordinates.
This way a position object can, for example, maintain relative heading follow the curvature of the road when updated by the default controller, while heading can be set explicitly in terms of absolute value by API calls.
pos.SetMode(Position::PosModeType::UPDATE, PosMode::Z_REL | PosMode::H_REL | PosMode::P_REL | PosMode::R_REL);
pos.SetMode(Position::PosModeType::SET, PosMode::Z_REL | PosMode::H_ABS | PosMode::P_REL | PosMode::R_REL);
There will be a default setting for both SET and UPDATE which will be applied until changed by SetMode() function:
| SET |
PosMode::Z_REL | PosMode::H_ABS | PosMode::P_REL | PosMode::R_REL |
| UPDATE |
PosMode::Z_REL | PosMode::H_REL | PosMode::P_REL | PosMode::R_REL |
Further examples
Specify only mode for Z, reuse whatever setting for the remaining components:
pos.SetInertiaPosMode(x, y, z, h, p, r, PosMode::Z_ABS);
Set/reset default value for pitch, reuse whatever setting for the other components:
pos.SetInertiaPosMode(x, y, z, h, p, r, PosMode::Z_MASK & PosMode::Z_DEF);
Mode is coded in four chunks of three bits. One chunk for each of the position components Z, Heading, Pitch and Roll. Chunks are padded and aligned to segments of four bytes.
| Z | Comment | Hex | Int | |
|---|---|---|---|---|
SKIP |
XXXX XXXX XXXX XXX0 |
first bit is zero |
0x0000 |
0 |
Z_DEF |
XXXX XXXX XXXX XX01 |
first bit is one, second is zero |
0x0001 |
1 |
Z_ABS |
XXXX XXXX XXXX X011 |
first and second bit is one, third zero |
0x0003 |
3 |
Z_REL |
XXXX XXXX XXXX X111 |
first, second and third bits are all one |
0x0007 |
7 |
| H | Comment | Hex | Int | |
|---|---|---|---|---|
SKIP |
XXXX XXXX XXX0 XXXX |
first bit is zero |
0x0000 |
0 |
H_DEF |
XXXX XXXX XX01 XXXX |
first bit is one, second is zero |
0x0010 |
16 |
H_ABS |
XXXX XXXX X011 XXXX |
first and second bit is one, third zero |
0x0030 |
48 |
H_REL |
XXXX XXXX X111 XXXX |
first, second and third bits are all one |
0x0070 |
112 |
| P | Comment | Hex | Int | |
|---|---|---|---|---|
SKIP |
XXXX XXX0 XXXX XXXX |
first bit is zero |
0x0000 |
0 |
P_DEF |
XXXX XX01 XXXX XXXX |
first bit is one, second is zero |
0x0100 |
256 |
P_ABS |
XXXX X011 XXXX XXXX |
first and second bit is one, third zero |
0x0300 |
768 |
P_REL |
XXXX X111 XXXX XXXX |
first, second and third bits are all one |
0x0700 |
1,792 |
| R | Comment | Hex | Int | |
|---|---|---|---|---|
SKIP |
XXX0 XXXX XXXX XXXX |
first bit is zero |
0x0000 |
0 |
R_DEF |
XX01 XXXX XXXX XXXX |
first bit is one, second is zero |
0x1000 |
4,096 |
R_ABS |
X011 XXXX XXXX XXXX |
first and second bit is one, third zero |
0x3000 |
12,288 |
R_REL |
X111 XXXX XXXX XXXX |
first, second and third bits are all one |
0x7000 |
28,672 |
Looking up values in the table above, modes can be specified in numerous ways.
For example: To set world position with relative Z and absolute Pitch, reusing setting for the other components, any of the following identical examples will work:
pos.SetInertiaPosMode(x, y, z, h, p, r, PosMode::Z_REL | PosMode::P_ABS);
=
pos.SetInertiaPosMode(x, y, z, h, p, r, 0x7 | 0x300);
=
pos.SetInertiaPosMode(x, y, z, h, p, r, 0x307);
=
pos.SetInertiaPosMode(x, y, z, h, p, r, 0x0307);
=
pos.SetInertiaPosMode(x, y, z, h, p, r, 7 | 768);
=
pos.SetInertiaPosMode(x, y, z, h, p, r, 7 + 768);
=
pos.SetInertiaPosMode(x, y, z, h, p, r, 775);
-
Simple 2D vehicle model. Report only X, Y and Heading. Align Z, pitch and roll to the road surface:
pos.SetMode(Position::PosModeType::SET, PosMode::H_ABS);
pos.SetInertiaPos(x, y, 0.0, h, 0.0, 0.0);
or use short version of the same function, applying zero to z, p, r:
pos.SetInertiaPos(x, y, h); -
Advanced 3D vehicle model. Report all 6 DOF: X, Y, Z, Heading, Pitch and Roll. I.e. no explicit alignment to road surface:
pos.SetMode(Position::PosModeType::SET, PosMode::Z_ABS | PosMode::H_ABS | PosMode::P_ABS | PosMode::R_ABS);
pos.SetInertiaPos(x, y, 0.0, h, 0.0, 0.0);
or use short version of the same function, applying zero to z, p, r:
pos.SetInertiaPos(x, y, h); -
Rotate a pedestrian 90 degree left to move forward facing the road driving direction, as on a hoverboard:
pos.SetHeadingRelative(1.5708);
pos.SetMode(Position::PosModeType::UPDATE, PosMode::H_REL);
Default controller will now move the pedestrian along the road but rotated 90 degree. -
Put pedestrian on a hoverboard moving sideways forward (heading=90 degrees) along the road, 1 meter above ground:
pos.SetHeadingRelative(1.57);
pos.SetZRelative(1.0);
pos.SetMode(Position::PosModeType::UPDATE, PosMode::Z_REL | PosMode::H_REL);
Default controller will now move the pedestrian along the road 1 meter above ground, with maintained heading relative to the road.
For convenience, esmini can calculate heading and align to the tangent of each segment (polyline and NURBS). Further, to smoothen motion esmini can interpolate corners of polyline trajectories. The behavior is based on omitted position components and followingMode settings.
In general specified angles (heading, pitch and roll) will be interpolated over the segment between two vertices/control points. However, in many cases it’s prefered to just align heading with the tangent of the curve. For these cases it should not be necessary to specify the angles, but rather have esmini calculate them.
First, when z value is missing, i.e. elevation is not specified, esmini will project the points to the road surface. Availability of z will also affect alignment of orientation. General rule is that ground trajectories (elevation not specified) will align pitch and roll to the road surface, while these components will be set to zero for floating (specified z values) trajectories.
Missing angles will be handled according to the table below:
Input |
Result |
|
missing angle component |
z specified? |
align orientation to |
heading |
Yes |
trajectory |
heading |
No |
trajectory |
pitch |
Yes |
trajectory |
pitch |
No |
road surface |
roll |
Yes |
0.0 |
roll |
No |
road surface |
While moving along polylines, which tangent is not continuous, it’s usually preferred to interpolate/smoothen orientation change at vertex corners.
The table below shows the result for combinations of input settings of polyline trajectories:
Input |
Result |
||
Angle specified? |
FollowingMode? |
Interpolate angle along segment |
Interpolate corners |
Yes |
position |
No |
No |
Yes |
follow |
Yes |
No |
No |
position |
No |
Yes |
No |
follow |
Yes |
No |
There is a code and scenario example illustrating the various interpolation and position modes:
Video clip:
This specific example includes four objects:
-
Pedestrian/skater on the road
Initially it’s standing still, aligned along the road x-axis (relative heading = 0). -
Box2 moving along a NURBS trajectory
Specified z will make it "floating" (not projected on ground). Omitted heading will have esmini calculate heading along the tangent. -
Box3 moving along another NURBS trajectory
Also floating. Demonstrating specified roll will be interpolated between control points. -
Box1 moving along a polyline trajectory
followingMode = position will make smoothly transitions at vertex corners.
The application will control and report the motion for the skater. At 5 seconds it will rotate 90 degree and move this way along the road, like riding a hoverboard. Then at 15 seconds it will move one meter up from road surface (z=1 relative road) and also rotate back 90 degrees (heading=0 relative road) and move this way along the road, like on a flyboard. During the complete scenario pitch and roll is set to 0 (absolute value - not aligned to road).
For the OpenSCENARIO RelativeRoadPosition and RelativeLanePosition the ds and dsLane attributes specifies an offset along the s-axis of the road and lane respectively. In case the offset results in a s-value outside the boundary of the current road, the target location will be searched via connected roads and lanes, even through junctions.
In case the action object (the one being positioned) has been assigned a route, the route will be respected, i.e. the target location will be along the route. If the action object lacks a route, any route of the referenced entity will be respected instead. If neither objects has a route assigned the default routing rules will apply.
In the image below two cars, red and blue, are being positioned by means of RelativeRoadPosition or RelativeLanePosition with the white car as the reference entity. They have both the same ds value, i.e. same longitudinal distance (along s-axis) from the white car. Since the red car has an assigned route, it ends up along that route. The blue car, lacking a route, will end up along the route of the referenced white car.
OpenSCENARIO provides the controller concept as a way to outsource control the motion and appearance of scenario entities. For the OpenSCENARIO description of the controller concept, see OpenSCENARIO User Guide.
Controllers can be assigned to object of type Vehicle or Pedestrian. Once assigned, controllers are activated for a given domain (e.g longitudinal, lateral, Lighting, Animation) using the ActivateControllerAction. It’s also possible to activate immediately as part of the AssignControllerAction.
While the ActivateControllerAction is executing, the Controller assigned to that object will manage specified domain(s). Controllers may be internal (part of the simulator) or external (defined in another file).
Intended use cases for Controllers include:
- Specifying that a vehicle is controlled by the system under test.
- Defining smart actor behavior, where a controller takes intelligent decisions in response to the road network or other actors. Hence, controllers can be used, for example, to make agents in a scenario behave in a human-like way.
- Assigning a vehicle to direct human control.
The Controller element contains Properties, which can be used to specify controller behavior either directly or by a file reference.
Although OpenSCENARIO from v1.2 supports assignment of multiple controllers to an object, esmini is currently only supporting one controller to be assigned (and hence activated) to an object at the time.
esmini version < 2.0 totally lacked support for controllers. Instead some similar functionality were implemented as part of different example-applications. For example, EgoSimulator provided interactive control of one vehicle. There was also an "external" mode which allowed for an external vehicle simulator to report current position for the typical Ego (VUT/SUT…) vehicle.
First, this approach made the example code of simple applications complex. Secondly, it limited the use cases of esmini since functionality was tightly embedded in the applications.
Controllers provides a much more flexible way of adding functionality to esmini, in a way harmonizing with the standard (OpenSCENARIO 1.X). A side effect of this "outsourcing" of functionality is that the former demo applications could be reduced to a minimum both in number and size.
Controllers was introduced in esmini v2.0. Briefly it works as follows:
There is a collection of embedded controllers coming with esmini. Each controller inherit from the base class Controller (Controller.h/cpp). In order for esmini to be aware of the existence of a controller it has to be registered. This is done through the ScenarioReader method RegisterController. It will put the controller into a collection to have available when the scenario is being parsed. So all controllers need to be registered prior to loading the scenario.
A controller is registered with the following information:
-
Its static name which is used as identifier.
-
A pointer to a function instantiating the controller.
This architecture makes it possible for an external module to create a controller and registering it without modifying any of esmini modules. In that way it is a semi-plugin concept, you can say.
Note: Even though ScenarioReader have a helper function for registering all the embedded controllers the RegisterController can be called from any module directly, at any time prior to scenario initialization.
esmini catalog framework supports controllers as well, so controllers can be defined in catalogs as presets, just like vehicles and routes for example.
Controllers are completely handled in the OpenSCENARIO file. With one exception: esmini provides the --disable-controllers option which totally ignore any controllers, just performing the scenario with DefaultControllers, which can be handy when previewing and debugging scenarios.
In terms of OpenSCENARIO a controller is assigned to an entity (object) in any of two ways:
-
As part of the Entities section and ScenarioObject definition, using the ObjectController element.
-
The AssignControllerAction which can be triggered as any action at any time during the simulation.
Once assigned the controller must finally be activated using the ActivateControllerAction which can be triggered at any time. It provides an attribute to specify which domain(s) to control: Lateral, Longitudinal or both.
Example 1: Implicit assignment of controller using the ObjectController element:
<Entities>
<ScenarioObject name="Ego">
<CatalogReference catalogName="VehicleCatalog" entryName="$HostVehicle"/>
<ObjectController>
<Controller name="MyController" >
<Properties>
<Property name="esminiController" value="InteractiveController" />
<Property name="speedFactor" value="1.5" />
</Properties>
</Controller>
</ObjectController>
</ScenarioObject>
</Entities>Example 2: As above, but using catalog reference:
<Entities>
<ScenarioObject name="Ego">
<CatalogReference catalogName="VehicleCatalog" entryName="$HostVehicle"/>
<ObjectController>
<CatalogReference catalogName="ControllerCatalog" entryName="interactiveDriver" />
</ObjectController>
</ScenarioObject>
</Entities>Example 3: Activate controller in the Init section:
<Init>
<Actions>
<Private entityRef="Ego">
<PrivateAction>
<TeleportAction>
<Position>
<LanePosition roadId="0" laneId="-3" offset="0" s="$EgoStartS"/>
</Position>
</TeleportAction>
</PrivateAction>
<PrivateAction>
<ActivateControllerAction longitudinal="true" lateral="true" />
</PrivateAction>
</Private>
</Actions>
</Init>Example 4: Assign and activate the controller in the Storyboard section:
<Event name="InteractiveEvent" maximumExecutionCount="1" priority="overwrite">
<Action name="AssignControllerAction">
<PrivateAction>
<ControllerAction>
<AssignControllerAction>
<CatalogReference catalogName="ControllerCatalog" entryName="interactiveDriver" />
</AssignControllerAction>
</ControllerAction>
</PrivateAction>
</Action>
<Action name="ActivateControllerAction">
<PrivateAction>
<ControllerAction>
<ActivateControllerAction longitudinal="true" lateral="true" />
</ControllerAction>
</PrivateAction>
</Action>
<StartTrigger>
<ConditionGroup>
<Condition name="" delay="0" conditionEdge="none">
<ByValueCondition>
<SimulationTimeCondition value="20" rule="greaterThan"/>
</ByValueCondition>
</Condition>
</ConditionGroup>
</StartTrigger>
</Event>To disable any controller simply apply ActivateControllerAction on no domain, like:
<ActivateControllerAction longitudinal="false" lateral="false" />That should work in all OpenSCENARIO supporting tools. In esmini you can also assign DefaultController which will disconnect any assigned controller.
The purpose of the ghost concept is to support external driver models. The idea is to launch a forerunner, which performs the maneuvers in the OSC file. The resulting trajectory (also called trail), including speed and heading info, is registered. The driver model can then use the ghost trail as guidance. More specifically, the driver model can probe the trail at any timestamp or distance from its current pos and use the result for both steering and speed target.
A typical use case is having an external Ego vehicle simulator, including System Under test (SUT) and a driver model. Instead of the driver model (in the external simulator) being aware of and execute scenario maneuvers it can hand over to esmini to play the maneuvers and then simply try to mimic it, following in ghost’s trail. This way the external simulator can benefit from esmini scenario engine and just concentrate on following the ghost.
The ghost feature is available in the following esmini embedded controllers:
There are two different methods to probe the ghost trail:
-
By time: Ask for ghost state (pos, heading, speed…) at specific timestamp
-
By distance: Ask for ghost state at a specific distance, from a position (typically current Ego position)
Both methods will return a RoadInfo structure including position details e.g. world position (absolute and relative Ego), road coordinates, road pitch, trail heading, heading angle to the target point and ghost speed at that point.
link:..//EnvironmentSimulator/Libraries/esminiLib/esminiLib.hpp[role=include]This approach is trivial, it will give the state of ghost at the specified timestamp using the API call:
The API call is:
The method needs some further explanation. The basic idea is to look specified distance along the ghost trail and receive information at that target point. The tricky part is where to start measuring the distance from. This is how it works:
-
From Ego, find closest point on trail (blue point)
-
From that point, look forward x meters along trail (red dot)
-
Get info from that ”target” point
Ideally, two points are used: One closer for target speed and one more distant for steering target.
Tuning parameters:
-
Lookahead distance along trail
-
Typically speed dependent (look further at higher speeds)
-
Potentially decrease distance with increased curvature (to stay on road)
-
-
Headstart time
-
As small as possible, but enough for required lookahead distance
-
Note: If lookahead distance is too large, the steering target angle becomes irrelevant, leading to driving off road
Most actions that affects motion will be transferred from the original entity (Ego) to its ghost, for example: SpeedAction, LaneChangeAction and TeleportAction. In addition, triggers will be affected in the following ways:
-
Entity triggers: Replace any occurrence of original entity with ghost in the list of
TriggeringEntities. For example, if the vehicle should brake when reaching a specified position, then it’s the ghost that should trigger the action - not the original entity. -
Other triggers: Restart ghost from where the Ego is now, in effect rewind ghost
headStartTimeseconds and recreate trajectory from there. The purpose is to enable ghost to react on triggers that happens in the "real" scenario (which plays outheadStartTimeseconds behind the ghost) -
SimulationTimetrigger conditions for events where Ego is the actor will be adjusted wrt ghostheadStartTimetime. For example, if vehicle should brake after 20 seconds of driving (simulationTime = 20), then it will be changed to simulationTime = 17 (for headstart 3s), since the ghost has been driving for 20 seconds at that point.
The following examples shows two different triggers for a lane change action. First example uses TraveledDistanceCondition. Ghost will be the triggering entity and since this is independent of other road users, there is no need to restart ghost.
The second example, on the other hand, uses TimeHeadwayCondition which calculates the time gap between the white Ego car and the red Target car, hence depending on other road users. Since ghost is running ahead of time it does not make sense to involve ghost in such conditions, hence Ego will be the triggering entity.
Example 1: Trigger based on traveled distance - no restart
<ByEntityCondition>
<TriggeringEntities triggeringEntitiesRule="any">
<EntityRef entityRef="Ego"/>
</TriggeringEntities>
<EntityCondition>
<TraveledDistanceCondition value="20"/>
</EntityCondition>
</ByEntityCondition>Example 2: Trigger based on time headway - restart ghost
<ByEntityCondition>
<TriggeringEntities triggeringEntitiesRule="any">
<EntityRef entityRef="Ego"/>
</TriggeringEntities>
<EntityCondition>
<TimeHeadwayCondition
entityRef="Target"
value="4.0"
freespace="false"
coordinateSystem="road"
relativeDistanceType="longitudinal"
rule="lessThan"/>
</EntityCondition>
</ByEntityCondition>Note that the strategy results in desired behavior in probably the majority of cases, but not all. There might be exceptions where another behavior is expected or required. The manipulation described above is handled by the methods SetupGhost() and ReplaceObjectInTrigger() in ScenarioEngine.cpp which can be modified to fit custom needs.
Timestamps in log are always of same reference. Whenever a ghost is involved the time will start at -headStartTime, in order for the actual scenario to start at simulationTime = 0. Example: Assuming headStartTime = 3s, a log message for an Ego action like:
-2.100: SpeedChange standbyState → startTransition → runningState
means that action SpeedChange (Ego action that has been moved to ghost) is performed 0.9 (-2.1 + 3) seconds after ghost was launched. This would, for example, be the case for a simulationTime = 0.9 condition.
Below is a listing of some of the available controllers in esmini. Note that only DefaultController is related to the OpenSCENARIO standard. The other ones are esmini-specific and will not work in other tools (so far there is no standard plugin-architecture for controllers to enable moving between tools). For updated and complete definition of controllers and their parameters, see ControllerCatalog.xosc.
Performs actions exactly as specified in the OpenSCENARIO file. Assigned to entities by default. Can be assigned at any point to "unassign" any currently assigned controller.
Domain: |
n/a (will always take over both Lateral and Longitudinal domains) |
Properties: |
None |
Example: |
n/a (any scenario not explicitly assigning and activating another controller) |
A simple vehicle model controlled by the user via keyboard (arrow keys).
Domain: |
longitudinal and/or lateral (independent) |
||||
Properties: |
|
||||
Example: |
A simple driver model. A ghost-twin performing the events a few seconds ahead. The entity will then do its best to mimic the motion and speed of the ghost by follow the trajectory. The primary purpose of this controller is to provide an example, but it can be useful to smoothen out the sometimes synthetic feel of pure default controller.
Domain: |
longitudinal and lateral (in combination only). |
||||||||||||
Properties: |
|
||||||||||||
Example: |
The entity will not be moved by the scenario. Instead its state (position, rotation …) is expected to be reported from external simulator via API, e.g. SE_ReportObjectPos. Ghost trajectory can optionally be created for an external driver model to use as reference.
Domain: |
longitudinal and/or lateral (independent). |
||||||
Properties: |
|
||||||
Example: |
A way of integrating SUMO controlled vehicles in a scenario. OpenSCENARIO vehicles are reported to SUMO, and SUMO vehicles are reported back to esmini. A reference to a SUMO config file is provided as input to the controller. See cut-in_sumo.xosc for an example.
Domain: |
longitudinal and lateral (in combination only). |
||
Properties: |
|
||
Example: |
Based and inspired by Regulation 157 Safety Models. Includes four safety models relating to the (UNECE ALKS regulation #157). The controller includes preliminary and experimental implementation of four models:
-
RegulationCharacteristics of expected system performance as stated in the UNECE Reg 157 Paragraph 5.2.5.2. -
ReferenceDriverReference model of a "typical" human driver as specified in UNECE Reg 157 Annex 4 - Appendix 3. -
RSSResponsibility-Sensitive Safety performance model proposed and described in Shalev-Shwartz, Shai, Shaked Shammah, and Amnon Shashua. "On a formal model of safe and scalable self-driving cars." arXiv preprint arXiv:1708.06374 (2017). -
FSMFuzzy Safety Model proposed and described in Mattas, Konstantinos, et al. "Fuzzy Surrogate Safety Metrics for real-time assessment of rear-end collision risk. A study based on empirical observations." Accident Analysis & Prevention 148 (2020): 105794.
Domain: |
longitudinal and lateral (in combination only). Note: No advanced steering, just maintaining current lane offset. |
||||||
Properties: |
Further properties see |
||||||
Examples: |
alks_r157_cut_in_quick_brake.xosc |
Run example alks_r157_cut_in_quick_brake.xosc:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/alks_r157_cut_in_quick_brake.xosc
Change model:
-
Open
resources/xosc/alks_r157_cut_in_quick_brake.xoscin a text editor -
In the ALKS_R157SM_Controller, change
<Property name="model" value="Regulation"/>to
<Property name="model" value="ReferenceDriver"/> -
Save and run again:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/alks_r157_cut_in_quick_brake.xosc
You can hardly see any difference. Let’s run the two variants again and record for post analysis.
-
Set model to "ReferenceDriver" (by edit the file as described above)
-
Run:
./bin/esmini --headless --fixed_timestep 0.02 --osc ./resources/xosc/alks_r157_cut_in_quick_brake.xosc --record sim-ref.dat -
Set model to "Regulation"
-
Run:
./bin/esmini --headless --fixed_timestep 0.02 --osc ./resources/xosc/alks_r157_cut_in_quick_brake.xosc --record sim-reg.dat -
View scenarios in parallel:
./bin/replayer --window 60 60 800 400 --res_path ./resources --dir . --file sim- --view_mode boundingbox -
Create a merged .dat file and plot it:
./bin/replayer --file sim- --dir . --save_merged sim_merged.dat
./scripts/plot_dat.py --param speed sim_merged.dat
Something like this should show:
Of course you can run also FSM and RSS models in similar way and merge all four .dat files. For more information about replayer, see "Plot merged .dat files" example in Plot scenario data section.
Implementation of the Lane Independent Routing Model (LIRM) developed as part of the master thesis project Olsson, Daniel; Rylander, Eric: "A Framework for Verification and Validation of Simulation Models in esmini".
From the report: "LIRM has solved the issue with lane dependent routing that esmini had, by implementing a new controller that extends the behavior of the pathfinding and route following to incorporate lane changes."
This controller enhance the capabilities for an entity to find a path through the road network according to a specified route (basically a set of waypoints). The main advantage, compared to the route handler of the default controller, is that lane changes will be injected by the path finding algorithm in search for optimal path. The model will also take route strategy (Shortest, Fastest or Least Intersections) into consideration, which the default controller does not yet support.
A good example is found in follow_route_with_lane_change.xosc. The vehicle start out in left-most lane and needs to make three lane changes in order to exit the highway.
./bin/esmini --window 60 60 800 400 --osc ./EnvironmentSimulator/Unittest/xosc/follow_route_with_lane_change.xosc --trail_mode 1
Domain: |
longitudinal and/or lateral (independent). |
||||||
Properties: |
|
||||||
Example: |
An UDP interface for external driver models or vehicle simulators. The idea is to allow external devices to control entities in an ongoing esmini simulation, over network.
Concept:
-
Assign the controller to any scenario vehicle (in the OpenSCENARIO file)
-
Send input to esmini controller
-
The vehicle will be updated by the controller accordingly (and bypassing esmini default controller)
-
receive ground-truth (OSI over UDP) from esmini
There are a few input mode options:
-
DRIVER_INPUT
Control a simple vehicle model in terms of pedals and steering input. Vehicle will apply latest received values until update is received.Parameters: throttle
brake
steeringAngle -
VEHICLE_STATE_XYZHPR
Report explicit complete state, ignoring road geometry. Useful when advanced vehicle dynamics is involved and vehicle for example is pitching as result of acceleration and rolling as result of steering. Optional flag to enable dead reckoning on esmini side, in which case esmini will move entity according to latest received heading and speed (basically extrapolating).Parameters: x
y
z
h(eading)
p(itch)
r(oll)
speed
wheelAngle(wheel yaw/stering angle)
deadReckon(flag) -
VEHICLE_STATE_XYH
Report explicit partial state.z,pitchandrollare aligned to the road geometry.Parameters: x
y
h(eading)
speed
wheelAngle(wheel yaw/stering angle)
deadReckon(flag) -
VEHICLE_STATE_H
Minimal explicit state, useful when the driver model output is only heading and speed.Parameters: h(eading)
speed
wheelAngle(wheel yaw/stering angle)
deadReckon(flag)
For more info see: UDPDriverController.pdf
Demo (running a somewhat outdated testUDPDriver.py):
A simple "follow-the-leader" controller which aims to keep a specified distance to a specified leading vehicle. The controller utilizes a simple 2D "bicycle" vehicle model. Each time-step the controller finds out the direction and distance to the lead vehicle. Based on this, the acceleration and steering input is calculated and provided to the vehicle model.
The term off-road indicates that the controller does not consider any road network features. Note that it still can run on roads, parking lots, just that it’s ignoring it in terms of lanes and road-marks.
Below is a video clip showing the controller in action on an open paved area. The white lead vehicle is controlled interactively by the user via arrow keys. The controller is assigned to the red car which then strives to follow the lead car at the distance of 20 meters.
Below are the steps to add new controller in esmini:
-
Create a sepreate controller module in the controller folder. Suggestion is to copy one of the existing and use as a starting point.
-
Register the controller in the LoadControllers() fuction in EnvironmentSimulator/Modules/ScenarioEngine/SourceFiles/ScenarioReader.cpp
-
Create Instantiate method in the new controller. The name shall be unique eg InstantiateControllerLooming
-
In the new controller, Type name shall be unique eg, CONTROLLER_LOOMING_TYPE_NAME
-
Add controller type and it shall be unique in EnvironmentSimulator/Modules/Controllers/Controller.hpp
-
Make sure GetTypeStatic() and GetTypeNameStatic() and methods returns the unique type and type name values
esmini make use of OpenSceneGraph (OSG) for visualization of the scenario. The OpenSCENARIO files can optionally refer to existing 3D models of the static environment (scene graph) and dynamic objects (entities). If the scene graph reference is missing, esmini will try to generate a basic model based on the OpenDRIVE road network description. Currently esmini only supports OSG native .osgb 3D file format. However, there are ways to convert 3D models as described next.
OpenSceneGraph (osg) includes readers and writers for quite a few 3D file formats. It comes with a demo application, osgconv, a command line tool that simply takes one file as input and outputs the same content in a different format.
Examples:
Convert from fbx to osgb format:
osgconv car.fbx car.osgb
Convert from osgb to fbx format:
osgconv car.osgb car.fbx
osgconv has a lot of useful options. Run osgconv -h for more info. Here’s a few examples:
osgconv in.fbx out.osgb --compressed will compress and embed textures
osgconv in.fbx out.osgb -t 1,5,7 will translate the model (x=1, y=5, z=7)
osgconv in.fbx out.osgb -o -90-1,0,0 will rotate the model (rotate -90 deg around X-axis)
osgconv in.fbx out.osgb -o -90-1,0,0 --use-world-frame as above but pivot point world origin
A useful environment variable is OSG_OPTIMIZER which affects the structure and content of the scene graph. For example, if the osgb file includes cloned sub trees, like motorway railings, these might not be populated in the fbx file. To solve that set:
OSG_OPTIMIZER = FLATTEN_STATIC_TRANSFORMS_DUPLICATING_SHARED_SUBGRAPHS
Exact syntax depends on your environment. Examples:
| bash |
|
| powershell |
|
Then run the osgconv command (in the same terminal where you defined the environment variable).
osg native format is osgb which stands for OpenSceneGraph binary format. It’s a stable and compact (quick to load) format, which also can embed textures and animations. Thanks to the osgconv tool, esmini can get away with only supporting the .osgb format and still indirectly support many other 3D formats.
There are pre-built binaries available, see below. But for full capabilities, including support for converting from 3D file formats .dae (collada) and .fbx (Autodesk Filmbox) you have to build OpenSceneGraph (osg) yourself.
The only prerequisite step is for Linux to install some dependecies:
-
Run
software-properties-gtk
or open the "Software & Updates" application -
then under the "Ubuntu Software" tab click "Source code" and add a server of your choice, e.g. within your country
-
Run
sudo apt-get build-dep openscenegraph
Reference Step 1 & 2: https://unix.stackexchange.com/a/614082
You can then try this script: compile_osg_apps.sh. It will first fetch and install FBX SDK and then build OSG with FBX and DAE support. Find more details and instructions in the header of the script.
For more info regarding building OSG for Linux, here’s a great guide:
https://vicrucann.github.io/tutorials/osg-linux-quick-install/
But note that it does not consider FBX support.
Find more info on COLLADA via this old link.
Windows
Binaries including FBX but not DAE support, see here:
https://objexx.com/OpenSceneGraph.html
Linux
sudo apt-get update
sudo apt-get -y install openscenegraph
Here follows an example of how to convert .osgb models into .fbx format for use in the Unity3D framework.
-
Open a command prompt in the folder where your model.osgb is
-
Run command:
osgconv model.osgb out/model.fbx -s 100,100,100
"-s …" is for scaling which typically is needed for fbx files. Potentially it also needs to be oriented according to Unity coordinate system. In that case try:
osgconv model.osgb out/model.fbx -s 100,100,100 --use-world-frame -o 120—0.5773503,-0.5773503,-0.5773503
A folder named "out" should have been created and including the model.fbx plus any texture files
-
Drag the resulting fbx file, and all textures, into a unity project, preferably an empty folder
-
Add the model to the Scene hierarchy
-
Select the model and int the "Inspector", select the "Materials" tab
-
Change the "Location" to "Use External Materials (Legacy)" and click "Apply"
-
Open the automatically created "Materials" folder (next to the model file)
-
Select all materials and change
for standard template: "Rendering Mode" to "Cutout" (good default option)
for HDRP tem 10BC8 plate: "Surface type" to "Transparent" and check "alpha clipping"
That should basically be it.
See issue 63 for brief info.
And these two scripts operating on textures and material respectively, can potentially provide some inspiration:
- bake_signs.sh
- fix_dae_materials.py
esmini use a very simple lighting model. Adding a global low ambient plus a stronger directed "sun" light. This is a simplistic model of a bright day where everything is visible but sides facing towards the sun gets brighter than those facing away.
The ambient intensity is 0.4. The "sun" x,y,z position is (-7500, 5000, 10000) and intensity 0.8.
Now, it has been reported that some custom models looks very dark on sides facing away from the sun. The main reason is often that the material definition do not have any ambient contribution.
There are two solutions:
Solution 1: Fix the material
Convert the model to Collada .dae format, example:
osgconv my_model.fbx my_model.dae
Then open the .dae file in a text editor. Look for <library_materials> and the material references inside that element. The actual material definitions are (typically?) found under <library_effects>. There might be many types, but two that we encountered are <phong> and <lambert>. They should have a diffuse element, something like:
<diffuse>
<color sid="diffuse">0.8 0.8 0.8 1.0</color>
</diffuse>`There might be a similar one for ambient. Check color values, if they are low, e.g. 0.2, try increase to max 1.0. If element is missing, just add it. In the end, it should look something like:
<ambient>
<color sid="ambient">1.0 1.0 1.0 1.0</color>
</ambient>`Save the file as my_model_fixed.dae.
There is a script that automates this process, but it will only recognize phong and lambert material types: fix_dae_materials.py.
Finally, convert the patched dae file to osgb:
osgconv my_model.dae my_model.osgb
Different formats have different orientation frames. Hence it may be necessary to fix rotation. In that case, try something like:
osgconv my_model.dae -o 90-1,0,0 --use-world-frame my_model.osgb
Solution 2: Add light source
Users can add up to two custom light sources, specifying position (x, y, z) and intensity. Examples:
Add one custom light source:
./bin/esmini --window 60 60 800 400 --osc .\resources\xosc\lane_change_crest.xosc --custom_light 4000,-6000,500,1.0
Add two custom light sources:
./bin/esmini --window 60 60 800 400 --osc .\resources\xosc\lane_change_crest.xosc --custom_light 4000,-6000,500,0.4 --custom_light 5000,5000,200,0.2
There are no fixed structure where scenario resources, e.g. vehicle 3D model or controller catalog, needs to be stored. Instead, additional search paths can be added by launch argument --path <path>. For example:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/slow-lead-vehicle.xosc --path ../../ --path c:/tmp/my_models
will add the relative path ../../ and absolute path c:/tmp/my_models to the list of places where to look for resources referred to by the scenario.
When esmini is downloaded as a zip file from Internet it might be blocked, i.e. prevented from being started, by Windows to protect your system. This is indicated by the following, or similar, message popping up when attempt to start:
"Microsoft Defender SmartScreen prevented an unrecognized app from starting. Running this app might put your PC at risk."
As long as the zip was downloaded from esmini official GitHub repository (https://github.com/esmini/esmini) it is perfectly safe to use.
The simplest solution is to unblock the zip file before unpacking it:
-
Right click
-
Click Properties
-
At bottom right, check "Unblock"
Note: This method also works for individual files after unzipping. But it can only be applied for one file at a time, why it’s simpler to unblock the complete zip before unpacking.
It is also possible to unblock from Powershell command line:
-
Single file:
Unblock-File esmini-demo_Windows.zip -
Multiple files recursively:
Get-ChildItem -Path esmini-demo_Windows -Recurse | Unblock-File
From esmini version 2.26.6 mac executables are universal binaries which means they work on both Intel and Apple Silicon based Macs. Following are some known issues and work arounds:
-
Window can’t be located in the upper part of the screen (some systems). If window won’t open, try to adjust the y value (second entry) of the window argument. 60 pixels usually works fine, e.g:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc -
Issue has been reported that when multiple screens are connected esmini just shows a bland window. Currently only solution is to unplug the additional montitors or unplug Mac from dock station.
-
Graphics can not run in separate thread. Hence the
--threadlaunch flag will have no effect. -
On Mac the zip-package might be put in quarantine, to release it:
xattr -d com.apple.quarantine file.zipor even better:xattr -c file.zip -
esmini executables are not signed with a developer ID - since we simply don’t have one. If you got the package directly from esmini release page on GitHub it is safe to unpack and run. In order to execute the files you need to do one of following from a terminal in the folder the demo package was extracted:
-
Remove quarantine flags:
xattr -c -r esmini-demo/bin, or -
Sign executables for use on local machine:
codesign -f -s - esmini-demo/bin/*
-
SUMO comes with a great tool, netconvert, that can convert road networks from and to various formats.
Prerequisite: Download and install SUMO from here.
Example: Convert an OSM map to OpenDRIVE for use in esmini:
netconvert --osm-files city.osm --opendrive-output city.xodr --no-turnarounds
Preview in odrviewer:
./bin/odrviewer --window 60 60 800 400 --odr city.xodr
Do either:
-
remove any
resources/modelsfolder -
from
./buildfolder, runcmake ..
or
-
get the package from here and unpack files into
./resources/models
In order to appear in the scenario, without need for AddEntity action, an entity must be represented in the Init section of the Storyboard. Common is to add actions to establish inital position and speed.
Example:
<PrivateAction>
<TeleportAction>
<Position>
<LanePosition roadId="1" laneId="-1" offset="0" s="50"/>
</Position>
</TeleportAction>
</PrivateAction>
<PrivateAction>
<LongitudinalAction>
<SpeedAction>
<SpeedActionDynamics dynamicsShape="step" dynamicsDimension="time" value="0.0"/>
<SpeedActionTarget>
<AbsoluteTargetSpeed value="30.0"/>
</SpeedActionTarget>
</SpeedAction>
</LongitudinalAction>
</PrivateAction>First time the cmake-script is run it will try to download pre-built libraries and model assets. If this step fails for some reason, e.g. "Couldn’t resolve host name" or certificate failure, you can change to another location.
Modify esmini/CMakeLists.txt (under esmini root), for example referring to dropbox instead of esmini:
set(FILE_STORAGE
"dropbox") # "dropbox", "google", "esmini" (limited GB/Day)
set(MODEL_STORAGE
"dropbox") # "dropbox", "google", "esmini" (limited GB/Day)Then to apply settings:
cd build
cmake ..In esmini, the behavior of the two position types RoadPosition and LanePosition differ in terms of heading.
-
RoadPosition: When not specified, the heading will align to the direction of the road (s-axis) -
LanePosition: When not specified, the heading will align to the driving direction of the lane
In the image below, the white cars are positioned by RoadPosition with t=-1.5 and t=1.5 respectively while the red cars are positioned by LanePosition with laneId="-1" and laneId="1" respectively.
Explicitly specified heading (in Orientation sub-element):
-
Absolute heading will be respected and treated in the same way for both position cases
-
Relative heading will be based on the default heading, i.e. road or driving direction respectively.
In next image, an Orientation element was added with absolute heading = 0.4 for all four cars.
In next image, an Orientation element was added with relative heading = 0.4 for all four cars.
Corresponding behavior applies also to RelativeRoadPosition and RelativeLanePosition.
In some (maybe most) use cases it’s preferred that vehicles align to the driving direction of the lane. But in other cases not, example: In an overtake maneuver, the overtaking car may start in the neighbor lane. The driving direction of the lane should not affect the heading of the car.
In OpenDRIVE definition of laneOffset: A lane offset may be used to shift the center lane away from the road reference line.
In esmini lane offset affects the reference line as well as center lane, which is a deviation from the OpenDRIVE standard. As consequence, t values are also shifted with the lane offset.
2+1 roads can be defined utilizing lane offset. Consider image below:
esmini
| white car | red car | |
|---|---|---|
laneId |
-2 |
-1 |
t |
-5.25 |
-1.75 |
OpenDRIVE
| white car | red car | |
|---|---|---|
laneId |
-2 |
-1 |
t |
-1.75 |
1.75 |
A 2+1 example example scenario is found here: resources/xosc/two_plus_one_road.xosc
Follow the following steps to install OSI (and dependent Google Protobuf) for use by some esmini scripts, e.g. osi2csv.py.
Protobuf for Python (skip if already installed)
-
Open a Powershell
-
pip install protobuf==3.20.2
protoc (Protobuf compiler)
-
Download protoc-3.20.2-win64.zip
(release page: https://github.com/protocolbuffers/protobuf/releases/tag/v3.20.2) -
Unzip (at least protoc.exe) to some location
OSI for Python
-
Open a Powershell in the to-be-parent folder of OSI
-
git clone https://github.com/OpenSimulationInterface/open-simulation-interface -
cd open-simulation-interface -
git checkout v3.5.0 -
./convert-to-proto3.sh -
$env:PROTOC=<location of protoc.exe>
for example:$env:PROTOC="C:/tmp/protoc-3.20.2-win64/bin/protoc.exe" -
pip install .
This should be it.
How to remove old versions (if needed)
-
pip uninstall protobuf -
pip uninstall open-simulation-interface
esmini can run in two different time modes: 1. Real-time and 2. Fixed-time. Default is real-time.
The following command, from esmini root, will run real-time:
./bin/esmini --window 60 60 800 400 --osc ./resources/xosc/cut-in.xosc
This one fixed-time:
./bin/esmini --window 60 60 800 400 --fixed_timestep 0.05 --osc ./resources/xosc/cut-in.xosc
The difference is basically that real-time will run with as small time-steps as possible (actually limit at 1 ms to avoid unnecessary CPU load), maxim zing precision but constrained by system clock. In other words, the scenario will play out in "natural" speed, good for previewing or demonstrating sc narios. Each timestep is calculated by esmini based on passed system-time since last frame.
The fixed-time is simpler, esmini will simply execute the scenario as fast as possible applying the specified delta-time for each step. Good for runni g tests for later post processing of resulting logs, data or OSI files.
A common use-case is to run esmini quick/headless and then utilize the replayer for viewing the result. See more info here: Replay scenario
The Issues tab at esmini GitHub page is a valuable source of questions and answers. To search in all issues, make sure to set filter: is:issue.
usage: plot_dat.py [-h] [--x_axis X_AXIS] [--equal_axis_aspect] [--derive] [--dots] (--list_params | --param PARAM) <filename>
positional arguments:
filename dat filename
optional arguments:
-h, --help show this help message and exit
--x_axis X_AXIS x-axis parameter
--equal_axis_aspect lock aspect ratio = 1:1
--derive derive values wrt x, i.e. dy/dx
--dots add dots
--list_params list available parameters in given file
--param PARAM parameter to plot (can be specified multiple times)
Note: In addition to <filename> one of the arguments --list_params and --param <PARAM> is requiredFrom esmini version 2.29.0 a set of cmake presets are provided. These can be utilized from Visual Studio Code and Visual Studio (from 2019 version 16.10 onwards, see more info here) with the CMake Tools extension.
CMake tool is used to create standard make configurations. A few example "create…" batch scripts are supplied (in scripts folder) as examples how to generate desired build setup from command line.
-
VisualStudio / win64 / Windows SDK v10 / Release and Debug
-
Ubuntu and Kubuntu (tested on 18.04) / gcc / Release and Debug
However, it should be possible to configure custom variants using cmake. For example to use Visual Studio 2019 run the following commands from command prompt (CMD or PowerShell), assuming starting point is esmini root folder:
mkdir build cd build cmake .. -G "Visual Studio 16 2019" cmake --build . --config Release --target install
This will first generate Visual Studio solution and then compile esmini using MSVC toolset v142 (default with Visual Studio 2019). Default architecture should be x64.
If you want to compile with MSVC 2017 toolset just add directive to the generator as follows:
cmake .. -G "Visual Studio 16 2019" -T v141
A complete list of supported toolsets are available here.
Provided 3rd party lib binaries are only provided for Windows x64 architecture. To make sure you build for x64 add -A x64, For example:
cmake .. -G "Visual Studio 16 2019" -T v141 -A x64
If you want to compile with MSVC 2015 (v140) toolset and for win32 use the following generator command:
cmake -G "Visual Studio 16 2019" -T v140 -A Win32 ..
Note:
Provided 3rd party libs only available for x64 builds. If you need to build full esmini for Win32 (32bit), you need to compile the 3rd party libraries yourself. Perhaps the "scripts/generate_*_libs.sh" scripts can be a starting point for such an endeavor.
Another possibility for Win32 is to build a slim esmini with a minimum of dependencies. See Slim esmini - customize configration.
Of course, building with a specific toolset requires it to be installed. Use Visual Studio Installer. Steps:
-
choose "Modify"
-
make sure Desktop Development with C++ is checked
-
go to tab "Individual components"
-
scroll down to "Compilers, build tools, and runtimes"
-
check the MSVC versions you need, e.g. "MSVC v140 - VS 2015 C++ build tools (v14.00)" and "MSVC v141 - VS 2017 C++ x64/x86 build tools (v14.16)"
All configurations defines an "Install" build target that compiles (if needed) and copies relevant binaries into a common "esmini/bin" folder recognized by the example scripts under the "esmini/run" folder.
Note:
-
For automatic downloading of external dependencies (OSG binaries) and 3D models, CMake version 3.11.4 or above is required (FetchContent_MakeAvailable was introduced).
-
In Windows, if you get an error like "the c compiler identification is unknown", then please make sure to install "Windows Universal CRT SDK" from the Visual Studio Installer tool.
esmini is designed to link all dependencies statically. Main reason is to have a all-inclusive library for easy integration either as a shared library/DLL (e.g. plugin in Unity, or S-function in Simulink) or statically linked into a native application.
Note: Nothing stops you from going with all dynamic linking, it’s just that provided build scripts are not prepared for it.
CMake scripts will download several pre-compiled 3rd party packages and 3D model resource files.
Default location for these resources is Google drive. If there is an issue, try switch to the backup location at Dropbox by changing the following line in EnvironmentSimulator/CMakeLists.txt:
set ( FILE_STORAGE "google" )
to
set ( FILE_STORAGE "dropbox" )
Links to all packages can be found in EnvironmentSimulator/CMakeLists.txt.
If you need to (re)build a 3rd party lib for some reason, e.g. for an yet unsupported system or need for a specific version, these build scripts might be a starting point:
or this script that builds all three libs, combining the above ones:
Linux Ubuntu 18.04
sudo apt install build-essential gdb ninja-build git pkg-config libgl1-mesa-dev libpthread-stubs0-dev libjpeg-dev libxml2-dev libpng-dev libtiff5-dev libgdal-dev libpoppler-dev libdcmtk-dev libgstreamer1.0-dev libgtk2.0-dev libcairo2-dev libpoppler-glib-dev libxrandr-dev libxinerama-dev curl cmakeAlso, g++ version >= 5 is needed for c++14 code support.
Windows and Mac: Install the cmake application
To run from the visual studio, the steps below might help with setup configuration.
Note: Make sure cmake.preset set to auto to recognize the presets (defined in the CMakePresets.json file). Open setting (Ctrl + ,) type cmake.preset, set auto in drop down box. This is a global setting.
To run:
-
Open the Command Palette (Ctrl+Shift+P) and run CMake: Configure. Select
ubuntu_release -
Shift + F7 to select and build specific target
-
e.g.
installto build and copy all targets to bin folder
-
-
Configure custom run arguments (create launch.json)
-
Open the run and debug window (Ctrl+Shift+D)
-
Select
create a launch.json file -
In the drop down list, select
C++ (GDB/LLDB) -
Select
Add Configuration…(in the lower right corner of the sub window) -
Select
C/C++: (gdb) Launch -
Change
programsetting as"${workspaceFolder}/build/EnvironmentSimulator/Applications/esmini/esmini" -
Change
argsas["--window", "60", "60", "800", "400", "--osc", "../resources/xosc/cut-in.xosc"]
-
-
Press F5
For Debug,
Repeat step 1 and select ubuntu_debug. Repeat step 2 to build debug binaries and then step 4. Breakpoints can be added to the program specified in the launch.json file (do step 3 if not already done).
Build target can also be changed in the status bar, click Set the default build target. If not cmake commands are visible in status bar, check global settings: Ctrl + , then write "cmake status bar" to find Status Bar Visibility setting. Set to visible.
Another way to set build target is via Ctrl+Shift+P, then write "set build target" to find CMake: Set Build Target setting. Select it and choose build target in the drop down list that should appear.
Make sure cmake.preset set to never to ignore the preset file (CMakePresets.json):
-
Open setting (Ctrl+,) type cmake.preset, select
neverin drop down box
Note: This is a global setting, i.e. will affect other VSCode instances. Reset to auto if needed. Another way to ignore the presets is simply to remove the CMakePresets.json file and restart VSCode.
To run:
-
Open the Command Palette (Ctrl+Shift+P) and run
CMake: Select a Kit. Select the compiler you want to use. For example GCC 11.3.0 -
Open the Command Palette (Ctrl+Shift+P) run the
CMake: Select Variantcommand. SelectRelease -
Open the Command Palette (Ctrl+Shift+P) run the
CMake: Set Build Target. Selectinstall -
Open the Command Palette (Ctrl+Shift+P) and run the
CMake: Buildcommand, or select the Build button from the Status bar. (If cmake commands are not visible in status bar, check global settings: Ctrl + , then write "cmake status bar" to findStatus Bar Visibilitysetting. Set tovisible.) -
To configure launch arguments, see step 3 in With preset.
-
Press F5
For debug:
Repeat step 2 and select Debug. Repeat step 3 to 6. Breakpoints can be added to the program specified in the launch.json.
For more info see: https://code.visualstudio.com/docs/cpp/CMake-linux
When linking esmini with software already dependent on Google protobuf there might be need for dynamic linking of shared protobuf library. This can be achieved by defining cmake symbol DYN_PROTOBUF as following example:
cmake .. -D DYN_PROTOBUF=True
Then build as usual. It will link with protobuf shared library instead of linking with a static library.
When running esmini, the protobuf shared library needs to be found. Set dynamic library path environment variable to the folder where the library is. Example:
Linux:
export LD_LIBRARY_PATH=./externals/OSI/linux/lib-dyn
macOS:
export DYLD_LIBRARY_PATH=./externals/OSI/linux/lib-dyn
Note:
The dynamic versions of protobuf were added Aug 31 2021. So you might need to update the OSI library package. Get the latest from following links:
The external dependencies OSG, OSI, SUMO and implot are optional. Also the unit test suite is optional, in effect making the dependecy to googletest framework optional as well. All these options are simply controlled by the following cmake options:
-
USE_OSG
-
USE_OSI
-
USE_SUMO
-
USE_GTEST
-
USE_IMPLOT
So, for example, to cut dependency to OSG and SUMO, run:
cmake .. -D USE_OSG=False -D USE_SUMO=False
To disable OSG, SUMO, OSI and googletest, run:
cmake .. -D USE_OSG=False -D USE_SUMO=False -D USE_OSI=False -D USE_GTEST=False USE_IMPLOT=False
All options are enabled/True as default.
Note:
Disabling an external dependency will disable corresponding functionality. So, for example, disabling OSI means that no OSI data can be created by esmini. Disabling OSG means that esmini can’t visualize the scenario. However it can still run the scenario and create a .dat file, which can be played and visualized later in another esmini build in which OSG is enabled (even on another platform).
esmini slim can be compiled and executed in the MSYS2 environment. Try following steps:
-
Download MSYS2 from: https://www.msys2.org
-
Install with default options
-
Start MSYS2 MinGW x64 (e.g. from start menu)
-
Update MSYS2 packages, run:
pacman -Syu --disable-download-timeout --noconfirm
(MSYS2 should close automatically) -
Restart MSYS2
-
Finalize update and install needed packages, run:
pacman -Su --disable-download-timeout --noconfirm
pacman -S --needed base-devel mingw-w64-x86_64-toolchain --disable-download-timeout --noconfirm
pacman -S mingw-w64-x86_64-cmake --disable-download-timeout --noconfirm-
Optional (not needed to compile or run esmini):
pacman -S git --disable-download-timeout --noconfirm -
Build esmini (from MSYS2 MinGW x64 command line):
cmake -G "MSYS Makefiles" -D USE_OSG=False -D USE_SUMO=False -D USE_OSI=False -D USE_GTEST=False ..
cmake --build . --config Release --target installFirst generate build configuration (see above)
Then it should work on all platform to build using cmake as follows:
cmake --build . --config Release --target install
Or you can go with platform specific ways of building:
Windows/Visual Studio
-
Open generated solution, build*/EnvironmentSimulator.sln
-
Select configuration, Debug or Release
-
Build CMakePredefinedTargets/INSTALL (right-click and select build)
macOS
To generate a Xcode project file, run the initial cmake command as follows:
cmake -G Xcode ..
Then build as usual:
cmake --build . --config Release --target install
or using Xcode directly:
xcodebuild -scheme install -configuration Release build
or open the generated project file in Xcode, and build from there.
To create bundles (shared library container), do from esmini root folder:
lipo -create bin/libesminiRMLib.dylib -output bin/esminiRMLib.bundle
lipo -create bin/libesminiLib.dylib -output bin/esminiLib.bundleLinux
Once cmake .. has created the build configuration, of course you can build by calling the gnu make applciation directly instead of going via cmake --build as described above.
cd build
make -j4 installThis will build all projects, in four parallel jobs, and copy the binaries into a dedicated folder found by the demo batch scripts.
CentOS 7 has some limitations, e.g. old versions of C/C++ compiler toolkits and runtimes. So it’s not possible to link with provided 3rd party binary libraries targeting Ubuntu 18++. However, by disabling some featuers in esmini, e.g. OSI and SUMO, it can still be used for previewing scenarios.
VirtualBox image for Windows host here:
https://www.linuxvmimages.com/images/centos-7/
Follow steps below to build and run esmini on CentOS 7.
sudo yum install git
sudo yum install cmake
sudo yum install gcc-c++
sudo yum install freeglut-devel
sudo yum install fontconfig-devel
sudo yum install libXrandr-devel
sudo yum install libXinerama-devel
sudo yum install epel-release
sudo yum install p7zip
git clone https://github.com/esmini/esmini
cd esmini
cd externals
mkdir OpenSceneGraph
cd OpenSceneGraph
curl -L "https://www.dropbox.com/s/mxztf6zbgojyntp/osg_centos.7z?dl=1" -o osg_centos.7z
7za x osg_centos.7z
rm osg_centos.7z
cd ../..
mkdir build
cd build
cmake -D USE_OSG=True -D USE_SUMO=False -D USE_OSI=False -D USE_GTEST=False ..
cmake --build . --target install --config Release
cd ..
./bin/esmini.exe --headless --fixed_timestep 0.01 --record sim.dat --osc ./resources/xosc/cut-in.xoscBrief info on how to contribute is found here: https://github.com/esmini/esmini/blob/master/CONTRIBUTING.md
Complementing the Build guide this section contains additional information regarding branch strategy, formatting and test.
From v2.30.0 the dev branch was introduced as the new top branch for continuous integration of features and bug fixes.
dev branch contains the latest integrated features and bug fixes. New features should be developed in branches created from dev. master is the default branch and primarily reserved for official releases.
Continuous integration job will be launched for pushed commits on the following branches:
-
master -
dev -
any other branch starting with
feature/, e.g.feature/support_emergency_vehicles
Pull requests should be targeting the dev branch.
Master branch represent the latest released software. It should be in synch with dev.
dev and master branches should be in perfect sync, i.e. based on common history. Unfortunately, to keep this true and to avoid messy history and lost tags, branch history might be changed in rare cases. That can lead to local branch clones out of synch. If that is the case, follow these steps to restore sync of your local branch:
-
git branch backup(only needed if you have local commits you want to preserve for later cherry-picking) -
git fetch origin -
git reset --hard origin/<branch name>
<branch name> might be master or dev, or whatever branch to be restored.
From esmini v2.30.0 the CI will include formatting checks, in addition to static analysis and tests. The code styles are defined by following schemes:
| C++ | |
| Python | |
| cmake |
To run formatting (both check and apply) locally, follow steps below:
esmini makes use of version 15, avoiding some known bugs in v14
Windows:
Download and install LLVM-15.0.7-win64.exe found here:
https://github.com/llvm/llvm-project/releases/tag/llvmorg-15.0.7
Linux:
Check if clang-format version 15 is installed:
clang-format-15
If you get the following message:
Command 'clang-format-15' not found, but can be installed with:
sudo apt install clang-format-15then clang 15 must be installed.
On Ubuntu systems prior to 22.04, first install llvm version 15:
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 15Source: apt.llvm.org
Then, clang v15 can be installed:
sudo apt install clang-format-15see also: ci.yml
./support/python/src/main.py run format --clang_format_checker
./support/python/src/main.py run format --cmake_format_checker
./support/python/src/main.py run format --black_format_checkerThere are two sets of tests on different frameworks:
-
Unit tests (white box)
-
Smoke tests (black box)
The unit tests suites are basically C++ code modules of test cases based on the Google Test framework. They range from true unit tests (testing individual classes or functions) to system level tests reading a scenario and inspecting output, e.g. log files or visualization pixel data. The key feature of the unit tests is the possibility to inspect internal objects and variables.
Example: Unittest/RoadManager_test.cpp
The smoke test suite is basically execution of a selection of scenarios with some strategic inspections of the result. Scenario elements and timing can be checked against log file while object info such as position, rotation and speed can be checked against dat file entries. A Python framework handles the execution and gathering of result for convenient checking.
The smoke test script is found here: test/smoke_test.py
For easy execution of all tests, both unit tests and smoke tests, run this main test script: scripts/run_tests.sh, from esmini root folder:
./scripts/run_tests.sh
Tip: On Windows, run it from GIT bash.
The main idea with esmini lib is to provide a limited, simplifed but useful API to the complex world of esmini, OpenSCENARIO and traffic simulation. By making use of only standard C types it has been possible to wrap and use the library in numerous environments, like for example MATLAB, Simulink and Unity (C#).
esmini lib is a, so called, shared library which is linked dynamically with a custom application. This means that it’s not linked statically with the application, instead it’s loaded whenever the application is launched. An obvious advantage with shared libraries is that they can be shared between multiple applications, reducing size and potentially simplify maintenance.
An additional important advantage with dynamic linking is that the library can be updated without need to recompile the custom application making use of the library (however it can’t be updated during runtime, only in between launches). An obvious requirement is that the API is compatible when replacing a library with a different version, since the API is established at link-time. On the other side of the "same coin", a disadvantage with dynamic linking is that you need to have the shared library present at run-time.
Although esmini lib is a shared library itself, it includes everything needed in terms of 3rd party functionality. It’s actally linking all dependencies statically. Think about it as a single container including everything needed, e.g. support for 3D graphics, XML, expressions, and SUMO traffic simulation. There are two advantages with this approach: 1. Getting rid of dependency to hundreds of 3rd party shared libraries and 2. The linker makes sure to only include relevant functions which reduce the size by extreme (compared to bundle individual and complete shared libraries).
Below is a typical interaction between a custom application (e.g. vehicle simulator) and esmini (providing simulation of the environment, like road and traffic).
| Frame | app time | app events | esmini time | esmini events |
|---|---|---|---|---|
0 (t=0.0) |
0.0 |
Init esmini("scenario.xosc") ⇒ |
0.0 |
|
0.0 |
0.0 |
Step(dt=0.0) |
||
0.0 |
0.0 |
|
||
0.0 |
0.0 |
|
||
0.0 |
0.0 |
|
||
0.0 |
0.0 |
|
||
0.0 |
0.0 |
|
||
0.0 |
Get Ego state (initial position, speed…) ⇒ |
0.0 |
||
0.0 |
0.0 |
⇐ Return Ego initial state |
||
1 (t=0.0→0.1) |
0.1 |
Update Ego(dt=0.1) |
0.0 |
|
0.1 |
|
0.0 |
||
0.1 |
Report Ego state(state) ⇒ |
0.0 |
||
0.1 |
0.0 |
Register state of Ego, protect it from default controller (but do not apply state yet) |
||
0.1 |
Step esmini(dt=0.1) ⇒ |
0.0 |
||
0.1 |
0.0 |
Step(dt=0.1) |
||
0.1 |
0.0 |
|
||
0.1 |
0.0 |
|
||
0.1 |
0.0 |
|
||
0.1 |
0.1 |
|
||
0.1 |
0.1 |
|
||
0.1 |
0.1 |
|
||
0.1 |
0.1 |
|
||
2 (t=0.1→0.2) |
0.2 |
Update Ego(dt=0.1) |
0.1 |
|
0.2 |
|
0.1 |
||
0.2 |
Report Ego state(state) ⇒ |
0.1 |
||
0.2 |
0.1 |
Register state of Ego, protect it from default controller (but do not apply state yet) |
||
0.2 |
Step esmini(dt=0.1) ⇒ |
0.1 |
||
0.2 |
0.1 |
Step(dt=0.1) |
||
0.2 |
0.1 |
|
||
0.2 |
0.1 |
|
||
0.2 |
0.1 |
|
||
0.2 |
0.2 |
|
||
0.2 |
0.2 |
|
||
0.2 |
0.2 |
|
||
0.2 |
0.2 |
|
||
and so on… |
No systematic documentation available. See header files esminiLib.hpp and esminiRMLib.hpp for API details.
The following section covers some specific use cases of the library.
Screenshots can be controlled via the API:
int SE_SaveImagesToFile(int nrOfFrames);nrOfFrames: -1 ⇒ continuously, 0 ⇒ stop, > 0 ⇒ number of frames, e.g. 1=next frame only, 2=next two frames
Images will be stored in current folder with name screen_shot_*.tga (* = counter) for post processing.
Images can also be saved to memory for instant processing. Control the feature with:
int SE_SaveImagesToRAM(bool state);state: true ⇒ capture images, false ⇒ don’t capture (default, might improve performance on some systems)
Grab image with:
int SE_FetchImage(SE_Image *image);image is a reference to a SE_Image struct which will contain the image meta and image data
See image-capture code example.
Note: In order to grab a rendered frame image, it must have been saved to RAM during the post rendering phase. First frame will be saved to RAM by default. The SE_SaveImagesToRAM() function must be used to enable the feature for frames > 0.
The reason for not enable always by default is performance savings. On some system the operation to read back rendered pixels from graphics frame buffer to application RAM memory is not neglectable (depending on whether color coding or byte order transformation is needed). So a compromize is to have the feature enabled for the first init frame, and then activate by user when needed.
Finally, to disable esmini image handling altogether and revert to OSG default handling, make the following call before SE_Init():
SE_SetOffScreenRendering(false);
Note that disabling the this will permanently disable the callback mechanism for the complete esmini session (overriding any SE_SaveImagesToRAM() call).
Similarly as with camera related launch arguments, camera positioning can be controlled via esmini lib API.
-
SE_AddCustomCamera()
Add a camera with relative position and orientation (heading and pitch) -
SE_AddCustomFixedCamera()
Add a fixed camera at custom global position and orientation (heading and pitch) -
SE_AddCustomAimingCamera()
Add a camera with relative position looking at current entity -
SE_AddCustomFixedAimingCamera()
Add a camera at fixed location but always looking at current entity -
SE_AddCustomFixedTopCamera()
Add a top view camera with fixed position and rotation -
SE_SetCameraObjectFocus
Set/change object to focus on
More info, see headerfile.
Any number of cameras can be defined. These cameras can then be activated via function: SE_SetCameraMode().
Applications can even loop through the camera modes to produce images from several cameras. See code example multiple-cameras.cpp.
OSI data can be retrieved in three ways:
-
Storing OSI tracefile
-
Receive in UDP packages
-
Receive via API call
Call SE_EnableOSIFile(filename) in order to create a OSI tracefile. Specify custom filename or set 0 or "" to use default filename (currently "ground_truth.osi").
By launch argument --osi_receiver_ip <ip addr> you can have esmini sending OSI packeges in UDP frames. The socket port is hardcoded but can of course be changed in the code (OSI_OUT_PORT in OSIReporter.cpp).
See receiver side code example osi_receiver.cpp.
Fetch OSI data by either one of the API calls:
-
SE_GetOSIGroundTruthRaw() -
SE_GetOSIGroundTruth(int* size)
The "raw" version will simply return a pointer to the already existing OSI data structure. No additional data copying will be performed. This is the preferered option if:
-
The application is coded in C++, making the Protobuf data structures fully compatible
-
The data is read before next call to
SE_UpdateOSIGroundTruth(), which will update the data
See code example: osi-groundtruth, also covered in User guide - Hello world tutorial OSI groundtruth.
The second function will fetch the OSI data serialized into a string. This will be a true copy and also generic format which can be parsed in any environment, independent of OS and programming language. The backside is additional resources needed for the copying and serialization.
See code examples in ScenarioEngineDLL test cases, e.g: receive_GroundTruth.
Note:
SE_UpdateOSIGroundTruth() needs to be called after each SE_Step* in order to prepare and populate OSI data. This function will also save the OSI frame to file and send over UDP, if requested. Only after this preparation the SE_GetOSIGroundTruth* methods will get updated OSI data.
An exception is when OSI file or UDP receiver is specified by corrsponding launch arguments --osi_file [filename] and --osi_receiver_ip <ip addr>. Then, OSI update will automatically be updated by esmini according to specified multiplier --osi_freq <number> (1 is default). For example, --osi_freq 3 will update OSI every third step.
The purpose of this explicit OSI update control is to support use cases where esmini is running at much higher frequency than needed for OSI data (e.g. for vehicle dynamics purposes). To optimize performance, OSI update can then be skipped most frames and only run when needed.
BTW: Any additional explicit calls to SE_UpdateOSIGroundTruth() will be ignored. Which means that the programmer does not have to worry about additional performance penalty introduced by multiple calls to this function.
OSI provides information needed for sensor models at bounding box level for both moving and stationary objects, including ones defined in OpenDRIVE. Sensor models can then detect overlap, occlusion and collisions between those bounding boxes. Although sensor models are not within scope of esmini, there is a code example on how to retrieve and extract OSI data. It can perhaps be a starting point: osi-groundtruth.cpp
For even simpler use cases esmini provides a sensor mechanism, basically collecting scenario objects which reference point is within a sensor view frustum. This is too rough for collision detection, but can be useful for low-pass features like adaptive cruse control (ACC) or just optimizing CPU load by filtering objects for some processing.
The view frustum is defined by a horizontal field of view combined with near and far limits. Basically an object is detected by a ideal sensor if the object reference point is within the view frustum, i.e:
-
angle from sensor mounting point is within the
field of view -
distance from sensor mounting point is larger than
nearand less thanfar
The sensors are added from the code, see the esmini-dyn code example. API is found here.
Limitations:
-
OpenDRIVE stationary objects not considered
-
Only reference point checking (part of the object can be inside frustum and still not detected)
Below example is just to show the simplicity of using esmini library. For more information and realistic examples, e.g. involving cmake, please go ahead to Hello World programming tutorial.
If not already done, download and unpack latest esmini demo. Make sure to pick demo, not bin, for your platform.
In esmini root folder, create a file named main.cpp and add the following lines:
Code:
#include "./EnvironmentSimulator/Libraries/esminiLib/esminiLib.hpp"
int main()
{
SE_Init("./resources/xosc/cut-in.xosc", 0, 1, 0, 0);
while (SE_GetQuitFlag() == 0)
SE_Step();
SE_Close();
return 0;
}Compile:
-
Linux:
gcc main.cpp ./bin/libesminiLib.so -Wl,-rpath,\$ORIGIN -o bin/my_player -
Windows (x64 Native Tools Command Prompt for Visual Studio):
cl main.cpp ./bin/esminiLib.lib /Fe:bin/my_player
Run:
from Linux terminal or Windows PowerShell:
./bin/my_player
With some limitations (see details here) esmini can play the example scenarios provided with the ASAM OpenSCENARIO v1.1 release bundle.
-
If you don’t have esmini already, download latest demo package for your platform from here.
-
Download the standard from ASAM here (register and download is free of charge).
-
Extract to any folder.
-
Run the examples from command line in esmini root folder, for example:
./bin/esmini --window 60 60 800 400 --osc ../../openscenario-v1.1.0/Examples/DoubleLaneChanger.xosc
or with absolute path:
./bin/esmini --window 60 60 800 400 --osc c:/stuff/openscenario-v1.1.0/Examples/DoubleLaneChanger.xosc
The source of this document is written in the asciidoc format. The html is generated with asciidoctor. Code snippets, images and other resources are included in the process. Also the version numbers are automatically inserted to avoid "human errors".
Install tools:
| Linux |
|
| Windows |
|
Generate html: asciidoctor user_guide.adoc
Full commit history is found here