Title: Customizable Embedded 3D Surround View Turn-Key Solution on Apalis iMX6 SoM
1- Customizable Embedded 3D Surround View Turn-Key
Solution on Apalis iMX6 SoM
- Author Thomas Hase, TES Electronic Solutions
GmbH
2Multi-Camera 3D Surround View systems are already
established in many passenger vehicles. Demand
for such systems now also grows for special
vehicles like construction machines, harvesters,
buses or military vehicles. In comparison to
personal cars, these vehicles are produced in a
rather moderate volume and have a high need for
customization of the optical system.
3In this blog post, we present a customizable 3D
Surround View Turn-Key solution based on the
Toradex Apalis iMX6 SoM based on the NXP i.MX 6
SoC, that can be quickly adapted concerning the
specific requirements of the targeted vehicle.
4Please watch this video from Embedded World 2017
in Nuremberg, Germany, where this system solution
was presented
Video Link https//www.youtube.com/watch?vY67DhP
6qaUs
5We also explain how to get familiar with the
topic of 3D surround view, evaluating the
solution via a step-by-step approach and finally
how to come to a product meeting your specific
needs.
TES 3D Surround View Stitching Library the
heart of the system solution TES Electronic
Solutions provides a 3D Surround View software
library designed for embedded systems, CPU/GPU
independency, and high customizability. Detailed
information can be found on the TES 3D Surround
View Website.
6- The library takes raw fish-eye pictures from
memory and computes a 3D View of the vehicle in
its maneuver situation. - TES 3D Surround View Highlights
- Increases safety and comfort through improved
situation awareness - Excellent stitching and 3D scene generation
- Watch your maneuver situation from any position
outside or inside the vehicle - Any number of predefined viewing positions
(Views) possible with animated transitions
between different views - Many parameters configurable per View
- Highly customizable and portable
- Independent from the used type of cameras (the
software expects raw fish-eye pictures in memory)
73D Surround View Turn-Key Solution In the
following, we present a complete HW/SW system
solution based on the Toradex Apalis iMX6Q IT
Module allowing easy hardware customization to
meet specific customer demands.
System Setup
8- The System setup consists of the following
components - Processor Toradex Apalis iMX6 IT Computer On
Module with Heatsink - Toradex Ixora Carrier Board
- Camera System
- First Sensor Mobility Area View Kit DCAVK
including - 4x 1MPixel MJPEG Ethernet Cameras (PoE),
DC3K-1-EAP - 4x 5m Ethernet cables (up to 25m possible for
each camera) - Micrel Gigabit Ethernet (PoE) Switch
- Metal Plate for Desktop Application, additionally
4 Vacuum cups for vehicle application included - Software
- TES 3D Surround View Software under Linux on
Apalis iMX6
9- Resource Usage
- The 3D Surround View algorithm mainly exploits
the GPU. The CPU is only used for system control
and in the case of view-changes. - Configuration 4 MJPEG IP cameras, combined
80Mbit data rate _at_30 fps and 1Mpx resolution. Low
polygon car model, 1280x1024 screen resolution at
60Hz refresh rate. - GPU 3-6 GFlops (proportional to pixel amount
depicted on screen) - RAM usage 60 MB in total
- 4 10 MB for camera data buffers
- 2 MB for 3D environment and car model
- 18 MB for camera video textures
10- This turn-key system solution can be rapidly
reproduced for evaluation and prototyping
purposes. Of course, also the separate components
can be exchanged or adapted to the target
application needs, e.g. - The cameras can be exchanged against other
types. - A dedicated application specific main board
can be designed taking the Ixora Carrier board as
a reference. - The user interface and HMI overlay of the 3D
Surround View software can be easily adjusted. - The 3D model of the vehicles can be easily
exchanged. - Both Toradex and TES, are open to provide the
necessary support.
11- Getting started and familiar with 3D Surround
View - 3D Surround View is not only a topic of running
an algorithm on an embedded platform and be done
with it. There are many aspects of the system
setup that need to be considered and require
customization for specific use cases, e.g. - Selection of the appropriate cameras regarding
lens, resolution, physical data transmission
(cable length!), power supply, number of cameras
etc. - Positioning of the cameras (usually the higher
the better) - Defining the appropriate viewing positions, i.e.
virtual position and viewing vector of the
observer in the 3D scene - Optimizing the 3D Surround View parameters, e.g.
Environmental model (shape and size of the
projection bowl), stitching method and
parameters. - User interface for the 3D Surround View System
- 3D Model of the vehicle, potentially with
transparency.
12Just consider an excavator compared to a personal
car On an excavator, you need significantly
longer cables to the cameras, the cameras are
mounted typically much higher. Latency is not
that much of a concern as the driving speed is
lower and maybe you just want to apply only 3
cameras or mount the cameras un-symmetrically
because the driver can see the left front part of
his vehicle very well while he cannot see the
right front part at all.
So it might be a good idea to first play with a
3D Surround View system to get an idea of such
application aspects. For this purpose or for
those who just want to give the TES software a
try there is an offline PC demonstrator available
at zero cost. TES provides this demonstrator on
request in a zip archive with no need of
installation (just unpack). Any modern Windows PC
should be sufficient to run this software. The
program comes with prerecorded MJPEG Ethernet
streams in .PCAP-format and runs these streams
through the TES 3D Surround View software.
13The PC keyboard and mouse can be used to change
the viewer position predefined views are
available on the number keys and free-flight is
possible via mouse and (Shift-) arrow keys. Also,
some of the 3D Surround View system parameters
like the bowl-size and -shape as well as the
stitching parameters can be adjusted. Do not
hesitate to get in touch with TES (coordinates
below) to get this Software.
First steps in the Lab and on the vehicle Now
that you have played with the software (and
assuming it does more or less what you need) you
might want to give it a try with real-life
cameras, either in lab conditions or right away
on the vehicle. The fastest way to do that is to
get a First Sensor Mobility Area View Kit
DCAVK (contents see above under System Setup)
and the TES 3D Surround View Windows PC Software
in Online-Mode (Evaluation Version). This kit
allows lab setups as well as on-vehicle
installation using a Windows Laptop. Up to 20m
cable length (between cameras and switch) has
been tested so far. Also here TES can assist you
in purchasing this setup.
14And now on the embedded hardware Having tested
the system under lab conditions and on the
vehicle that target platform aspects now get
focus (again). I.e. you want to see your setup
running on the potential target hardware. At that
point, you may want to take your Apalis iMX6
module with Ixora carrier board (or buy some from
Toradex if you do not have those, yet) and get
the TES 3D Surround View software for that target
platform (prototype version). I.e. you get rid of
the PC and use the Apalis/Ixora platform instead.
The TES prototype software for that target
platform has the same functionality as the
evaluation version for PC and you can re-use the
configurations that you have setup at the earlier
stage now on the target.
15Getting to a product For a product, you now have
to customize the system to meet your specific
application requirements. I.e. you use the Apalis
iMX6 module as core and might re-design the
carrier board based on the Ixora design data to
meet your specific requirements regarding,
interfaces, power supply, mechanics etc. In that
step or even before you also may want to exchange
the cameras with other types. Again, TES can
assist you also in the hardware and system
customization, development and (if wanted)
manufacturing process.
16Summary We have introduced an embedded 3D
Surround View Turn-Key Solution and explained a
process to get you familiar with this topic
starting at zero cost. Following this process,
your product development risk is significantly
reduced because you can evaluate the system in a
step-by-step approach and at the same time
determine your application specific requirements.
Finally, we have introduced the steps to come to
a product based on this turn-key solution. Just
get started!
17Thank you