Customizable Embedded 3D Surround View Turn-Key Solution on Apalis iMX6 SoM

1

• Customizable Embedded 3D Surround View Turn-Key
  Solution on Apalis iMX6 SoM

- Author Thomas Hase, TES Electronic Solutions
GmbH
2
Multi-Camera 3D Surround View systems are already
established in many passenger vehicles. Demand
for such systems now also grows for special
vehicles like construction machines, harvesters,
buses or military vehicles. In comparison to
personal cars, these vehicles are produced in a
rather moderate volume and have a high need for
customization of the optical system.
3
In this blog post, we present a customizable 3D
Surround View Turn-Key solution based on the
Toradex Apalis iMX6 SoM based on the NXP i.MX 6
SoC, that can be quickly adapted concerning the
specific requirements of the targeted vehicle.
4
Please watch this video from Embedded World 2017
in Nuremberg, Germany, where this system solution
was presented
Video Link https//www.youtube.com/watch?vY67DhP
6qaUs
5
We also explain how to get familiar with the
topic of 3D surround view, evaluating the
solution via a step-by-step approach and finally
how to come to a product meeting your specific
needs.
TES 3D Surround View Stitching Library the
heart of the system solution TES Electronic
Solutions provides a 3D Surround View software
library designed for embedded systems, CPU/GPU
independency, and high customizability. Detailed
information can be found on the TES 3D Surround
View Website.
6

• The library takes raw fish-eye pictures from
  memory and computes a 3D View of the vehicle in
  its maneuver situation.
• TES 3D Surround View Highlights
• Increases safety and comfort through improved
  situation awareness
• Excellent stitching and 3D scene generation
• Watch your maneuver situation from any position
  outside or inside the vehicle
• Any number of predefined viewing positions
  (Views) possible with animated transitions
  between different views
• Many parameters configurable per View
• Highly customizable and portable
• Independent from the used type of cameras (the
  software expects raw fish-eye pictures in memory)

7
3D Surround View Turn-Key Solution In the
following, we present a complete HW/SW system
solution based on the Toradex Apalis iMX6Q IT
Module allowing easy hardware customization to
meet specific customer demands.
System Setup
8

• The System setup consists of the following
  components
• Processor Toradex Apalis iMX6 IT Computer On
  Module with Heatsink
• Toradex Ixora Carrier Board
• Camera System
• First Sensor Mobility Area View Kit DCAVK
  including
• 4x 1MPixel MJPEG Ethernet Cameras (PoE),
  DC3K-1-EAP
• 4x 5m Ethernet cables (up to 25m possible for
  each camera)
• Micrel Gigabit Ethernet (PoE) Switch
• Metal Plate for Desktop Application, additionally
  4 Vacuum cups for vehicle application included
• Software
• TES 3D Surround View Software under Linux on
  Apalis iMX6

9

• Resource Usage
• The 3D Surround View algorithm mainly exploits
  the GPU. The CPU is only used for system control
  and in the case of view-changes.
• Configuration 4 MJPEG IP cameras, combined
  80Mbit data rate _at_30 fps and 1Mpx resolution. Low
  polygon car model, 1280x1024 screen resolution at
  60Hz refresh rate.
• GPU 3-6 GFlops (proportional to pixel amount
  depicted on screen)
• RAM usage 60 MB in total
• 4 10 MB for camera data buffers
• 2 MB for 3D environment and car model
• 18 MB for camera video textures

10

• This turn-key system solution can be rapidly
  reproduced for evaluation and prototyping
  purposes. Of course, also the separate components
  can be exchanged or adapted to the target
  application needs, e.g.
• The cameras can be exchanged against other
  types.
• A dedicated application specific main board
  can be designed taking the Ixora Carrier board as
  a reference.
• The user interface and HMI overlay of the 3D
  Surround View software can be easily adjusted.
• The 3D model of the vehicles can be easily
  exchanged.
• Both Toradex and TES, are open to provide the
  necessary support.

11

• Getting started and familiar with 3D Surround
  View
• 3D Surround View is not only a topic of running
  an algorithm on an embedded platform and be done
  with it. There are many aspects of the system
  setup that need to be considered and require
  customization for specific use cases, e.g.
• Selection of the appropriate cameras regarding
  lens, resolution, physical data transmission
  (cable length!), power supply, number of cameras
  etc.
• Positioning of the cameras (usually the higher
  the better)
• Defining the appropriate viewing positions, i.e.
  virtual position and viewing vector of the
  observer in the 3D scene
• Optimizing the 3D Surround View parameters, e.g.
  Environmental model (shape and size of the
  projection bowl), stitching method and
  parameters.
• User interface for the 3D Surround View System
• 3D Model of the vehicle, potentially with
  transparency.

12
Just consider an excavator compared to a personal
car On an excavator, you need significantly
longer cables to the cameras, the cameras are
mounted typically much higher. Latency is not
that much of a concern as the driving speed is
lower and maybe you just want to apply only 3
cameras or mount the cameras un-symmetrically
because the driver can see the left front part of
his vehicle very well while he cannot see the
right front part at all.
So it might be a good idea to first play with a
3D Surround View system to get an idea of such
application aspects. For this purpose or for
those who just want to give the TES software a
try there is an offline PC demonstrator available
at zero cost. TES provides this demonstrator on
request in a zip archive with no need of
installation (just unpack). Any modern Windows PC
should be sufficient to run this software. The
program comes with prerecorded MJPEG Ethernet
streams in .PCAP-format and runs these streams
through the TES 3D Surround View software.
13
The PC keyboard and mouse can be used to change
the viewer position predefined views are
available on the number keys and free-flight is
possible via mouse and (Shift-) arrow keys. Also,
some of the 3D Surround View system parameters
like the bowl-size and -shape as well as the
stitching parameters can be adjusted. Do not
hesitate to get in touch with TES (coordinates
below) to get this Software.
First steps in the Lab and on the vehicle Now
that you have played with the software (and
assuming it does more or less what you need) you
might want to give it a try with real-life
cameras, either in lab conditions or right away
on the vehicle. The fastest way to do that is to
get a First Sensor Mobility Area View Kit
DCAVK (contents see above under System Setup)
and the TES 3D Surround View Windows PC Software
in Online-Mode (Evaluation Version). This kit
allows lab setups as well as on-vehicle
installation using a Windows Laptop. Up to 20m
cable length (between cameras and switch) has
been tested so far. Also here TES can assist you
in purchasing this setup.
14
And now on the embedded hardware Having tested
the system under lab conditions and on the
vehicle that target platform aspects now get
focus (again). I.e. you want to see your setup
running on the potential target hardware. At that
point, you may want to take your Apalis iMX6
module with Ixora carrier board (or buy some from
Toradex if you do not have those, yet) and get
the TES 3D Surround View software for that target
platform (prototype version). I.e. you get rid of
the PC and use the Apalis/Ixora platform instead.
The TES prototype software for that target
platform has the same functionality as the
evaluation version for PC and you can re-use the
configurations that you have setup at the earlier
stage now on the target.
15
Getting to a product For a product, you now have
to customize the system to meet your specific
application requirements. I.e. you use the Apalis
iMX6 module as core and might re-design the
carrier board based on the Ixora design data to
meet your specific requirements regarding,
interfaces, power supply, mechanics etc. In that
step or even before you also may want to exchange
the cameras with other types. Again, TES can
assist you also in the hardware and system
customization, development and (if wanted)
manufacturing process.
16
Summary We have introduced an embedded 3D
Surround View Turn-Key Solution and explained a
process to get you familiar with this topic
starting at zero cost. Following this process,
your product development risk is significantly
reduced because you can evaluate the system in a
step-by-step approach and at the same time
determine your application specific requirements.
Finally, we have introduced the steps to come to
a product based on this turn-key solution. Just
get started!
17
Thank you