TelePresence Microscopy at CSIRO Minerals

1
TelePresence Microscopy at CSIRO Minerals

• Colin MacRae
• CSIRO Minerals - Microbeam Laboratory
• www.minerals.csiro.au/microbeamlab

2
Talk outline

• What and why?
• History of TelePresence at CSIRO Minerals
• 3rd generation TelePresence system
• System architecture
• Control interfaces
• Video streaming technology
• Security features
• Conclusions, where to from here?

3
What is TelePresence?

• Tele-conferencing
• Video Audio
• Tele-control
• Microscope controls
• On-line data collection analysis
• Tele-control is provided by some manufacturers
• Ideally, TelePresence should be
• Interoperable with many platforms
• Shrink-wrapped software
• Looks professional, with easy to use interfaces
• Fast and responsive
• Stable and secure

4
Why do we want TelePresence?

• Internal clients (Brisbane, Hobart, Perth)
• External clients (Melbourne, Sydney, Brisbane)
• For analyses, our clients must either
• Mail / courier samples, and wait for our report
  (slow)
• Travel to our laboratory (expensive)
• TelePresence can help
• Provide results to client in real-time, and also
  allows instant feedback from client
• Attract new clients

CSIRO Minerals
CSIRO Minerals, Exploration Mining
BHP-Billiton
CSIRO Marine Research
Comalco RioTinto
5
What do we want to connect?

• Scanning Electron Microscopes
• Electron Microprobes Proton Microprobe
• X-ray detectors EDS WDS
• Cathodoluminescence systems
• Optical microscopes

6
What do we want to offer

• Clear fast images
• On-line microanalysis
• On-line x-ray spectra
• Large area optical views
• Streaming maps
• Security
• Voice over IP
• Data storage
• Data analysis/presentation software

7
What do we need from TelePresence?

• Provision of
• Real-time video from multiple sources (SEM, room
  camera)
• Streaming EDS spectra and other data (
  controls)
• Control of microscope stage column (lower
  priority)

• Minimal hardware requirements
• PC or laptop (no workstations)
• ADSL or cable modem, 1Mbit/s (not Internet2,
  GRID)

• Shrink-wrap software
• Easy to set-up
• Very easy to learn and use, with generic
  controls/interface
• Very, very stable
• Extraordinary security
• Low maintenance

• Must not affect normal operation of instruments!

8
What do we need from TelePresence?

• Provision of
• Real-time video from multiple sources (SEM, room
  camera)
• Streaming EDS spectra and other data (
  controls)
• Control of microscope stage column (lower
  priority)
• Minimal hardware requirements
• PC or laptop (no workstations)
• ADSL or cable modem, 1Mbit/s (not Internet2,
  GRID)
• Shrink-wrap software
• Easy to set-up
• Very easy to learn and use, with generic
  controls/interface
• Very, very stable
• Extraordinary security
• Low maintenance
• Must not affect normal operation of instruments!

9
TelePresence history

• TelePresence Microscopy 1999
• Nestor J. Zaluzec, Argonne National Laboratory
• Web-based, pervasive
• Web based notebook
• Video with GTS streaming technology

10
TelePresence history

• TelePresence Microscopy 1999
• Nestor J. Zaluzec, Argonne National Laboratory
• Web-based, pervasive
• Java system 2001
• Platform independent, safe code
• MJPEG video (TCP-IP)
• Web based notebook with embedded Java applets for
  Images, Spectra and maps
• Successful, but limited

11
TelePresence history

• TelePresence Microscopy - 1999
• Nestor J. Zaluzec, Argonne National Laboratory
• Web-based, pervasive
• Java system - 2001
• Platform independent, safe code
• MJPEG video
• Web based notebook with embedded Java applets for
  Images, Spectra and maps
• Successful, but limited
• C system 2004 ?
• Widely used language, with broad library support
• MPEG-4 video compression in software, changeable
  video size
• New, secure architecture

12
Video latency

• Definition
• Latency time displayed time captured
• What do we want?
• lt500ms for focussing (or better)
• lt200ms preferred
• Causes of latency
• Video compression / decompression
• Hardware accelerated rendering (done)
• Hardware compression (cheap MPEG-4 hardware
  compression coming soon)
• Router delays (congestion)
• Ultra-broadband internet (eg GRID)
• Reduce stream size (better codec)
• inter-router delays (speed / distance)
• More bandwidth wont help
• Use faster internet protocol (i.e. TCP ? UDP)

13
Another real life problem The Firewall
Firewall
Firewall
Internet
Client network
Our network
port 8080 (UDP), from specified addresses
Server
Client
Video
port 8080 (TCP)
Host
DMZ
14
Comparison of platforms
Web browser
Java
C
15
System architecture
Password
Client
Host
Server
(TCP socket)
commands data

• JEOL 8500F FEG-Electron Microprobe
• JEOL 8900R
• Electron Microprobe
• FEI Quanta
• FEG-Environmental SEM
• Hitachi S-5000
• In-Lens Cold FEG SEM
• PIXE
• Proton Microprobe Melbourne Uni

Video (UDP stream)
Server
Server
Generic interface
Instrument specific
Drivers
16
DirectShow video streaming

• Very easy to configure and re-configure graphs
• Other stream designs are possible, eg
• Saving stream to HDD
• Motion detection triggering
• Adaptable to as yet unforseen requirements
• All components are upgradeable and replaceable
• New codecs can be adopted as they become
  available
• Future proof
• We didnt have to write any of these components
  (only wire them together)
• Some DirectShow components are hardware-accelerate
  d
• Fast video rendering

17
TelePresence interfaces

• Video
• Camera control
• Tilt-pan-zoom controls
• Used in conjunction with video window
• EDX spectrometers
• Generic, easy to use interface
• Fast (2-5 updates/s)
• Text streams
• Miscellaneous others
• Stage and Electron optical System

18
Video streaming technology

• Microsoft DirectShow
• Widely supported standard with pre-compiled
  modules
• Hardware-accelerated rendering
• MPEG-4 encoding
• 320 ? 240 at 25 fps is 500kbit/s (ADSL)
• (12 Mbit/s by MJPEG)
• (59 Mbit/s for raw video)
• UDP transmission
• Lowest possible latency over IP (200ms)

19
Video cameras

• Canon VC-C1 and VC-C3 cameras
• 450 x 350 pixels at 30 frames/s.
• 90º pan, 25º tilt, 10? zoom
• Good low-light performance

20
Bench-top camera

• ELMO camera
• XGA video, 20 frames/s
• www.elmousa.com/presentation

21
Client feedback

• Problem Video stream is one-way. How do clients
  tell us what they want to see?
• Verbal feedback (telephone)
• that thing, over to the left. Up, say, half an
  inch
• Not very effective
• Graphical feedback
• Video overlay marker
• Visible to all other clients and the operator
• More tools to follow

22
Security

• Firewalls
• Blocks all direct communication with servers and
  drivers
• Host and servers are in separate DMZ network,
  not mounted on CSIRO file system
• Socket encryption
• Sockets are wrapped with Rijndael cipher, NIST
  advanced encryption standard (AES 2001).
• Certified for all levels of classification, up to
  and including top-secret.
• Hackers must break AES to read data, send false
  data, or issue commands

23
On-line help
24
Off-line data analysis
25
Elemental Data Set
Ti
Fe
26
Three colour map - Chimage
2500um
Chimage ref. MM
27
Hand selection of phases
28
Phase patched image
29
Electron induced cathodoluminescence map with
selected area and smaller inserts showing U, Th,
Yb and Y. X-ray maps produced on the nuclear
microprobe.
MacRae et al. Heavy Minerals 2003, Johannesburg,
South African Institute of Mining and Metallurgy.
pp139-143
30
Lines profiles on a spectral map
Distance
Frequency (nm)
340
925
31
Summary

• TelePresence allows Microscopy facilities
  providing a alternative service to clients
• High-quality real-time video is achievable over
  cable modem using MPEG-4 codec's and UDP
  transmission
• Internet security is ensured using firewalls and
  encryption of communications
• Architecture is extensible and applicable to any
  instrument for which DLLs or control code are
  available
• Off-line data analysis using Chimage

32
Acknowledgements

• CSIRO Minerals
• Aaron Torpy
• Nick Wilson
• CSIRO ITS
• Tony Sanderson
• Andrew Smith
• Engineering Firm Cymontkowski
• Marc Cymontkowski
• Further information
• TPM web reference
• http//minerals.tpm.csiro.au/software/teleprese
  nce/
• Chimage web reference
• http//www.minerals.csiro.au/em-unit/chimage_5_in
  dex.htm

33
Questions?