On-line acoustic source localisation with VoxNet

1
On-line acoustic source localisation with VoxNet

• Michael AllenCogent Computing Applied Research
  Centre, Coventry University

2
What is acoustic source localisation?
Estimate distance or angle to acoustic source at
distributed points (array) Calculate intersection
of distances or crossing of angles
Given a set of acoustic sensors at known
positions and an acoustic source whose position
is unknown, estimate its location
X
3
Motivating application

• Bioacoustics
• Understanding animal behaviour based on acoustics
• Specifically marmots (Blumstein, UCLA)
• End user knowledge gain
• why did animal x make a call at position y?
• How a wireless, embedded sensing system
• rapid deployment, in-network processing,
  interaction
• Why on-line, real-time?
• Enable actuation user taking photo
• Why develop a flexible instrument?
• General applicability
• React to events, change physical network
  configuration

4
Compelling and challenging?

• Acoustic sensing requires high sample rates
• Cannot simply sense and send
• Implies on-node, in-network processing
• Indicative of generic high data rate applications
• A real-life application, with real motivation
• Real life brings deployment and evaluation
  problems which must be resolved

5
Basic marmot localisation processing chain
Raw detection data transfer
Direction of Arrival Processing
Data fusion
Distributed event detection
(NB 2-3 detections are required to estimate
position)
Observation
6
Existing processing components

• On-line event detector (Trifa and Girod, UCLA)
• Energy in frequency bands
• Angle of arrival estimator (Chen et al, UCLA)
• Approximated Maximum Likelihood (AML)
• Data fusion technique (Collier and Ali, UCLA)
• Pseudo-likelihood map

Ali, M., Azgari, S., Collier, T., Allen, M.,
Girod, L., Hudson, R., Yao, K., Taylor, C.,
Blumstein, D., "An Empirical Study of
Collaborative Acoustic Source Localization,
Journal of Signal Processing Systems (JSPS),
November 2008.
7
So where are the problems?

• Integrating existing components into end-to-end
  system
• Glue and new supporting components were needed
• Early CENS work (2007)
• Catering for on-line performance
• Main achievement 2008
• Timeliness results must be gathered so they are
  still useful
• Adaptive techniques to deal with network
  unreliability
• Providing a generic approach
• Main achievement 2008
• Rather than just a marmot localisation system
• Framework for near-real-time high data rate
  applications - autonomy

8
Contributions

• Multi-level evaluation cycle iterative
  prototyping
• Combination of in-situ deployment, controlled
  experimentation and simulation
• Instantiation of on-line acoustic source
  localisation system and adaptive behaviour
• Suitable for proof of concept evaluation purposes
• Framework to support more general acoustic source
  localisation and high data rate apps.
• Evaluated against existing literature

9
Talk Roadmap

• An on-line marmot localisation system
• The VoxNet platform (and its deployment)
• Reduction of latency for position estimation
• Proof-of-concept techniques to reduce latency
• Acoustic localisation framework
• Generic approach to acoustic localisation

10
What is the VoxNet platform?

• Hardware and software platform
• For prototyping acoustic sensing applications
• Rapidly deployable small size, internal battery
• 32-bit, 400MHz ARM processor, runs Linux
• 802.11b, 4 channel 48KHz mic. array

VoxNet node
Ad-hoc Network of Deployed Nodes
Gateway
Control Console/Sink
Allen, M., Girod, L., Newton, R., Madden, S.,
Blumstein, D., Estrin, D., "VoxNet An
Interactive, Rapidly-Deployable Acoustic
Monitoring Platform", (IPSN 2008)
11
VoxNet software stack

• High level VoxNet programs written using
  WaveScript
• Program defined as a directed graph of
  communicating stream operators which data flow
  through
• Underlying Networked streams allow program flow
  across multi-hop network
• TCP-based streams
• DSR-based multi-hop networking
• Sink-side control console allows network level
  interaction with nodes
• Extensively used for controlled experimentation
  in this work

12
Deployment

• VoxNet was deployed at the Rocky Mountain
  Biological Laboratory (RMBL) in Summer 2007
• In-situ marmot eventdetection
• Naïve grouping logic rendered localisationresult
  s unusable
• Controlled experimentation
• Multi- and single-hop data requests (0.8, 32,
  128KB) using control console

13
Application-related lessons learned

• Deployment density/coverage what is optimal?
• In-situ interaction, visualisation, reprogramming
• Integration of off-line processing
• Latency is affected by environment, network
  factors
• Important to understand where the problems are
• Better grouping algorithm required
• Transmitting raw data for any detection is not
  good if detectors have high false positive rate

14
Talk Roadmap

• An on-line marmot localisation system
• The VoxNet platform (and its deployment)
• Reduction of latency for position estimation
• Proof-of-concept techniques to reduce latency
• Acoustic localisation framework
• Generic approach to acoustic localisation

15
Reduction of Latency

• Recall on-line system aim is timeliness
• We observed latency in-situ (RMBL)
• Main possible sources of latency
• On-node (event detector)
• Network transfers (transmitting data)
• Sink-side processing (AML processing, data
  fusion)
• Network incurs the biggest latency
• Chose to investigate node and network components

16
Observed latency-inducing factors

• Application level Bursts of events can cause
  queued traffic in the network, increasing latency
• Multi-hop topology Number of hops a node is from
  sink and number of nodes at each hop will affect
  latency
• Physical level the lower the gain of the antenna
  at the sink/gateway, the shorter the separation,
  increased loss
• Other potential factors data transport, data
  rate

17
How can latency be reduced?

• By using application-specific knowledge
• Axioms
• Only send data that ensures information gains
• Solution Lazy Grouping
• Process locally when beneficial (aids timeliness)
• Solution Adaptation
• Evaluated solutions in simulation

18
What is Lazy Grouping?

• An algorithm for data grouping, data collection
• Nodes send small detection notifications with
  detection timestamp, sink groups them together
• Sink requests full data if necessary
• Treat network as expensive resource
• Reduce data transmitted no useless data
• Reduce time taken to gather a groups data
• Free the channel for other events

19
Walk through
T1 100
Identity consistency checkNode B already in
group?
Identity consistency checkNode C already in
group?
T2 200
Temporal consistency check T2 mean lt fuzz?
Temporal consistency check T3 mean lt fuzz?
mean
0
100
150
200
T3 300
fuzz
0
100
150
400
Node A, 100
Node B, 200
Node C, 300
20
Walk through
Data Request
Detection data
mean
0
100
150
200
fuzz
0
100
150
400
Node A, 100
Node B, 200
Node C, 300
21
Evaluation of Lazy Grouping

• Methodology simulation and experiments built
  around an in-situ trace
• Event detections at RMBL
• Experiment one reduction in data transmitted
• Run off-line version of grouping on data trace
• Experiment two reduction in collection time
• Model data collection using controlled
  experimental results

22
Experiment one data reduction
Reduced amount of data that would have been sent
by 64 (grouped 126 out of 353 detection events)
23
Experiment Two Data Collection

• Gathered controlled experimental data at RMBL
• Time take for all nodes to send 800B or 128KB
• Modelled the Lazy Grouping collection as
• notification, request mean of the 800B
  transfers
• raw data mean of the 128KB transfers
• queue number of raw detections in nodes send
  queue
• Evaluated for all groups made in Experiment One

latency notification request (raw data
queue)
24
Experiment Two Data Collection
Data collection time potentially reduced by up
to 20 seconds
25
Adaptation

• AML can be carried out at sink or on node
• Would it be better to process locally, send less
  data?
• Is this always so?
• Raw data 32KB, Sink AML 0.04s
• Node AML 2.2s, Data 800B
• Controlled multi-hop experiment at RMBL

26
Simulating local processing trade-off
NETWORK LATENCY
AML PROCESSING TIME
Process locally, send 800B
Send raw data, process at sink
As hops from sink increases, benefit of
processing DoA locally is seen
DoA processing
27
Adaptation algorithm

• (Assume that nodes have a queue for processing
  AML algorithm)
• Predict latency of transfer
• Compare to latency to process AMLs in queue
• Process locally if it will take less time,
  otherwise send raw detection data

28
Predicting latency

• Approach - Dynamic estimation
• Use previous transfer latencies to predict next
  transfer latencyd items in queue h hops
  from sink s size of itemg mean rate of k
  prev. transfer(s) 1/k . ? (hs/t)

prediction dhs/g
29
Adaptation evaluation

• Methodology simulation using experimental data
  traces
• RMBL (actual data trace, single-hop)
• Royce Hall, UCLA (Controlled data requests,
  multi-hop)
• Laboratory, UCLA (Controlled data requests,
  multi-hop)
• Latency, data size, node ID recorded for each
  transfer
• Latency predicted for each recorded transfer
• k 3 for dynamic estimation calculation

30
Performance

• How many choices did algorithm get right?
• RMBL 70.5, Royce 96.3, Lab 97.0
• How accurate were the predictions?
• Percentage within 25 of actual latency
• RMBL 49.0, Royce 47.7, Lab 63.5
• Disappointing performance
• Further simulation found that network latency and
  local processing time affected one another

31
Arbitrary local processing times
Results are less accurate when local processing
time median transfer time Explains difference
between data sets
Median transfer latenciesRMBL 2.9s, Royce
0.5s, Lab 0.5s
32
(No Transcript)
33
Summary of findings

• Lazy Grouping
• Application gain only useful information
  gathered in more timely manner
• Adaptation Dynamic Estimation
• Further improvement from Lazy Grouping
• General appeal for high data rate applications
• Do in-network processing as needed and formalise
  the procedure
• Exploits the node resources, frees the network
• Enhanced observable/reported event density
• If local processing time is similar to mean
  transfer latency, expect reduced performance
• Prediction accuracy can be further improved upon
  re-deployment of adaptive network

34
Roadmap

• An on-line marmot localisation system
• The VoxNet platform (and its deployment)
• Reduction of latency for position estimation
• Proof-of-concept techniques to reduce latency
• Acoustic localisation framework
• Generic approach to acoustic localisation

35
Conceptual flow of generic on-line acoustic
source localisation system
Distributed event detection
Event grouping
Data collection
Data fusion
Information representation
Examples of each stage have been instantiated and
evaluated (if not previously evaluated)
36
General class of application

• General class of application - high data rate,
  on-line
• Apps need to feature
• Filtering behaviour - the ability to determine
  which data is important to the overall goal of
  the system
• Dynamic behaviour - the ability to adapt to
  changing conditions within the network
• Instantiated by Lazy Grouping and Adaptation
• Also seen in other high data rate frameworks
  (VanGo, Lance)

37
Contributions

• A generic framework for acoustic localisation
• Identified important behavioural features for
  class of applications
• Instantiation of components within framework
• Proof of concept, adequate for validation
• VoxNet platform (collaboration)
• Lazy grouping algorithm
• Adaptation algorithm
• Multi-level evaluation suited to real-life
  problem
• In-situ experimentation, controlled experiments,
  simulation

38
Acknowledgements

• Coventry Elena Gaura, James Brusey
• CSAIL/MIT Lewis Girod, Ryan Newton, Sam Madden
• CENS/UCLA Travis Collier, Deborah Estrin, Dan
  Blumstein
• EE/UCLA Andreas Ali, Kung Yao

39
Missing features and evaluation

• Priority
• How to prioritise data for collection
• Predict data collection timeouts
• Sink side processing
• Have not considered contribution of sink-side
  latency, particularly data fusion
• Investigation for future work

40
Example energy calculation
ch1
VoxNetAudio
rewindow
hanning
fft
// acquire data from source, assign to four
streams(ch1, ch2 ch3, ch4) VoxNetAudio(44100)
// window ch1 data stream into 32 sample chunks,
// passing through hanning then fft
operatorsfreq fft(hanning(rewindow(ch1,
32)))// apply bandpass filter to
freqbpfiltered bandpass(freq, 2500, 4500)//
calculate energy in bandpassed dataenergy
calcEnergy(bpfiltered)
41
Splitting up a logical program
freq
toNet(frequencies)
fromNet(frequencies)
bandpass
send(frequencies)
arrived(frequencies)
// acquire data from source, assign to four
streams(ch1, ch2 ch3, ch4) VoxNetAudio(44100)
freq fft(hanning(rewindow(ch1, 32)))//send
data to sinktoNet(frequencies,freq)//
receive data from node streamfreq2
fromNet(frequencies) // apply bandpass filter
to freqbpfiltered bandpass(freq2, 2500,
4500)// calculate energy in bandpassed
dataenergy calcEnergy(bpfiltered)
Node
Sink
42
Rejection from groups
Detections that are not close enough to mean are
rejected
43
Breakdown of choice percentage
Royce Hall data set
RMBL data set
44
What are the network effects?
45

• Data transport level TCP interprets wireless
  loss as congestion (causes back-off)
• Link level variable data rate in 802.11 affected
  by loss

46
Bursts of events can cause build up of traffic in
the network
47
As events aggregate in network, sending latency
increases
48
Number of nodes at each hop and number of hops
can increase latency(message forwarding, etc)
49
Choice of antenna at sink matters Lower gain
antenna increased latency and higher variability
50
VoxNet software stack
Node application
Sink application
Control console
Node-side control logic
Networked streams
Networked streams
High level application is supported by lower
level components (C-based)
TCP/IP
TCP/IP
Multi-hop Routing (DSR)
Multi-hop Routing (DSR)
Time synchronisation
Time synchronisation
Audio data acquisition
51
Networked streams, multi-hop communication

• Networked streams allow WaveScript programs to
  flow across network boundaries
• Nodes communicate with sink using network streams
• Publish/subscribe mechanism
• Wrapped TCP connections
• Buffered outgoing message queues
• Multi-hop communication
• Application level Dynamic Source Routing (DSR)
• IP packet forwarding enabled in kernel

52
Control console

• Allows communication with nodes below high level
  application layer
• Start/restart/pause programs
• Disseminate new programs
• Receive status updates, control information from
  nodes
• Used extensively in this work to perform
  controlled experiments
• Data requests to nodes (send back X KB)

53
Current VoxNet node hardware

• Resource rich, rapidly deployable
• 4 channel mic. array at 48KHz, 802.11b
• Linux, 400 MHz ARM/64MB/8GB
• Small form factor, internal battery
• Rapidly deployable
• Field tested over several months(synchronised
  audio recording) in Mexico and Colorado

With thanks to Travis Collier
54
Lazy Grouping algorithm
Nodes must be unique in a group
All detections in a group must be within x
milliseconds of the mean