John Ford

1
CICADA Project

• John Ford
• August 5, 2008

The NRAO is operated for the National Science
Foundation (NSF) by Associated Universities, Inc.
(AUI), under a cooperative agreement.
2
Outline

• CICADA
• Program organization
• Assets
• GUPPI
• Future directions

3
CICADA

• Configurable Instrument Collection for Agile Data
  Acquisition
• FPGA based data acquisition and processing
• Uses CASPER tools and hardware
• Umbrella program for organizing FPGA projects
• Purchase/obtain boards, software, development
  systems
• 2 BEE2, 5 iBOB, 6 ADC, 10 GbE switches, servers,
  etc.

4
CICADA Projects

• Spectrometers
• KFPA backend
• Replacement for GBT Spectrometer
• Event capture
• Pulsar machines
• Incoherent machines
• Spigot replacement
• Coherent dedispersion machines

5
Basic KFPA Spectrometer 2 GHz bandwidth, 32K
channels

• CASPER Hardware
• 14 ROACH/ROACH-II boards, ADC's
• Needs to be in receiver room, fiber to lab
• EVLA Station boards plus CASPER software
• Uses standard EVLA hardware
• Directly compatible with EVLA digitizers
• External supplier

6
GUPPI History

• GBT Future Instrumentation Workshop, September
  2006
• University of Cincinnati Group worked on it until
  May, 2007, Produced report and basic design
• 2 WVU summer students and Glen Langston built
  event capture device over the summer
• Scott Ransom yells at us to stop planning and
  get to work in August, 2007
• October 29th, 2007, Held workshop to brainstorm
  the project and get started on detailed design
  and implementation
• April 4th, first pulsar observation (under test
  conditions with 43m telescope)?

7
Another pulsar backend? We already have 5!

• BCPM 4-bit, 48-96MHz, 96 chan, public
• Spectral Proc 6-bit, lt40MHz, 1024 chan, public
• Spigot 3-level, 50/800MHz, 1024/2048 chan,
  public
• GASP 8-bit, lt100MHz, coherent dedisp, private
• CGSR2 2-bit, lt100MHz, coherent dedisp, private
• The only machines to give Full Stokes are GASP,
  CGSR2, and the Spectral Processor

GUPPI 8-bit, 800MHz, 4096 chan, Full Stokes,
and coherent dedisp
8
GUPPI's Advantages

• Searching 800MHz of BW, 4096 chan, and
  RFI-resistance (from polyphase filterbank) will
  make GUPPI a Super-Spigot that will be
  unbeatable for searches from 1-5 GHz (previously
  the BCPM and Spigot)?
• High Dynamic-Range Studies 8-bit sampling, high
  spectral resolution and Full Stokes will make
  GUPPI perfect for scintillation studies, HI
  absorption, Faraday rotation measurements,
  polarization studies, and singlepulse studies
  (previously the Spectral Processor)?
• Ultra-High Precision Timing 8-bit sampling,
  800MHz of BW, and RFI resistance will allow
  unprecedented timing precision from 1-3 GHz for
  millisecond pulsars (i.e. the NANOGrav project
  and the search for nHz gravitational waves
  previously GASP/CGSR2)?

9
GUPPI Team

• NRAO- CV
• Paul Demorest, Ron Duplain, Rich Lacasse, Scott
  Ransom
• NRAO-GB
• Patrick Brandt, Glen Langston, Randy McCullough,
  Jason Ray
• Others
• Casperites
• Glenn Jones

10
Original UC design
11
GUPPI BEE2 signal processing chain

• Each 1.6 GS/s stream uses 1 FPGA for signal
  processing, 1 IBOB for data acquisition
• 4096 point PFB/FFT, with data streaming in and
  out, diagnostics in Block Ram
• Combines signals from each polarization,
  calculates stokes parameters, accumulates, and
  packages data, transmits over 10 Gb Ethernet to
  host.
• Minimum 50 microsecond accumulations
• 100 - 200 MB/sec data rate

12
GUPPI iBOB Design

• Uses 1 ADC module and 2 10 Gbit XAUI links to
  digitize and transmit data streams to BEE2
• Has room for diagnostics, digital downconverter

13
Sampler
14
GUPPI BEE2 signal processing design

• Each 1.6 GS/s stream uses 1 FPGA for signal
  processing
• 4096 point PFB/FFT, with data streaming in and
  out, diagnostics in Block Ram

15
Signal Processing
16
Xaui Alignment Block
17
GUPPI BEE2 signal combining FPGA

• Combines signals from each polarization,
  calculates stokes parameters, accumulates, and
  packages data, transmits over 10 Gb Ethernet to
  host.
• Minimum 50 microsecond accumulations
• 100 MB/sec data rate

18
Signal combining/output
19
Software Connections

• Server
• Controller access

• Client
• external interface

• Controller
• core logic
• parameter exposure
• Data Acquisition interface

• Data Acquisition
• data storage
• data quick look
• data status

• Demux
• common parameter access
• fully qualify parameter names

The dashed box contains those modules which will
run on GUPPI host, beef.

• IBOB Interface
• parameter access
• (TinySH client)?
• (Runs on IBOB.)?

• BEE2 Interface
• parameter access
• (client-server)?
• (Runs on BEE2.)?

20
User client module functions

• Simple command line interface
• Allows users to set and get all parameter values
• Allows users to start and stop FPGA processes
• Python based, extensible with standard Python
  functionality
• Users can write/run their own scripts to control
  observations
• Client modules can be seamlessly integrated into
  GBT MC system, or any system that can open a
  connection to a Python SimpleXMLRPCServer

21
Hardware Parameters

• hardware designexcerpt (right)?
• software register
• BRAM

22
GUPPI Command Prompt

• simple command prompt wrapped aroundPython
  interpreter
• tab completion for functions and parameter names
• four basic functions
• get and set for parameters
• load and unload for FPGA profiles

23
Just add Python

• write simple scripts to build more functionse.g.
  use Python execfile
• import Python modules for extensibility
• e.g. matplotlib (pylab) for plotting

24
Data Acquisition Software

• Multi-threaded shared memory architecture
• C program
• Connects to 10 Gb Ethernet using UDP
• Buffers data, provides quick-look functions
• Streams data to disk in PSRFITS format
• Handles 300 MB/S data stream with Myricom 10 Gb
  Ethernet card, Tyan Motherboard, Opteron
  Processors and AMCC hardware RAID
• Interfaces to Python based controller through
  shared memory command buffer

25
Portability and Extensibility

• Controller written in Python
• Data Acquisition software written in C
• Host is 64 bit Linux system, BEE2 runs 32 bit
  Linux system
• All connections to hardware are portable to
  newer/different interfaces
• All code written by NRAO staff GPL licensed

26
First Light
27
GUPPI Future Directions

• Add more diagnostics
• Add other configurations, narrow bandwidths, more
  channels, wider/narrower outputs, etc.
• Add coherent dedispersion modes
• Long FFT's needed to implement inverse ISM filter
• Possibly brute force better? Using convolution
  instead of FFT-gtMultiply-gtIFFT
• Maybe better to stream out to computers for
  calculations.
• Make it robust enough to release for everyday use
• Integrate with GBT observing system

28
Conclusions

• Reconfigurable Computing platforms make for quick
  hardware development
• GUPPI started in earnest in October, 2007. First
  light was in April 2008.
• Standard software interfaces make for quick
  control interface development
• The BEE2 and IBOB platforms use a common shell
  interface to the FPGA parameter space, allowing
  for easy portability between all hardware
  subsystems
• Python for the development language allowed the
  power of the Python interpreter to be used to
  provide complex functions easily