Utbildning

1
Fysikalisk Systemteknik Christian
Bohm Overview About the group Overview of
projects What is an FPGA The major
projects Summary
2
Fysikalisk Systemteknik Personell Professor
Christian Bohm Lecturer Sam Silverstein Part time
lecturer Magnus Engström Adjunkt Eddie
Ahlestedt Forskningsingenjör (emeritus) Hans
Eriksson Forskningsstuderande Jonas
Klereborn Abdelkader Bousselham Attila
Hidvegi Florian Bauer (external) New
Instrumentation Physics
Experimental Physics
Technology
We collaborate with experimental physics groups,
focusing on the development of new
instruments. Make it easier to develop and
maintain useful engineering skills while
retaining an active grasp of the relevant
physics. Use experience from projects to solve
new problems. Look for general solutions and
methodologies which are easier to carry over to
new problems.
3
Different instrumentation projects In
collaboration with particle physics,
SU Digitizing electronics for ATLAS TileCal The
Jet/Energy-sum processor for the ATLAS first
level trigger In collaboration with
astroparticle physics, SU Participation in the
development of IceCube In collaboration with
physicists at KI Development of a SPECT
camera Development of a PET-Camera In
collaboration with molecular physics,
SU Frequency stabilisation of semiconductor las
ers for laser trapping and cooling of atoms
4
What is a Field Programmable Gate Array?
Firmware
Configuration memory
Logic with Data path switches
5
FPGA
Configurable Logic Blocks
6
Configuration memory pattern defines circuit
001011100-------1--0---01000000-------1---0-110-10
10
a
b
c
e
d
f
g
clk
h
b
c
a
e
d
clk
7
Modifying the memory content changes the circuit
001101100-------0--0---11111000-------0---1-000-10
10
a
b
c
e
d
f
g
clk
h
b
a
e
d
clk
8
FPGAs have been around since mid-1980s Early
components were programmed at a bit-level using
graphic editors Increased complexity required
better methods High level languages (VHDL),
or Schematic specifications
9
When designing complex circuits with FPGAs one
has to consider Does the design fit? Is it
fast enough? Is it too expensive?
10
FPGA design process
High level description (VHDL)
11
State-of-the-art FPGAs

• Very complex
• Many gates 8 milion gates
• many i/o pins 600
• flexible interconnects 7 metal layers
• high costs 50 kkr
• Multiple clocks - 8
• Embedded memories 64 MB
• Embedded multipliers - 64
• Embedded processors 4 PowerPC
• High speed IO 3 GB/s

12
Efficient tools required

• Re-use of previously developed code blocks
• Intellectual Property blocks
• IP-blocks can be
• In-house developed
• Commercially available
• Freely available open-core
• Part of the design can be accomplished by
• assembling compatible IP-block
• Processors (embedded or IP)
• Memories
• Busses
• Interfaces
• Etc.

13
When designing complex FPGA modules one must
decide

• What to implement in logic
• What to implement in processor software
• Hardware software co-design
• VHDL ? System-C or Handle C

14
ATLAS Tile Calorimeter Read-out
Many prototypes test beam tests
earliest ATLAS subsystem lots of firsts
production experience 2000 boards this year
15
The ATLAS TileCal Digitizer collaboration with
particle physics SU
Task to digitize pre-amplified PMT-pulses and to
transfer data selected by the L1-trigger to the
higher level triggers.

• 16-bits dynamic range with limited precision
• L1 buffer memory 2.5 us
• Storage of selected data
• Format data
• send to level 2
• Physical layout
• Noise control
• Radiation tolerance
• Reliability (physical chain electrical star)
• We also made a optical link with matching
• reliability

16

• 16-bits dynamic range with limited precision

• Reliability

10
17
Experiences from the digitizer project Large
scale system design System aspects timing and
grounding Reliability Radiation tolerant
design Production
18
ATLAS data flow

• Large detector sizes and high
• precision implies large number
• of channels and large event sizes
• Low storage rates and therefore
• high selectivity

LHC physics looks for rare events 1 in 1014 ?
High event rates
The first level trigger must in the worst case be
able to reject all but 1 event in 10000
The task of the first level trigger is to process
all events i.e. new data every 25 ns
The first level trigger uses data with low
spatial Resolution (64x64) and restricted dynamic
range (1-256) from calorimeters and muon detector
Since all data must be stored while waiting for
the L1 decision the processing must be quick 1ns
The first level trigger trigger decisions
and Region Of Interest information to the second
level
19
ATLAS first level trigger collaboration with
particle physics SU
Looks for typical features for event selection
Calorimeter trigger
L1 accept
Analog Input signals
Central Trigger (CERN)
Muon Trigger (Italy)
ROI info
The calorimeter trigger is a Birmingham-London -Ru
therford-Stockholm-Heidelberg-Mainz collaboration
Looks for isolated clusters resembling
single Electrons/hadrons in the ECAL and HCAL
Calorimeter trigger
Electron/Tau Processor (GBR)
64x64 8-bit
64x64 Analog signals
Preprocessor (Heielberg)
32x32x2 8-bit
Jet (Sthlm) and Missing energy (Mainz) processor
Looks for energy balance
Digitizes determines amplitudes and pulse starts
20
Analog
Tower 0.1 x 0.1
Calorimeter LAr, Tile
S
Realtime data path
Pre-Processor Timing alignment 10-bit
FADC FIFO BCID Lookup Table BC-MUX Sum 2x2
Pre-Processor RODs (DAQ)
Cluster Processor (e/g and t/had) Cluster Finding
Region Of Interest Builder (L2)
.1 x .1
Count
Level-1 CTP
Jet/Energy-Sum Processor

.2 x .2
ET EX EY
Jets
CP/JEP RODs (DAQ)
Count
SET, ET
21
The JET/energy sum trigger
Look for .4x.4, .6x.6 and .8x.8 energy
clusters centered around a local .4x.4 maximum
Form global sums of total Et and missing
Et Process 1024 .2x.2 jet elements in
parallel 32 processor boards with large FPGA for
Jet and missing energy processing, sharing
overlapping environment data Latency
(processing time) 200 ns
Many different module types
Standardized modules
22
The JET trigger
We have built a 18 layer backplane with gt20 000
pins for the Jet and the E/t processors
VME - - Communication with neighbors Report
results
23
The JET trigger
We have participated in the design of the
JEM Processor board.
And developed firmware for the algorithms and
control functions
24
Experiences from the trigger project Large
scale system design Massive pipelined parallel
processing Reliability Large FPGA design (gt 1
Mgates) Draw on experience from earlier
bit-serial trigger project to do pipelined
processing of multiplexed data -gt more efficient
use of logic and interconnects!
25
SU SPECT Collaboration with Karolinska hosptal
The design of a SPECT camera with an
innovative cylindrical crystal started 1992
72 PMTs around crystal position
determination via light sharing Earlier design
based on transputers discontinued Pulse
detection sampling ADCs Digital pulse
processing digital trigger Firewire
network Xilinx FPGAs Texas Instrument DSP TMS
320 6000 family
26
ICE-CUBE Collaboration with the astroparticle
physics group at SU
1400 m
1000 m
60 modules/string
80 strings Volume 1 km3
Digital Optical Module (designed by D. Nygren)

• Self-triggers on each pulse
• Captures waveforms
• Time-stamps each pulse
• Digitizes waveforms
• Performs feature extraction
• Buffers data
• Responds to Surface DAQ
• Set PMT HV, threshold, etc
• Noise rate in situ 500 Hz

27
ICE-CUBE
The DOM circuit board
2 ATWD - 4 channel transient waveform recorder
300 MHz 256 samples 2 channels hi and lo gain
from PMT Symmetric timing pulses between hub and
DOM sampled at 20 MHz 10bits Supports a higly
stable local clock 3.3 ns rms FPGA and CPU
combined in new Altera FPGA
28
ICE-CUBE
Experimental Requirements IceCube

• Time resolution lt5 ns rms
• Waveform capture
• gt250 MHz - for first 500 ns
• 40 MHz - for 5000 ns
• Dynamic Range
• gt200 PE / 15 ns
• gt2000 PE / 5000 ns
• Dead-time lt 1
• OM noise rate lt 500 Hz (40K in glass sphere)

Proposed IceCube DAQ Network Architecture
Pair
DOM
String Subsystem
60 DOMs
20
"DOM
kB/sec
HUB"
N x 20
kB/sec
N pairs
Strings
8
0
String LAN
100
BaseT
Total traffic 0.6 MB/sec
String
Processor
All Hits -
0.6 MB/sec
String Coincidence
Lookback Requests
Messages - 170
kB/sec
Fulfill
Lookback
Event
Fulfill Lookback Messages
Messages
Builder
Event LAN
0.6 MB/sec
String
100
BaseT
Coincidence
Total traffic 1.6 MB/sec
Messages
Built events 1 MB/sec
all event builders)
Global
(
Trigger
Event Triggers /
Lookback Requests for
Online LAN
all Strings
- 0.8 MB/sec
BaseT
100
Satellite
Total traffic 1MB/sec
Offline
SAN
Data
(Network
Handling
Disk Storage)
Tape
--
29
Digital Laser Control Collaboration with Anders
Kastbergs group
Modulation
Absorption cell
detector
laser
Lock on w frequency component 0 ? lock on maximum
Lock-in amplifier
Aim to design a simple laser control that
can manage a large number of units Our solution
use an FPGA based lock-in module
Asin(wtf)
Asin(2wtf)-Asinf
Asinf
x
cos(wt)/2
Cordic algorithm to produce sine and
cosine waveforms second order Butterworth low
pass Filter Hardware design based on SPECT
module
30
SUMMARY