Implementing%20White%20Rabbit%20in%20your%20design

1
Implementing White Rabbit in your design

• Peter Jansweijer

2
Outline

• My background
• KM3NeT / FPGA / Serial Communication
• White Rabbit introduction
• Technology / Implementations
• WR PTP Core in detail
• How to implement WR in your design

3
KM3NeT Artist impression detection method
Cherenkov
m
n

• 11.520 DOMs in the deep sea at 3-5 km depth

4
The very beginning Measuring propagation delay
over a coded serial communication channel using
FPGA.
011101011000001010110111010110000010101101110101
0
3
1
RxRecClk
BitSlide(40)
0 0001 1
0 0000 0
0 0011 3
Start/Stop delay
Presented at the VLVnT09, October 15, 2009 in
Athens Elsevier Nuclear Instruments and Methods
http//dx.doi.org/10.1016/j.nima.2010.04.126FPGA
transceivers propagation delay study
http//www.nikhef.nl/pub/services/biblio/technical
reports/ETR2010-01.pdf
5
White Rabbit Introduction
6
Layer 1 Syntonization

• Syntonization The adjustment of two electronic
  circuits or devices in terms of frequency
• All network devices use the same physical layer
  clock.
• Clock is encoded in the Ethernet carrier and
  recovered by the receiver chip.
• Phase detection allows sub-ns delay measurement

7
Precision Time Protocol (IEEE 1588)

• Frame-based synchronization protocol.
• Synchronizes local clock with master clock.
• Link delay evaluated by measuring and exchanging
  frames with tx/rx timestamps

8
Digital Dual Mixer Time Difference (DDMTD)
9
Using the DDMTD as phase detector and phase
shifter
10
White Rabbit an extension of Ethernet

• Two separate services
• (enhancements to Ethernet) provided by WR
• Synchronization accuracy better than 1 ns
  precision (tens of ps sdev skew max)
• Deterministic, reliable and low-latency Control
  Data delivery

11
Standardization

• Ongoing standardization effort to include WR into
  PTPv3
• Extension to PTP (IEEE1588) defined as PTP
  profile
• Compatible with standard PTP
• According to ISPCS Plug Fest results
• White Rabbit is the most accurate PTP
  implementation in the world!

12
White Rabbit Switch http//www.ohwr.org/projects/w
hite-rabbit/wiki/Switch

• Central element of WR network
• 18 1000BASE-BX10 ports
• Open design (H/W and S/W)
• Commercially available

13
White Rabbit Node example 1 SPEC http//www.ohwr.
org/projects/spec/wiki

• FMC-based Hardware Kit
• All carrier cards are equipped with a White
  Rabbit port.
• Mezzanines can use the accurate clock signal and
  TAI (synchronous sampling clock, trigger, time
  tag, ).
• Starting kit http//www.ohwr.org/projects/wr-star
  ting-kit/wiki

14
Other White Rabbit Node examples
CUTE-WR (LHAASO)
Central Logic Board (KM3NeT)
CRIO-WR (CERN)
SVEC (CERN)
SPEXI (CERN)
15
White Rabbit PTP Core http//www.ohwr.org/projects
/wr-cores/wiki/Wrpc_core
16
White Rabbit PTP Core in detail
17
White Rabbit PTP Core in detail
PHY Interface to the physical network, with
deterministic phase relationship between gigabit
clock and system clock
18
White Rabbit PTP Core in detail
Endpoint Ethernet MAC, with precision time
stamping capabilities of in- and out-bound
Ethernet frames
19
White Rabbit PTP Core in detail
Normal packets
Redirector Redirects all PTP Ethernet packets to
the Mini-NIC
Receives the payload of the PTP packets and
stores payload data in memory
20
White Rabbit PTP Core in detail
Embedded Processor system (LatticeMico32 Bus
interface Memory) running a PTP-daemon
21
White Rabbit PTP Core in detail
Now the CPU has knowledge of time. It can
adjust the local time and update the proper TAI
time and control the Pulse Per Second (PPS)
generation
22
White Rabbit PTP Core in detail
The local oscillator that is used for TX is
locked onto the RX receive clock. A PLL is made
using Digital Dual Mixer Time Difference (DDMTD)
technique and software on the LatticeMico32
(hence Soft-PLL) to tune the oscillator. DDMTD
allows for lt 1 ns phase adjustment under control
of the LatticeMico32 gt a result of calculations
of the round trip delay (Timing Synchro)
23
White Rabbit PTP Core in detail
1-Wire is used to fetch a unique MAC addressUART
is used for debug only
24
How to implement WR in your design
25
How to implement White Rabbit in your design
FPGA
DATA (1 Gbps)
PHY
SFP
Unique-ID (MAC-addr) (optional)
GTREFCLK
SPI
Your own stuff
VCXO
DAC
125 MHz
CLK125M_PLLREF
REF clock generator
CLK_DMTD
CLK20M_VCXO
VCXO
DAC
PLL
20 MHz
DMTD clock generator
Timing (TAIns,REFCLK)
http//www.ohwr.org/projects/white-rabbit/wiki/WRR
eferenceDesign
26
White Rabbit CLBv2(KM3NeT)
FPGA
IP/UDP Packet Buffer Stream Selector (IPMUX)
31 TDCs
Start Time Slice UTC Offset counter since
Fifo
TDC0
Time Slice Start
RxPort 1
RxPacket Buffer 64KB
31 PMTs
RxPort 2
Rx Stream Select
Rx_mac2buf
Rx_buf2data
Fifo
TDC 30
Flags
RxPort_m
Management Control
S
State Machine
Management Config.
Pause Frame
ADC
Fifo
Hydrophone
TxPort 1
TxPacket Buffer 32KB
TxPort 2
Tx Stream Select
Tx_pkt2mac
Tx_data2buf
Management Control
Flags
TxPort_m
S
S
Nano Beacon
M
M
M
S
M
M
M
M
M
Debug LEDs
S
S
S
I2C
GPIO
I2C
Xilinx Kintex-7
Data
UTC time Clock (PPS, 125 MHz)
Debug RS232
Temp
Compass
Tilt
SPI Flash
Control
Point to Point interconnection
Wishbone bus
27
White Rabbit Group http//www.ohwr.org/projects/w
hite-rabbit/wiki
Picture taken _at_ 4th WR workshop (2011), GSI
Darmstadt

• WR Users http//www.ohwr.org/projects/white-rabbit
  /wiki/WRUsers

Many slides in this presentation orginate from
the WR group Thanks!
28
Thank you