Fermilab Control System ACNET - PowerPoint PPT Presentation

Loading...

PPT – Fermilab Control System ACNET PowerPoint presentation | free to view - id: bbeb1-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Fermilab Control System ACNET

Description:

Console applications to save/restore to/from database on user request ... Eight used at A0 photo-injector. Variety of functions. A few at TTF for modulators ... – PowerPoint PPT presentation

Number of Views:63
Avg rating:3.0/5.0
Slides: 46
Provided by: kevin437
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Fermilab Control System ACNET


1
Fermilab Control System(ACNET)
  • J. Patrick
  • February 17, 2005

2
Fermilab Control System
  • Overview
  • Applications
  • Central Services
  • Front-ends/Instrumentation
  • Timing
  • Security and Remote Access
  • Summary

3
Fermilab Control System
  • aka ACNET
  • Unified control system for the entire complex
  • 400 MeV Linac
  • 8 GeV Booster Synchrotron
  • 120 GeV Main Injector Synchrotron
  • 1 TeV Tevatron Synchrotron
  • antiproton source target/debuncher/accumulator
  • antiproton Recycler storage ring
  • fixed target lines
  • Simultaneous operation of
  • Tevatron proton-antiproton collider (storage
    ring)
  • Antiproton production and storage (0.5 Hz)
  • 120 GeV fixed target to Meson lab (0.1 Hz)
  • 120 GeV fixed target to NUMI/MINOS (0.5 Hz)
  • 8 GeV fixed target to MiniBoone (5 Hz)

4
Fermilab Accelerator Complex
5
Control System Overview
Console Applications
Java Applications
Web Applications
Database
Application
CentralServices
Open Access Clients
Servlets
Consolidators
Central
ethernet
IRM Front-Ends
MOOC Front-Ends
Labview Front-Ends
Front-End
field bus VME, SLD, Arcnet, ethernet,
Field Hardware
CAMAC, VME, PMC, IP, Multibus, CIA, GPIB,
6
History
  • Originally developed for Tevatron in early 80s
  • Substantial evolution over the years
  • CAMAC field hardware -gt very broad diversity
  • PDP-11/Mac-16 front-ends-gt x86 Multibus-gt VME
    based processors Labview
  • DEC PCL network-gt IEEE 802.5 token ring-gt
    ethernet
  • VAX database -gt Sybase
  • VAX central services -gt Java
  • PDP-11 applications -gt VAXstations-gt Java
    application infrastructure-gt Port of VAX code to
    Linux in progress

7
System Scale
  • 200,000 devices/350,000 properties
  • 475 front-ends
  • 90 Sun Netras 12 Linux PCs that run Java based
    data loggers, servlets, open access clients
  • VAXes to run VAX central services
  • 70 VAXes that run high console level applications
  • 70 Linux PCs to to parallel VAXes until VAX
    retirement.
  • X-displayed out to desktop PCs

8
Device Model
  • lt 8 character devices names
  • e.g. TBEAM
  • Informal conventions provide some rationality but
    no enforcement
  • More verbose hierarchy layer created, but not
    utilized/maintained
  • Each device has one or more properties
  • Reading, setting, analog digital alarms,
    digital status control
  • Device descriptions stored in Sybase database
  • Scaling transformations defined in device
    database
  • Add new devices with DABBEL command language on
    VAX
  • Templates available for various device types
  • View/modify device definitions with GUI
  • Monotype data, arrays are transparently supported
  • Structured data is often done, though support is
    not transparent
  • Application (library) layer must properly
    interpret the data from the front-end and perform
    required data translations

9
Device Database GUI
10
Communication
  • Applications/Central Services/Front-ends
    communicate via ACNET protocol
  • Custom protocol now exclusively based on UDP
  • Extensive network/switches etc
  • Asynchronous request/reply request multiple
    reply
  • Timeout/retry mitigates UDP unreliability.
  • RETDAT/SETDAT protocol for readings/settings
  • Can have multiple devices/packet
  • Fast Time Plot (FTP) returns blocks of readings
    taken at up to 1440 Hz every 0.5 seconds
  • Snapshot protocol returns blocks of 2048 points
    taken at an arbitrarily high rate

11
Communication
  • Application communication with front-ends passes
    through central layer
  • Datapool Manager on VMS via shared memory
  • Data Acquisition/Server Engine (DAE) in Java (via
    RMI)
  • Consolidation of requests to avoid overloading
    front-ends
  • Centralizes security, monitoring, error logging
  • Issues include
  • Limited packet size and internal device address
    space.
  • Timestamps not part of protocol packet, applied
    on data receipt
  • GPS sync clock event sent to all front-ends, can
    be included in structured devices
  • New get/set32 protocol addresses the above, only
    on centrals now
  • Lack of transparent support for structured data.
  • Bridging to network devices that dont directly
    support ACNET is via front-ends or Open Access
    Clients

12
Database
  • Central Sybase database
  • Three parts
  • Device and node database
  • Application database
  • Used by applications as they wish
  • Save/Restore/Shot Data database
  • MySQL used for data loggers, and for some other
    specialized purposes. Not for general application
    use

13
Application Environment
  • Two major high level application environments
  • Console environment
  • VAX based
  • C language ( legacy FORTRAN)
  • Port to Linux in progress
  • Expect some operational utilization in the fall
  • Java environment
  • Platform independent
  • Windows/Sun/Linux/Mac all supported and utilized
  • Full access to entire control system
  • Both environments have a standard application
    framework that provides a common look and feel
  • All applications launched from Index Pages
  • Currently separate pages for Console, Java
    environments
  • Also some browser applications of varying styles

14
Index Page (Console)
15
Application Index (Java)
Web Browser
Standalone Application
16
Application Overview
  • Utility applications
  • Parameter Page
  • Fast Time Plot
  • Data Logger viewer
  • Database Viewer
  • Synoptic displays
  • Diagnostics
  • Save/Restore
  • Alarms
  • Machine Specific Applications
  • 600 Primary 125 Secondary applications in
    console environment
  • gt100 applications in Java environment
  • Mostly utility, shot data analysis

17
Parameter Page (Console)
18
Parameter Page (Java)
19
Fast Time Plot (Java)
20
Synoptic Display (Console)
  • Graphical BuilderLex Draw

21
Synoptic Builder (Java)
22
Synoptic Display (Web)
23
Sequencer (Console)
  • Automatically Sequence Collider Shots, other
    activities
  • Relies on ACL scripting language

24
Finite State Machine (Java)
  • Graphical Builder
  • Primary current usage is Pelletron control
  • Keeps track of HV sparks and beam related trips

25
Sequenced Data Acquisition
  • Automated acquisition of data for collider shots
  • Configure readings for various stages of collider
    shot
  • Snapshots/FTP as well as single scalar readings
  • SDAEdit tool makes it easy to add new things
  • Data acquired automatically and stored in Sybase
    database
  • Summary html reports generated when store goes in
  • Step efficiencies, error reports, luminosity
  • Supertable captures 200 most popular items
  • Html, Excel, Java Analysis Studio formats
    automatically generated
  • Suite of tools to view detailed data
  • Data can be directly read by Excel, Java Analysis
    Studio, potentially other external tools
  • Simple Java API for user written code

26
Alarms Display
27
Scripting
  • Accelerator Control Language (ACL)
  • Console environment
  • Access to much of the functionality in the
    control system
  • VMS DCL style syntax
  • Extensively used in Sequencer
  • Jython
  • Java API in principle accessible to Jython
  • Some but limited testing/use to date

28
External Tools
  • Excel
  • Excel based parameter page
  • Easy cut/paste of Java tables into Excel
  • Excel data can be imported into a variety of
    tools
  • MathCAD, OptimA used locally
  • MATLAB
  • MATLAB supposedly has a standard Java interface,
    in theory it should be straightforward to
    integrate with the control system
  • However we havent tried it
  • An attempt to do this with Mathematica gt 1 year
    ago was not successful, it does not use a
    standard Java distribution
  • Java Analysis Studio
  • Currently data logger and shot data plug-ins

29
Excel Parameter Page
30
Java Analysis Studio
31
Save/Restore
  • Devices may be assigned to save/restore lists
  • Console applications to save/restore to/from
    database on user request
  • Scheduled saves of the entire control system 4
    times/day

32
Electronic Log
  • Based on Oak Ridge product
  • Web browser client
  • Perl/cgi back-end
  • Shift log machine / subsystem logs
  • Used by A0
  • API to add entries programmatically, including
    images
  • Both Console and Java environments
  • Java application framework includes standard menu
    item to make posts.

33
Central Layer
  • Open Access Clients (virtual front-ends)
  • Java Servlets
  • Bridge requests from outside firewall
  • Request Consolidation
  • Alarms server
  • Front-end download
  • Other things

34
Open Access Clients
  • Processes with no user interface, always running
  • Obey same communication protocol as front-ends
  • Compare to EPICS Soft IOC
  • Several classes
  • Utility Data loggers, scheduled data
    acquisition, virtual devices
  • Calculations Both database driven and custom
  • Process control Autotune of fixed target lines
  • Bridges to ethernet connected scopes,
    instrumentation, control systems
  • Easy access to devices on multiple front-ends
  • Friendlier programming environment than VxWorks
    front-ends
  • Framework transparently handles ACNET
    communication
  • Access to database, other high level operating
    system features
  • Do not provide hard real-time response
  • Clock events via ethernet multicast
  • gt 100 Almost all run in Java framework VAX
    framework deprecated

35
Data Loggers
  • 70 instances running in parallel on central
    nodes
  • 80 GB compressed storage per node in MySQL
    database
  • Circular buffer wraparound time depends on
    device count/rates
  • Data at maximum 1 Hz rate extracted to Backup
    logger every 24 hours
  • Backup logger is available online gt data
    accessible forever
  • Generally allocated to various departments to
    configure as they wish
  • No central management of archiving lists, so
    inevitable duplication
  • Tasks follow the Open Access Client architecture
  • Simple GUI to specify logging lists
  • Single readings or snapshots on clock events or
    at fixed rates to 15 Hz
  • Programmatic logging interface available for
    custom loggers
  • Variety of retrieval tools available
  • Standard console applications, Console and Java
    environments
  • Web applications
  • Java Analysis Studio
  • programmatic APIs

36
Front-Ends
  • Interface field hardware to the high level
    control system
  • Several architectures
  • MOOC Minimally Object-Oriented Communication
  • Very wide diversity of functionality
  • Processors all VME (VXI) based
  • Very wide diversity of field hardware supported
  • Links to many other form factors such as CAMAC,
    etc.
  • MVME 162 (68k) and 2400 (Power PC)
  • VxWorks Operating System
  • 275 in the system

37
Front-Ends - IRM
  • IRM Internet Rack Monitor
  • MVME 162 PSOS operating system
  • Chassis includes 64 channel x1 KHz 16 bit
    continuous digitizer
  • 8 x 10-20 MHz option available
  • 8 bytes external digital I/O
  • 8 x 12 bit DAC outputs
  • Expandable via IndustryPak I/O
  • 125 in the system
  • Eight used at A0 photo-injectorVariety of
    functions
  • A few at TTF for modulators

38
Front-Ends
  • Linac front-ends
  • VME based Power PC processors
  • Non-MOOC infrastructure tied to 15 Hz Linac
    cycle rate
  • 50 in the system
  • HRM Hotlink Rack Monitor
  • Evolution of IRM architecture
  • I/O chassis separate from processor chassis
  • 64 x 10 KHz ADC up to 2 per chassis
  • 10-20 MHz snapshot board
  • Timestamps included with ADC data
  • 8 byte digital I/O per chassis
  • Multiple chassis per PPC processor
  • VxWorks (not MOOC)
  • Initial build of 15 units in progress

39
Labview
  • Front-Ends
  • General control/acquistion from high level
    controlsystem
  • Interact via ACNETprotocol written inLabview VI
    language
  • No FTP/Snap
  • Non-local consoleaccess viaPCAnywhere,Timbuktu
    etc.
  • 25 in system
  • Instrumentation
  • Flying Wires
  • Sample Bunch Display
  • Both PCs Macs

40
Field Hardware
  • Wide variety of hardware supported
  • CAMAC, VME, VXI, Multibus, CIA,
  • Too extensive to describe here
  • Support for wide variety of commercial
    instrumentation
  • Scopes, spectrum analyzers, signal generators,
    DVMs,
  • Ethernet-gtGPIB direct ethernet
  • Through front-ends through OACs direct in some
    cases
  • Mostly in console environment, ports to Java as
    needed
  • Siemens-Moore APACS PLC supervisor
  • Used for cryogenic control
  • Interface through Open Access Client using
    library from company
  • Other PLCs also supported through front-ends

41
Bridge to External Systems
  • Accelerator NETwork control system (AccelNET) is
    a software package designed for control of
    electrostatic particle accelerator systems.
  • Runs on PC hardware under the Linux operating
    system
  • Modular Linux programs that communicate to each
    other through a central database server using
    TCP/IP.
  • CAMAC- GROUP3 I/O
  • ACNET??AccelNET Bridge

42
Timing and Control Networks
  • TCLK (Clock Events)
  • 10 MHz Manchester encoded signal with 8 2
    bits/event
  • 256 possible events defined for the complex
  • Decoding hardware/signal generators for variety
    of buses
  • Standalone version of this system used at A0
    photo-injector
  • MDAT (Machine Data)
  • 10 Mbit/sec serial link 8 bit type 16 bits
    data per frame
  • Originally developed to rapidly distribute ramp
    currents around the main ring and Tevatron usage
    has broadened
  • Beam Sync revolution markers derived from RF
  • States
  • State devices defined
  • Any node may issue transition request, gets
    multicast everywhere
  • Can be distributed via MDAT
  • Beam permit/abort
  • CAMAC interfaces.

43
Security
  • Control system network inside a firewall
  • Restricted access from/to outside the accelerator
    division network
  • Privilege classes assigned based on account,
    service, (node)
  • Control from where applications may be run
  • Control which programs/users/nodes may do
    settings
  • Emphasis on accountability as well as security
  • Setting log
  • Application log

44
Remote Access
  • Console environment via remote X display
  • Administrative controls on usage, particularly
    settings etc.
  • Java applications can be launched from anywhere
    from a browser via Web Start
  • Access to inner parts of control system only via
    VPN
  • Applications to be run remotely w/o VPN must be
    designed for that
  • Applications that work via servlets
  • Secure Controls Framework Subset of data
    access API implemented in this way, better
    performance than servlets/xml
  • Access from outside the standard application
    environments
  • xml-rpc is preferred method
  • Readings of any device setting of virtual
    devices
  • No setting of hardware associated devices
  • Used by all current experiments

45
ACNET Summary
  • Should meet requirements for SMTF
  • Has been/is being continually modernized
  • Extensive experience at FNAL.
  • Less time to get a system running
  • A broad range of people could contribute
    quickly
  • Limited experience outside of Fermilab
  • Remotely accessible but full system not portable
  • Documentation
About PowerShow.com