Best Ever Archive Utility, Yet - PowerPoint PPT Presentation

1 / 19
About This Presentation
Title:

Best Ever Archive Utility, Yet

Description:

Compile-time defaults usually in 'preferences.ini' and 'plugin_customization.ini' ... Reset statistics ..../restart: Stop, re-read configuration, start again ... – PowerPoint PPT presentation

Number of Views:67
Avg rating:3.0/5.0
Slides: 20
Provided by: xnv
Category:

less

Transcript and Presenter's Notes

Title: Best Ever Archive Utility, Yet


1
Best Ever Archive Utility, Yet
  • Creating a BEAST was easy.
  • BEAUtY seems harder.
  • kasemirk_at_ornl.gov
  • April 2009

2
Previous Setup
IOC
ASCII Config.
Channel Access
Archive Engine
config.xml
Binary Data Files
Data Server
XML-RPC
  • History
  • 2000Started byBob Dalesio
  • 2003Index Tools,Data Server
  • 2007CSS Client

CSS-based OPI
3
Details
  • Older Java archive viewer also available.
  • Archive Engines running on sampling computer,
    data server on serving machine.
  • Scripts to restart engines, copy data, update
    indices.
  • Headaches with data maintenance
  • Index time grows with data
  • Stuck when index files reach 2GB
  • SNS Users faced with 80 sub-archives
  • No clue what needs fixing after network/power
    problems
  • No idea who contributes how many samples
  • No way to remove selected channels or time ranges
  • Improving on this means implementing an RDB

4
New Setup
IOC
Channel Access
ArchiveEngine
Samples
Config.
config.xml
RDB(Oracle/MySQL)
EngineConfig -Import
Other tools for config samples
CSS-based OPI
5
CSS Data Browser Handles Both
  • New URL
  • Just oneRDBsub-archive
  • Old and newdata can becombined inone plot

6
Configure Sample Engines
  • Configuration is in RDB
  • Directly use SQL
  • EngineConfigImport for legacy config files
  • View/Edit via web
  • Hierarchical (as before)
  • Sampling engine (name, where to run, )
  • Groups
  • Channels
  • No more duplicate channels!

7
Web Interface to Engine Config in RDB
  • Tomcat/JSP/Servlets to view and edit

8
Web Config View Channel Stats
9
Web Config View Sample Stats
??
OK
10
Stats
  • 34 sample engines, 79400 channels
  • Host that runs sampling engine
  • CPU load 45, zero disk I/O wait, very responsive
  • Oracle performance
  • 8000 samples/sec peak in write tests
  • Operationally maybe ¼ of that
  • Better configurationwould likely havefewer
    samples/sec

11
Report Example, SNS Beam Instr. Group
  • May be SNS-specific, but shows usefulness of data
    in RDB

12
Summary
  • Still testing, many Oracle setup issues
  • Ran since Oct 08, then..
  • Oracle cluster setup since March
  • Updated partitioning in April
  • Plan for about a year now
  • Make operational in maybe 2 months
  • Thanks go to
  • Oracle Jeff Patton
  • JSP Katia Danilova
  • Beam Instr. Reports Mariano Padilla, Jeff,
    Douglas Edwardson
  • IT Greg Lawson

13
Stuff
14
Hurdles
  • Months Get new Oracle server configured
  • Interface cards for storage array, fiber switches
  • Firewall holes for office access, backup, admin
  • Changes in 10g
  • SELECT MIN(stamp), MAX(stamp) -gt NULL, NULL
  • No range partitioning on Index-Organized-Tables
  • Configuration issues
  • ORA-01654 unable to extend
  • ORA-00257 archiver error
  • What used to be impossible is now easy, but
    still expensive
  • DELETE FROM SAMPLES WHERE

15
Other Ideas
  • Jlabs MyA
  • Operational, but
  • Wrapper code around MySQL to create cluster
  • Handles less meta info (units, limits, )
  • Viewer is one-of C/TCL/Tk
  • Gabriele Carcassi mentioned RDDTool
  • Toolset for logging data with data-aging
  • Command-line RPN tools, web viewer
  • May be faster than general-purpose RDB
  • but lacks advantage of gpp RDB

16
SNS Setup for EngineConfigImport Tool
  • Imports existing XML config files into RDB
  • On srv02 in /imported_config
  • .xml files copied from archive1, includessome
    editing to remove duplicate channels
  • Review .log files
  • 'chmk/chmkhebt/PPS_HEBTPLC_BChmk100AlarmContactA
    already in 'pps/pps_hebt
  • Maybe edit your .xml file, refer to import.sh
    to re-import

17
Getting Sources, Building binaries
  • Get sources from DESY CVS repository
  • Maybe binaries real soon from ChannelArchiver
    web page
  • Build binaries
  • Find somebody who understands Eclipse RCP, how
    to use
  • org.csstudio.archive.rdb/EngineConfigImport.produc
    t
  • org.csstudio.archive.engine/ArchiveEngine.product
  • org.csstudio.sns.product/SNS_CSS.product (or DESY
    CSS.product)
  • Settings
  • Compile-time defaults usually in
    preferences.ini and plugin_customization.ini
    files.
  • Either change, or prepare file for
    -pluginCustomization command-line option

18
RDB Setup
  • Currently supported MySQL or Oracle
  • Main table difference Oracle TIMESTAMP has
    nanosecs, MySQL has separate nanosecs col.
  • All tools auto-configure based on JDBC URL
  • Table Schema
  • ERD, MySQL DBD in org.csstudio.archive.rdb/dbd
  • Setup for Oracle (table types, partitioning) not
    final

19
Run Archive Engine
  • Command-line tool similar to previous engine
  • More settings (RDB, Logging, JCA/CAJ, )
  • Compiled-in defaults from preferences.ini and
    plugin_customization.ini
  • Can specify overrides via
  • -pluginCustomization my_plugin_customization.ini
  • Mandatory command-line options for instance
  • engine_name to locate config in RDB
  • port for consistency, must match settings in RDB
  • Built-in web server for status, restart, stop
  • http//hostport/ Empty page (used to be main)
  • ./main Main page
  • ./reset Reset statistics
  • ./restart Stop, re-read configuration, start
    again
  • ./stop Stop
Write a Comment
User Comments (0)
About PowerShow.com