Oxford University Particle Physics Unix Overview - PowerPoint PPT Presentation

Loading...

PPT – Oxford University Particle Physics Unix Overview PowerPoint presentation | free to view - id: 75c932-MmVkN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Oxford University Particle Physics Unix Overview

Description:

Oxford University Particle Physics Unix Overview Pete Gronbech Senior Systems Manager and GridPP Project Manager 11th October 2012 Graduate Lectures * – PowerPoint PPT presentation

Number of Views:86
Avg rating:3.0/5.0
Slides: 22
Provided by: Physic171
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Oxford University Particle Physics Unix Overview


1
Oxford University Particle Physics Unix Overview
  • Pete Gronbech
  • Senior Systems Manager and
  • GridPP Project Manager

2
  • Strategy
  • Local Cluster Overview
  • Connecting to it
  • Grid Cluster
  • Computer Rooms
  • How to get help

3
Particle Physics Strategy The Server / Desktop
Divide
Virtual Machine Host
Servers
General Purpose Unix Server
Linux Worker nodes
Group DAQ Systems
Web Server
Linux File Servers
NIS Server
torque Server
Win XP PC
Win XP PC
Win 7 PC
Win 7 PC
Linux Desktop
Desktops
Approx 200 Desktop PCs with Exceed, putty or
ssh/X windows used to access PP Linux systems
4
Particle Physics Linux
  • Unix Team (Room 661)
  • Pete Gronbech - Senior Systems Manager and GridPP
    Project Manager
  • Ewan MacMahon Grid and Local Systems
    Administrator
  • Kashif Mohammad Grid Support
  • Sean Brisbane Local Server and User Support
  • Aim to provide general purpose Linux based
    systems for code development and testing and
    other Linux based applications.
  • Interactive login servers and batch queues are
    provided.
  • Systems run Scientific Linux which is a free Red
    Hat Enterprise based distribution.
  • Systems are currently running SL5, this is the
    same version as used on the Grid and at CERN.
    Students should use pplxint5 and 6.
  • A Grid User Interface is provided on the
    interactive nodes to allow job submission to the
    Grid.
  • Worker nodes form a PBS (aka torque) cluster
    accessed via batch queues.

5
Current Clusters
  • Particle Physics Local Batch cluster
  • Oxfords Tier 2 Grid cluster

6
PP Linux Batch Farm
Scientific Linux 5
pplxwn32
16 AMD Opteron 6128 cores
pplxwn31
16 AMD Opteron 6128 cores
Users log in to the interactive nodes Pplxint5
6, the home directories and all the data disks
(/home area or /data/group ) are shared across
the cluster and visible on the interactive
machines and all the batch system worker nodes.
Approximately 350 Cores each with 4GB or RAM
memory.
pplxwn30
16 AMD Opteron 6128 cores
pplxwn29
16 AMD Opteron 6128 cores
pplxwn28
16 AMD Opteron 6128 cores
pplxwn27
16 AMD Opteron 6128 cores
pplxwn26
16 AMD Opteron 6128 cores
pplxwn25
16 AMD Opteron 6128 cores
pplxwnnn
8 Intel 5420 cores
pplxwnnn
8 Intel 5420 cores
pplxwnnn
8 Intel 5420 cores
pplxwnnn
8 Intel 5420 cores
pplxwnnn
8 Intel 5420 cores
pplxwn5
8 Intel 5420 cores
pplxwn4
8 Intel 5420 cores
pplxint5
Interactive login nodes
pplxint6
7
PP Linux Batch Farm
Data Storage
NFS is used to export data to the smaller
experimental groups, where the partition size is
less than the total size of a server.
NFS Servers
9TB
The data areas are too big to be backed up. The
servers have dual redundant PSUs, RAID 6 and are
running on uninterruptable powers supplies. This
safeguards against hardware failures, but does
not help if you delete files.
Data Areas
pplxfsn
19TB
Data Areas
The home areas are backed up to by two different
systems nightly. The OUCS HFS service and a local
back up system. If you delete a file tell us a
soon as you can when you deleted it and its full
name. The latest nightly backup of any lost or
deleted files from your home directory is
available at the read-only location
"/data/homebackup/username The home areas are
quotad but if you require more space ask us.
Store your thesis on /home NOT /data.
pplxfsn
30TB
Data Areas
pplxfsn
19TB
Home areas
pplxfsn
8
Particle Physics Computing
The Lustre file system is used to group multiple
file servers together to provide extremely large
continuous file spaces. This is used for the
Atlas and LHCb groups.
df -h /data/atlas Filesystem Size
Used Avail Use Mounted on /lustre/atlas

183T 147T 27T 85 /data/atlas df -h
/data/lhcb Filesystem
Size Used
Avail Use Mounted on /lustre/lhcb

58T 40T 16T 72 /data/lhcb
9
(No Transcript)
10
Strong Passwords etc
  • Use a strong password not open to dictionary
    attack!
  • fred123 No good
  • Uaspnotda!09 Much better
  • Better to use ssh with a passphrased key stored
    on your desktop.

11
Connecting with PuTTY
  • Demo
  • Plain ssh terminal connection
  • With key and Pageant
  • ssh with X windows tunnelled to passive exceed
  • ssh, X windows tunnel, passive exceed, KDE
    Session
  • http//www.physics.ox.ac.uk/it/unix/particle/XTunn
    el20via20ssh.htm
  • http//www.howtoforge.com/ssh_key_based_logins_put
    ty

12
(No Transcript)
13
Puttygen to create an ssh key on Windows
Paste this into /.ssh/authorized_keys on
pplxint If you are likely to then hop to other
nodes add ForwardAgent yes to a file called
config in the .ssh dir on pplxint Save the
public and private parts of the key to a
subdirectory of your h drive
14
Pageant
  • Run Pageant once after login to load your
    (windows ssh key)

15
SouthGrid Member Institutions
  • Oxford
  • RAL PPD
  • Cambridge
  • Birmingham
  • Bristol
  • Sussex
  • JET at Culham

16
Current capacity
  • Compute Servers
  • Twin and twin squared nodes
  • 1300 CPU cores
  • Storage
  • Total of 700TB
  • The servers have between 12 and 36 disks, the
    more recent ones are 2TB capacity each. These use
    hardware RAID and UPS to provide resilience.

17
Get a Grid Certificate
Must remember to use the same web browser to
request and retrieve the Grid Certificate. Once
you have it in your browser you can export it to
the Linux Cluster to run grid jobs. Details of
these steps and how to request membership of the
SouthGrid VO (if you do not belong to an existing
group such as ATLAS, LHCb) are here http//www.gr
idpp.ac.uk/southgrid/VO/instructions.html
18
Two New Computer Rooms provide excellent
infrastructure for the future
The New Computer room built at Begbroke Science
Park jointly for the Oxford Super Computer and
the Physics department, provides space for 55
(11KW) computer racks. 22 of which will be for
Physics. Up to a third of these can be used for
the Tier 2 centre. This 1.5M project was funded
by SRIF and a contribution of 200K from Oxford
Physics. The room was ready in December 2007.
Oxford Tier 2 Grid cluster was moved there during
spring 2008. All new Physics High Performance
Clusters will be installed here.
19
Local Oxford DWB Physics Infrastructure Computer
Room
Completely separate from the Begbroke Science
park a computer room with 100KW cooling and
gt200KW power has been built. 150K Oxford
Physics money. Local Physics department
Infrastructure computer room. Completed September
2007. This allowed local computer rooms to be
refurbished as offices again and racks that were
in unsuitable locations to be re housed.
20
Cold aisle containment
21
The end for now
  • Ewan will give more details of use of the
    clusters next week
  • Help Pages
  • http//www.physics.ox.ac.uk/it/unix/default.htm
  • http//www2.physics.ox.ac.uk/research/particle-phy
    sics/particle-physics-computer-support
  • Email
  • pp_unix_admin_at_physics.ox.ac.uk
  • Questions.
  • Network Topology

22
Network
  • Gigabit JANET connection to campus July 2005.
  • Second JANET gigabit connection Sept 2007.
  • JANET campus connection upgraded to dual 10
    gigabit links August 2009
  • Gigabit Juniper firewall manages internal and
    external Physics networks.
  • 10Gb/s network links installed between Tier-2 and
    Tier-3 clusters in 2012.
  • Physics-wide wireless network. Installed in DWB
    public rooms, Martin Wood, AOPP and Theory. New
    firewall provides routing and security for this
    network.

23
Network Access
Super Janet 4
2 10Gb/s with Super Janet 5
Physics Firewall
Physics Backbone Router
1Gb/s
OUCS Firewall
1Gb/s
10Gb/s
Backbone Edge Router
10Gb/s
100Mb/s
Campus Backbone Router
1Gb/s
10Gb/s
depts
Backbone Edge Router
depts
100Mb/s
depts
100Mb/s
depts
24
Physics Backbone
Linux Server
10Gb/s
Server switch
10Gb/s
Linux Server
1Gb/s
Linux Server
1Gb/s
Physics Firewall
Server switch
1Gb/s
Win 2k Server
1Gb/s
1Gb/s
Particle Physics
1Gb/s
100Mb/s
Physics Backbone Router
100Mb/s
1Gb/s
desktop
Clarendon Lab
100Mb/s
1Gb/s
desktop
1Gb/s
1Gb/s
100Mb/s
Astro
Atmos
Theory
25
Future Physics Backbone
Linux Server
10Gb/s
Server Switch S4810
Frodo
10Gb/s
Super FRODO
Linux Server
10Gb/s
1Gb/s
Server switch
10Gb/s
Physics Firewall
1Gb/s
Win 2k Server
1Gb/s
1Gb/s
Particle Physics Dell 8024F
10Gb/s
1Gb/s
Physics Backbone Switch Dell 8024F
1Gb/s
10Gb/s
desktop
Clarendon Lab Dell 8024F
1Gb/s
10Gb/s
desktop
10Gb/s
10Gb/s
1Gb/s
Astro Dell 8024F
Atmos Dell 8024F
Theory Dell 8024F
About PowerShow.com