?????????????? ?????????? ? ??????? ??????

1
?????????????? ?????????? ? ??????? ??????  

• ????????? ???????? ??????????

???????? ??????????? ?????????????? ?????????? ???? ???. ???????? ?????????????? ?????????????-?????????????? ??????? ???????????? ????? ???????????? ??????????? ???????? ?????????? ? ????????? ??????? ?????? ??? ??. ?.?. ????????? ?????? ? ??? ??. ?.?. ????????? 29 ???????? 2016 ????
2
???? ?????????? ???? ? ??????
?? ????????? ?? ?????? ????????? ???????????
?????? ?? ???????? ?????? ?????? ???????? ?????
????? ????? ????? ?????? ????-?????????? ????? ??
???? ??????? ?????? (?????? ? ??????????? LHC ?
??????????? ???????????).
??? ??????????? ????-?????????????? ?? LHC ????
?? ?????????? ???????????? ? ??????? ????????????
????? ??????, ??????????? ? ??????????, ? ??????,
????????? ??????? ????????. ??????? ??? ?? ????
??????? ?????? ?? ?????????? ??? ?????????????
?????????????? ?????????????? ??? ?????????
??????.
3
Grids, clouds, supercomputers..
Grids, clouds, supercomputers, etc.

• Supercomputers

• Grids
• Collaborative environment
• Distributed resources

• Volunteer computing
• Simple mechanism to access millions CPUs

Clouds
Ian Bird
3
4
Real World Problems Taking UsBEYOND PETASCALE
SUM Of Top500
1 ZFlops
1
100 EFlops
10 EFlops
1 EFlops
100 PFlops
10 PFlops
1 PFlops

• Example Real World Challenges
• Full modeling of an aircraft in all conditions
• Green airplanes
• Genetically tailored medicine
• Understand the origin of the universe
• Synthetic fuels everywhere
• Accurate extreme weather prediction

What we can just model today with lt100TF
100 TFlops
10 TFlops
1 TFlops
100 GFlops
10 GFlops
1 GFlops
100 MFlops
1993
2017
1999
2005
2011
2023
2029
5
Reach Exascale by 2018From GigFlops to ExaFlops
2018
2008
Sustained ExaFlop
1997
Sustained PetaFlop
1987
Sustained TeraFlop
Note Numbers are based on Linpack Benchmark.
Dates are approximate.
ustained GigaFlop
S
The pursuit of each milestone has led to
important breakthroughs in science and
engineering.
Source IDC In Pursuit of Petascale Computing
Initiatives Around the World, 2007
6

• TOP500 List June 2016

Rank Site System Cores Rmax (TFlop/s) Power (kW)
1 National Supercomputing Center in Wuxi China Sunway TaihuLight - Sunway MPP, Sunway SW26010 260C 1.45GHz, Sunway NRCPC 10,649,600 93,014.6 15,371
2 National Super Computer Center in Guangzhou China Tianhe-2 (MilkyWay-2) - TH-IVB-FEP Cluster, Intel Xeon E5-2692 12C 2.200GHz, TH Express-2, Intel Xeon Phi 31S1P NUDT 3,120,000 33,862.7 17,808
3 DOE/SC/Oak Ridge National Laboratory United States Titan - Cray XK7 , Opteron 6274 16C 2.200GHz, Cray Gemini interconnect, NVIDIA K20x Cray Inc. 560,640 17,590.0 8,209
4 DOE/NNSA/LLNL United States Sequoia - BlueGene/Q, Power BQC 16C 1.60 GHz, IBM 1,572,864 17,173.2 7,890
5 RIKEN Advanced Institute for Computational Science (AICS) Japan K computer, SPARC64 VIIIfx 2.0GHz, Tofu interconnect Fujitsu 705,024 10,510.0 12,660
6 DOE/SC/Argonne National Laboratory United States Mira - BlueGene/Q, Power BQC 16C 1.60GHz, Custom IBM 786,432 8,586.6 3,945
7 DOE/NNSA/LANL/SNLUnited States Trinity - Cray XC40, Xeon E5-2698v3 16C 2.3GHz, Aries interconnect Cray Inc. 301,056 8,100.9
8 Swiss National Supercomputing Centre (CSCS) Switzerland Piz Daint - Cray XC30, Xeon E5-2670 8C 2.600GHz, Aries interconnect , NVIDIA K20x Cray Inc. 115,984 6,271.0 2,325
9 HLRS - Stuttgart Germany Hazel Hen - Cray XC40, Xeon E5-2680v3 12C 2.5GHz, Aries interconnect Cray Inc. 185,088 5,640.2
10 King Abdullah University of Science and TechnologySaudi Arabia Shaheen II - Cray XC40, Xeon E5-2698v3 16C 2.3GHz, Aries interconnect Cray Inc. 196,608 5,537.0 2,834
41 Moscow State University Russia Lomonosov 2 - T-Platform A-Class Cluster, Xeon E5-2697v3 14C 2.6GHz, Nvidia K40m T-Platforms 42,688 2,102.0 1,079
7
????????? ????

• ???? - ??? ???????, ??????? 
• ???????????? ????????????? ???????? ???
  ??????????
• ????????????????? ?????????? ????? ?????????
• ?????????? ???????????, ????????, ?????????????
  
• ????????? ? ??????????.
• ???????????? ?????????????????? ????????????
• (Ian Foster "What is the grid? ", 2002 ?.)

• ?????? ????
• Distributed Computing
• High-Throughput Computing
• On-Demand Computing
• Data-Intensive Computing
• Collaborative Computing
• ????????????????? ???????? ???? ???????????
  ?????????? ??????????? ? ?????? ??????? ???????,
  ???????????, ?????????????, ????????,
  ????????????, ????????? ?????????? ? ??????
  ???????????.
• ??????????? ??????????? (VO)

8
???? - ??? ???????? ??? ??????????? ?????????????
?????????????? ????????? ? ???????? ??????
??????????? ?????????
9
Some history

• 1999 Monarc Project
• Early discussions on how to organise distributed
  computing for LHC
• 2001-2003 - EU DataGrid project
• middleware testbed for an operational grid
• 2002-2005 LHC Computing Grid LCG
• deploying the results of DataGrid to provide a
• production facility for LHC experiments
• 2004-2006 EU EGEE project phase 1
• starts from the LCG grid
• shared production infrastructure
• expanding to other communities and sciences
• 2006-2008 EU EGEE-II
• Building on phase 1
• Expanding applications and communities
• 2008-2010 EU EGEE-III
• 2010-2014 - EGI-InSPIRE

10
????????????? ???????? ?????????????
Origins of the Terms Both first commonly used
in the context of IT in the early 1990s Grid
metaphor for distributed computing in science and
engineering Cloud metaphor for a computer
network in the context of network management
What they have come to mean Grid Computing
virtual organizations share compute resources
over a network Cloud Computing anybody can
access data and compute services over the
internet
11
?????? ????????????
12
??????????? ??? ? ???????
13
Some Desktop Grids

• World Community Grid - IBM (http//www.worldcomm
  unitygrid.org/) 300.000 PCs
• Leiden Classical Grid - Education on Grid
  (http//boinc.gorlaeus.net/) 17.000 PCs
• SZTAKI - Hungarian initiative(http//www.desktopg
  rid.hu/) 70.000 PCs
• AlmereGrid (http//almeregrid.nl) 3.000 PCs
• PS3GRID (Based on Playstations)
• (http//ps3grid.net/)

14
WHAT DESKTOP GRIDS ARE

• Use free CPU cycles
• Volunteer computing
• Use master-slave model of distributed computation
• Major technologies
• BOINC
• XTREMWEB-HEP
• OURGRID
• CONDOR
• X-COM
• SARD

BOINC Berkeley Open Infrastructure for Network
Computing

• BOINC is an open source middleware system for
  deploying a desktop grid based on volunteer
  computing.
• Originally created from the SETI_at_home project
• Now has over 22 full scale production projects
• Projects are based on different scientific
  disciplines

15
??? ????? ??????? ??????

• ??????? ??????????? ??????? ?????? ??, ???
  ??????? ?????? ? ??????, ????? ?? ????? ????
  ?????????? ?????????, ???????? ? ????????????????
  ???????????? ?????????? ????????? ??? ?????? ?
  ???? ????????? ?????? ????????, ???????? ?
  ?????????.
• ????? ??????, ??????? ????????? ? ?????? ??
  ??????????????. ??? ? 2001 ???? ???? ????? ?????
  ? ???????? ???????????? ????????????? ??? ???????
  ?????? ??? ?????????? ??? V
• ????? (volume, ? ?????? ???????? ???????????
  ??????),
• ???????? (velocity ? ??????? ??? ????????
  ????????, ??? ? ????????????? ????????????????
  ????????? ? ????????? ???????????),
• ???????????? (variety, ? ?????? ???????????
  ????????????? ????????? ????????? ?????
  ????????????????? ? ??????????????????? ??????
• ??????, ????? ????? ????? ?????? ??????
  ???????????????, ?????????? ?????? ???, ?? ????
  ????????????? ??????????????? ?????????, ? 2014
  ???? ???? ??? "3V" ?????? ?????????? ?????????,
  ???????? ????? ? ????? V,
• Validity (??????????????, ????????????),
• Veracity (?????????????),
• Value (????????, ??????????),
• Visibility (???????????, ??????????? ?
  ????????????) ? ?. ?.

16
??????? ?????? ??????? ????, ?????? ???
?????????? ??????? ????????? ?????? ????
????????? ??????
?????????? ? ????? ?????? ?????? ????????? ?
????????????? ??? ????????? ??????, ? ??
???????????? ???? ?????????????????,
???????????????????, ?????????
Social Media
Website
Billing
ERP
Network Switches
CRM
RFID
17
?????? ?????? ?? ??????? ??????
18
??? ????? ????????? ??????? ???????  
??????????? ?????????? ?????? ????????? ?????????
? ?????? ????????? ?????????? ??????, ? ?????????
??????? ???? ??????, ?????????? ???? ??? ?????
???????, ? ????????? ??????? ??????????? ??????
?????? ????????? ? ??????? ??????, ?????????????
???????????? ??? ?????????? ?????? ??????????.
?? ???? ???? ??????? ???????????? ? ???????
????????? ??????? ?????? ??????????? ??????
??????? ? ???????????????? ?????? (??????????????
??????????, ?????? ????????? ?????, ??????????
??????, ?????????????? ??????, ?????????????
??????, ????????? ????, ???????????? ?????????,
???????? ?????????, ????????????? ??????? ? ??.).
? ????????? ???????? ? ??????? ?????? ? ???????,
???????????, ????? ??? ?????? ??????
??????????????? ?? ???? ???? ?? ?????????? ?????
????????????????? ???????.
19
?????????? ? ??? Big Data
We are BIG!
????????????? ????????? ?????????????? ??????
???????? ??????????, ??? ???????????? ? ?????
???? ? ???????? ??????? ??????.
20
Large Hadron Collider and Detectors
pp, B-Physics, CP Violation (matter-antimatter
symmetry)
General Purpose,proton-proton, heavy
ions Discovery of new physics Higgs,
SuperSymmetry
Exploration of a new energy frontier in p-p and
Pb-Pb collisions
LHC ring 27 km circumference
Heavy ions, pp (state of matter of early universe)
21
The ATLAS Experiment at the LHC
3000 scientists 174 Universities and Labs From 38
countries More than 1200 students
??????? ?????? ???? 14 TeV ??????? ????????????
?????? 40 MHz ??? 7000 ????, ??????? 25 ?,
????? 44 ? ?????????? ?????????????? ??????? 140
000 000

• ATLAS has 44 meters long and 25 meters in
  diameter, weighs about 7,000 tons. It is about
  half as big as the Notre Dame Cathedral in Paris
  and weighs the same as the Eiffel Tower or a
  hundred 747 jets

22
Data Collection and Archiving at CERN
Data flow to permanent storage 4-6 GB/sec
200-400 MB/sec
1-2 GB/sec
4 GB/sec
1-2 GB/sec
23
?????? ??????????? ?? LHC

• Tier-0 (CERN)
• ????? ??????
• ????????? ????????????? ??????
• ????????????? ??????
• Tier-1 (11 centres)
• ?????????? ???????? ??????
• ????????????? ? ?????????
• ??????
• Tier-2 (gt200 centres)
• ?????????????
• ?????????? ??????

24
The Worldwide LHC Computing Grid
WLCG An International collaboration to
distribute and analyse LHC data Integrates
computer centres worldwide that provide computing
and storage resource into a single infrastructure
accessible by all LHC physicists
25
CERN Computer Centre

• CERN ?omputer Centre
• Built in the 70s on the CERN site (Geneva)
• 3000 m2 (3 main machine rooms)
• 3.5 MW for equipment
• New extension
• Located at Wigner (Budapest)
• 1000 m2
• 2.7 MW for equipment
• Connected to the Geneva CC with 2x100Gb links

26
???????? ?????? ???????????

• ?????????? ???????????? ???????? ?? ????
  ????????????? ??????? ???????????? ???????? (HLT,
  Clouds, HPC)
• ????????? ?????? ??????????? ? ??????
  ????????????, ? ????? ??????????? ?????????????
  ????????
• ???????????? ?????? ???????????? ? ????????
  ???????????? ???????????, ????? ???????? ?????
  ?????????????????? ??? ????????????? ???????????
  ?????????? (??????????????, GPU)
• ??????????? ????????? ?????????, ??????????
  ?????????? ?????? ?????? ? ??.

27
Country Normalized CPU time 2015-2016
All Country - 45,196,250,152 Russia-
2,140,123,800 Job 1,196,456,588
45,334,211
27
28
Russian Data Intensive Grid infrastructure
(RDIG)

• The Russian consortium RDIG (Russian Data
  Intensive Grid), was set up in September 2003 as
  a national federation in the EGEE project.
• Now the RDIG infrastructure comprises 10
  Resource Centers with
• gt 12000 CPU and gt 10000 TB of disc storage.

• RDIG Resource Centres
• ITEP
• JINR-LCG2 (Dubna)
• RRC-KI
• RU-Moscow-KIAM
• RU-Phys-SPbSU
• RU-Protvino-IHEP
• RU-SPbSU
• Ru-Troitsk-INR
• ru-IMPB-LCG2
• ru-Moscow-FIAN
• ru-Moscow-MEPHI
• ru-PNPI-LCG2 (Gatchina)
• ru-Moscow-SINP
• - Kharkov-KIPT (UA)
• BY-NCPHEP (Minsk)
• UA-KNU
• - UA-BITP

29
???????? ??????????? ???????????? ???

• ???????? ??-??????????????
• ??????????????, ??????????????? ? ???????????
  ????????? ????????????, ?????????? ? ???? ?
  ???????-??????????
• ????????????? ?????????????? ??????? ????

• 
  
  ??? ????????
  ??????
  ???????, ?????????, ??????????

?????????????? ?????????????, ??????, ????????? ?
?????????
????? ??????????? 324 ?? 18 ????? ????, 30
???????? ????, 65-????????? ????, ? 2015 ????
???????????? 270 ?????? ? ??????????????
????????, 2 ??????????, 112 ?????? ?
?????? ???????????
30
RO-LCG 2015, Cluj-Napoca, 28 - 30 Oct. 2015
31
Network and Telecommunication at JINR
JINR Local Area Network Comprises 8146 computers
nodes Users 4379, IP 13436 Remote VPN users
780 E-library- 1475, mail.jinr.ru-2400 High-spee
d transport (10 Gb/s)
Controlled-access at network entrance. General
network authorization system involves basic
services (Kerberos,AFS, batch systems, JINR LAN
remote access, etc.) IPDB database - registration
and the authorization of the network elements and
users, visualization of statistics of the network
traffic flow, etc.
The upgrade of the JINR-Moscow telecommunication
channel for data transfer up to 100 Gb/s speed
has been approved
32
?????????? ????
33
??????????? ????? ????????? Tier2 ????
1. ??????????? ?????????????? ?????? ???????? ?
????????? ??????. 2. ?????? ??????
?????????????-?????????????? ?????? ???
????????????? ???????? ?????????????? ??????????
? ?????????? ??????????? ??????????? ? ?????????
?????????????. 3. ????????? ?? ???? ????????????
??????????? ????????????? ???????? ? ?????????
???????? ????????????? ? ?????????? ???????
??????? ? ?????????? ??????.

• ??????? ? ???????? ??????????
• 2600 ?????????????? ?????,
• ???????? ????????? 2 PB.
• ? 2011-2016 ????? ????????? ????? 32 ?????????
  ????? ?????????????,??????? ???? ?????????????
  940 ???. ??????????????? ?????.
• ?????????? ? ??????????? 99

34
????? ?????? Tier1 ? ???? ??? ???????????? CMS
?? LHC  
??????? ???????????? ? ????????? 3500
?????????????? ????? 3.5 ?? ???????? ?????????
5.4 ?? ????????? ?????????? 100 ??????????? ?
??????????, 70 ?? ?????????? ??????? ??????
Managing, Computational and disk servers
TIER1 TIER1 CPU time
CH-CERN CH-CERN 49,792,481
DE-KIT DE-KIT 26,191,924
ES-PIC ES-PIC 23,738,814
FR-CCIN2P3 FR-CCIN2P3 37,987,697
IT-INFN-CNAF IT-INFN-CNAF 48,838,655
RU-JINR-T1 RU-JINR-T1 50,947,347
UK-T1-RAL UK-T1-RAL 18,621,006
US-FNAL-CMS US-FNAL-CMS 207,293,643
Total 463,411,872 463,411,872
Tape library
System module of uninterrupted power
35
?????????? ? ?????????? ???????????? ????
??? ????????? ???????????????? ?????????
?????????? ??????????? ????????? ???? ????? ?
???????? ?? ??????? ????????????????? ??
???????????????? ????????? ??????????,
????????????? ?? ????????? ? ???????? ??????? ?
???????? ?????? ??? ???????????? ?????????????.
UPS
36
???????? ?????????????? ????
IP Baku, AZ
BITP Kiev, UA
JINR cloud ????????????? ? ?????????
???????????????? ???????????-?????????
PRUE Moscow,RU
EGI Federated Cloud
????????? OpenNebula, ?????????? ???????????
?????? ??? ?????? ???????? (NICA, PANDA, NOVA,
BESIII), ? ????? ?????????????? ???????
??????????????
37
?????????? ???????? ?????????????
JINR cloud Platform OpenNebula CPU cores
200 RAM 400 GB Storage 16 TB
PRUE cloud Platform OpenNebula CPU cores
48 RAM 128 GB Storage 14 TB
IP cloud Platform OpenNebula CPU cores 24 RAM
256 GB Storage 16 TB
BITP cloud Platform OpenStack CPU cores
32 RAM 32 GB Storage 8 TB

• JINR cloud integration with clouds of partner
  organizations
• Institute of Physics of Azerbaijan National
  Academy of Sciences IP (Baku, Azerbaijan)
• Bogolyubov Institute for Theoretical Physics of
  the National Academy of Sciences of Ukraine
  BITP (Kiev, Ukraine)
• Plekhanov Russian University of Economics PRUE
  (Moscow)
• EGI Federated cloud

38
???????????? ?????????????? ?????????? ????

• ????????? ??????????? ??????????? ??? ??????????
  ???????????? ????????? ???????????? ????????,
• ???????????? ?????????????? ??? ?????????
  ?????????? ?????? ?????????? ???????? ?
  ?????????????? ?????????.
• ????????????? ???????????? ????????? ???????
  ????????? ????? ? ?????????????? ?????????
  ???????????? ??????????.

• ???????? ??????? ????????
• 224 CPU-cores,
• 57216 CUDA-cores
• 182 PHI-cores
• 1920 GB RAM
• 60 TB HDD.
• ????? ??????????????????
• 140 TFlops ??? ???????? ? ????????? ?????????
• 50 TFlops ??? ???????? ? ??????? ?????????.

? ?????????? ??????? ?????????????? ????????
???????? 120 ???????, ? ??? ????? 26 ??
?????-???????? ???? ? 19 ?? ??????????
?????????????.
39
Parallel computing on HybriLIT
Parallel computing for QCD problems  F.
Burger(IP, HU, Berlin, ), M. Müller-Preussker
(IP HU, Berlin, Germany), E.-M. Ilgenfritz
(BLTP VBLHEP, JINR),  A. M. Trunin (BLTP JINR) 
http//theor.jinr.ru/diastp/summer14/program.html
posters
Parallel computing for investigation of
Bose-systems Alexej I. Streltsov (Many-Body
Theory of Bosons group at CQD, Heidelberg
University, Germany), Oksana I. Streltsova (LIT
JINR)
http//MCTDHB.org
Parallel computing for Technical problems  A.
Ayriyan (LIT JINR), J. Busa Jr. (TU of Kosice,
Slovakia), E.E. Donets (VBLHEP, JINR), H.
Grigorian (LIT JINR,Yerevan State University,
Armenia),   J. Pribis  (LIT JINR TU of Kosice,
Slovakia) 
arXiv1408.5853
40
In 2015 year was presented MCTDHB-Lab package
solver of the many-boson Time-Dependent
Schrödinger Equation. This solver can be used for
theoretical investigations of the
highly-non-equilibrium quantum dynamics realized
in trapped systems of ultra-cold atoms and
molecules. MCTDHB-Lab is available as a
FREE-for-download, cross-platform software with a
mouse-click interface
http//QDlab.org
Heterogeneous Computations HybriLIT - team
Many-body theory of bosons group Heidelberg,
Germany
41
HybriLIT tutorials
Tutorials on the basis of HybriLIT Regular
tutorials on parallel programming technologies
both for the institute staff and for students and
young scientists from JINR member-states
organized by the UC Specialized courses from
the leading software developers.
Specialized courses and seminars within
conferences and schools organized by JINR. In
Dubna University, Sofia university, National
University of Mongolia, Technical University of
Moldova. In particular within GRID2014,
MPAMCS2014, The Helmholtz International Summer
School 2014 MMCP2015, NEC2015, AIS-GRID2015,
MCE2016, May 2016, Master Class on Parallel
Programming at University of Presov. Planned
GRID-2016.
All information (lectures, materials) of past and
upcoming events can be found on
http//indico-hybrilit.jinr.ru/
42
Worldwide LHC Computing Grid Project (WLCG)

• ???????? ??????? ??????? WLCG ???????? ????????
  ?????????? ?????????????? ???????????? ???????
  ??? ?????????, ???????? ? ??????? ??????
  ?????????? ????????????? LHC.
• ????-?????????? ???????? ??????? ?????????? ????
  ??????????????.
• ???????? ????? ????, ??????? ? ???? ?? ??????? ?
  ??????? LCG ??? ???????? ? 2003 ????.
• MoU ?? ??????? ? ??????? WLCG ??? ???????? ?
  2007 ????.
• ?????? ?????????? ??????? ? ???? ? ??????? WLCG ?
  2016 ????
• ???????? ??????? ?????????? ?????????
  ??????????????? ??????????? ??????
• ????????? ???????? WLCG ??? ?????????????
• ???????? ?????? ??????????? WLCG
• ???????? ??????? ????????????? ???
  ?????????????
• ???????? ??????????? Tier1 ?????? ? ??????

43
??????? ? ??????? WLCG

• ???????? software ??? ????????????? ATLAS,
  ALICE, CMS
• ???????? WLCG Dashboard
• Global data transfer monitoring system for WLCG
  infrastructure
• ?????? NOSQL ????????
• ?????????? GRID, Cloud, HPC
• ?????????, ?????????????, ???????? ???????????
  Tier3 ???????
• ???????? DDM, AGIS for ATLAS
• GENSER MCDB
• ???????? ?????? Tier1 ? ??????

44
WLCG Google Earth Dashboard
45
??????????? ??????? ??????????????? ????????
?????? ???????????? ATLAS

• ?????????? ????? ??????????? ???????
  ???????? ?????? ??? ??????????? ???????????
  ??????????????? ???????? ?????????? ????????????
  ATLAS.
• ??????????? ??????? ?? ????????,
• ?????????? ???????????? ????????,
• ???????????? ???????????????? ?
  ?????????????????? ???????,
• ?????????? ????????? ??????????, ??????????? ?
  ???????? ??????,
• ????????? ??????? ???????? ? ?????? ?????????????
  ???????.
• ???????????
• ????? ????????? ????? ???????????? ???????
• ??????? ????? ????? ???? ??????,
• ??????????? ???? ???????,
• ??????? ?????????? ? ????????? ?????????
  ????????,
• ??????????????? ??????? ??????????? ??????
  ???????.

????????? ?????? ???????????? ???????????
???????? ?????????? ? ?????????????
?????????????? ?????. ?????? ???????????? ATLAS
???????????? ?????, ??? ?? 130 ????-?????? ?
????? ??????? ????????? ???????????? ????? 120
????????, ? ??????? ???????? ????? ?????????
??????. ????????? ????? ????????? ??????
?????????? 2 ?? (20 000 000 ??????).
46
??????? ??????????? Tier3-??????? ??? ???????
?????? ????????????? ???
??? ???????????? ??????? ?????? Tier3 ? ???? ????
??????????? ??????????? ???????? ??????????????,
??????? ????????? ????????? ????????? ?????????
???????????? ??????? ?????? Tier3. ? ???????????
?????????? ????????????? ????? ??????????????
???? ??????????? ?? ???? ??????????? ?????????,
??? ????????? ??????????? ??? ??????? ????????
???????????? ????????????, ????????? ???
??????????? ???????? ?????????? ???????????,
?????????? ?????? ???????????? ?? ??????? ?????
?????????? ??? ??????????? ??????????? ???????
?????? Tier3.
?????????? ??????? ??????????? Tier3 ???????
(??-????) ????? ???????? ???????? ??? ???????????
????? ? ?????? ??????????? ???????????, ??? ???
?????????????? ?????????? ?????? ?? ????? Tier3
?????? ? ?????????????? ???????.
Poster reports at the Conference Computing in
High Energy and Nuclear Physics (CHEP) 2012
21-25 May 2012 New York City, NY, USA
47
?????????? ??????? ??????????? ???????? ?????? ?
?????????????? WLCG

• ???? ??????? ??????? ? ????????
  
  ????????????? ??????? ???????????,
  ?????????
  ???????? ??????????
• ? ?????? ???????? ??????
  
  (????? 1 ????????? ? ????),
• ?????????? ?? ?????? ?????????????
  
  ???????? (????????? ?????????? ?
  
  ???????? ???????? ??????,
  FTS, xROOTd),
• ??????? ?????????? ?????? (Tier-0,Tier-1,Tier-2,Ti
  er-3)
• ?????????????? ?????? ???????????? ???????????
  ??????????? 
• ?????????? ? ??????? ???????? ??????????
  ????????? ?????????? ? ??????????? ?????????
  (ORACLE, HADOOP)
• ???????????? ????????? ?????? ??? ??????????????
  ????????? ???????????? ? ??????????? ??????????
  ??? ????????? ??????.
• ??????? ????????? ?????????
  ????????????? ??????????? ? ??????????
  ????????? ????? ?????????????  ? ???????????????
  ?????????????? WLCG.

48
ATLAS FAX Infrastructure (From Rob Gardner)
Provides a global namespace Unifies dCache, DPM,
Lustre/GPFS, Xrootd storage backends Xrootd an
efficient protocol for WAN access Main Fall-back
use case in production at many sites Regional
redirection network provides lookup scalability
Today uses xrootd protocol soon will also use
http
49
DIRAC

• DIRAC has all the necessary components to build
  ad-hoc grid infrastructures interconnecting
  computing resources of different types, allowing
  interoperability and simplifying interfaces.
• This allows to speak about the DIRAC interware.

50
BES-III Distributed Computing

• What have been done in computing
• Grid monitoring system developed from scratch
• JINR cloud was integrated in BES-III
  infrastructure
• 6 of all jobs was done in JINR during the past
  year
• Planning to continue participate in BES-III
  experiment by
• Improving monitoring
• Research on clouds in grid
• Providing storage and CPU cores

51
NICA Complex New era in the hot dense matter
science

• Data transfer rate 4.7 GB/s
• 19 billions of events per year 30 PB
• 8.4 PB data to store per year after processing
  and analysis

Cryogenic Complex
52
NICA Project Management Information
System (PMIS)
53
Simulation of NICA-MPD-SPD Tier0-Tier1 computing
acilities

• Working at TB scale the NICA MPD-SPD experiments
  will face with great challenges in distributed
  computing
• large increase of CPU and network resources
• combined grid and cloud access
• Intelligent dynamic data placement
• distributed parallel computing
• renewal most of simulation and analysis software
  codes.
• The program SyMSim (Synthesis of Monitoring and
  SIMulation) for simulation of grid-cloud
  structures is developed.
• The originality consists in combining a
  simulation program with a real monitoring system
  of the grid/cloud service in frame of the same
  program.

Data storage and processing scheme of Tier0-Tier1
level
Estimated rate of NICA-MPD experimental data
from Tier0 to be transferred to Tier 1 is about
24 PB by one month. Simulation result shows what
happened in the grid/cloud system if the data
volumes are grow up to 1,5 times for example.
Simulation result allows one to understand how
the intensity of the input stream determines the
reserves of the system capacity
Number of DAQ data files stored on output disk
buffer for growing data volumes
54
CBM_at_GSI Methods, Algorithms Software for Fast
Event Reconstruction

• Tasks
• - global track reconstruction
• - event reconstruction in RICH
• - electron identification in TRD
• - clustering in MVD, STS and MUCH
• - participation in FLES (First Level
• Event Selection)
• - development of the Concept of
• CBM Databases
• - magnetic field calculations
• - beam time data analysis of the
• RICH and TRD prototypes
• - contribution to the CBMROOT
• 
  development
• - D0-, vector mesons, J/??ee- and
• J/??µµ- reconstruction

AuAu_at_25 AGeV
J/??ee-
a S/Bg2s, b Efficiency (), c J/? per hour
(10 Mhz)
a b c
pC_at_30GeV 14 22 11
pAu_at_30GeV 18 22 27
AuAu_at_10AGeV 0.18 18 64
AuAu_at_25AGeV 7.5 13.5 5250
STS

• Modern parallelization involves multiplicative
  effects coming from
• Vectorization (SIMD - Single Instruction
  Multiple Data) factor 2 to 4
• 2) Multithreading factor 4/3 3) ? -Many core
  processor factor ?. Total 4 ?

RICH
TRD
STS CA STS Kalman Filter RICH ring reconstruct. TRD track reconstruct. TRD el. id. ?(k,n) criterion KFPar -ticle
164.5 0.5 49.0 1390 0.5 2.5
Average time per core (µs/track or µs/ring) of
SIMD-algorithms (besides track reconstruction in
the TRD) for data processing. Global throughput
increases linearly with the number of cores.
55

• PanDa ? ???????????? ATLAS

• ? ???????????? ATLAS ?? ??????? ????????
  ?????????? ??????????? ????????? ??? ??????????
  ??????????????? ????????? PanDA Workload
  Management System (WMS), ??????? ????????
  ?????????? ?????????????
• ?????? PanDA ??????? ? 2005 ???? ????????
  BNL ? UTA - Production and Data Analysis
  system.
• ?????????????????? ? ?????? ??????? ??????????
  ?????????, ??????? ????? ?????????? ???????
  ?????????????? ??????? ?????????? ????????????.
• ? ??????? PanDA, ?????? ????? ??????
  ?????????????? ??????, ??????? ???????????? ???
  ????????? ?????? ????????????, ???? ????
  ????-?????? ?????????? ?? ????? ????
• PanDa ????????? ??????? ?? ???????????
  ???????????, ?????????? ? ??????????????
  ???????????? ??????????? ? ?????? ???????????????
  ??????????, ????????? ? ????????????????? ???? ?
  ????????????.
• ?????????????? ?????? ????????????? ?????????????
  ? ???????????. ????? ??????????? ????????? ??????
  ???????

• ? ????????? ????? PanDa ????????????
• ????? ???? - ??????? ? 50 ??????? ????
• ????? ????? ?????????????? ?????
• ????? ????????? ??????? ? ???
• ?????? ?????????????

56
PanDA Workload Management
PanDA server
Data Management System (DQ2)
Production managers
LocalReplicaCatalog (LFC)
https
production job
LocalReplicaCatalog (LFC)
LocalReplicaCatalog (LFC)
https
https
submitter(bamboo)
LoggingSystem
define
pull
task/jobrepository (Production DB)
https
https
NDGF
https
analysis job
pilot
https
OSG
job
pilot
arc
submit
ARC Interface (aCT)
EGEE/EGI
pilot
condor-g
pilot scheduler (autopyfactory)
End-user
Worker Nodes
57
?????????? ?????????? ??????????????
?????????? ??? ?????????? ???????? ???????

• PanDa, ??? ?????????, ?????????????? ????????????
  ???????? ????????, ????????? ? ?????????? ???????
  ??? ?????????? ? ???????? ???????? ?????? ?
  ?????????? ????????????.
• ???????? PanDA ? ??????????? ??????????
  ????????? ?????? ?????????????? ? ????????????
  ?????????? (????, cloud, ????????, ???,
  ???????????????) ? ????? ???????? ?????????????
  ????????? ??? ??????? ???????? ??????????
  ???????? ??????? ? ?????, ???????????????
  ??????????, ????????, ?????????????????????
  ??????????????, ???????.

??????????????? ?15, 2013, ???.56
58
?????? ????????????? ??????? PanDA
(????????????? Titan ? Oak Ridge)

• ???????????? ??????????????? Titan ? ?????
  ?????????? ????????? ? ??????? ????????? PanDA

59
BigPanDA. Generalizing PanDA. Beyond Grid and
ATLAS
OLCF

• ??????? ? ?????????????????? ???????? ??? ??,
  ????, ??????? (?????)
• ?????? ??????? ?????? ? ??? ??, ??? ????, ??? ??.
  ?????????

60
??????? ? ??????? ?????????????? ??????????

• Worldwide LHC Computing Grid (WLCG)
• EGI-InSPIRE
• RDIG Development
• Project BNL, ANL, UTA Next Generation Workload
  Management System for BigData
• Tier1 Center in Russia (NRC KI, LIT JINR)
  
• 6 Projects at CERN
• CERN-RFBR project Global data transfer
  monitoring system for WLCG infrastructure
• BMBF grant Development of the grid-infrastructure
  and tools to provide joint investigations
  performed with participation of JINR and German
  research centers
• Development of grid segment for the LHC
  experiments with South Africa
• Development of grid segment at Cairo University
  and its integration to the JINR GridEdu
• JINR - FZU AS Czech Republic Project The grid
  for the physics experiments
• NASU-RFBR project Development and implementation
  of cloud computing technologies on grid-sites at
  LIT JINR and BITP for ALICE experiment
• JINR-Romania cooperation Hulubei-Meshcheryakov
  programme
• JINR-Moldova cooperation (MD-GRID, RENAM)
• JINR-Mongolia cooperation (Mongol-Grid)
• JINR-China cooperation (BES-III)
• Cooperation with Belarus, Slovakia, Poland,
  Bulgaria, Kazakhstan, Armenia, Georgia,
  Azerbaijan

61
LIT traditional conferences
The International Conference Mathematical
Modeling and Computational Physics, 2015
Distributed Computing and Grid-technologies in
Science and Education
LIT schools
Mathematics. Computing. Education
62
??????????? ???????? ?????????? ? ?????????
??????? ?????? ??? ??. ?.?. ?????????

• ???????? ???????????
• ????????????? ???????????? ? ???????? ?????????
  ??????? ?????? ??? ??????? ???????? ????? ?????,
  ? ??? ????? ? ??????? ????????? ?
  ???????????????? ??????????
• ???????? ?????????????? ???????? ?????????? ???
  ??????? ????????????????? ? ???????????????
  ???????? ???
• ??????-???????????? ?????? ? ???????
  ?????????????? ?????????? ? ????????? ???????
  ??????
• ??????? ????????????? ??????????????, ???????
  ?????????? ???????, ???????????, ????????????
  ????? ? ?.?.
• ?????????? ???????????? ? ??????? ?????????? ?
  ?????????? ????????
• ????????? ? ?????? ???????????
• ???????????, ????????? ? ???????? ???,
  ??????????? ??? ???? ? ???????????? ????????? ??
  ?????? ??????????? ????????? ?????.

63
????? ????? ?????? ?????????? Big Data
64
?????????????-??????????????? ???? ???????
????????? ???????? ??????? ?? ?????? ???????? Big
Data
????????? ??????????
?????????? ????
??????????, ??????????????? ??????????
??????????? ??????????
??????
???-?????
NoSQL DB (MongoDB, Hbase, Neo4j, Cassandra)
???? ?
????????
?????????????? ?????????? ?????? (Flume,
Elasticsearch)
???????
SQL DB (PostgreSQL,MySQL)
?????????? ?????? (HDFS, lizardfs, dCache, DPM)
????????? ????????? (Avro, Sqoop)
?????????? ??????? ?????
????????
???????????


?????????
?????????? ??????? ????? (Dirac, Panda)
????????????? ?????? (MapReduce, Hive, Pig)
??????????????? ??????? ????? (MapReduce, Hive,
Pig)
???????????????? ?????? (Data Mining, Mahaut)
????????


??????
???????????? (Table au, Microstrategy)
????????????? ???????????
?????????? ?????????? ???????????
???????? ???????
?????????
????????? ? ???????????????
????????????? ???????????
????????????? ???????
?????? ????????????
???????? ???????
65
?????????? ?????????????????? ??????? ?? ???????
?????? ??????????? ???????????? ????? ????? ?
??????????? ?????
???????? ?????????, ?????????? ????????????????
???????? ?????????? ? ??? ??. ?.?. ????????? ??
????? ????? (?? ?????? 2016 ????)
?????????? ????? ?????? ???????????? ??????????
????????????? ????? ?? ??????? ?? ?????? ????????
???????? ? ???????? ?????????? ? ?????? ?
???????????? ? ???????? ??????? ???????? ????,
????????? ?????????? ? ??????????? ???????, ?
??? ????? ?????????? ? ???????????, ??????? ?????
??????? ??????????? ? ???? ??????, ? ???????
??????????? ????? ?????? ???????????.
?????????? ????, ?????????? ????? ?????
???????????????? ????????????? ? ????????????
????? ?????, ???????????? ? ???? ?????? ???????
???????? ?????????, ?????????? ?????????????
???????????????? ??????????? ??? ? ?????????
??????? ??