SLAC is focusing on the modeling and simulation of DOE accelerators using highperformance computing

1
Parallel 3D Finite Element Particle-In-Cell
Simulations with Pic3P Arno Candel, Andreas
Kabel, Zenghai Li, Cho Ng, Liequan Lee, Greg
Schussman, Kwok Ko Advanced Computations
Department, SLAC Ilan Ben-Zvi, Jorg Kewisch, BNL
Abstract SLAC's Advanced Computations Department
(ACD) has developed the parallel 3D Finite
Element electromagnetic Particle-In-Cell code
Pic3P. Designed for simulations of beam-cavity
interactions dominated by space charge effects,
Pic3P solves the complete set of Maxwell-Lorentz
equations self-consistently and includes
space-charge, retardation and boundary effects
from first principles. Higher-order Finite
Element methods with adaptive refinement on
conformal unstructured meshes lead to highly
efficient use of computational resources.
Massively parallel processing with dynamic load
balancing enables large-scale modeling of
photoinjectors with unprecedented accuracy,
aiding the design and operation of
next-generation accelerator facilities.
Applications include the LCLS RF gun and the BNL
polarized SRF gun.
Causal Moving Window
Overview

• SLAC is focusing on the modeling and simulation
  of DOE accelerators using high-performance
  computing
• The performance of high-brightness RF guns
  operating at the space-charge limit is affected
  by retardation and wakefield effects which are
  omitted in standard static tracking codes
• Conventional PIC codes based on Finite Difference
  methods have problems dealing with complex
  geometries, and are restricted to low
  particle-field coupling accuracy
• SLAC has developed the first parallel 3D PIC code
  Pic3P that uses higher-order Finite Element
  methods on conformal unstructured meshes for
  unprecedented modeling accuracy

21D spacetime diagram indicating the causal
field domain for an accelerated particle bunch
inside the gun at a snapshot in time. Fields
outside this causal domain can be neglected for
the gun simulations this leads to a significant
speedup.
LCLS RF Gun 1.6 cell, 2.856 GHz, 120 MV/m, 5.8
MeV, copper cathode, elliptical iris, high
cylindrical symmetry of operating mode with
dual-feed race track design, measured initial
distribution, Q1nC
Pic3P Emittance Calculations for the LCLS RF
gun and the BNL polarized SRF gun
Parallel 3D Finite Element PIC Code Pic3P
Dynamic Load Balancing

• Pic3P includes space charge, wakefield and
  retardation effects from first principles
• Pic3P is supported by SciDAC and designed for
  massively parallel operation on leadership-class
  supercomputers
• Pic3P solves the full set of Maxwell-Lorentz
  equations self-consistently in time domain with
  unconditional stability of the field solver in
  the time step

Measured initial distribution at the cathode and
laser profile (Courtesy LCLS Commissioning Team)
Realistic Particle Distributions

• Pic3P obtains higher-order particle-field
  coupling accuracy by using higher-order Finite
  Element vector basis functions Ni

LCLS RF Gun Pic3P 3D emittance results for a
cylindrical bunch agree perfectly with 2D results
obtained with Pic2P and MAFIA. PARMELA results
differ since retardation and wakefield effects
are neglected in the electrostatic model.
Particle phase space at exit of the LCLS RF gun
as calculated by Pic3P, starting from measured
initial particle distribution shown above.
Snapshot of causal moving window PIC simulation
of the LCLS RF gun. Colors indicate parallel
partitioning of fields and particles to
processors. Dynamic load balancing enables the
solution of large problems for unprecedented
modeling accuracy.
BNL polarized SRF Gun ½ cell, 350 MHz, 24.5 MV/m,
5 MeV, 7 Gauss solenoid, recessed GaAs cathode
(70 K) inserted via choke joint, cathode spot
size 6.5 mm, Q3.2 nC, initially ellipsoidal bunch

• The operating mode of a RF gun is calculated with
  ACDs parallel frequency-domain Finite Element
  code Omega3P to higher accuracy than machining
  tolerances and can be directly loaded into Pic3P
  for optimum field quality

Summary

• SLAC is focusing on the modeling and simulation
  of DOE accelerators using high-performance
  computing
• The Advanced Computations Department has
  developed the first parallel Finite Element PIC
  code Pic3P for self-consistent simulations of
  space-charge dominated beam-cavity interactions
• Pic3P has been extensively benchmarked and
  delivers state-of-the-art modeling accuracy for
  simulations of next-generation accelerator
  structures
• Pic3P has been applied to calculate emittance
  growth effects in the LCLS RF gun and the BNL SRF
  gun

Close-up of mesh in cathode region
Bunch distribution at gun exit, colored by energy
Left Unstructured mesh model of the LCLS RF
gun Right Drive fields and particles on symmetry
plane
Bunch transit through the BNL polarized SRF gun
as simulated with Pic3P, scattered self-fields
are shown
This work was supported by DOE Contract No.
DE-AC02-76SF00515 and used resources of NERSC
supported by DOE Contract No. DE-AC02-05CH11231,
and of NCCS supported by DOE Contract No.
DE-AC05-00OR22725.