Hansang Bae, Steve Clark, Gerhard Klimeck, Sunhee Lee, Maxim Naumov, Faisal Saied,

1
Large Scale Simulations of Nanoelectronic devices
with NEMO3-D on the Teragrid
Network for Computational Nanotechnology (NCN)

• Hansang Bae, Steve Clark, Gerhard Klimeck, Sunhee
  Lee, Maxim Naumov, Faisal Saied,
• Gerhard Klimeck
• Technical Director
• Network for Computational Nanotechnology
• Teragrid Conference
• June 5, 2007

Berkeley, Univ. of Florida, Univ.of Illinois,
Norfolk State, Northwestern, Purdue, Stanford,
UTEP
2
NEMO 3-D Technical Approach
DesignedOptical Transitions Sensors
Quantum Dot Arrays Computing

• Problem
• Nanoscale device simulation requirements
• Cannot use bulk / jellium descriptions, need
  description of the material atom by atom gt use
  pseudo-potential or local orbitals
• Consider finite extend, not infinitely periodic
  gt local orbital approach
• Need to include about one million atoms. gt need
  massively parallel computers
• The design space is huge choice of materials,
  compositions, doping, size, shape.gt need a
  design tool

• Approach
• Use local orbital description for individual
  atoms in arbitrary crystal / bonding
  configuration
• Use s, p, and d orbitals.
• Use genetic algorithm to determine material
  parameter fitting
• Compute mechanical strain in the system.
• Develop efficient parallel algorithms to generate
  eigenvalues/vectors of very large matrices
• Develop prototype for a graphical user interface
  based nanoelectronic modeling tool (NEMO-3D)

Realistic material description at the atomic
level enables simulation of realistic
nanoelectronic devices.
3
Parallel Eigenvalue Solver on a Cluster

• Result / Demonstrations / Impact
• 52 million atom strain - volume (101nm)3 matrix
  of order O(1x109)
• Studied
• Alloy disorder in QD
• Long range strain in self-assembled QDs
• Valley splitting in Si
• Valley splitting in Si/Ge
• P impurities in Si

• Divide Simulation domain into slices.
• Limiting operation sparse matrix-vector
  multiplication
• Communication only from one slice to the next
  (nearest neighbor)
• Communication overhead across the surfaces of the
  slices.
• Future 2D and 3D domain decomposition

4
Scaling on a variety of platforms

• Relatively small system 2 million atoms
• Clusters with 1GB ethernet are competitive with
  high-end PSC/XT3

5
16 million atoms - Scaling up to 512 CPUs

• Clusters have enough memory can store the
  Hamiltonian gt speed
• Blue Gene small memory need to recompute matrix
  on the fly
• PSC/XT3 best for NEMO3D

6
NEMO 3-D towards Peta-Scale ComputingBenchmarks
under constant computational load

• Blue Gene scales well down to 64 atoms/core

• Cray XT3 scales well down to 4,096 atoms/core

• NEMO 3-D will scale to 10,000 cores without
  problem
• With Block-Lanczos, multiple vectors we can
  project running to 100,000 cores
• gt Enable nano-science and nano-engineering runs
  in tens of minutes rather than days!

7
How to Treat Disordered Systems?

• In and Ga atoms are randomly placed
• Real alloys are spatially disordered
• Different samples look differentgt statistical
  properties

• Typical approach VCA
• average over In and Ga
• create an average atom InGa

8
What do interfaces REALLY mean now?

• In/Ga atoms are randomly placed
• Interface is rugged in 3-D
• gt Confinement changes!
• Material not homogeneousgt fluctuations in
  material!
• VCA ignores all that!

9
Inhomogeneous Broadening due to Alloy Disorder

• Problem
• Cations are randomly distributed in alloy.
• Does alloy disorder limit electronic structure
  uniformity for dot ensembles?
• Requires atomistic simulation tool.
• Approach
• Simulate a statistical ensemble of dots.
• Identical in size and shape
• Different only in cation ordering.

In0.6Ga0.4As Lens Shaped Dot
Diameter30nm,Height5nm, GaAs embedded
1,000,000 Atom Simulation, sp3s basis
In and Ga atoms are randomly distributed
Inhomogeneous Broadening?
10
Inhomogeneous Broadening due to Alloy
Disordergt1,000 quantum dot ensemble

• Problem
• Cations are randomly distributed in alloy.
• Does alloy disorder limit electronic structure
  uniformity for dot ensembles?
• Requires atomistic simulation tool.
• Approach
• Simulate a statistical ensemble of dots.
• Identical in size and shape
• Different only in cation ordering.

Simulated G1 - 5meV No size or
shape fluctuations
Measured G34.6 meV (R. Leon, PRB, 58, R4262)
In0.6Ga0.4As Lens Shaped Dot
Diameter30nm,Height5nm, GaAs embedded
1,000,000 Atom Simulation, sp3s basis
In and Ga atoms are randomly distributed
Inhomogeneous Broadening?
11
AlGaAs Bulk AlloyVirtual Crystal Approximation -
VCA

• How big does a system have to be such that the
  average is good enough?
• Is all the physics captured?
• Does the disorder matter?
• Can we do a super-cell calculation that includes
  the disorder?

12
Alloy Disorder

• VCA gap is too large!
• Fluctuations in placement of Al atomsgt
  configuration noise
• Fluctuations in number of Al atomsgt
  concentration noise
• How important are these fluctuations?
• How do they depend on system size?

13
Disorder in Bulk BandstructureHow Large is Large
Enough?
Al0.2Ga0.8As

• Dependence on system size may be non-monotonic!
• Bulk starts in this point of view for gt100,000
  atoms (12.5nm)3 !
• Beware Most bandstructures are computed with
  lt100 atoms!

14
Network for Computational NanotechnologyNCN -
nanoHUB

• Overcoming barriers to quantum mechanics
  simulations in physics, chemistry, biology, and
  materials to migrate nano-science to
  nano-technology.

NCN Develop Deploy Methods, Tools and
Training atomistic gt mesoscopic gt systems
15
An NCN ExampleNEMO 3-D - Nanoelectronic Modeling
NCN Develop Deploy Methods, Tools and
Training atomistic gt mesoscopic gt systems
16
nanoHUB has a global following!
50 in the US
17
online simulation
and more
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
nanowireUsage Statistics
Released May 19, 2006 544 Users 5,093 Simulations
25
The software and your support have proven to be
effective tools in stimulating the interest of
these students in these nanoelectronics topics
and in enabling them to investigate and learn
about the new physics that these materials and
devices entail. In the case of the doctoral
student, his work utilizing the software has
already resulted in a conference paper presented
at the IEEE Nano 06 Conference in July 2006 and
submission of a journal paper with other papers
likely to follow.
Kenneth P. Roenker Prof. of Electrical
and Computer Engr. University of Cincinnati
nanowire
MOSFET
FETToy
CNTbands
26
By comparing the experimentally obtained DOS with
the simulated ones which I obtain using the
CNTbands tool, I have been able to determine the
chirality of the CNTs I have been probing
experimentally. I indeed use the CNTbands tool
very frequently and I am considered to be among
the top users of such a tool an indication of
how important it is in my research.
Noureddine Tayebi Department of Electrical and
Computer Engineering Beckman Institute for
Advanced Science and Technology University of
Illinois, Urbana-Champaign
An EXPERIMENTALIST !!!!
CNTbands
27
nanoHUB in publications 151 citationsnanoHUB
alongside commercial apps
TCAD simulations using SCHRED 15 or ISE, .,
were used to support our analysis and compute the
inversion carrier profiles in the devices.
Effect of channel positioning on the 1/ f noise
in silicon-on-insulator metal-oxide-semiconductor
M von Haartman, M Oestling - Journal of Applied
Physics, 2007 - link.aip.org?...
Google scholar nanohub Schred finds 30
citations
28
Annual nanoHUB Usage is Exploding
Simulation users with at least one simulation gt
5,579 users gt 211,501 simulations (May.
2007) Interactive simulations introducedin
April 2005 gt 5x increase
Total users Simulation users an IP with gt15
minute session time gt 23,433 users gt 3 hrs avg
session time / user Interactive presentations
introducedin August 2003 gt 23x increase
29
Cyberinfrastructure
30
Web-enabling Tools
31

• System Architecture

Maxwells Daemon
ContentDatabase
0101
1011
1001
nanowire job
nanowire job
nanowire job
nanoHUB cluster
Rendering Farm
Violin
32
Rappture Toolkit
Rappture

• Created by NCN in Nov 2004
• Open Source
• Create standard desktop apps
• Works with your favorite
• programming language

33

• Over 50 tools online!
• 50 more in the pipeline

34
Evolution of Scientific Computing
110011011101001011100
110011011101001011100
portals
gt22,523 total users
(E H)G I
35
nanoHUB in 2012
110011011101001011100