DC and AC High Dynamic Range pixels

1
DC and AC High Dynamic Range pixels
Intertech Pira Image Sensor Conference London
19-20 March 2008
Bart Dierickx1, Nick Witvrouwen1, Benoit Dupont1,2
1Caeleste CVBA, Antwerp, Belgium
www.caeleste.be contact bart_at_caeleste.be
2Institut dElectronique Fondamentale,
Université Paris Sud Orsay, France
2
About Caeleste

• Founded dec 2006
• 4 partners are ex-FillFactory (founders)
• Technology scope
• Hybrid and monolithic CMOS image sensors
• Smart pixels in-pixel analog and digital
  intelligence
• Extreme performance dynamic range, time
  resolution, temperature, radiation tolerance
• Market scope
• Scientific Metrology
• Space applications
• Molecular imaging

3
Contents

• Why do we need a high dynamic range sensor?
• Definitions of DR
• Obtain a high DC dynamic range by non-linearity
• AC high dynamic range pixel
• Conclusions

4
Outline

• Why do we need a high dynamic range sensor?
• Definitions of DR
• Obtain a high DC dynamic range by non-linearity
• AC high dynamic range pixel
• Conclusions

5
A high dynamic range scene
René Magritte, l Empire des lumières 1954
Ceci nest pas une High dynamic range scene
6
Capture the whole scene, and then try to recover
detail and contrast over the full scene dynamic
range
7
Why do we need a wide dynamic range sensor?

• DC
• To catch highlights
• To allow us to be lazy and not adjust camera
  speed to the scene
• To discriminate objects in any part (dark/bright)
  of the scene / picture
• ?Catch the whole scene / range

8
AC high dynamic range
Not capture the whole range (DC) But
capture the time varying small signal of interest
in the presence of a large DC background
9
Why would we need a wide dynamic range sensor?

• AC
• To extract AC information only from a scene
• To recover weak AC information buried in a large
  DC background
• Narrow band exchange noise noise bandwidth
• For specific purposes
• Distance ranging
• time of flight method
• Time gating
• making the sensor sensitive during precise times
  spans
• Patterned light 3D imaging

10
Outline

• Why do we need a high dynamic range sensor?
• Definitions of DR
• Obtain a high DC dynamic range by non-linearity
• AC high dynamic range pixel
• Conclusions

11
Dynamic Range definition?

• DRwikipedia
• Applies to the scene, not to the sensor

Wikipedia Dynamic range is a term used
frequently in numerous fields to describe the
ratio between the smallest and largest possible
values of a changeable quantity, such as in sound
and light.

• Our changeable quantity is P, light
• W, W/m2, photons, lux
• signal, S, is the measurement result
• V, ADC bits

AC or DC
12
Natural scenesmay have a huge dynamic range
Moon surface
Sun illuminant
Sun surface
Full Moon illuminant
Natural scenes
100000lux
1lux
light power W/m2
1e-4
1e-5
1e-1
1e-2
1e-3
1
10
100
1e4
1000
1e7
1e6
1e5
Sunlit snow 95
Outdoor night objects
indoor objects
surface
Sunlit average object (albedo) 18
indoor shadow objects
office light illuminant ()
Sunlit charcoal 3
Sunlit window illuminant ()
Outdoor shadow objects
13
linear dynamic range definition

• Linear response sensor
• S/N or SNR Dynamic Range?
• typical Between 1000 1 60 dB Smax1V,
  N1mVRMS
• extreme high end 100001 80 dB Smax2V,
  N200µVRMS

dynamic range definition no.1 DR is in light
power domain what Smax/Nmin is in voltage
(signal) domain
14
Image sensor detection chain
lamp surface (emittance)
Illuminant (illuminance)
object reflection (emittance)
scene
15
Image sensor detection chain
lamp surface (emittance)
Dynamic range of the Scene
Illuminant (illuminance)
Signal ______ Noise
object reflection (emittance)
scene
16
DR S/N ?
Saturation
Linear response (slope responsivity)
log output signal V or noise VRMS
signal
Noise
Log Illumination (lux, or other)
Dark Noise (Read noise, kTC, FPN,DSNU,DCSN)
Photon Shot Noise Limited
PRNU
Pmax
NEPmin
17
How to push DR beyond Smax/Nmin
In a linear, DC coupled, system, Dynamic Range is
very closely related to Signal/Noise

• Ways out
• Non-linear response
• AC signal detection ?AC dynamic range

18
Maybe there are S/N definitions suitable for
non-linear systems?

• Signal, S, Vsignal V
• The maximum signal Smax, Vmax-Vdark
• At the working point, referred to Dark S,
  V-VDark
• Linearized around working point
• Take into account ?V/?Illumination or PR
• The time dependent or filtered AC part of the
  signal
• Noise, N, Vnoise VRMS
• At dark Nmin
• Where S/N1
• At the working point N

19
Constant N.E.Contrast - linear
Log Signal S, Noise N V, VRMS
Signal
NECSNR
Photon shot noise
Read noise
Log Illumination P W, W.s, W/m2, lux, photons
20
In a non-linear system SNR?NEC
Signal saturates
Log Signal S, Noise N V, VRMS
SNR is finite?
Signal
Total noise / photo response?
Total noise
Read noise
Log Illumination P W, W.s, W/m2, lux, photons
21
Noise Equivalent Contrast - linear
general
Log Signal S, Noise N V, VRMS
Signal
Hypothetical noise for a constant noise
equivalent contrast
Log Illumination P W, W.s, W/m2, lux, photons
22
DC Dynamic Range definitions

• Further attempts for definition
• The range of light intensity levels that can be
  captured by the image sensor within a single
  frame
• The range of illumination levels on a similar
  object within the same frame, for which the
  object is recognizable (decent contrast, after
  image processing)
• The range of intensities that can be captured,
  for which the SNR has at least a certain value
• The range of intensities that can be captured for
  which the Noise Equivalent Contrast (NEC) has at
  least a certain value

DRSNRmax DRSNR10 DRNEC10
23
Summary of definitions for DC dynamic range

24
Outline

• Why do we need a high dynamic range sensor?
• Definitions of DR
• Obtain a high DC dynamic range by non-linearity
• AC high dynamic range pixel
• Conclusions

25
Obtain high DC high dynamic range by
non-linearity

• non-linear response
• a way to increase the sensors capability to
  capture a wide dynamic range scene
• a way to exploit the fact that the noise level
  depends on the scene contents

26
We do not need a high DRWe need a high NEC

• Goals
• make a pixel that can capture a high DC dynamic
  range means actually
• reach a constant or minimal NEC over the largest
  possible dynamic rangewikipedia definition
• Assumptions
• NEC is needed to allow recovery of details in all
  parts (dark, bright) of the scene
• Unproven underlying hypothesis the largest
  range is obtained when NEC is just large enough,
  i.e. constant

27
In search for a high DR

• Exercise of thought
• Obtain the constant NEC by exploiting non-linear
  response
• Increase DR by sacrificing NEC where it is
  sufficient
• Non-linear response is obtained by
• A non-linear transconductance, gain or Ceffective
• A non-linear integration time tint

28
(No Transcript)
29
Non-linear Vsignal(Q)
Vsignal
Qphoto
Ceff
Vsignal
Qphoto
slope tint(Q)/Ceff(Q)
Vsignal
Ceff
Q
Qphoto
30
NEC as function of Vsignal(Q)
SRSpectral Response A/W
31
NEC constant, excercise

• Postulate NEC constant
• Hence
• Will impose a relation for S(P), via tint(P) or
  Ceff(P)
• This relation depends on N or N(P)
• Note SVsignal, NVnoise

32
keep NEC constant by varying tint or Ceff during
integration
tint varies
Ceff varies
Nature of noise N
Constant chargeRMS
tint1/Q
CeffQ2
kTC noise
Constant voltageRMS
tint1/Q
CeffQ
EMI, read noise, ADC...
vpower
tint1/Q
No solution
PSN
vtint
tint1/Q2
No solution
DCSN
power
Always fulfilled
Always fulfilled
PRNU
tint
No solution
No solution
DSNU
Ref. B. Dierickx, Wide dynamic range the
pixels standpoint, ISSCC 2008
33
Interpretation
The relations tint1/Q and CeffQ found are
essentially logarithmic responses
Is a consequence of imposing a constant NEC
34
Outline

• Why do we need a high dynamic range sensor?
• Definitions of DR
• Obtain a high DC dynamic range by non-linearity
• AC high dynamic range pixel
• Conclusions

35
Examples of AC information

• Extract a modulated light source from a DC
  background
• E.g. recognize an IR remote control or an IR
  transmitter in a scene
• Artificial light source flicker detection
• Time of flight
• Ranging sample and time stamp the light
  returning from an illuminator
• Time gating
• Acquire light only during precise fractional time
  spans
• Acquire only light from a certain distance as
  reflected from a short illuminator pulse or
  signals at very precise moments or an accurate
  global shutter.

36
Extract AC from DC

• How to extract AC information from a huge dynamic
  range scene
• Brute force acquire multiple DC frames, and
  demodulate off line
• More subtle Subtract DC part from the signal,
  acquire the AC part only and demodulate off-line
  or in electrical domain
• Best demodulate in optical or charge domain and
  acquire that image

Sensor must handle full DR and many
frames Uncorrelated noise accumulates
Sensor must only handle AC Uncorrelated noise
accumulates
Sensor must only handle AC No uncorrelated noise
37
Image sensor detection chain
AC Signal
Demodulation of the output voltage
Demodulation of the photocharge
Demodulate the light
Subtract DC photo current
object reflection
scene
38
time gating pixeldemodulating in charge domain
39
Continuously tunable sensitive volume




p
n
n
n
n
Depletion layer
0
40
Continuously tunable sensitive volume
Patent pending




p
n
n
n
n
Depletion layer
Sensitive volume
0
41
Time gating
1 integration time
illumination
sensitivity
sensitivity
Q

time


Q


42
Preliminary device specs
Geometrical
Electro-optical

• Array 360x720 pixels
• 30 fps nominal
• Pixel pitch 20 µm
• Technology 0.35um CMOS
• Substrate backside thinned, 5E12/cm3 p-type
• Gating / switching speed ltlt1ns effective
• Gating on/off cycle gt100kHz

• Full well 50000 e-
• Read noise 20e- (below kTC) per pixel/frame
• QE gt 80 visible range
• PS(?) Parasitic sensitivity when gate is off lt 1
  in BSI
• DC/AC suppression factor minimum of PS and duty
  cycle.
• Shortest global shutter time lt 1 ns

43
Outline

• Why do we need a high dynamic range sensor?
• Definitions of DR
• Obtain a high DC dynamic range by non-linearity
• AC high dynamic range pixel
• Conclusions

44
Worth remembering

• High dynamic range is a property of the scene and
  light source.
• The sensor has to accommodate.
• DC When you want to acquire the scene
• One must apply some form of non-linear response
• ? have a sufficient NEC in all parts of the
  image/scene
• AC When you want to extract AC information from
  the large DC background
• Demodulate as early as possible in optical or
  charge domain, better than in voltage domain or
  off-line
• have as high as possible NEC for the AC signal
  only

45

• Thank you