Two-scale Tone Management for Photographic Look

1
Two-scale Tone Management for Photographic Look
Soonmin Bae, Sylvain Paris, and Frédo Durand MIT
CSAIL
2
Ansel Adams
Ansel Adams, Clearing Winter Storm
3
An Amateur Photographer
4
A Variety of Looks
5
Goals

• Control over photographic look
• Transfer look from a model photo

For example, we want
with the look of
6
Aspects of Photographic Look

• Subject choice
• Framing and composition
• ? Specified by input photos
• Tone distribution and contrast
• ?Modified based on model photos

Input
Model
7
Tonal Aspects of Look
Ansel Adams
Kenro Izu
8
Tonal aspects of Look- Global Contrast
Ansel Adams
Kenro Izu
High Global Contrast
Low Global Contrast
9
Tonal aspects of Look - Local Contrast
Ansel Adams
Kenro Izu
Variable amount of texture
Texture everywhere
10
Related Work Scale/Frequency Manipulation

• Used for audio visual equalizer
• controls sound ambiance
• Not really used yet for images
• Exception Kais Power Tools

11
Related Work - Example-based style transfer

• Non-photorealistic styles
• Hertzmann 01 Efros 01 Drori 03 Rosales 03
• mimics brush strokes or textures
• but does not target photorealistic style

Hertzmann 01
12
Related Work - Tone Mapping

• Reduce global contrast
• Pattanaik 98Tumblin 99Ashikhmin 02Durand
  02Fattal 02Reinhard 02Li 05
• Seeks neutral reproduction
• Little control over look
• In contrast, we want to achieve particular looks

Durand 02
13
Related Work Professional tools

• Image editing software
• e.g. Adobe Photoshop
• need skills
• tedious
• Photo management tools
• e.g. Adobe Lightroom, Apple Aperture
• optimizes user efficiency (workflow)
• but has limited control

Adobe Photoshop
Adobe Lightroom
14
Our work
Model
Input Image
Result

• Transfer look between photographs
• Tonal aspects

15
Our work
Global contrast
InputImage
Result
Local contrast

• Separate global and local contrast

16
Overview
Global contrast
Carefulcombination
Split
Post-process
InputImage
Local contrast
Result
17
Overview
Global contrast
Carefulcombination
Split
Post-process
InputImage
Local contrast
Result
18
Split Global vs. Local Contrast

• Naïve decomposition low vs. high frequency
• Problem introduce blur halos

Halo
Blur
Low frequency
High frequency
Global contrast
Local contrast
19
Bilateral Filter

• Edge-preserving smoothing Tomasi 98
• We build upon tone mapping Durand 02

After bilateral filtering
Residual after filtering
Global contrast
Local contrast
20
Bilateral Filter

• Edge-preserving smoothing Tomasi 98
• We build upon tone mapping Durand 02

BASE layer
DETAIL layer
After bilateral filtering
Residual after filtering
Global contrast
Local contrast
21
Global contrast
BilateralFilter
Carefulcombination
Post-process
InputImage
Local contrast
Result
22
Global contrast
BilateralFilter
Carefulcombination
Post-process
InputImage
Local contrast
Result
23
Global Contrast

• Intensity remapping of base layer

Remapped intensity
Input intensity
Input base
After remapping
24
Global Contrast (Model Transfer)

• Histogram matching
• Remapping function given input and model histogram

Modelbase
Inputbase
25
Global contrast
Intensitymatching
BilateralFilter
Carefulcombination
Post-process
InputImage
Local contrast
Result
26
Global contrast
Intensitymatching
BilateralFilter
Carefulcombination
Post-process
InputImage
Local contrast
Result
27
Local Contrast Detail Layer

• Uniform control
• Multiply all values in the detail layer

Input
Base 3 ? Detail
28
The amount of local contrast is not uniform
Smooth region
Textured region
29
Local Contrast Variation

• We define textureness amount of local contrast
• at each pixel based on surrounding region

Smooth region? Low textureness
Textured region? High textureness
30
Textureness 1D Example
Input signal
31
Textureness
Textureness
Input
32
Textureness Transfer
Model textureness
Step 1 Histogram transfer
Input textureness
Desired textureness
Hist. transfer
x 0.5
Step 2 Scaling detail layer (per pixel) to
match desired textureness
x 2.7
x 4.3
Input detail
Output detail
33
Global contrast
Intensitymatching
BilateralFilter
Carefulcombination
Post-process
InputImage
Texturenessmatching
Local contrast
Result
34
Global contrast
Intensitymatching
BilateralFilter
Carefulcombination
Post-process
InputImage
Texturenessmatching
Local contrast
Result
35
A Non Perfect Result

• Decoupled and large modifications (up to 6x)
• Limited defects may appear

result after global and local adjustments
input (HDR)
36
Intensity Remapping

• Some intensities may be outside displayable
  range.
• Compress histogram to fit visible range.

correctedresult
remappedintensities
initialresult
37
Preserving Details

• In the gradient domain
• Compare gradient amplitudes of input and current
• Prevent extreme reduction extreme increase
• Solve the Poisson equation.

correctedresult
remappedintensities
initialresult
38
Effect of Detail Preservation
uncorrected result
corrected result
39
Global contrast
Intensitymatching
BilateralFilter
ConstrainedPoisson
Post-process
InputImage
Texturenessmatching
Local contrast
Result
40
Global contrast
Intensitymatching
BilateralFilter
ConstrainedPoisson
Post-process
InputImage
Texturenessmatching
Local contrast
Result
41
Additional Effects
model

• Soft focus (high frequency manipulation)
• Film grain (texture synthesis Heeger 95)
• Color toning (chrominance f (luminance) )

aftereffects
beforeeffects
42
Global contrast
Intensitymatching
BilateralFilter
ConstrainedPoisson
Soft focus Toning Grain
InputImage
Texturenessmatching
Local contrast
Result
43
Recap
Global contrast
Intensitymatching
BilateralFilter
ConstrainedPoisson
Soft focus Toning Grain
InputImage
Texturenessmatching
Local contrast
Result
44
Results

• User provides input and model photographs.
• Our system automatically produces the result.
• Running times
• 6 seconds for 1 MPixel or less
• 23 seconds for 4 MPixels
• multi-grid Poisson solver and fast bilateral
  filter Paris 06

45
Input
Model
Result
46
Result
Input
47
Model
Input
Result
48
Comparison with Naïve Histogram Matching
Model Snapshot, Alfred Stieglitz
Input
Naïve Histogram Matching
Our result
Local contrast, sharpness unfaithful
49
Comparison with Naïve Histogram Matching
Model Clearing Winter Storm, Ansel Adams
Input
Our Result
Histogram Matching
Local contrast too low
50
Color Images

• Lab color space modify only luminance

Input
Output
51
Limitations

• Noise and JPEG artifacts
• amplified defects
• Can lead to unexpected results if the image
  content is too different from the model
• Portraits, in particular, can suffer

52
Conclusions

• Transfer look from a model photo
• Two-scale tone management
• Global and local contrast
• New edge-preserving textureness
• Constrained Poisson reconstruction
• Additional effects