The%20Development%20of%20Image%20Completion%20and%20Tutorial%20Plug-ins%20for%20the%20GIMP

1
The Development of Image Completion and Tutorial
Plug-ins for the GIMP

• By Cathy Irwin
• Supervisors Shaun Bangay and Adele Lobb

2
Project Aims

• Automatic image completion plug-in for GIMP (GNU
  Image Manipulation Package)
• Tutorial for general plug-in development

3
Image Completion

• Remove scratches, defects, writing objects,
  reconstruct damaged images - Manual techniques
  are painstaking
• Automate process user only selects region to be
  removed
• Plug-in realistically fill in background regions
  by analysing known regions

4
Related Work

• Texture synthesis
• Image inpainting

Efros Freeman (2001)

• Bertalmio, Sapiro, Caselles Ballester (2000)

5
Image Completion Plug-in

• Fragment-Based Image Completion
• Drori, Cohen-Or Yeshurun (2003)

6
My Approach

• Re-implementing proven technique BUT
• Incorporating it into the GIMP
• Inevitable differences due to language,
  environment, coding style
• Verify authors results
• Scenarios
• Textured regions
• Smooth regions
• Geometric shapes

7
Fragment-Based Image Completion Algorithm

• Fast Approximation
• Confidence Map
• Candidate Position Map
• Search
• Composite Fragments

8
Implementation in the GIMP

• GIMP 2.0
• Use empty plug-in templates as basis for
  implementation
• The implementation language C
• Gimp and GTK libraries
• manipulate the GIMP-specific elements images,
  drawables, channels and layers

9
Fast Approximation
X
10
Fast Approximation

• Each step is added as a new layer to the original
  image
• Manipulate and view independently of one another

11
Confidence Map

• Certainty with which the colour value of each
  pixel is known
• Gaussian falloff value used to decreases the
  confidence level towards the center of the
  unknown region

12
Candidate Position Map

• Takes confidence map as input
• Pixel with the maximum value in the candidate map
  is concluded to be the next most appropriate
  target pixel to be used in the search and
  composite phases

13
Search Phase

• Iterates through each pixel in a 20 x 20 pixel
  block around the target to find an appropriate
  source pixel
• Average value of the pixels around each potential
  source it is taken to determine the best possible
  match with the neighbourhood surrounding the
  target pixel
• Size of this neighbourhood region determined by
  the user at run time but is usually about 8
  pixels wide
• The search formula finds the pixels which have a
  higher confidence in the source than in the
  target regions and correspond well in terms of
  colour

14
Compositing Fragments

• Superimpose that source fragment over the target
  fragment so that the detail from the source
  merges seamlessly with the region around the
  target.
• Transparent edges - blend in convincingly
• Use existing GIMP functions
• Selections
• Copy
• Paste

15
(No Transcript)
16
Results

• Suited to
• Stochastic (random) texture that occurs often in
  the surrounding region
• Effective at making the reconstructed area blend
  into the known regions

17
Results cont

• 3D structure not taken into account
• Complete image according to the direction the
  line was taking at the edge of the gap and not
  according to mathematical principles

18
Results cont

• Parameter values depend on the size of the gap
  and the type of texture in the picture
• 160 x 120 pixel image between 12 minutes and an
  hour to process
• Smaller gaps need a smaller blur radius and
  smaller neighbourhood search region. Larger,
  smooth or stochastic regions can be accurately
  completed using a larger neighbourhood region
  that also decreases the processing time

19
Results cont
20
On-line Plug-in Tutorial

• General tutorial for writing plug-ins for the
  GIMP in C Tutorial Page
• Provide a starting point for beginners, not a
  definitive guide on every aspect of plug-in
  development
• Areas covered
• areas that I found problematic during the
  development of the plug-in
• features that were discovered to be particularly
  useful
• references to where more specific information can
  be found

21
Conclusion

• Plug-in development aided by availability of
  templates and documentation regarding GIMP
  specific functions
• Some common image manipulation operations complex
  to implement
• Not possible to re-implement the original paper
  exactly within the GIMP environment due to the
  tool and function constraints and the extra
  processing required for certain operations
• A plausibly similar implementation has been
  achieved - takes advantage of GIMP functions to
  approximate the intentions of the original paper

22
(No Transcript)