VIPS: An Image Processing Library for Large, and Not So Large, Images

1
VIPS An Image Processing Library for Large, and
Not So Large, Images
Kirk Martinez University of Southampton Southampto
n, UK

• John Cupitt
• Imperial College
• London, UK

Presented by Nicolas Robidoux Université
Laurentienne Sudbury ON
http//www.vips.ecs.soton.ac.uk
LGM Montréal 2009
2
Introduction

• VIPS is a 2D scientific image processing system.
• Needs little memory, runs quickly
• Optimised for multi-CPU machines
• Good support for large images and for colour
• Mostly C with some C. Python interface
• Library runs on any Unix and on Windows. LGPL
• Has an advanced GUI, nip2, a blend of a
  spreadsheet and a paint program. GPL

3
Speed and memory use
Load, crop, shrink, sharpen and save a 5k x 5k
pixel RGB tiled TIFF image. Fastest real time of
three runs on a quiet system. Tests run on a 2
CPU Opteron server running Ubuntu 8.10.
ImageMagick and GraphicsMagick compiled with
Q16. FreeImage does not have a sharpening
operation, so that part of that test was skipped.
nip2 seems slow because it has a long start-up
time once it starts, it processes at the same
speed as the Python and C versions. Source-code
for the various implementations is on the VIPS
website.
4
SMP scaling
Load, crop, shrink, colour-correct, sharpen and
save a 10k x 10k pixel CIELAB image in VIPS
format. Number of CPUs on horizontal axis,
speed-up on vertical. Run on a 64-CPU Itanium2
supercomputer (SGI Origin 2000). Details of the
benchmark and source-code are on the VIPS
website. This is after a substantial tuning
effort on the SMP system as part of the
development of the PARSEC benchmark.
5
History

• VIPS was started in 1990 as the image processing
  system for the VASARI project (multispectral
  imaging of old-master paintings to detect
  long-term colour change).
• 10,000 x 10,000 pixels, seven colour bands, 16
  bits per band, up to 1.6 GB for the final image.

Our Sun4 had 64 MB of RAM and a 25 Mhz processor.
Ouch!
6
History

• VIPS looked something like this

Several operations run at once and images are
pulled through the pipeline by demand from the
sink. A simple, lightweight system combines
operations without the need for large
intermediate images.
7
History

• We added SMP support in 1993

Run a pipeline in each thread, sinks arrange
synchronisation.
8
History

• We added large file (gt2GB) support in 2002

We map a small window into each input file. These
windows are shared between threads when possible.
Window positions are calculated using 64-bit
arithmetic.
9
History

• We improved SMP scaling in 2005

A buffer manager decouples workers from the image
write library so they never have to wait. We also
added a system for quickly sharing and recycling
pixel buffers.
10
General principles

• 2D colour images only no video, no volumes
• Any number of (pseudo-)colour bands
• All band elements in an image must have the same
  pixel format (eg. 32-bit signed int)?
• All operations are non-destructive
• Images are uninterpreted arrays. Alpha channels,
  CMYK, layers, etc. must be implemented on top of
  VIPS
• Pipelines are static. Apps have to destroy and
  rebuild to make a change.

11
API

• The C and Python APIs are very similar to PIL.
• import sys
• from vipsCC import
• im VImage.VImage (sys.argv1)?
• im im.extract_area (100, 100, im.Xsize () -
  200, im.Ysize () - 200)?
• im im.similarity (0.9, 0, 0, 0)?
• mask VMask.VIMask (3, 3, 8, 0,
• -1, -1, -1,
• -1, 16, -1,
• -1, -1, -1)?
• im im.conv (mask)?
• im.write (sys.argv2)?
• Load, crop, shrink, sharpen, save. Python binding
  is generated automatically by SWIG.

12
Other features

• Pixel can be 8/16/32-bit integer, signed and
  unsigned, 32/64-bit float, 64/128-bit complex
• XYZ, Lab, Yuv, Yxy, Lch, RGB colour spaces ICC
  colour management with lcms
• Operation database, plugins, metadata, many file
  formats, memory, disc and screen sinks
• 350 operators, mostly simple filters rank,
  Fourier, morphological operators, convolutions,
  histogram operations, colour, arithmetic, affine
• Simple 10k lines of C for the core, 50k in
  operators

13
Comparing VIPS and GEGL

• Low-level
• VIPS images are uninterpreted 3D arrays. VIPS has
  no explicit support for alpha channels, no CMYK
  image type, no layers, ... Applications built on
  VIPS are responsible for presenting images to the
  user. Operators do set hints about the possible
  interpretation a pixel might have, for example
  this image is a histogram. GEGL directly
  supports features like alpha channels.
• Static
• VIPS pipelines are fixed you can't alter any
  settings after calling an operator, you can only
  evaluate pixels. If an application wants to
  change a parameter, it has to destroy and rebuild
  the pipeline. You only need to destroy and
  rebuild from the change onwards. GEGL has a
  dynamic, interactive graph.
• No caching
• VIPS does almost no caching for you. There is a
  cache operator, but you have to explicitly add it
  to the graph yourself. By contrast, GEGL
  automatically caches every pixel which
  contributes to the display.

14
Comparing VIPS and GEGL

• Operators are polymorphic
• VIPS operators have to each handle all 10 element
  types and any number of image bands. This is done
  with metaprogramming usually, an operator will
  have 2 or 3 implementations and use macros or
  templates to generate the code for all the cases.
  GEGL operators only see float arrays and use BABL
  to convert to and from the real data type.
• Analogy
• VIPS Xlib, GEGL canvas widget.
• You can imagine a version of GEGL which uses VIPS
  as a backend.

15
GUI

• nip2, the VIPS GUI, is a spreadsheet where each
  cell can be a complex object an image, plot,
  widget, matrix, etc.
• nip2 has its own lazy, higher-order, pure
  functional language with classes, somewhat like
  dynamically typed Haskell. Spreadsheet cells are
  class instances. Cells are joined with snippets
  of this language.
• As the spreadsheet recalculates it builds
  optimised VIPS pipelines behind the scenes. Image
  generation is then pure VIPS.
• Fast, low memory use, huge images.

16
GUI
17
GUI

• You can use nip2 for quite large, complex
  applications. We have a set of linked workspaces
  which analyze four-dimensional images (volumes
  over time) from PET scanners to calculate tissue
  inflammation indices.
• The workspaces read 300MB of image data, process
  7,000 images, generate 60 GB of intermediate
  images and take 2 minutes to completely
  recalculate. They need only 400 MB of RSS to run.
• Useful tool for technical users. Not aimed at
  general audience.

18
GUI
19
Bad things about VIPS

• Somewhat old-fashioned, clunky C API
• Uses manpages. We get a lot of complaints about
  hard-to-navigate docs.
• Limited range of operators. It would be nice to
  have segmentation, for example.
• Awkward to extend, despite a plugin system.
  Operators have a fixed set of arguments and it's
  difficult to add functionality.

20
TODO

• We've started moving VIPS to GObject.
• The current stable version has several
  GObject-based systems. We plan to move most of
  the VIPS types to GObject in the next version,
  then start rewriting operations in the version
  after. We will switch to gtk-doc for API docs.
• This should give us a sane, extensible,
  well-documented, easy to bind API with hopefully
  similar performance to the current version.
• We'll aim to have a vips7 compatibility layer.

21
For more information
http//www.vips.ecs.soton.ac.uk
Kirk Martinez University of Southampton Southampto
n, UK
John Cupitt Imperial College London, UK