OpenCV

1
OpenCV

• Computer Vision API for everyone

Vadim Pisarevsky Intel Corporation
2
What is OpenCV?

• OpenCV stands for Open Source Computer Vision
  Library
• It is a collection of computer vision, image
  processing and general-purpose numeric
  algorithms.
• Very efficient (implemented in C/C)
• Has BSD-like license (that is, absolutely free
  for academic and commercial use)
• Available at http//sourceforge.net/projects/openc
  vlibrary

3
Why OpenCV?

• Provide de-facto standard API as a base for image
  processing and computer vision applications.
  Create new usage models for PCs, mobile
  platforms.
• Make IA attractive for such applications by using
  Intel Performance Libraries to improve OpenCV
  performance

4
The Project History

• CVL project was started the main
  goals/features
• Human-Computer interface is a main target
• Real-Time Computer Vision Library for using by
  UI Developers, Videoconferences, Games
• Highly optimized for IA.

OpenCV beta 1 with stereo support released and
presented at CVPR01
OpenCV beta 4 released.
OpenCV alpha 3 released and presented at CVPR00
1999/01
2000/06
2000/12
2004/08
Continuous development and various research
projects
5
Community

• There is a large community including people from
  major companies (IBM, Microsoft, Intel, SONY,
  Siemens, Google,) and research centers
  (Stanford, MIT, CMU, Cambridge, INRIA etc.)
• gt12000 members of the forum OpenCV_at_yahoogroups.com
  , with average daily traffic 10-20 messages.
• Contributions bug reports, patches, new features
  (video acquisition, 3d tracking, textures, Python
  interface)

6
Statistics

• Code Volume (beta 4)
• the code 500 functions, 170K lines, 6MB
• documentation (HTML) 13K lines, 650K
• tests 51K lines, 1.7MB
• Downloads
• 46000 beta3.1 downloads (36000 Win32, 10000
  Linux), 23000 beta 4 downloads (18000 Win32,
  5000 Linux)
• (since beta4 users could also
• take the latest updates from CVS)

7
Supported Platforms, Tools

• The processing (algorithmic) part
• Windows
• Microsoft Visual C (6.0, .NET 2003), Intel
  Compiler, Borland C, Mingw (GCC 3.x) via
  project files and command-line Makefiles.
• Linux GCC (2.9x, 3.x), Intel Compiler
  ./configure-make-make install, RPM (spec file
  is provided)
• C and light C are used. A few and well
  localized if and pragmas (i.e. very portable)
• GUI, Video/image acquisition
• Windows DirectShow, VFW, MIL, CMU1394
• Linux V4L2, DC1394, FFMPEG
• Documentation plain HTML

8
OpenCV and Performance Libraries

• Intel Performance Libraries have a long history
  (since mid-90s)
• Currently include IPP (low-level API for signal
  image processing, media codecs etc.) and MKL
  (native LAPACK and FFTPack)
• Non-commercial versions for Linux and trial
  versions for Windows and Linux are freely
  available from Intel site.
• Optimized for Pentium 4, Itanium-based, Xscale
  (IPP only), SMP systems (the best version for
  particular configuration is automatically
  selected).
• OpenCV can automatically detect IPP and MKL at
  runtime and use them to speedup processing.

CV
CXCore
IPP
MKL
9
Performance Boost with IPP
Approximate speed-ups using assembly optimized
IPP over the embedded optimized C in OpenCV on
Pentium 4 processor
10
OpenCV Architecture
IPP MKL are not required to build and use
OpenCV!
11
The Functionality in Words

• Core (cxcore, partly used by Intel Open Source
  Probabilistic Network Library as well)
• Simple operations on dense arrays
• Matrix algebra, math functions, RNG
• DFT, DCT
• Serialization to XML/YAML (data persistence)
• Drawing functions (2D graphics)
• Complex data structures sparse matrices, growing
  sequences, graphs
• Vision (cv)
• Basic image processing (filters, geometrical
  transformations, color space transforms)
• Image analysis (feature selection, morphology,
  contour retrieval, histograms)
• Structural analysis (shape descriptors, planar
  subdivisions)
• Motion analysis and object tracking
• Object/face detection
• Camera calibration and elements of 3D
  reconstruction
• IO/GUI (highgui)
• Image/video acquisition.
• Simple GUI facilities (all OpenCV visual samples
  use HighGUI)
• Experimental/obsolete functions (cvaux)
• 3D vision stereo calibration, trifocal tensor,
  bundle adjustment

12
The Functionality in Pictures
13
The future OpenCV directions/plans

• The strategy shifts and expands from
  a-man-in-front-of-camera paradigm towards
  generic visual information processing
  (cameraphones, digital home, security, medical
  applications, media retrieval etc.)
• Next major release is scheduled for 2005Q4
• Will remain free and open source
• Even better stability (to call it 1.0?) and
  performance
• Better use of IPP (gt yet higher performance)
• More open development model
• Some new functionality to be transferred from
  past/present research projects

14
The all-time OpenCV Crew
15
Useful Links

• http//www.intel.com/research/mrl/research/opencv/
  (OpenCV at Intel)
• http//sourceforge.net/projects/opencvlibrary/
  (the project page at SF. Take the latest release
  or CVS snapshots here)
• http//groups.yahoo.com/group/OpenCV/ (The forum.
  Subscribe and ask your questions here)
• http//www.opencv.org/ (OpenCV Wiki pages, still
  being formed)
• http//www.cs.ru.ac.za/research/sharpercv/ (C
  interface for OpenCV)
• http//www.ient.rwth-aachen.de/team/asbach/opencv-
  python.html (more wrappers for OpenCV. To be
  included into the official distribution)
• http//lush.sourceforge.net/ (Lisp-like language
  by Yann LeCun with interface to OpenCV)
• http//www.intel.com/software/products/ (Intel
  Software page. Take IPP, Intel Compiler and other
  tools here)
• http//sourceforge.net/projects/openpnl (OpenPNL
  - Bayesian Nets library from Intel shares some
  code with OpenCV)
• http//www-2.cs.cmu.edu/cil/v-source.html (a lot
  of links to open source computer vision and image
  processing software)
• http//www.google.com/search?qopencv (find more
  OpenCV resources with Google)

16
BackupSample OpenCV code
17
The simplest OpenCV program
sample.cpp
include ltcv.hgt include lthighgui.hgt int main(
int argc, char argv ) IplImage image
if( argc ! 2 ) return -1 image
cvLoadImage( argv1 ) if( !image ) return
-1 cvNamedWindow( Sample, 1 )
cvShowImage( Sample, image )
cvWaitKey() return 0
(the screenshot is from Windows version)
Build it and run
g o sample pkg-config --cflags opencv
sample.cpp pkg-config -libs opencv
./sample lena.jpg
18
Using contours
include ltcv.hgt include lthighgui.hgt IplImage
image 0 int thresh 100 void
on_trackbar(int) IplImage gray
cvCreateImage( cvGetSize(image), 8, 1 )
CvMemStorage storage cvCreateMemStorage(0)
CvSeq contours 0 cvCvtColor( image,
gray, CV_BGR2GRAY ) cvThreshold( gray, gray,
thresh, 255, CV_THRESH_BINARY )
cvFindContours( gray, storage, contours )
cvZero( gray ) if( contours )
cvDrawContours( gray, contours,
cvScalarAll(255),
cvScalarAll(255), 100 ) cvShowImage(
"Contours", gray ) cvReleaseImage( gray )
cvReleaseMemStorage( storage ) int main(
int argc, char argv ) IplImage image
if( argc ! 2 !(image cvLoadImage(argv1))
) return -1 cvNamedWindow(
"Contours", 1 ) cvCreateTrackbar(
"Threshold", "Contours",
thresh, 255, on_trackbar ) on_trackbar(0)
cvWaitKey() return 0
19
Detecting faces in Video
include "cv.h" include "highgui.h" int main(
int argc, char argv ) static
CvMemStorage storage 0 static
CvHaarClassifierCascade cascade 0
CvCapture capture 0 int optlen
strlen("--cascade") if( argc ! 3
strncmp( argv1, "--cascade", optlen ))
return -1 cascade (CvHaarClassifierCascade
)cvLoad( argv1 optlen ) capture
cvCaptureFromAVI( argv2 ) if( !cascade
!capture ) return -1 storage
cvCreateMemStorage(0) cvNamedWindow(
"Video", 1 ) for() IplImage
frame cvQueryFrame( capture ), img
CvSeq faces if( !frame )
break img cvCloneImage(frame)
img-gtorigin 0 if( frame-gtorigin )
cvFlip(img,img)
cvClearMemStorage( storage ) faces
cvHaarDetectObjects( img, cascade, storage,
1.1, 2, CV_HAAR_DO_CANNY_PRUNING, cvSize(20,
20) ) for( int i 0 i lt (faces ?
faces-gttotal 0) i ) CvRect r
(CvRect)cvGetSeqElem( faces, i )
cvRectangle( img, cvPoint(r-gtx,r-gty),
cvPoint(r-gtxr-gtwidth,r-gtyr-gtheight),
CV_RGB(255,0,0), 3 )
cvShowImage( "Video", img )
cvReleaseImage( img ) if( cvWaitKey(10)
gt 0 ) break cvReleaseCapture(
capture ) return 0
./facedetect -cascadeopencv/data/haarcascades/
haarcascade_frontalface_alt2.xml screetcar.avi