ITACs Digital Imaging Poster Tips and Tricks Print this page for reference check Scale to Fit, then

1
Interactive Picture Database Website Featuring
Multiple Organizational Hierarchies Brendan
ElliottDepartment of Electrical Engineering and
Computer Science, Case Western Reserve University
ABSTRACT Digital cameras are being increasingly
used as a replacement for film cameras. As
personal digital photo libraries increase in
size, specific pictures may become harder to
find, just as it can be difficult to retrieve a
print among a shelf full of traditional photo
albums. This project, Risukun.com 1, began
approximately two and a half years ago as an
automated website to share and organize my
pictures. Since its conception it has grown from
about 1,600 images to roughly 30,000 images,
including pictures posted by friends and family.
The site has been redesigned several times during
this period in order to answer the challenges
posed by the increasingly large scale. Two of
the major concerns are 1) allowing the user to
efficiently find a specific image or images that
they wish to retrieve, and 2) provide means for
the user to discover new pictures that may be of
interest to him or her. The first problem is
currently addressed through the use of modern
database technology to index and provide an
efficient framework for querying. There are
several approaches that I have been investigating
to facilitate the second issue. One of the main
approaches was to design the system to allow
images to be placed in an arbitrary number of
conceptual hierarchies. The user can then switch
between different contexts in which a picture is
being browsed. Other features of this site
include a framework for doing image similarity
searches, support for users to manage their own
content, the ability to post comments, and access
to image metadata.
HIERARCHIES The system supports more than just
pictures links to websites, HTML pages, video
files, word documents, and even zip files can be
added as leaf nodes into the database. These
leaf nodes can appear in multiple
semantic/organizational hierarchies or even
multiple times in the same hierarchy. All files
are automatically placed exactly once in a
special Files hierarchy that reflects the file
system, but many appear freely in other
hierarchies as well. For example, these
hierarchies include Places, People, Animals, etc.
The current implementation also includes a
second automatically generated hierarchy, Date,
which organizes objects by the year/month/day
they were taken. For example, say I meet up with
a friend in Tokyo for a bit of site seeing and we
both post pictures on the site. Using the Date
hierarchy, you can browse his pictures from the
same day inside of the same conceptual folder as
my pictures without physically moving files with
the OS's file system. This possible because nodes
are distinct from their various node instances.
Furthermore, when viewing a particular object, it
is possible to switch the context in which you
are viewing it via a list of links to all other
node instances. This connects the hierarchies
allowing a seamless browsing experience. This
browsing/exploration process is illustrated by
Figures 2-6, which shows how the user can easily
go from browsing a set of holiday pictures to
looking at pictures taken at the same location or
involving the same person or with a similar
object.
USERS COMMENTS Content-owning users may log
in to perform management operations on their
files/folders. Comment may be posted anonymously
or by full users. Comment text is immediately
indexed for keyword searching.

• IMAGE SIMILARITY SEARCH (WITH CASEY KRETSCHMAN)
• As a class project for EECS 405 (Prof. Z. Meral
  Ozsoyoglu), we examined and implemented two
  methods of doing content-based image search. As
  opposed to a traditional keyword search, the
  input to the query is a picture and the goal is
  to find other pictures like it. To do this in
  a fully automated manner, one needs to come up
  with a similarity function that operates on
  low-level features. A common approach is to
  convert an image into a feature vector in the
  form of k numbers and then treat it as a point in
  k-dimensional space. Then a distance function,
  such as Euclidean distance, can be used to
  determine how close pictures are to each other
  and the items are placed in an appropriate
  high-dimensional/metric index structure.
• More specifically, the similarity metrics we
  compared were
• RGBAvg 2 A very simple metric that takes the
  average amount of red, blue and green in an image
  and generates a 3-dimensional feature vector.
• Color-Shape Histogram 3 Cuts each image into a
  number of cells (say, 8x8) and reduces the color
  space (to, say, 64 colors), recording the amount
  of each color in each cell.
• These were each put into an M-Tree 4, which is
  an insert-friendly metric index structure with a
  standard implementation available.
• As is expected for such simple similarity
  metrics/feature extraction techniques, the
  results were fairly poor semantically, especially
  for RGBAvg. The Color-Shape histogram did
  provide generally better results, but also took
  an order of magnitude longer to calculate (a
  couple seconds versus a couple of minutes). One
  factor that contributed to the slow performance
  of the CSH approach was the failure of the index
  structure due to the data distribution and the
  much higher number of dimensions involved (up
  to 4096 dimensions versus 3 dimensions) and
  deteriorated to comparing the query object to
  every item in the database. This is evident in
  the sample results shown in Figure 14.

Post
Figure 3Browsing a picture Currently the user
is browsing a picture organized by the file
system (Figure 2) and links are given to the
items location in other hierarchies.
Figure 8Picture with comments
Figure 7User has logged on
IMAGE METADATA
Organize
Figure 4Location Hierarchy View
Figure 10Viewing JPEG metadata
Figure 1Front Page
Figure 2Sample Folder with thumbnails
Figure 9Owner View of a folder
KEYWORD SEARCH First, all available textual
data associated with files/folders is
preprocessed to build a keyword index. During
this process, words are extracted and run through
a stemming function to remove plurality, verb
conjugations, etc. This cuts down on the number
of unique keywords and improves search results.
When a visitor tries to find something, all the
search terms are run through the same standard
stemming algorithm. If a user is logged in, they
have the option of doing a special hierarchy
builder search, that facilitates the process of
organizing many images at once.
DESIGN IMPLEMENTATION The site is implemented
primarily in Microsoft Visual C 7.1, storing
system logic in a shared C DLL with the web
pages created in ASP.NET. The database used is
Microsoft SQLServer 2000 with wrapper layer to
provide a persistent object model. The actual
image file remain on the file system and new
images can be posted by locally placing files in
a special folder tree on the file system, using
FTP, or uploading via the website. When the
system discovers a new image, it is indexed and
image metadata is extracted. Visitors to the
site can post comments and content owners can
perform certain management tasks including file
renaming and placing images in semantic
hierarchies. When a folder or image is browsed
for the first time, thumbnails and web-friendly
versions of pictures are automatically created.
Other types of content including movie files,
html pages, and text documents are also indexed.
Keyword search is provided to find pictures by
their caption or comment text. For some pictures
that have been specially preprocessed (currently
about 6000 images), content-based similarity
search can be performed. While the .NET
framework provides a rich base for creating
interactive multimedia applications, I am
currently making use of two external C
libraries. These include an insert-friendly
metric index for similarity search and a stemming
library to reduce words to their grammatical
stem to improve the quality of keyword search.
To integrate these libraries, I created wrappers
using Microsofts Managed Extensions for C to
allow them to be trivially and efficiently called
from C code. In addition to the website, there
is also a command line program using the shared
library to facilitate initial data population and
preprocessing tasks. I am currently also
developing an integrated FTP server that is
aware of the special content folder (to
auto-index on upload) and can also share account
information and allow browsing of the virtual
hierarchies, as well as a rich windows client to
further facilitate hierarchy building via an
intuitive drag-and-drop interface.
Figure 14Image similarity output
Figure 13Image similarity input
FUTURE WORK This is very much still a work in
progress and I am continuously posting content,
refining things and adding new features. In
addition to work on an integrated FTP server and
rich client for hierarchy creation, I am
currently investigating techniques for
semi-automating image labeling and hierarchy
construction as well as adding relevance feedback
to improve search results.

• REFERENCES
• Elliott, Brendan, RisuPicWeb, http//www.risukun.c
  om
• Faloutsos, C, Ron Barber, Myron Flickner, Wayne
  Niblack, Dragutin Petkovic, and William Equitz.
  Efficient and effective querying by image
  content. J. of Intelligent Information Systems,
  3(3/4)231-- 262, July 1994. http//citeseer.ist.p
  su.edu/faloutsos94efficient.html
• Stehling, R, Mario A Nascimeto, and Alexandre X.
  Flacao, Color-Shape Histograms for Image
  Representation and Retrieval, In Proc. Of the
  Intl. Workshop on Multimedia Information
  Retrieval, USA, 2000.
• Ciaccia, P, M. Patella, and P. Zezula, "M-tree
  An efficient access method for similarity search
  in metric spaces," Proceedings of the 23rd VLDB ,
  August 1997. http//citeseer.ist.psu.edu/article/c
  iaccia97mtree.html

Figure 5Plant Hierarchy View
Organize
Figure 6People Hierarchy View
Address http//www.risukun.com
Figure 11Build hierarchy search
Figure 12Search with two keywords