Understanding SVG Diagrams

1
Understanding SVG Diagrams

• Interim Presentation
• Vivek Ramakrishna
• Honours Program, Computer Science
• vivekr_at_csse.monash.edu.au

2
Motivation

• Vast amounts of information stored in diagrams
• Information not accessible to all users
• Blind people
• Search engine agents
• Should be able to retrieve information in a
  diagram, given a query

3
Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
4
Diagram Representations and Types

• Rasterised versus vector graphics
• Scalable Vector Graphics (SVG)
• Vectors have types and attributes
• ltrect x0 y 0 width50 height20
  colorredgt
• Project will investigate bar charts, organisation
  hierarchies and flow charts in SVG format.

5
A simple bar chart in SVG
6
Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
Interpreter Understands the meaning of the
diagram and produces a useful description
7
Constraint Multiset Grammars

• In visual domain, spatial relationships more
  important than sequential.
• P P1, , Pn where C
• Non terminal symbol P can be recognised by
  multiset of symbols P1,, Pn whenever the
  attributes of the symbol satisfy the constraints
  C.

8
CMG for a bar

• A bar comprises of a rectangle and a text label.
• The label is aligned centrally below the bar.
• Bbar() Rrectangle, Ttext
• where ( R.mid T.mid
• R.start gt T.start
• )
• We will use CMGs to produce grammars to interpret
  our diagrams.

9
CIDER

• Toolkit developed at Monash by Tony Jansen.
• Interprets diagrams based on CMG specifications.
• We will use CIDERs powerful compiler to parse
  SVG to recognise diagram tokens.

Bar Chart
Bars
Axes
Bar
X axis
Y axis
Rectangle
Text
Line
Line
10
Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
Semantics The key issue when extracting
descriptions is the format to specify them.
11
RDF A Description Logic

• Resource Description Framework
• Can express knowledge of a domain by first
  defining its terminology.
• Equipped with formal, logic based semantics.
• Allows complex relationships to be inferred
  despite not having been explicitly stated
• In an organisation chart, can infer all superiors
  of an employee, without stating each relationship
• Disadvantage DLs cant handle fuzzy predicates

12
Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
Query Interface Communicate question to system
and answer to user. Different requirements for
each user. We will use existing interfaces for
our trials.
13
Plan

• Considering SVGs of bar charts, organisational
  hierarchies and flow charts.
• CIDER used to interpret CMG for each diagram.
• Descriptions generated in RDF.
• Evaluation
• Get corpus of sample diagrams
• Evaluate with corpus of sample queries
• Limitations
• Grammars are inherently brittle
• Description Logics are rigid.
• We wish to investigate when and why failure occurs

14
Questions?