Virtual Dart: An Augmented Reality Game on Mobile Device

1
Virtual Dart An Augmented Reality Game on Mobile
Device

• Supervisor Professor Michael R. Lyu

Prepared by Lai Chung Sum Siu Ho Tung
2
Outline

• Background Information
• Motivation
• Objective
• Methods
• Results
• Future Work
• Q A

3
What is Augmented Reality (AR)?

• A combination of real world and computer
  generated data
• Add computer graphic into video

4
Background Information

• Most mobile phones equipped with cameras
• Games written in J2ME proprietary development
  platform

5
Background Information

• Typical mobile games

6
Background Information

• Mobile games employed Augmented Reality

7
Motivation

• How can the game remember external environment?
• ? Save external environment information

8
Objectives

• Demonstrate how a game remember its external
  environment for Augmented Reality (AR)
• Virtual Dart is just a game for demonstration of
  the proposed methodology

9
Problems to be solved

1. What information should we store?
2. How does the game recognize the information?
3. How does the game perform motion tracking?

10
Introduction to Mobile Video Object Tracking
Engine (mVOTE)

• Convert the camera movement into translational
  movement and degree of rotation

Feature Selection (Find a feature to trace)
Motion Tracking of Translational Movement
Motion Tracking of Rotational Movement
11
What is a feature?

• Section of an image that is easily highlighted
  for the purpose of detection and tracking
• Have a high contrast in relation to its immediate
  surroundings

X
12
What does our program need?
Functions needed for our program Can mVOTE do this?
Feature Selection (What information should we store?)
Feature Recognition (How does the game recognize the information?)
Motion Tracking of Translational Movement (How does the game perform motion tracking?)

13
Program Flow Initial Algorithm
14
Program Flow Initial Algorithm
15
Experiment of Feature Selection

• Feature Selection in mVOTE VS FAST Corner
  Detection Algorithm
• Testing Environment
• Normal lighting
• Insufficient lighting

16
Normal Lighting Condition
Feature Selection in mVOTE FAST Corner Detector

17
Normal Lighting Condition
Feature Selection in mVOTE FAST Corner Detector

18
Normal Lighting Condition
Feature Selection in mVOTE FAST Corner Detector

19
Insufficient Lighting Condition
Feature Selection in mVOTE FAST Corner Detector

20
Insufficient Lighting Condition
Feature Selection in mVOTE FAST Corner Detector

21
Analysis

• Normal Lighting
• ? Both algorithms worked reasonably well
• Insufficient Lighting
• ? Only mVOTEs Feature Selection could produce
  output
• Occasionally, Feature Selection in mVOTE selected
  some flat regions as features
• FAST Corner worked better in terms of accuracy

22
Experiment of Initial Approach
Selected Features Recognized Features

23
Experiment of Initial Approach
Selected Features Recognized Features

24
Experiment of Initial Approach
Selected Features Recognized Features

25
Initial Feature Recognition Conclusion

• Accuracy?? LOW!

Selected Features Recognized Features

26
Analysis

• Matching accuracy is very bad
• Store 3 25x25 pixels features blocks
• Feature Recognition on the 3 Blocks
• More than 1 point have same SSD

27
Program Flow Enhanced Feature Recognition
Algorithm
28
Enhanced Feature Recognition
Set 1 Set 2

29
Enhanced Feature Recognition
Set 3 Set 4

30
Enhanced Feature Recognition
Set 5 Set 6

31
Analysis

• Totally 10 set of sample photos
• 3 trials at each set
• Each run would produce a slightly different
  result
• May come from a small vibration during image
  capturing or maybe due to a small change in light
  intensity

32
Algorithms Comparison

• Initial Feature Recognition VS Enhanced Feature
  Recognition
• Initial Approach 3 Features
• New Approach Whole selection area
• Reason for LOW accuracy (Initial Approach)
• ? Features may not be descriptive enough

33
Improvement of Feature Selection

• Two conditions of a Good Feature
• Descriptive
• Large internal intensity difference
• ? Corner Detector can help us to find out good
  features

34
FAST Corner Detector

• Examine a small patch of image
• Considering the Bresenham Circle of radius r
  around the candidate pixel which is called p
• Intensities of n continuous pixels on the circle
  are larger than p or smaller than p by barrier
• ? Potential corner

35
e.g. r 3, n 12, barrier 25
215 65 150 gt 25 barrier ? Marked by red 65
39 26 gt 25 barrier ? Marked by Blue
215 255 200
39 167
20 132
16 65 152
91 153
101 165
114 182 167
36
e.g. r 3, n 12, barrier 25
215 63 21
84 57
108 12
65 90 52
31 15
35 43
57 58 62
37
e.g. r 3, n 12, barrier 25
215 63 21
84 57
108 12
65 90 90
31 15
35 43
57 58 62
38
FAST Corner Detector

• The typical values of r and n are 3 and 12
  respectively
• For the value of barrier, we did an experiment to
  choose the value
• We chose 25 after the experiment (for what?)

39
FAST Corner Detector

• Advantage
• Fast
• Disadvantages
• Cannot work well in noisy environment
• Accuracy depends on parameter barrier

40
FAST Corner Detector
barrier 10 barrier 40

41
How does Feature Recognition works?

• Full screen as search window
• Use Sum Square Difference (SSD) to calculate the
  similarity of blocks
• Still slow in current stage (20 60sec)
• Tried to use a smaller image and scale up to full
  screen
• Scaling step is too time consuming

42
Motion Tracking during the game

• Keep track of three features
• Use two features to locate dart board
• The last feature point is used for backup
• Use if either one of the feature points fail
• Condition for a feature point failure
• Feature point is at the edge of the screen
• Two feature points are too close

43
Future Works

• Allow users to load saved features
• Increase the speed of feature recognition
• Add physical calculation engine

44

• Q A