Image Recognition Features

1
Image Recognition Features
2
Image recognition Plugin
The Image Recognition feature provides two new
commands that use state-of-the-art image
recognition technology

• IMAGESEARCH Search for an image anywhere on the
  website, including inside of Java and Flash
  applets. Very useful to get state information
  from Java or Flash-applets.
• IMAGECLICK Send mouse clicks to an image, not a
  fixed X/Y position. This development solves the
  problem of moving buttons in interactive
  websites.
• The coordinates of the image are available in the
  !IMAGEX and !IMAGEY variables for further
  use.

3
IMAGESEARCH command

• IMAGESEARCH IMAGEc\testimage.bmp  CONFIDENCE99
  
• Two cases
• Confidence 100, a high confidence level (100)
  means that an exact copy of the test image has to
  be found on the website. Extremly fast algorithms
  (less 1 sec)
• Confidence lt 99, This command will trigger an
  advanced image recognition algorithm. The lower
  confidence levels allow for color changes. Search
  focuses on shape of object (letters, characters)

4
IMAGECLICK

• IMAGECLICK IMAGEc\buttonimage.bmp
• CONFIDENCE99 CONTENTHelloltSPgtWorld
• IMAGECLICK IMAGESEARCH WINCLICK
• The CONTENT attribute has the same function as
  for the normal DS CMDCLICK command. Use it, for
  example, to fill an input box inside a FLASH
  applet.
• If the image is found, the image recognition
  module will return the coordinates of the middle
  of found image to iMacros

The coordinates are in the !IMAGEX and
!IMAGEY variables
5
Test Chat Flash Applet - Blue
6
Test Chat Flash Applet - White
7
Test Chat Flash Applet - Black
8
Test Case

• Function testing the chat applet
• Send a text to the input box
• 2. Click the "Send" button
• 3. Verify that text was received and is displayed
  in main window.
• This can be achieved with the following command
  sequence
• IMAGECLICK IMAGEinputbox.bmp CONFIDENCE99
  CONTENTWelcomeltSPgttoltSPgtHeidelberg!
• IMAGECLICK IMAGEsendbutton.bmp CONFIDENCE99
• IMAGESEARCH IMAGEtext.bmp CONFIDENCE99

9
Create Input Images
inputbox.bmp
sendbutton.bmp
text.bmp
10
Image Recognition Tips

• Processing time image recognition 2-3 seconds
  typically
• Processing time increases with input image size
• Processing time 100 matching lt 1 second
• False detection if confidence level is set too
  low
• MS Paint useful for creating input images. MS
  Paint is part of Windows OS.

11
Demo

• Write a macro that adds 4 plus 1 at
  http//www.iopus.com/iim/demo/v6/flash
• Enter Hello World into banner chat at
  http//www.123flashchat.com/
• Create macro that logs into http//mywallop.com/
  and automatically adds a new blog entry.