20072008 Semester 1 SD5943 Multimedia Literacy Week11 Cognitive and HCI

1
2007/2008 Semester 1 SD5943 Multimedia Literacy
Week11 Cognitive and HCI

• Prepared by Wilson YUEN
• sdwilson_at_polyu.edu.hk

2
Agenda

• Part 1 Introduction to HCI
• Part 2 HCI and Digital Gaming

3
Part 1 Introduction to HCIReference
Professor Benjamin Lok, The Department of
Computer and Information Science and Engineering,
University of Florida
4
Introduction

• What is a user interface?
• Why do we care about design?
• We see this all the time.
• Whats good about the design of this error box?
• The user knows there is an error
• Whats bad about the design of this error box?
• Discouraging
• Not enough information
• No way to resolve the problem (instructions or
  contact info)

5
Why HCI is important?

• The study of our interface with information.
• It is not just how big should I make buttons or
  how to layout menu choices
• It can affect
• Effectiveness
• Productivity
• Morale
• Safety
• For example think about if a car with poor HCI
• Take 10 minutes for everyone to write down one
  common device with substantial HCI design choices
  and discuss with the neighbor the pros and cons.
  How does it affect you or other users?

6
Apple iPod

• iPod by Apple Computers
• Pros
• portable
• power
• ease of use
• GUI
• Cons
• scratches easily
• no speech for car use
• proprietary
• too colorful

7
HCI Communities

• Academics/Industry Research
• Taxonomies
• Theories
• Predictive models
• Experimenters
• Empirical data
• Product design
• Other areas (Sociologists, anthropologists,
  managers)
• Motor
• Perceptual
• Cognitive
• Social, economic, ethics

8
HCI Tools

• Sound
• 3D
• Animation
• Video
• Devices
• Size (small-gtvery large)
• Portable (PDA, phone)
• Plasticity
• Context sensitive/aware
• Personalizable
• Ubiquitous

9
Usability Requirements

• Goals
• Usability
• Universality
• Usefulness
• Achieved by
• Planning
• Sensitivity to user needs
• Devotion to requirements analysis
• Testing

10
Bad Interfaces

• Encumbering
• Confusing
• Slow
• Trust (e.g. windows crashing)
• What makes it hard?
• Varies by culture
• Multiple platforms
• Variety of users
• Think of a game youve played with a bad
  interface
• UNIX

11
Goals

• Telepresence
• Direct Manipulation
• Immersion (Virtual Reality and Augmented Reality)

12
Requirements Analysis

• Ascertain users needs
• Ensure proper reliability
• Promote appropriate standardization, integration,
  consistency, and portability
• Complete projects on schedule and within budget

13
Ascertain Users Needs

• Define tasks
• Tasks
• Subtasks
• Frequency
• Frequent
• Occasional
• Exceptional
• Repair
• Example
• Difference between a space satellite, car engine,
  and a calculator

14
Reliability

• Actions function as specified
• Data displayed must be correct
• Updates done correctly
• Leads to trust! (software, hardware, information)
  case Pentium floating point bug
• Privacy, security, access, data destruction,
  tampering (CIA)

15
Standardization, Integration, Consistency,
Portability

• Standardization common user-interface features
  across multiple applications
• Apple
• Web
• Windows
• Integration across application packages
• file formats
• Consistency common action sequences, terms,
  units, layouts, color, typography within an
  application
• Portability convert data and interfaces across
  multiple hardware and software environments
• Word/HTML/PDF/ASCII

16
Case Study Library of Congress Database Design

• http//catalog.loc.gov/
• Two interfaces
• Catalog New Books
• 3-6 hour training course - staffers
• Search Catalog of Books
• General public too complex, command language
  and complex cataloging rules
• Solution
• Touch screen
• Reduced functionality
• Better information presentation
• Eventually Web based interface
• Same database and services, different interfaces

17
Usability Measures

• How can we measure the goodness of an
  interface?
• What are good metrics?
• ISO 9241 Usability Standard
• Effectiveness
• Efficiency
• Satisfaction
• Ben Schneiderman Approach
• Time to learn
• Speed of performance
• Rate of errors
• Retention over time
• Subjective satisfaction

18
Usability Motivations (1)

• Time to learn
• Speed of performance
• Rate of errors
• Retention over time
• Subjective satisfaction

• Life-Critical systems
• Applications air traffic, nuclear reactors,
  military, emergency dispatch
• Requirements reliability and effective (even
  under stress)
• Not as important cost, long training,
  satisfaction, retention
• Industrial and Commercial Use
• Applications banking, insurance, inventory,
  reservations
• Requirements short training, ease of
  use/learning, multiple languages, adapt to local
  cultures, multiplatform, speed
• Office, Home, and Entertainment
• Applications E-mail, ATMs, games, education,
  search engines, cell phones/PDA
• Requirements Ease of learning/use/retention,
  error rates, satisfaction
• Difficulties cost, size

19
Usability Motivations (2)

• Time to learn
• Speed of performance
• Rate of errors
• Retention over time
• Subjective satisfaction

• Exploratory, Creative, Collaborative
• Applications Web browsing, search engines,
  simulations, scientific visualization, CAD,
  computer graphics, music composition/artist,
  photo arranger (email photos)
• Requirements remove the computer from the
  experience,
• Difficulties user tech savvy-ness (apply this to
  application examples)
• Socio-technical systems
• Applications health care, voting, police
• Requirements Trust, security, accuracy,
  veracity, error handling, user tech-savvy-ness

20
Universal Usability

• Interface should handle diversity of users
• Backgrounds
• Abilities
• Motivation
• Personalities
• Cultures
• Question, how would you design an interface to a
  database differently for
• A. right-handed female, Indian, software
  engineer, technology savvy, wants rapid
  interaction
• B. left-handed male, French, artist
• Does not mean dumbing down
• E.g. Helping disabled has helped others (parents
  w/ strollers, elderly)
• E.g. Door handles
• Goal Address the needs of more users - unlike
  yourself!

21
Physical Variation

• Ability
• Disabled (elderly, handicapped, vision,
  ambidexterity, ability to see in stereo)
• Speed
• Color deficiency
• Workspace (science of ergonomics)
• Size
• Design
• Field of anthropometry
• Measures of what is 5-95 for weight, height,
  etc. (static and dynamic)
• Large variance reminds us there is great
  variety
• Name some devices that this would affect.
• note most keyboards are the same
• screen brightness varies considerably
• chair height, back height, display angle
• Multi-modal interfaces
• Audio
• Touch screens

22
Cognitive and Perceptual Variation

• Blooms Taxonomy
• knowledge, comprehension, analysis, application,
  synthesis, evaluation
• Memory
• short-term and working
• long-term and semantic
• Problem Solving and reasoning
• Decision making
• Language and communication
• Search, imagery, sensory memory
• Learning, skill development, knowledge
  acquisition
• Confounding factors
• fatigue, cognitive load, background, boredom,
  fear, drugs/alcohol

23
Personality

• Computer anxiety
• Gender
• Which games do women like?
• Pac-man, Donkey Kong, Tetris
• Why? (Hypotheses less violent, quieter
  soundtracks, fully visible playing fields, softer
  colors, personality, closure/completeness)
• Can we measure this?
• What current games are for women?
• Style, pace, top-down/bottom-up, visual/audio
  learners, dense vs. sparse data
• No simple taxonomy of user personality types.
  E.g. Myers-Briggs Type Indicator
• Extrovert vs. introvert
• Sensing vs. intuition
• Perceptive vs. judging
• Feeling vs. thinking
• Weak link between personality types and
  interfaces
• Think about any of applications, and see if user
  personality is important!

24
Cultural and International Diversity

• Language
• Date / Time conventions
• Weights and Measures
• Left-to-right
• Directions
• Telephone no and address
• Names, titles, salutations
• ID, passport no, SSN
• Sorting
• Icons, buttons, colors
• Etiquette
• Evaluation
• Local experts/usability studies

Which one is Fit?
25
Users with Disabilities (1)

• Federal law to ensure access to IT, including
  computers and web sites. (1998 Amendment to
  Rehabilitation Act, USA). In Hong Kong, we have
  e-Inclusion programme.
• Disabilities
• Vision
• Blind (bill-reader)
• low-vision
• color-blind
• Hearing
• Deaf
• Limited hearing
• Mobility
• Learning
• Dyslexia
• Attention deficient, hemisphere specific, etc.
• Keyboard and mouse alternatives
• Color coding
• Font-size

26
Users with Disabilities (2)

• Contrast
• Text descriptors for web images
• Screen magnification
• Text to Speech (TTS)
• Check email on the road, in bright sunshine,
  riding a bike
• Speech Recognition
• Head mounted optical mice
• Eye Gaze control
• Learning what helps those with disabilities
  affects everyone
• Present procedures, directions, and instructions
  accessible to even poor readers
• Design feedback sequences that explain the reason
  for error and help put users on the right track
• Reinforcement techniques with other devices
• Good topics for you final year project!

27
Elderly

• Reduced
• Motor skills
• Perception
• Vision, hearing, touch, mobility
• Speed
• Memory
• Other needs
• Technology experience is varied (How many
  grandmothers use email? mothers?)
• Uninformed on how technology could help them
• Practice skills (hand-eye, problem solving, etc.)
• Touch screens, larger fonts, louder sounds

28
Children

• Technology experts??
• Age changes much
• Physical dexterity
• (double-clicking, click and drag, and small
  targets)
• Attention span
• (vaguely) Intelligence
• Varied backgrounds (socio-economic)
• Goals
• Educational acceleration
• Socialization with peers
• Psychological - improve self-image,
  self-confidence
• Creativity art, music, etc. exploration
• Teenagers are a special group
• Next generation
• Beta test new interfaces, trends
• Cell phones, text messages, simulations, fantasy
  games, virtual worlds
• Requires Safety
• They
• Like exploring (easy to reset state)

29
Accommodating Hardware and Software Diversity

• Support a wide range of hardware and software
  platforms
• Software and hardware evolution
• OS, application, browsers, capabilities
• backward compatibility is a good goal
• Three major technical challenges are
• Producing satisfying and effective Internet
  interaction (broadband vs. dial-up wireless)
• Enabling web services from large to small (size
  and resolution)
• Support easy maintenance of or automatic
  conversion to multiple languages

30
HCI Goals

• Influence academic and industrial researchers
• Understand a problem and related theory
• Hypothesis and testing
• Study design
• Interpret results
• Provide tools, techniques and knowledge for
  commercial developers
• competitive advantage (think ipod)
• Raising the computer consciousness of the general
  public
• Reduce computer anxiety (error messages)
• Common fears
• Ill break it
• Ill make a mistake
• The computer is smarter than me
• HCI contributes to this!

31
Part 2 HCI and Digital Gaming Reference
Chris Nelson, Eric Whalen, Ben Hansen, University
of Wisconsin-Eau Claire
32
Why HCI and Video Games?

• Since Developers and Analysts are always creating
  products for end-users, it is important to know
  what the human factors should be considered for
  the final product.
• Video games serve as a glance into the possible
  future of interface design.

33
Overview

• Productivity Paradox
• What is HCI?
• Past input/output HCI in video games
• Present and Future HCI in video games
• Summary

34
Productivity Paradox

• Economists have found a disappointing lack of
  correlation between the amount of money invested
  in information technology and changes in industry
  productivity measures.
• Gary M. Olson (School of Information, University
  of Michigan)
• Productivity is the fundamental economic measure
  of a technologys contribution.
• Erik Brynjolfsson, MIT
• Has resulted in CEOs and line managers to
  question their huge investments in computers and
  related technologies.
• Erik Brynjolfsson, MIT

35
Paradox?

• The Economist

36
Productivity Paradox (Cont.)

• Labor Productivity is calculated as the level of
  output divided by a given level of labor input
• Erik Brynjolfsson, MIT
• Multifactor Productivity is calculated as the
  level of output for a given level of several
  inputs, typically labor, capital and materials
• Erik Brynjolfsson, MIT
• Typically Multifactor is a better indicator of
  productivity since it adjusts for shifts among
  inputs
• Erik Brynjolfsson, MIT

37
Four Explanations For Paradox

• Mismeasurement of outputs and inputs
• Quality of work life goes up ? wage will go down
• Lags due to learning and adjustment
• Benefits can take several years to show up
• Redistribution and dissipation of profits
• Benefits individual firms, but not industry as
  whole
• Mismanagement of information and technology
• Building inefficient systems
• (by Erik Brynjolfsson, MIT)

38
Human Computing Interaction (HCI)

• The study of interaction between people and
  computers
• Wikipedia
• Relates Information Technology with many other
  fields of study and research
• Wikipedia
• Interaction between users and computers occurs at
  the user interface which includes both software
  and hardware
• Wikipedia
• The major focus of HCI is the design and creation
  of computer systems
• Gary M. Olson (School of Information, University
  of Michigan)
• Goal of HCI is to produce hardware and software
  that is useful, usable, and aesthetically
  pleasing
• Gary M. Olson (School of Information, University
  of Michigan)

39
Interdisciplinary Aspects of HCI

• Aesthetics
• Artificial intelligence
• Cognitive science
• Computer vision
• Design
• Ergonomics
• Library and information science
• Philosophy
• Psychology
• Social psychology
• Sociology
• Speech-language pathology

40
Science of Human Interaction

• Perceptual-Cognitive-Motor process
• Moment-by-moment
• Gary M. Olson (School of Information, University
  of Michigan)
• Distributed cognition
• Technical interaction of teammates
• Gary M. Olson (School of Information, University
  of Michigan)

41
Cognitive Modeling (1)

• Cognitive, Perceptual, and Motor Components in
  the moment-by moment interaction a person
  encounters when working at a computer
• Gary M. Olson (School of Information, University
  of Michigan)
• Contrasts a lot of traditional psychological
  research since, multiple components of behavior
  interact
• Gary M. Olson (School of Information, University
  of Michigan)

42
Cognitive Modeling (2)

• Time it takes to do a task the sum of a number
  of Cognitive Engineering Time Parameters
  including mental and physical acts
• Gary M. Olson (School of Information, University
  of Michigan)
• Typical errors include
• Loss of items from short-term memory
• A motor slip
• Gary M. Olson (School of Information, University
  of Michigan)

43
Cognitive Modeling (3)

• EPIC (Executive Process Interactive Control)

Gary M. Olson (School of Information, University
of Michigan)
44
Distributed Cognition

• Addresses the more social and contextual aspects
  of work that the cognitive modeling line does
  not.
• Gary M. Olson (School of Information, University
  of Michigan)
• Stems from anthropological and sociological
  studies of real-world work situations.
• Gary M. Olson (School of Information, University
  of Michigan)
• Recognizes how peoples actions are intimately
  intertwined with the artifacts of their work
  their team member roles, responsibilities, and
  actions and even their cultural and historical
  setting
• Gary M. Olson (School of Information, University
  of Michigan)

45
Why Use HCI In Video Games?

• As a matter of necessity, game designers must
  create cognitive models and rule sets to guide
  artificial intelligences and bound interactive
  behavior. This makes games a perfect test-bed
  for HCI explorations.
• Keeker, The Untapped World of Video Games

46
Summary

• Productivity Paradox
• HCI increasing usability of systems, in turn will
  increase productivity
• Video game developments are the cutting edge in
  interface design

47
Ergonomics in HCI

• Human anthropometry
• Arrangement of displays and menu controls
• Human cognitive and sensory limits
• Sensory and perceptual effects of CRT and other
  display technologies
• Controls

http//sigchi.org/cdg/cdg2.htmlH3
48
Anthropometry

• measurement of humans
• in physical anthropology, refers to the
  measurement of living human individuals for the
  purposes of understanding human physical variation

http//en.wikipedia.org/wiki/Anthropometry
49
Design

• Display / Environment
• Menu
• Audio
• Video
• Controls
• Keyboards, Mice
• Controllers, Joysticks

50
Menu and Interface

• Menu Based
• Auto-Loading, Non-Linear
• MLB 2005
• Menu Based
• TouristTrophy
• Auto-Loading, Non-Linear
• Still Menu Driven
• GTA Vice City
• Auto-Loading, Non-Linear

51
Output Devices

• Audio
• Mono
• All output on one channel
• Stereo
• Dual channel output
• 5.1 / 7.1
• Multiple channels of output

52
Output Devices (cont.)

• Video
• CRT (cathode ray tube)
• Plasma / LCD

http//en.wikipedia.org/wiki/Cathode_ray_tube
53
Input Devices

• Mechanics of Particular Devices
• Performance Characteristics
• (human and system)
• Handwriting and Gestures
• Stylus, EyeToy, Wiimote, WiiFit
• Speech Input
• SOCOM, etc.
• Eye Tracking
• Military Developed

http//sigchi.org/cdg/cdg2.htmlH3
54
Characteristics of Input Devices

• Weight
• Portability
• Speed / Performance
• IR
• Wireless

http//sigchi.org/cdg/cdg2.htmlH3
55
The Beginnings

• Corded
• Box
• Simple
• D-pad Control

http//wii.advancedmn.com/article.php?artid6355
56
Form Follows Function

• Additional Buttons
• D-pad with 8 way control
• Rounded, better shaped control
• Lighter

http//wii.advancedmn.com/article.php?artid6355
57
Ergonomics are Considered

• First leap to form fitting design
• Feedback through the controller
• More input possibilities
• Analog joystick(s)
• Some of the first wireless

http//wii.advancedmn.com/article.php?artid6355
58
In Good Hands

• More development into design
• Smaller, but heavier
• Available in wireless
• D-pad plus at least one analog joystick
• Analog buttons

http//wii.advancedmn.com/article.php?artid6355
59
Now

• Same Evolution
• Wireless
• Xbox360, PS3 with standard USB
• Also have wireless packages
• Revolution all wireless
• Different Directions

http//wii.advancedmn.com/article.php?artid6355
60
The Five Senses Sight

• Cameras
• PS2 EyeToy

EyeToy Example
http//video.google.com/videosearch?qeyetoy
61
The Five Senses Touch

• Guns
• Game Ex.? Duck Hunt
• Racing Wheels
• Game Ex.? Gran Turismo, NASCAR Thunder
• Pads
• Game Ex.? Dance Dance Revolution
• Guitar
• Game Ex.? Guitar Hero

62
The Five Senses Hearing

• Headsets
• Microphones

Lets Take a Quiz!!!
http//www.pbs.org/kcts/videogamerevolution/arcade
/name/index_flash.html
63
The Five Senses Smell Taste

• Sony Corp. has been granted a patent for beaming
  sensory information directly into the brain.
• The technique could one day be used to create
  videogames in which you can smell, taste, and
  touch, or to help people who are blind or deaf.
• A technique for aiming ultrasonic pulses at
  specific areas of the brain to induce sensory
  experiences

http//www.usatoday.com/tech/news/techinnovations/
2005-04-06-sony-patent_x.htm
64
Past Gaming Sony Playstation2
http//www.angelfire.com/extreme/paulsplace/PS2.ht
ml
65
Past Gaming Microsoft Xbox
http//www.angelfire.com/extreme/paulsplace/x-box.
html
66
Past Gaming Nintendo GameCube
http//www.angelfire.com/extreme/paulsplace/gamecu
be.html
67
Present Gaming Sony Playstation 3

• Processor IBM Cell 3.2 GHz
• Media Type Blu-ray
• Memory/Storage 256 MB RAM Installed
• Sound Dolby 5.1 Digital Surround Sound
• Input/Output 6 x USB, 3 x Ethernet, Wi-Fi IEEE
  802.11 b/g, Bluetooth Capable

Playstation 2
http//ps3.ign.com/articles/614/614682p1.html
http//reviews.cnet.com/Sony_PlayStation_3/4507-64
64_7-31355103.html?tagtxt
68
Present Gaming Microsoft Xbox 360

• Processor IBM PowerPC 3.2 GHz
• Media Type DVD-ROM
• Memory / Storage 512 MB RAM Installed
• Video Output ATI Xbox 360 GPU, 256-bit 2D/3D
  graphics acceleration
• Rendering Rate 500 million triangles/sec
• Sound 16-bit Surround Sound
• Input/Output 3 x USB ports, 1 x Ethernet port

Original Xbox
http//reviews.cnet.com/Microsoft_Xbox_360/4507-64
64_7-31355096.html
69
Present Gaming Nintendo Wii

• Processor CPU IBM Broadway GPU ATI
  Hollywood
• Memory Unknown amount of RAM,512 Mb of Built-In
  Flash Memory
• Input/Output 2 x USB, Wireless Accessible

Gamecube
http//en.wikipedia.org/wiki/Nintendo_Revolution
70
Where we could go from here???

• Virtual Mall?
• Making your own pizza?
• The possibilities are endless!!!

71
Conclusion

• Productivity Paradox
• What Is HCI?
• Past input/output HCI in video games
• Present HCI in video games
• The Five Senses in Video Games
• Future HCI in video games
• Whats Next???

72
References

• ISO 9241 - The ergonomics of human system
  interaction
• http//www.system-concepts.com/articles/updateson
  standards/article0138.html
• Ben Shneiderman - Founding Director of the
  Human-Computer Interaction Laboratory at the
  University of Maryland at College Park
• http//www.cs.umd.edu/ben/
• Blooms Taxonomy
• http//www.officeport.com/edu/blooms.htm
• Myers-Briggs Type Indicator
• http//en.wikipedia.org/wiki/Myers-Briggs_Type_In
  dicator
• Web Accessibility Guide
• http//www.iproa.org/webcare/guide.pdf