Expert Group: System Design and Evaluation

1
Expert GroupSystem Design and Evaluation

• Millie ABELL,
• Ron A. COLE, GAO Liming,
• JIANG Minghu (Leader of Chinese Experts),
• LIU Dailin, Bill NELSON, WANG Qiong,
• YOU Yue, ZHANG JianMin, Tanja SCHULTZ
  (Leader of American Experts)

2
Outline

• Answers, Comments, and Suggestions to the
  questions asked
• Develop the platform
• Develop the software
• Evaluate the system
• What can be done, What could be done
• Challenges

3
Task

• Target user middle school (12 yrs and up)
• Focus Students
• Scenario Olympic Games 2008 in China

4
Client / Server Design
ELLS Central Server
Distributed Server
Distributed Servers
School Servers
Clients
5
Platform
6
Develop the Platform Q1

• How do we integrate the four functions of the
  ELLS (delivery, communication, feedback, and
  management) into a virtual learning environment?
• Delivery
• scalable system from high-speed internet to stand
  alone (CD/DVD)
• Communication
• central server - broadcasting to local server
  (updates)
• local server - client
• Management
• Hierarchical structure of servers Global -
  central - local
• Teacher manages his/her own students on school
  level
• Storage of studends data on the local server
  level
• Feedback
• Within session feedback on the client side
• Between sessions upload to local server

7
Develop the Platform Q2

• How do we accommodate different bandwidth in one
  system?
• Flexible system which is scalable according to
  the bandwidth available
• ranging from high-speed internet connection to
  stand alone system using CD/DVD
• CD/DVDs needed to be available for all kinds of
  Operating Systems (Windows, Unix/Linux, Mac)
• Low speed connection at home one solution could
  be downsizing the applications (text instead of
  speech)
• Future development highspeed connections
  everywhere

8
Develop the Platform Q3

• What browser should the system be built for?
• We strongly suggest to build the platform
  independent from the Operating System
• gt ALL browsers will be supported
• Netscape, Internet Explorer, ...
• Augmented communication
• Peer to Peer
• What tools? Netmeeting? IP6? Multicasting?
• We suggest to talk to providers in order to make
  sure that tools are available which allow
  peer-to-peer on all kinds of platforms

9
Develop the Platform Q4

• How do we ensure that the Chinese and American
  efforts develop a single compatible ELLS
  platform?
• following the established standards
• CMI
• Integrated project
• Establishing a strong cooperation
• Encourage communication
• Design together

10
Goal Develop the Software

• Q1 Is the summary of the capabilities accurate?
• Q2 What technologies are the most effective
  for teaching and how can they be integrated?
• Q3 What kind of technology can we affort?
• Q4 What are the roles of on-line mentors and
  on- site teaching personnel and how are their
  roles integrated into the ELLS?

11
Develop the Software Q1

• Is the summary of the capabilities accurate?
• Synthesis okay
• Speech recognition for small sentences, problem
  of non-accented speech, children, NLP not good
  for Chinese, incomplete in terms of video and
  animation, more tech available in NLP which is
  not described at all,
• Our suggestion Set up Paper about what
  technology can do.

12
Develop the Software Q2

• What technologies are the most effective for
  teaching
• speaking, listening, reading, and writing and how
  could they be integrated
• SR for speaking, reading and feedback, video,
  TTS, web is important, pentabloid, dynamically
  change speed of speech, animation
• and how can they be integrated?
• Integration in terms of platform client server
• Download of RT applications to client
• High bandwidth to support Audio and video
• Focus on high bandwidth solutions

13
Develop the Software Q3

• What kind of technology can we affort?
• Very limited resources in terms of time and money
  - very little
• Benchmark 10 times more than originally
  estimated
• Research

14
Develop the Software Q4

• What are the roles of on-line mentors and on-site
  teaching personnel and how are their roles
  integrated into the ELLS?
• Role is to give feedback,
• Tandem solution nice but problem of unbalanced
  system
• Chinese speakers in US
• Computer assisted or fully automatic feedback
• Teach on-line mentors to give the right feedback.

15
Evaluate the Software

• How do we build in an effective system of pilot
  testing to ensure that the components work as
  designed?
• participatory design work out specs together
  with content and pedagogic people, build a
  prototype, invite potential users to use the
  system, do quantity and quality evaluation
• Formative and summative evaluation
• have independent evaluators,
• cost effectiveness, measures of effectiveness
  students performance, students perception,
  teacher perceptions asked by questionaires,
  quantity and quality evaluation
• suggestion refer this to people who know more
  about, too short of a time frame to do long-term
  evaluation of learning effects

16
CU Communicator Galaxy architecture