Project Topic 2: Virtualized Classroom

1
Capstone Vision, Video and VR
CSC 59866CD Fall 2004

• Lecture 3
• Project Topic 2 Virtualized Classroom

Zhigang Zhu, NAC 8/203A http//www-cs.engr.ccny.cu
ny.edu/zhu/ Capstone2004/Capstone_Sequence2004.ht
ml
2
Virtualized Classroom  Automated Production
Media Integration and User-Customized
Presentation

• Zhigang Zhu1,2, Chad McKittrick1 and Weihong Li2
• Video Computing Laboratory
• Department of Computer Science
• 1City College and 2Graduate Center
• The City University of New York

3
Outline

• Background
• A Bigger Picture
• Multimedia Virtual Classroom (UMassCCNY)
• CCNY Virtualized Classroom
• Automatic capture
• intelligent media integration
• user-customized presentation
• Conclusions and Discussions

4
hypothesis, strategy motivation
pedagogy
5
In cooperation with

• Ellen Smiley
• Dept of Psychology, School of Social Science
• City College of New York
• W. Richards Adrion
• James F. Kurose
• Edward M. Riseman
• RIPPLES Group and Vision Lab
• University of Massachusetts Amherst
• Dr. Parviz Kermani
• e-Learning Lecture Navigator (ELN)
• Polytechnic University of New York
• and IBM T.J. Watson Research Center

6
CD-MANIC the best of both worlds
UMass Multimedia Asynchronous Networked
Interactive Courseware

• CDs
• for bandwidth intensive content
• internet for
• content updates
• logging student behavior
• collaboration
• search
• external links
• overcomes any bandwidth constraints provides
  convenience
• contents dont change often
• use anywhere
• planes, trains, behind firewalls

7
CUNY and CCNY

• CUNY 20 colleges
• Ph.D. programs in GSUC
• CCNY 20, 000 students
• MS programs only
• Minority Serving Institute
• In 2002, URM 66 in engineering, 35 in CS
• US News 2003 No 2 in diversity
• large percentage of work students
• Questions
• Effective learning tools for recruiting and
  retention

8
Outline

• Background
• A Bigger Picture
• Multimedia Virtual Classroom (UMassCCNY)
• CCNY Virtualized Classroom
• Automatic MM capture
• intelligent media integration
• user-customized presentation
• Conclusions and Discussions

9
Multimedia Virtual Classroom

• Automatic data collection and indexing
• Audio/video
• PPT, handwriting
• online materials, etc
• Collaborative Learning Environment (CLE)
• Active
• Coversational
• Constructive reflective
• Intentional contextual

10
MVC as an active CLE

• active
• Goal active interface, navigation
• base index, simple search
• extended search, active objects (text,
  images)
• learner-controlled interface

11
MVC as an conversational CLE

• conversational
• goal support collaboration
• base threaded discussions
• CLIMANIC

12
MVC as a constructive reflective CLE

• constructive reflective
• goal notation, collaboration
• base none
• notes on slides, threaded discussions, save
  CLIMANIC collaborations, integrated reporting

13
MVC as an intentional contextual CLE

• intentional complex contextual
• goal problem-focused pedagogy active classroom
  strategy, realistic problem domains
• base problem-focused learning
• CASA, on-campus distance learning
  integration, design tools environment
  integration

Integrated Tools
14
MVC supporting learner logger

• important tool for assessment
• goal assessing individual and course learning
  goals
• base quizzer, logger
• integrate with new tools extended search,
  notation, collaboration
• extend to integrated design tools environment
• integrate with CMS (OWL, WebCT, Blackboard)

15
Outline

• Background
• A Bigger Picture
• Multimedia Virtual Classroom (UMassCCNY)
• CCNY Virtualized Classroom
• Automatic MM capture
• intelligent media integration
• user-customized presentation
• Conclusions and Discussions

16
CCNY Virtualized Classroom

• Automated MM data capture
• Sensing, synchronization and recording
• Intelligent MM data integration
• Video, PPT slides, Mimio handwriting pages
• Flexible User Interface
• Course adaptive, user customized
• Virtualized Classroom (VC)
• A new e-learning environment generated and
  enhanced from materials captured from real
  classrooms.

17
Outline

• Background
• A Bigger Picture
• Multimedia Virtual Classroom (UMassCCNY)
• CCNY Virtualized Classroom
• Automatic MM capture
• intelligent media integration
• user-customized presentation
• Conclusions and Discussions

18
VC Automatic MM capture
19
Outline

• Background
• A Bigger Picture
• Multimedia Virtual Classroom (UMassCCNY)
• CCNY Virtualized Classroom
• Automatic MM capture
• intelligent media integration
• user-customized presentation
• Conclusions and Discussions

20
VC Media Integration

• PPT_at_ Slide Mimio page registration
• Pre-calibration
• Dynamic calibration
• Panoramic Video Mosaicing
• From a panning camera
• PPT_at_ Slide video image registration
• Toward a 3D virtual environment

21
VC Media Integration (1)

• PPT_at_ Slide Mimio page registration

A simple approach mark four corners
22
VC Media Integration (1)

• PPT_at_ Slide Mimio page registration

A simple approach mark four corners
23
VC Media Integration (1)

• PPT_at_ Slide Mimio page registration

Question What if one of the devices move?
24
VC Media Integration (1)

• Dynamic registration of PPT_at_ Slide Mimio page
  because
• Hard to mark in some cases
• due to occlusion and height
• Instructor may forget to do it
• Devices may not be able to keep stationary
• Camera, projector, Mimio capture bar
• completely automatic and mind-free approach

25
(No Transcript)
26
VC Media Integration (2)
Panoramic mosaic from video
27
VC Media Integration (2)
Panoramic mosaic from video
28
VC Media Integration
Toward an immersive VC interface
Question Window, ToC, etc
29
VC Media Integration (3)
Replacing the slide projections
30
VC Media Integration (3)
Image Alignment
From Video
From Digital Slide (PPT)
31
VC Media Integration (3)
Image Alignment
32
VC Media Integration (3)
Instructor Extraction
33
VC Media Integration (3)
Instructor Extractionraw
34
VC Media Integration (3)
Instructor Extractionrefined
35
VC Media Integration (3)
Instructor Extractioncontour
36
VC Media Integration (3)
Instructor Extractionand integration
video
image
shadow
contour
37
VC Media Integration
Toward an immersive VC interface
Question Window, ToC, etc
38
Outline

• Background
• A Bigger Picture
• Multimedia Virtual Classroom (UMassCCNY)
• CCNY Virtualized Classroom
• Automatic MM capture
• intelligent media integration
• user-customized presentation
• Conclusions and Discussions

39
User Customized Presentation

• Virtualized Classroom Presentation System (VCPS)
  in Java, consists of
• An authoring tool VCPS creator
• An presentation interface - VCPS player
• Features
• Customized sizes and locations
• Pop up ToC for indexing
• Synchronized play, next, last

40
User Customized Presentation

• PPT panoramic view popup ToC

41
User Customized Presentation

• PPT Mimio integrated Popup ToC

42
User Customized Presentation

• PPT Mimio Combined images New Buttons

43
User Customized Presentation

• PPT Mimio Combined images New Buttons

44
User Customized Presentation

• PPT Mimio Combined images New Buttons

45
User Customized Presentation

• PPT Mimio Combined images New Buttons

46
User Customized Presentation

• PPT Mimio Combined images New Buttons

47
User Customized Presentation

• PPT Mimio Combined images New Buttons

48
Outline

• Background
• A Bigger Picture
• Multimedia Virtual Classroom (UMassCCNY)
• CCNY Virtualized Classroom
• Automatic MM capture
• intelligent media integration
• user-customized presentation
• Conclusions and Discussions

49
Conclusions and Discussions

• CCNY Virtualized Classroom
• automatic MM capture
• intelligent media integration
• user-customized presentation
• Multimedia Virtual Classroom (UMassCCNY)
• Active and collaborative learning
• Content-based searching and indexing
• Pedagogy and assessment

50
Demos

• CCNY Virtualized Classroom System
• UMass CD-MANIC with enhanced searching

51
Capstone Project

• Java and Java 3D
• Sun ONE Studio 4 CE
• Java 3D has a high level implementation of OpenGL
• http//java.sun.com/products/java-media/3D/
• Technical Aspects Do and Dont
• Interface 2D or 3D
• Video, and Omnidirectional Video
• Indexable, Searchable pages for active learning
• Learner log for evaluation
• Internet enabled vs Standalone
• Reporting and Publications
• Conference and Journal Papers

52
References

• Z. Zhu, C. McKittrick and W. Li, Virtualized
  Classroom  automated production, media
  integration and user-customized presentation, The
  4th Int. Workshop on Multimedia Data and Document
  Engineering (with CVPR04), July 2, 2004,
  Washington DC
• W. Li, H. Tang and Z. Zhu, Automated registration
  of high-resolution images from slide
  presentations and whiteboard handwritings via a
  low-cost digital video camera. The 2nd IEEE
  Workshop on Image and Video Registration (with
  CVPR04), July 2, 2004, Washington DC
• W. R. Adrion, J. Kurose, E. Riseman, Z. Zhu, The
  Multimedia Virtual Classroom Developing and
  Evaluating A Constructivist Learning Environment,
  Fifth International Conference on Information
  Communication Technologies in Education, July
  1-3, 2004, on Samos Island, Greece

Links of the first two are on Prof. Zhus
Publication List
53
Next

• After Class
• Update the system design
• Next Lecture
• Visual Sensors