Projectos de ensino do Portugus e traduo falaparafala no mbito do Programa Carnegie MellonPortugal

1
Projectos de ensino do Português e tradução
fala-para-fala no âmbito doPrograma Carnegie
MellonPortugal
Isabel Trancoso
2
L2F Spoken Language Systems Lab
3
Core technologies

• Speech processing
• Text-to-speech synthesis
• Automatic process for building new voices
• Limited domain synthesis
• Expressive speech synthesis
• Audio-visual synthesis
• Automatic speech recognition
• Robust speech recognition
• Speaker adaptation
• Large vocabulary continuous recognition
• Rich transcription of spontaneous speech
• Speech coding
• Speech enhancement
• Speaker and language identification

• Text processing
• Morphological analysis
• Syntactic analysis
• Semantic analysis
• Discourse analysis
• NL Generation
• Named entity extraction
• Information retrieval
• Summarization
• Question answering
• Machine translation

• Spoken language processing
• Speech understanding
• Spoken dialog systems
• Speech-to-Speech machine translation
• Summarization of spoken documents
• Question answering on spoken documents
• Classification of multimedia documents
• Language tutoring
• etc.

4
European Projects (on-going)

• LIREC
• LIving with Robots and intEractive Companions
• VIDIVideo
• Interactive semantic video search with a large
  thesaurus of machine learned audio-visual
  concepts
• I-DASH
• The Investigators Dashboard (Safer Internet)
• COST 2102
• Cross-Modal Analysis of Verbal and Non-verbal
  Communication
• COST 2103
• Advanced Voice Function Assessment
• ECESS
• European Center of Excellence on Speech Synthesis

5
National Projects

• Other varieties of Portuguese
• POSTPORT Project
• Cooperation with Brazil
• Speech-to-speech machine translation
• PT-STAR Project
• Cooperation with CMU
• Computer Aided Language Learning
• REAP.PT Project
• Cooperation with CMU

6
Speech Translation Advanced Research to and
from Portuguese
PT-STAR
PI Luísa Coheur
7
Main tasks
8
Proof of concept

• Prototype development
• Broadcast news
• Lecture domain

9
Cross-lingual voice conversion
TTS
ASR
MT
10
REAP.PT
Computer Aided Language Learning (CALL) Reading
Practice
PI Nuno Mamede
11
Architecture
technologyfrom seed
L2 F - Spoken Language Systems Laboratory
11
12
Reading Session
13
Main Tasks

• Porting REAP to a new language involves several
  tasks
• Adaptation of the interface
• Integration of new linguistic resources
  (dictionaries, web corpora, target word lists)
• Integration of new tools (such as topic
  classifiers)
• Making the necessary adaptations for
  typologically different languages
• REAP.PT is not at all limited to porting to
  Portuguese, involving many open research
  challenges
• Readability
• Cloze question generation
• Oral comprehension

14
Oral comprehension

• Very strong vowel reduction in European
  Portuguese
• Text-to-speech synthesis of any selected text
• Audio playback of pre-recorded audio books
• (automatically aligned)
• Audio playback of broadcast news shows\
• (manually or automatically transcribed)

15
Demos
technologyfrom seed

• http//call.l2f.inesc-id.pt/reap.pt/
• http//www.l2f.inesc-id.pt/

Instituto de Engenharia de Sistemas e
Computadores Investigação e Desenvolvimento em
Lisboa
L2 F - Spoken Language Systems Laboratory
15
16
technologyfrom seed