Seminar Topics and Projects

1
Seminar Topics and Projects

• Giuseppe Attardi
• Dipartimento di Informatica
• Università di Pisa

2
Deep Learning Tokenizer

• Depling 2016 challenge requires tokenizer for any
  of the Universal Dependency TreeBank
• Build a DL tokenizer using Keras based on the
  approach of
• Basile, Valerio and Bos, Johan and Evang, Kilian
  A General-Purpose Machine Learning Method for
  Tokenization and Sentence Boundary Detection
  (2013), http//gmb.let.rug.nl/elephant/

3
Deep Learning POS for UD

• Depling 2016 challenge requires tokenizer for any
  of the Universal Dependency TreeBank
• Build a DL POS using CNN, for example a LSTM that
  uses word embeddings and possible charcater
  embeddings.

4
Deep Learning Morph Analyzer

• Depling 2016 challenge requires tokenizer for any
  of the Universal Dependency TreeBank
• Build a DL morphological analyzer that copmutes
  the morphology of each word, using Keras and
  charcaher embeddings.

5
UD extensions

• Write scripts to extract additional relations
  from the analysis of UD parse trees

6
Convolutional Networks for Sentiment Analysis

• Annotated Data SemEval training set
• Unannotated Data 50 million tweets
• Code DeepNL, https//github.com/attardi/deepnl
• Article A. Severyn, A. Moschitti.UNITN Training
  Deep Convolutional Neural Network for Twitter
  Sentiment Classification

7
POS tagging using Word Embeddings

• Data Evalita 2016
• Embeddings http//tanl.di.unipi.it/embeddings/
• Article Stratos, M. Collins. Simple
  Semi-Supervised POS Tagging.http//www.cs.columbi
  a.edu/stratos/research/naacl15semipos.pdf

8
Negation/Speculation Extraction

• Determine the scope of negative or speculative
  statements
• The lyso-platelet had no effect
• MnlI-AluI could suppress the basal-level activity
  
• Approach
• Classifier for identifying cues
• Classifier to determine scope
• Data
• BioScope collection

9
Corpus of Product Reviews

• Download reviews from online shops
• Classify as positive/negative according to stars
• Train classifier to assign score

10
Relation Extraction

• Exploit word embeddings as features extra
  hand-coded features
• Use the Factor Based Compositional Embedding
  Model (FCM)http//www.cs.jhu.edu/mrg/publication
  s/finere-naacl-2015.pdf
• SemEval 2014 Relation Extraction data

11
Entity Linking with Embeddings

• Experiment with technique
• R. Blanco, G. Ottaviano, E. Meiji. 2014. Fast and
  Space-Efficient Entity Linking in Queries.
• labs.yahoo.com/_c/uploads/WSDM-2015-blanco.pdf

12
Extraction of Semantic Hierarchies

• Use word embeddings as measure of semantic
  distance
• Use Wikipedia as source of text
• http//ir.hit.edu.cn/jguo/papers/acl2014-hypernym
  .pdf

Organism
Plant
Ranuncolacee
Aconitum
13
Suggested Topics for Seminars
14
Neural Reasoning

• B. Peng, Z. Lu, H. Li, K.F. WongToward Neural
  Network-based Reasoning
• A. Kumar et al.Ask Me Anything Dynamic Memory
  Networks for Natural Language Processing

15
Question Answering

• Bowl Competition (QANTA vs Jennings)
• https//www.youtube.com/watch?vkTXJCEvCDYk
• Iyyer et al. 2014 A Neural Network for Factoid
  Question Answering over Paragraphs
• IBM Watson
• http//www.aaai.org/Magazine/Watson/watson.php
• TAC
• http//www.nist.gov/tac/2008/qa/index.html

16
Image Understanding

• H. Y. Gao et al. Are You Talking to a Machine?
  Dataset and Methods for Multilingual Image
  Question Answering, NIPS, 2015.

17
Deep Learning Applications

• Character RNNs on text and code
• http//karpathy.github.io/2015/05/21/rnn-effec8ven
  ess/
• Morphology
• Better Word Representations with Recursive Neural
  Networks for Morphology Luong et al.
• Polysemous words
• Improving Word Representa8ons Via Global Context
  And Multiple Word Prototypes by Huang et al. 2012
• Natural language Inference (Logic)
• Question Answering
• Image Sentence mapping

18
Entity Linking

• Entity Kierarchy Embeddings
• http//www.cs.cmu.edu/zhitingh/data/acl15entity.p
  df

19
Deep Learning tsunami over NLP

• C. Manning. 2015. http//www.mitpressjournals.org/
  doi/pdf/10.1162/COLI_a_00239

20
Opinion Mining

• B. Liu. Sentiment Analisis and Subjectivity.
  2010. Handbook of NLP. http//www.cs.uic.edu/liub
  /FBS/NLP-handbook-sentiment-analysis.pdf

21
Semantic Role Labeling

• http//ufal.mff.cuni.cz/conll2009-st/task-descript
  ion.html

22
DL for NLP

• Neural Machine Translation
• D. Bahdanau, K. Cho, Y. Bengio. Neural machine
  translation by jointly learning to align and
  translate.http//arxiv.org/pdf/1409.0473v6
• Natural Language from scratch
• Zhang, X., LeCun, Y. (2015). Text Understanding
  from Scratch.http//arxiv.org/abs/1502.01710