Bilingual Dictionaries for Australian Languages: User studies on the place of paper and electronic dictionaries

1
Bilingual Dictionaries for Australian Languages
User studies on the place of paper and electronic
dictionaries

• Miriam Corris, Christopher Manning,
• Susan Poetsch, and Jane Simpson
• Linguistics, University of Sydney
• CS and Linguistics, Stanford University
• http//www.sultry.arts.usyd.edu.au/SULTRY/dict.htm
  l

2
Background

• Dictionaries have long been seen as an essential
  contribution by linguists to work on endangered
  languages more useful than grammars, papers
• Many field linguists produce dictionaries there
  has been an explosion of work in this area in
  recent decades in Australia
• But what is the actual use and usability of
  dictionaries of endangered languages?
• This has been little explored
• Most have been produced without much consultation
  with speakers about their structure

3
Background (2)

• We were interested in innovative computer
  interfaces for electronic dictionaries within
  this domain
• But this required us to investigate the actual
  and potential uses of paper or electronic
  dictionaries for indigenous languages by various
  user groups
• Who might use an electronic interface?
• How might they use it? For what purpose?
• Would they be able to use it?

4
The promise of e-dictionaries

• Why electronic dictionaries?
• There are many needs for EL dictionaries, but
  little money/human labour available
• Can make multiple versions at low cost
• Can provide learner supports, active learning
• Computers are engaging and exciting
• Why not computers
• Cost (of computers)
• Need electricity
• Cant use it in camp, under a tree, in the bath

5
Existing dictionary use studies

• Almost entirely for world languages
• mainly paper dictionaries, though increasing work
  on electronic dictionary usability
• Almost entirely highly-educated people (usually
  university students or dictionary society
  members)
• Studies appropriate to our domain were required
• Though see (Lindstrom 1985, Hansford 1991)

6
(Australian) endangered language (EL) dictionaries

• Usually alphabetical (some thesaurus/semantic)
• All bilingual or trilingual (makers usually not
  speakers)
• Mostly ELLWC organization only sometimes have
  LWC-EL finderlist
• Sometimes organized via semantic domains
• A few have vernacular definitions (incl.
  Warlpiri)
• Bigger ones have examples, structured entries
• Some are little more than wordlists
• Structure has often been taken for granted

7
Endangered language dictionaries

• Reasons for EL-LWC organization
• Useful for speakers of LWC, usually including
  lexicographer, in decoding EL
• Idea of documentation of EL in its own terms
• Symbolic value
• Doesnt always fit with user needs
• Commonly more need for encoding the EL
• Competing goals
• documentation vs. maintenance/learning
• symbolic vs. practically useful organization

8
Testing

• 76 speakers, semi-speakers, and learners of
  Australian Aboriginal languages (from primary
  school age through adult literacy workers)
• Alawa Feb 1999 at Minyerri dictionary
  workshops and task-based activities
• Warlpiri Feb 1999 demonstrations and
  observation May 1999 demonstrations and
  task-based activities Feb 2000 demonstration
  June 2000 task-based activities, educational
  courses, demonstration, and observation
• Warumungu May 1999 and June 2000 courses,
  task-based activities, and observation

9
Orientation

• Warlpiri active bilingual programs in three
  schools. Children learn Warlpiri as their first
  language. Young people can write Warlpiri.
• Alawa children and young adults do not speak as
  first language people are most likely to be
  literate in English, but mainly speak Kriol. LOTE
  program at Minyerri.
• Warumungu children and young adults do not speak
  as a first language some can write it through
  Batchelor College courses. No LOTE program.

10
The dictionaries

• Warlpiri dictionary (unpublished, but various
  xeroxed versions over 2000 pages of material)
  10,000 headwords incl. subentries, lengthy
  definitions, exemplification. Also shorter
  beginners dictionary (100 pages)
• Electronic interface Kirrkirr
• Alawa-Kriol-English dictionary draft, A-K-E main
  dictionary, semantic domain list, K-A-E and E-A-K
  finderlists, about 250 pp A4.
• Warumungu draft Warumungu-English dictionary,
  currently lacks E-W finderlist

11
Current dictionary use

• Observed natural uses of dictionaries
• Finding meanings for old Warlpiri words for
  translation and art documentation
• Making materials for school language programs
• Browsing
• Checking spelling symbolic function
• Factors inhibiting use
• Low levels of vernacular literacy
• Lack of availability of dictionaries not in
  classrooms
• They are not standardly perceived or chosen as a
  source of information about language

12
Sources of information

• One day I was sitting around the table with
  three young women. We had done two two-hour
  sessions on different activities with the
  dictionary, and then I proposed that we should
  find all the Alawa words for different kinds of
  kangaroo. They said, Oh, theres the plains
  kangaroo, . They wrote down all the words in
  Kriol, and then they said, Well go home and ask
  the old people. Even though the dictionary was
  sitting there.
• Susan Poetsch, Alawa, Feb 1999

13
Method

• Qualitative/ethnographic study
• We demonstrated and then observed people using
  paper and electronic dictionaries, and got their
  feedback on them
• We gave people dictionary-use tasks, and observed
  their performance
• Tasks often completed with assistance
• We have attempted to directly compare the
  usability of paper vs. e-dictionaries on tasks
• but confounding factors are too great

14
Rationale

• Language skills, literacy, educational skills,
  life experiences of subjects varied greatly
• In general, very limited literacy, education, and
  dictionary skills, particularly for older people
• Subjects required a great deal of time to
  complete tasks (e.g., commonly 45-60 mins for a
  12 lookup crossword puzzle)
• Subjects often required assistance/training
• No established approaches to indigenous
  dictionary usability pilot studies were needed

15
Task-based testing

• Designed 21 task-based activities
• Constructed assuming low levels of literacy
• In the end many were never used
• Order words alphabetically
• Lookup for filling in crosswords
• Recognizing parts of entries
• Translation understanding long entries
• Sorting by semantic domain using
  cross-references
• Checking spelling

16
Example task Part of Task 4a

• 3. How do you write bark (of a tree) in Alawa?
• 7. How do you write bull ant in Alawa?

17
Results

• We concentrate here on practical considerations
• Learning to use a dictionary people generally
  lacked the necessary literacy and
  dictionary-specific skills to make adequate use
  of dictionaries
• Ongoing opportunities for training are required
• Keeping to conventional dictionary structure
  allows crossover of skills from LWC (English)

18
Alphabetical order

• People know words are ordered, but not how
• A separate skill from literacy often not
  mastered
• A big obstacle to effective lookup
• The flick randomly strategy
• The proceed from the beginning strategy
• Linguists often regard digraphs like ng, ny as
  single letters in alphabetization
• this retards skills transfer from English
• stick to alphabetization of LWC community
  choice
• Effective typography helps cut index?
  expensive

19
Alphabetical order

• E-dictionary avoids reliance on alphabetical
  order by providing multiple routes to dictionary
  information
• typing, fuzzy spelling, lists, links sight
  words
• The ability to type in words was quickly adopted
• can just type a few letters
• In an e-dictionary, it is easy to provide
  multiple alphabetizations, but there is unlikely
  to be value in doing this standardization

20
Comprehensiveness

• Linguists, lexicographers, and sometimes older
  (commonly illiterate) speakers want to include as
  much information as is known
• For most potential users, with emerging literacy
  skills, short entries with illustrations are
  essential
• information overload of large dictionaries is
  overwhelming
• even a shorter dictionary will be a challenge to
  use effectively

21
Comprehensiveness

• Competing needs
• Teachers felt that comprehensive printed
  dictionaries were too long and detailed for
  school use
• But they remarked with frustration that the
  beginners dictionary didnt have enough words to
  be useful
• Suggests a lot of words, but short entries?

22
Macro-structure

• EL-LWC effective for proficient speakers decoding
  hard/old words
• EL-EL dictionary possible or preferred where EL
  literacy is strong(er) but this is rare
• For others, LWC-EL order is more useful, and
  helps improve their English literacy
• This commonly meant that they were using the
  finder list as a dictionary
• Usage didnt match dictionary design a LWC-EL
  dictionary would have been far superior

23
Citation forms

• Some found the concept hard to grasp
• Would have liked inflected forms in dictionary
• E-dictionaries easily can include all inflected
  forms as headwords
• Being able to do morphological analysis is a form
  of learner support

24
Pronunciation

• Lack of confidence in pronouncing words was a
  problem in Alawa, and discouraged use of
  dictionary (what does it sound like?, where is
  the stress?)
• E-dictionaries can solve this through use of
  audio
• some concerns on use of individuals voices
• synthesis may be best, though questionable for
  language learning

25
Extracting relevant info

• Long entries were hard/overwhelming to process
• People want simple entries, even if
  lexicologically questionable
• one word correspondences easiest for users
• even school teachers queasy on notion of word
  senses
• Either need sufficient user training or
• Several versions of dictionaries designed for
  different users
• need spacious clear entries for beginners

26
Microstructure details

• Sub-entries were confusing
• converting them to a crossreference better
• Definitions often overly technical
• People were confused by conventions such as
  reversal in word lists (kangaroo, plains)
• Part of speech abbreviations puzzling read as
  part of definition (minimally put at end)
• Cross-reference abbreviations (syn., ant.)
  confusing
• Small font size a problem (literacy level,
  eyesight)

27
E-dictionary use

• E-dictionaries have novelty, engaging
  interactiveness
• Children willing to spend time with it, even when
  they didnt understand everything
• Some teachers have thought our interface too
  advanced for primary school children, but in our
  experience, they underestimate the facility of
  todays young (indigenous) children with
  computers they manipulate them with ease
• E.g., one (bright) 10 yo returned after a demo
  and played with interface for 2 1/2 hours

28
Teachers

• Generally enthusiastic
• Would encourage kids to learn to read Warlpiri
• Would help teachers to prepare lessons they
  found the interface more efficient to use for
  lesson preparation than a domain-specific
  dictionary extract.
• Felt could use as a basis for classroom
  activities (better with some further development
  games and puzzles)

29
Primary school kids

• One major benefit is that it was on computer.
• It maintained interest
• They were enthusiastic about the computer side
  of things and negotiated the interfaces various
  windows easily
• e.g., wanted back button
• Sometimes, working on sense relations and
  definitions was of less interest than moving
  things around
• Word list and English search was found
  helpful (can compensate for poor spelling)

30
Older children

• More thoughtful had dictionary experience
• Still really liked word list
• Interested in word relationships
• Liked finding pictures and audio
• Students used it voluntarily during lunch breaks
  (confounding factor air-conditioning)

31
Adult literacy workers

• Less interested in graphical interface
• Mainly interested in looking at definitions
• Started discussing and disagreeing with them
  immediately
• although they had and used paper printout of
  dictionary, first real chance to see what was
  there?
• They wanted to, and were able to, annotate the
  definitions with notes

32
E-dictionary links

• Promoting subentries to entries appeared very
  effective People enjoyed exploring and
  explaining relation of derived terms to main word
  (even if sometimes folk etymologies?)
• Synonyms, and alternate forms used and liked
• Concept of antonym seemed confusing
• The semantically uninterpreted cf link was
  confusing

33
User-initiative, computer support

• One of the introductory Warlpiri literacy
  students, who had not been very interested in the
  literacy class, spent nearly 3/4 hour looking at
  Kirrkirr apparently in absorbed concentration.
  She wasnt especially interested in the sound and
  picture possibilities. She moved between words,
  scrolling along the list, typing in the search,
  clicking on the words in the network pane. She
  wasnt even put off when the dictionary
  definitions stopped appearing looking at the
  networks of words instead. After the Kirrkirr
  demo she asked if she could have a printed
  dictionary to take away with her to use in camp
  to learn the words. I interpret this as a desire
  to learn words in her own time and place.
  Jane Simpson Lajamanu May 1999

34
Conclusions

• Regular dictionary users (especially, compilers!)
  grossly underestimate the time they have spent
  becoming familiar with dictionary structure
• If a dictionary is going to be made for a speech
  community, then the people in that community need
  to feel confident in using it.
• Most of our users need a different type of
  dictionary from the current documentation
  dictionaries, (implicitly?) prepared to meet the
  needs of professionals My First Alawa
• Would probably be used by adults as well as kids

35
Conclusions

• E-dictionary showed great promise, though still
  room for improvement
• Make more interactive (more for kids to do)
• Add in more views of dictionary (e.g., search by
  colour)
• But it did show how an electronic dictionary can
  effectively offer multiple pathways and user
  supports in mediating between well-structured
  dictionary data and users needs for
  searching/browsing and presentation