How to get the best paper award

1
How to get the best paper award

• Hans-Dieter Burkhard
• Humboldt University Berlin

2

• I dont know
• (but it happens)

3
What is a good paper?

• Would you read it?
• Why
• Interesting topics
• Surprising results
• Nice story
• What is the message?
• What to learn from the paper?

4
What is a good paper?

• Examples
• New efficient quality measure in SE.
• New facts about neural control in animals.
• A first correctness proof.

5
What is a good paper?

• Who decides
• Reviewers of journals/conferences
• Readers (citations of the paper)

6
What is a good paper?

• Typical criteria of reviewers
• Contents
• Contribution
• Originality/novelty
• Significance to theory and practice
  
• Relevance to call of papers/to journal

7
What is a good paper?

• Typical criteria of reviewers
• Technical aspects
• Technically sound
• Readability and organization
• Appropriate length of the paper
• Content style and clarity of writing
• Quality of English

8
Highly ranked papers

• Articles in main journals
• (what are the most important journals in your
  research field?)
• Contributions in main international conferences
• (what are the most important conferences in your
  research field?)
• What is the acceptance rate

9
Acceptance rate

• From a mail by the program chairs
• We have almost completed the decision phase for
  the 531 papers that we received. We have
  followed the recommendations in 96 of the
  cases. There have been 21 papers out of 531
  decisions where we changed between reject
  poster or poster - accept compared to
  recommendations we received. In each case, it
  was done only after reading all reviews, the
  discussion, and the paper itself
• Accepted 119
  (22)
• Posters 133
  (25)
• Rejected 277
  (52)
• Transferred to Industrial Track 2

10
Highly ranked papers need time

• Leading researchers can produce 1-2 high quality
  papers per year
• Result of work in their group
• Result of discussions with others

11
Other papers

• Other (peer reviewed!) journals
• Other (peer reviewed!) international conferences
• National journals
• National conferences
• Workshops

There are so many conferences/journals
Quality Check Which kind of proceedings (e.g.
by Springer)
12
Why workshop papers?

• Workshop papers are not highly ranked.
• But
• Dont start publication at the highest level
• Early presentation of your work
• Discussion critics and information by others
  (hopefully by reviewers)

13
What about books and book articles?

• Large differences in ranking
• Fundamental book (cited by everyone)
• yet another textbook (among 100
  others)
• (What are the standard books in your field?)

14
Organization of conference reviews

• Program Chairs invite
• members of the Program Committee (PC)

15
Organization of conference reviews
Dear Anton, The 5th International Conference on
Multi-Agent Systems will be held in Leipzig
(Germany), the 25-27 September 2007. CEEMAS
conferences (http//www.ceemas.org/) take place
every second year in the Central and Eastern
European region. The programme committee of the
conference series consists of well known
researchers from the region and renowned
international colleagues. The conference
proceedings are planned to be published by
Springer Verlag in the LNAI series as
before. In view of your expertise on the field,
the Chairs would like to invite you to take part
in the Program Committee of the conference. In
addition to the task of reviewing a few papers in
April/May 2007, your duties as a PC member will
be to actively promote the workshop and to
solicit paper submissions from colleagues. Please
let us know if you are able, as we hope, to
accept this invitation. Best regards

• Chairs invite Program Committee (PC)

16
Thank for the invitation unfortunately I am
totally overloaded right now and so much decline.
I will be honored to join the PC as it was the
case for the previous ones. Please count on me.
Thank you for the invite. I will be happy to
help out on the PC.
Sure! What a pleasure to join you, my dear
friends.
17
Organization ConfMaster System

• Web based system (same as for authors)
• Authors upload their papers
• Reviewers can download assigned papers
• Each paper is usually assigned to 3 PC members
• PC members usually get 4-10 papers to review
• PC members ask their colleagues, assistants
• (look for the list of reviewers in the
  proceedings )

18
How papers are assigned to reviewers

• By Chairs only.
• By competencies
• PC members indicate their interests/expertise by
  given keywords
• By bidding for papers
• PC members indicate their interests/expertise
  based on paper abstracts

19
Format of reviews

• Fixed Attributes
• Ranking, e.g. between 1 5 (best)

Example Evaluation of work and contribution
4 Significance to theory and practice
4 Originality novelty
3 Relevance to the call of papers
3 Readability and organization
4 Overall recommendation
4 Reviewer familiarity
4
The important one
20
Format of reviews

• Additional text fields

-- Comments to the author(s) I was trying
to find why it is so important to build
coalitions and in what dynamic environments
you are trying to find them. Is it for a
plant? What are the application fields of
your coalition formation? What is an example for
a noisy environment? -- Comments to the
PC -- Summary
21
Remarks of the reviewers

• General remarks
• Generally, I dont understand what the
  intention of the authors is. Why do they call it
  data mining if, as I understand, it ends up as a
  specific way of searching? Also, the paper does
  not mention any related work, most notably from
  the case-based reasoning area where applications
  of that kind have been developed more than 10
  years ago.
• Contributions of the paper
• (here you can read what the reviewer has
  understood)
• What are the ways in which the paper should be
  improved?
• Firstly, name the problem precisely. Do you
  address search or data mining? If the latter,
  what is the result of the data mining process?
  Investigate related work and then describe what
  really is unique and new.

22
Remarks of the reviewers

• The idea of using relative information between
  objects/landmarks
• is not new. At least, it has been used in SLAM
  (Simultaneous
• Localization and Mapping) for years. See the line
  of research on
• Relative Mapping (e.g. Smith et al., 1990,
  ).
• In the paper, it does not describe if the
  observations are
• communicated to ALL robots (in the team). If it
  is the case, would
• it be better to have a central control where the
  observation are
• collected/computed and results send to all robots
  ?
• Detailed comments are as following
• "Line point" is mentioned many times in the
  paper. I do not
• understand. It is true that the robot observes
  line segments ?
• I do not understand the figures 4a. Please
  explain in more details.
• The figure 5b is too small to see. Please make it
  larger and
• explain in a better way. Please clarify the last
  paragraph of page
• 3, column 1.

23
Final decision

• By Chairs only
• (mainly based on overall evaluation)
• By discussion between reviewers (try to find
  consensus)

Overall evaluation 1 strong reject 2 weak
reject 3 balanced 4 weak accept 5 strong
accept
24
Tight schedule (!) for conferences

• Bidding Period January 22 to February 1PC
  Members/Reviewers notified of their allocation 5
  FebruaryDue dates for referee reports March
  2Consensus Discussions March 2 to 12Author
  notification March 23
• (leaves some time for preparing final version)

25
Best paper award

• Candidates
• Best ranked papers
• Proposals of PC members
• Decision (vote) by all PC members
• Inofficial rule The next year award is often not
  given to same topic, country, university,
• Award is subject to scientific/political goals

26
What you can learn by reviews

• What do others think about your work
• References to other interesting work
• Open questions in your paper
• Presentation problems
• Scientific problems

27
Writing papers

• You should already have some results
• The good case
• Problems become more clear while writing
• The bad case
• You still dont understand what you are writing
• NEEDS TIME!!!
• Write it two times
• (with a delay of 4 weeks or more)

28
What are interesting topics?

• Exploration of new fields
• (may arise by practical questions)
• Main stream? Private research field?
• (who reads your paper)

29
What are interesting topics?

• Improvements of other work
• New insights
• Better estimations, better experiments,
• Fewer preconditions
• Corrections
• Interdisciplinary Go to other communities

Why should someone read my paper?
30
What a paper should contain

• What people (reviewers) read first
• Abstract, introduction, conclusion, references
• Figures (readability!), tables
• They want to get an idea
• What are the main contributions?
• For their later reconsideration (for fixing
  ranking)
• What were the main contributions?

31
What a paper should contain

• Good structure
• outline/explanation in the introduction
• Clear (formal) Definitions and Results (!)
• If appropriate Evaluation by experiments
• with discussion!
• Related work what is new in your paper
• Open problems
• Dont forget to give the credits to other work

32
Credits, Acknowledgements, Citations

• Strict honesty with regard to contributions of
  partners, competitors and predecessors

33
How to find relevant papers

• Libraries
• References in other papers
• Citation indices
• Links to citations of a paper
• Citeseer
• Google Scholar

34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
Use of citation indices

• Investigation of more recent work (citing
  articles are newer than the cited article)
• Relevance of work (count of citations)
• Additionally Who reads your papers?

39
Technical aspects

• Given formats
• paper size
• pages
• citation styles
• Works best with LaTeX
• Use the correct terminology
• Include a nice story

40

• I. INTRODUCTION
• The classic example of negative information
  was described in the Sherlock Holmes case Silver
  Blaze. In this case, a house has been broken
  into. Under such circumstances, one would expect
  the watch-dog to bark. The curious incident of
  the non-barking of the dog in the nighttime
  provides Holmes with the information that the dog
  must know the burglar, allowing him to solve the
  case. Applied to mobile robot localization, this
  means that conclusions can be drawn from expected
  but actually missing sensor measurements 5.

41
Open problems
will be left for future research

• What can it mean to the reader
• You didnt like to work more on it?
• Reviewer answers (hopefully not)
• Present the paper if you are finished.
• But of course Be honest with respect to unsolved
  questions and further plans.
• Why is it unsolved?

42
Co-authorship

• Cooperative work can help a lot.
• Authorship
• all cooperating people
• order by (inverse) alphabet or better by
  contribution
• (clarify role/contribution of co-authors)
• Co-Authorship by recognized people can help to be
  accepted
• Inclusion of the professor (if she/he agrees)?

43
How to be visible by others

• Be present on the Web
• provide papers for down load
• Use appropriate key words
• Discuss with others on the conferences
• Join scientific networks!

44
Write more papers

• Why?
• want to have larger publication list
• institution pays travel expenses
• Consistently question your own findings
• Co-authoring
• Reuse of papers ?
• cut-and-paste-papers ?

45
If the paper was rejected

• Consider it as in sports
• a step to future success.
• referees are also humans.
• learn from comments (hopefully they explain the
  reasons).

46
If the paper was accepted

• Final version
• Follow the hints of reviewers.
• Journals further review after revision.
• Conferences no time for further review.
• Exception after conference proceedings
• Technical aspects
• No problem with LaTeX styles.

47
Presentation at conference

• Keep the time
• Prepare parts (proofs, remarks, explanations,
  slides, )
• near the end that you can easily
• add for remaining time
• skip for missing time
• Preparation
• Test the talk with clock and audience
• (with audience it will take more time than
  expected)

48
Presentation at conference

• Technical aspects
• Common Fonts, not less than 24 points
• Color
• consider physiological aspects
• and projector may change color
• Sufficient size of figures
• (readable inscriptions!)
• Parts of programs (implementation details) are
  usually not interesting and not readable

Can you read ?
red on blue ?
yellow on white?
49
Presentation at conference

• Get the interest of the audience by
• (a lot of papers are presented!)
• if possible recall other talks from the session
• Discussion
• Source of new questions for future work
• If you are lucky You will be invited for common
  work

50
Poster-Presentation at conferences

• Highly competitive
• Try to get attention
• (provocative) thesis
• interesting picture
• Stay at your poster for explanation/discussion
• Keep ready business card
• and printed papers (if available)

51

• Good luck!

52
After discussion

• It might be the case that I talked too much about
  introduction and conclusion, but not enough about
  the main part in between. It is like eating The
  proof of the meal is to eat. The proof of the
  paper is to read. But to get hungry, you need a
  good guess
• And if you write honestly the introduction and
  the conclusion, then you will learn a lot what
  your paper is really like.

The important thing is not to stop questioning.
(A. Einstein)
53
(No Transcript)
54
Another topic

• RoboCup 2009
• will be in Graz (Austria)
• i.e. not so far from here

55
Another topic
By the way Not only Soccer Robots, but Rescue
Robots, Robots for HomeGarden