Review%20Process%20-%20How%20to%20review

1
Review Process - How to review

• Fausto Giunchiglia
• By Fausto Giunchiglia and Alessandro Tomasi

2
Index1. Review Form 12. Review Form 23.
Answer to the Reviews4. Review Process
3

• 1. Review Form 1

4
1. Review Form 1

• REVIEWER CODE
• 1) Title
• 2) Author(s)
• 3) Paper Summary short description of the
  message and maybe of how it has been developed.
  What if more than one message or no message?

5
1. Review Form 1

• 4) Type of Paper/Research described
• Chose one of the options below and explain
• Research (with original results) of which kind
  
• (Theory, expemerimental, ...)
• Application (case study, ...)
• Synthesis of recent advances
• Other. Please specify

6
1. Review Form 1
5) General Ratings Rate within
Bad/Weak/Fair/Good/Excellent 0/1/2/3/4/5 Put
the score and some text motivating your score
about 5a) Relevance (with respect to the
reference community) 5b) Originality
(incremental, new work, ...) 5c) Significance
of the work (how big the gap from the state of
the art) 3d) Technical soundness 3e)
References 3f) Presentation
7
1. Review Form 1
6) Technical Soundness Chose one of the options
below and some text motivating the choice -
Technically correct - Minor errors (indicate
them) - Major errors (indicate them) -
Unsupported claims (provide a detailed
explanation)
8
1. Review Form 1
7) Presentation Rate within Yes/Somewhat/No Pu
t the score and some text motivating your
score 7a) Are the title and abstract
appropriate? 7b) Is the paper well-organized
(discuss course and fine grained structure)? 7c)
Is the paper easy to read and understand? 7d)
Are figures/tables/illustrations sufficient? 7e)
Is the English acceptable? 7f) Is the paper free
of typographical/grammatical errors? 7g) Is the
reference section complete?
9
1. Review Form 1
8) General Recommendation Please, chose one of
the options below - Very strong accept
(beautiful paper!) - Strong accept (excellent
and important contribution) - Weak accept (good
paper, some new interesting ideas) - Weak reject
(marginal, weak content, would require a major
revision) - Strong reject (unreadable, nothing
new, premature, contains major errors)
10
1. Review Form 1
9) Main Reason for your Decision For accept
choices please indicate one of the options below
- accept because of the originality (good
ideas, sound presentation) - accept because of
the quality of the proposed synthesis (useful
review on recent advances) - other For reject
choices please indicate one of the options below
- reject because it is not relevant for the
conference - reject because of the presentation
(unreadable, unstructured) - reject because the
content is too premature for really making sense
- reject because of the lack of originality
(results already known, or similar overview
already published) - reject because of major
errors
11
1. Review Form 1
10) Your Level of Expertise (Compared to Level of
Others) - I am an expert of the field and know
the relevant literature - I understand the
problem, I know some of the state of the art - I
only have a superficial understandings of the
issues 11) Does the paper qualify for the best
paper award Y/N 12) Comments to the Author(s)
Please, provide here a clear justification of
your ratings, in particular with regards to the
overall recommendation
12
1. Review Form 1
13) Additional Comments to the Author (after
circulation of reviews among reviewers) (Can be
empty, cannot change previous review) 14)
Additional Comments as Answer to Author's Answer
and Modifications (Can Be Very Short -
Evaluation Of Author Answer - Value Judgement
- Final Score (Possibly Changed))
13

• 2. Review Form 2

14

• 2. Review Form 2

REVIEWER CODE 1) Title 2) Author(s)
15

• 2. Review Form 2

3) Main Message Relevance How relevant is the
paper to the workshop? 0 not relevant at all
1 rather not relevant 2 relevant 3 very
relevant Technical Quality What is the
technical quality of the paper? 0 really bad
1 bad 2 good 3 really good
16

• 2. Review Form 2

Presentation What is the overall presentation
of the paper? 0 really bad 1 bad 2 good
3 really good Overall Ranking What is your
overall recommendation? 0 strong reject 1
reject 2 weak reject 3 weak accept 4
accept 5 strong accept
17

• 2. Review Form 2

Confidence Reviewer's expertise in the area 0 I
know little about this area 1 I know enough
about this area 2 I have good expertise in this
area Why to accept? What are the most
important reasons to accept this paper? (1-3
sentences) Why to not accept? What are the
most important reasons NOT to accept this paper?
(1-3 sentences) Comments Detailed comments on
the paper (primarily for the authors)
18
3. Answer to the Reviews
19
3. Answer to the Reviews
ltBrief introductiongt LIST OF ltgeneral comment
quoted from reviewsgt ltyour answer arguing how you
have accordingly modified the papergt Moving
now to the more specific comments LIST OF
ltspecific comment quoted from reviewsgt ltyour
answer arguing how you have accordingly modified
the paper, providing detail but not too
muchgt ltConcluding sentencegt
20
4. Review Process
21
4. Review Process
0) Abstract (send it to tomasi_at_dit.unitn.it not
later than a week before the presentation) 1)
Presentations (Fortunate situation where you may
know of what the paper is about) 2) All Papers
Submitted by June, 13th 3) Review Allocation (by
Program Chair) by June, 20th 4) Reviews Due by
July, 4th 5) Reviews circulated to Reviewers for
additional comments 6) Reviews send to Author by
July, 11th 7) Author sends back answer and
modified paper by July, 23rd 8) Reviewers
provide final answer 9) Exam Pass/No Pass (of
Authors and Reviewers) by July, 29th
22
4. Notice!!!
In the paper you have to use the same version of
the stylefiles. It is available by downloading it
from http//www.springeronline.com/sgw/cda/frontpa
ge/0,11855,3-111-2-124365-0,00.html Avoid using
stylefiles acquired from other sources as these
may not be correct
23
How to dopresentations
24
Structure

• 1. Presentation methods
• 2. Attitude towards the audience
• 3. How to select content
• 4. How to structure
• 5. Introduction
• 6. Stage fright
• 7. Main part

25
Structure

• 8. Using pictures
• 9. Visual aids
• 10. Interposed questions, interruptions
• 11. Final discussion
• 12. The end
• 13. Optical acoustic factors
• 14. Ancillary conditions

26
1. Presentation methods

• Rule No. 1
• Control effect
• Rule No. 2
• Integrate audience

27
1. Presentation methods

• Seminar
• Lecture
• Presentation with final discussion
• Presentation with intermediate discussion

28
Comparison of some presentation methods
Time to prepare / h
What remains
80
4,0
Open discussion
Presentation with
Intermediate
60
2,5
discussion
Presentation with
30
1,7
Final discussion
Lecture
20
1,0
29
2. Attitude towards the audience

• Who is the audience
• What do they know
• What are they interested in
• What do they understand

30
2. Attitude towards the audience

• Contact with eye
• Simple and understandable languages
• Concentrate on the important
• Credibility
• Reply to objections
• Avoid techno language

31
3. How to select content

• How much?
• From where?
• What is interesting?
• What is the objective?
• What should the audience do?
• Do not want too much!

32
4. How to structure catch words

• Speak as you think follow the thinking
• Use written concept of catch words

33
5. Introduction

• Try to find common understanding with the
  audience
• Lead to the subject
• Initially create attention

34
5. Introduction

• Say at the beginning
• The structure
• The length
• Invite to a dialogue
• But at the beginning
• Who are you

35
attention
Hope that it will end soon
Length of presentation
36
6. Stage-fright
Afraid to speak

• Is natural!
• Everybody has it !
• Is not forever!

37
Reduce Stage-fright

• Be well prepared!
• Learn important parts by hard!
• Relax!
• Look for a positive point
• Do some contacts before presentation!

38
Stucked?
39
7. Main part

• Introduction
• Main part
• Current situation
• Potential solution
• How to realise
• End

40
8. Using pictures / figures
Picture

• Take from the context of the audience
• Has to increase the message
• To explain the issue
• Does not be an end on itself

41
8. Metaphorical language

• To be a comparison
• Should be practial
• Story
• Citation
• Joke

42
9. Visual aids

• Black board
• Projector
• Film
• Videobeam
• Overhead-Display

43
Presentation programs

• Grafical Layout
• Colours Contrast Background
• Clipart-files
• Fonts (size)
• Produce hand outs

44
What we keep

• 10 Reading
• 20 Hearing
• 30 Seeing
• 50 Hearing Seeing
• 70 Reporting
• 90 Do it yourself

45
10. Questions

• Question
• Are welcome
• Should be answered immediatly
• Bit can also be delayed or forwarded to somebody
  else
• Interupptions
• ignore!
• Look at the person!
• Stopp speaking!

46
11. Final discussion

• Announce already in the introduction
• Write up issues of discussion
• Keep sequence of questions
• Repeat questions (if necessary)
• Draw conclusions

47
12. End

• Avoid
• New questions which do not help in the
  conclusions
• Main ideas
• Should be summarized

48
12. End

• Main idea review objective how to realise
• Good end

49
13. Acoustic factors

• No speaking smoking"
• Speak loudly and slowly
• You should vary volume and speed
• Make breaks
• Try presentation beforehand

50
Behave natural!

• With your body
• With your hands
• With your eyes

51
14. Other circumstanceas

• Room size
• Lighting
• Ventilation
• Sockets
• Other means (Microfon)
• Breaks

52
Reading - How to read

• Fausto Giunchiglia
• Literature
• Bruno Buchberger, Thinking Speaking Writing
• By Fausto Giunchiglia and Alessandro Tomasi

53
Index1. The Role of Literature2. What is
literature 2.1 Classify the results 3. The
Documentation of Literature4. Steps in the Use
of Literature
54

• 1. The Role of Literature

55
1. The Role of Literature

• Keep Re-Inventing the wheel

• For many problems
• the solutions is already in the literature
• the literature has solutions in similar problems

56
1. The Role of Literature

• It is necessary to know
• how the literature is organized
• within your research community
• outside your research community
• translate other concepts
• in our community

Necessary
Useful but not Necessary
57
1. The Role of Literature

• There is an interleaving of
• Try to solve the problem yourself
• Read the Literature

58
1. The Role of Literature
Interactive Process of Reading Papers Step
1 Step 2 Step 3
Quantity
Depth
59
1. The Role of Literature
How do I know when I can stop reading?
When I know the 90 of the papers cited by
relevant papers
60
1. The Role of Literature

• To keep Scholarship
• go to conference
• talk to people (best opportunity in conference)
• journals
• proceedings
• web

61

• 2. What is Literature

62

• 2. What is Literature

1. Monographs / Books
2. Journal Articles
3. Articles in Collections
4. Papers in Proceedings of Conference
5. Papers in Proceedings of Workshop
6. Technical Reports
7. Grey Literature (Ex Web Sites)

Main Way to do Career
More important than papers
63
2. What is Literature

• Each research community has
• 1-2 Top Journals
• 1-2 Top Conference

64

• 2.1 Classify the results

65

• 2.1 Classify the results

1. Monographs / Books

Note the research is finished
Contents extensive/complete description of
research
Originality None (2/3 years old)
Quality Control Very Good
66

• 2.1 Classify the results

1. Journal Articles

Note Come regularly
Contents Very good, very crisp, specialized
Originality Full of two years ago (in CS is
obsolete)
Quality Control Very High (Many shots)
67

• 2.1 Classify the results

1. Articles in Collections

Note Not regularly
Contents Very good, very crisp, specialized
Originality Full of two years ago (in CS is
obsolete)
Quality Control Lower
68

• 2.1 Classify the results

1. Conferences

Note Is Refereed
Contents New results, almost complete, semi-final
Originality Good (6 months ago)
Quality Control Medium (only one shot accept or
reject)
69

• 2.1 Classify the results

1. Workshop

Note Is Not Refereed
Contents New results, almost complete, semi-final
Originality Good (6 months ago)
Quality Control Medium (only one shot accept or
reject)
70

• 2.1 Classify the results

1. Technical Reports

Note ---
Contents Detailed about a specific topic
Originality Maybe
Quality Control Low (No Refereeing Process)
71

• 2.1 Classify the results

1. Grey Literature (Ex Web Sites)

Note The main way to publish our results
Contents Random
Originality Random
Quality Control Random
72
3. The Documentation of Literature
73
3. The Documentation of Literature

• Library
• - author files
• - keywords files
• Review Journals
• - Computing Journal
• - Artificial Intelligence Review
• - Scientific Magazines
• - Scientific Magazines for Spreading
  Communications
• - Journal for Publish new Results
• Bibliographies
• Citation index
• Research index

74
3. The Documentation of Literature
Research Community are organized into
societies Produce Organized Conferences,
Journals, Magazines Examples of Societies ACM,
IEEE, ECCAI, VLDB,
75
3. The Documentation of Literature

• Citations Index
• defines journals
• cites titles, authors, abstract of all paper
• Example
• ISI, the most important for Science

76
3. The Documentation of Literature

• Research Index
• the new game
• is on line
• how many times you are cited in Internet
• Example
• http//citeseer.nj.nec.com

77
4. Steps in the Use of Literature
78
4. Steps in the Use of Literature

• Use CiteSeer (http//citeseer.nj.nec.com)
• Ask to Colleagues, Advisor,
• Friends, by e-Mail
• Consult References
• Consult Review Journals
• Ask Authors