Folie 1

1
Intro
The Evaluation of the NEFIS Project focused on

• Metadata

• Data Rights and Data Rights Management

• User Needs and User Expectations - VTK

2
Intro
Evaluation based on two approaches

• Standardised questionnaires

• metadata and metadata guidelines
• data rights and data rights management
• user needs and user expectations - VTK

• Expert Statements

• metadata and metadata guidelines
• Prof. Dr. Keith Rennolls (Greenwich), Dr.
  Jarmo Saarikko (Metla)
• keyword list and thesaurus
• Renate Prüller (IUFRO), Gillian Petrokofsky
  (CAB International)
• data rights and data rights management
• Andreas Schuck (EFI), Tim Green (EFI)
• EFIS an electronic system for periodic
  reporting
• Prof. Dr. Michael Köhl (UHH), Aljoscha
  Requardt (UHH)

3
Metadata
1. Elaboration of metadata records a step-wise
iterative process
- preliminary records according previous schema
- metadata schema developed further to its
final version (D3, D5) - revise preliminary
records make adjustments according new schema
2. Questionnaire Metadata and Metadata
Guidelines - evaluate experienced
applicability and usability
3. Questionnaire results represented and
discussed at the Hamburg Meeting

• Expert Statements
• - metadata and metadata guidelines
• - keyword list and thesaurus

Evaluation Metadata
4
Metadata
Questionnaire Metadata and Metadata Guidelines

• Knowledge and experiences of the respondent

• Workload

• Metadata Guidelines

• NEFIS Metadata Schema

• Specific Metadata Elements and Refinements

• Metadata Functionality and Structure

• Future Perspectives

5
Metadata
General
79 Yes
Are the NEFIS metadata the first contact you
have had with metadata?
Partners knowledge and experience of elaborating
metadata
Partners knowledge and experience using metadata
for data source retrieval
6
Metadata
Workload
Do you consider the workload to enter metadata
records as
Level of complexity to prepare and enter metadata
29 complex
71 acceptable
Time Investment - most partners between 5-10
hours in total
7
Metadata
Guidelines
Are the guidelines easy to understand?
Would a metadata tutorial or training course be
helpful and appropriate?
71 Yes
8
Metadata
Schema
Metadata records without problems
9
Metadata
Schema
Workload time to prepare and enter required
information
10
Metadata
Schema
Relevance - data and resource documentation
11
Metadata
Schema
Relevance - data retrieval
12
Metadata
Specific Issues

• Type

DatasetGeoreferenced

• Format

Reference System - encoding scheme for
georeferenced data

• Coverage

Point / Box encoding scheme

• Quality Report

• Subject

• Audience

13
Metadata
Quality Report
"Quality" addressed within DCMI elements,
refinements and encoding schemes?
Named elements which address Quality within the
DCMI schema Creator, Description, Publisher,
Coverage, Source, Date, Audience
14
Metadata
Quality Report
Relative value of a Quality Report
Quality Report under the element Descriptions?
15
Metadata
Quality Report
Value of listed options to describe and structure
Quality Report"
16
Metadata
Quality Report
Value of quantitative measures of quality
(standard error, sample size, sampling unit,
resampling for measurement control)

• Which of these quantitative measures are
  relevant and appropriate for which type of data?
• it could become very complicated and time
  consuming to collect such information in a
  standardised way.

17
Metadata
Subject
Can the dataset be appropriately described by
using the NEFIS terms?

• sometimes difficult to group the term under a
  theme
• very time consuming to go through the lists
  more structure/ hierarchy would be helpful.

18
Metadata
Subject
How important is the option to add "Nominated
Terms"

• it is the way to improve the term lists
• very important during the development phase
• it allows the provider to demonstrate which
  additional terms are seen necessary to describe
  the particular resource more accurately

19
Schema - Resume
Metadata
Metadata Element
1. Title
2. Creator
3. Subject
4. Description
5. Publisher
6. Contributor
7. Date
8. Type
9. Format
10. Identifier
11. Source
12. Language
13. Relation
14. Coverage
15. Rights
16. Audience
Applicability and Usability
?
??
??
?
??
?
?
?
?
?
20
Conclusion
Metadata
Obstacles within the challenging tasks as the
development of keyword lists as well as the
quality report could have been clearly reflected
by the results of the metadata evaluation
Subject, Description and Format caused problems
for most partners

• low experience by some partners

• complicated structure and defined procedures

NEFIS metadata schema is applicable in its
current form

• from the perspective as a data provider
  dataset documentation

• from the perspective as a data user data
  resource retrieval

21
Data Rights
Data provider has to state what data are
accessible and for whom, and which type of data
rights and which levels of access should be
assigned to which kind of data.
Data provider should have options to assign
different levels of access to individual datasets
in order to protect sensitive data by restricting
access to them.
Prerequisite to guarantee that legal restrictions
to data access will be satisfied
22
Data Rights
Why to talk about Data Rights?
Data rights provoked several discussions
Data Rights, does it concern the actual data?
Should not all data presented in NEFIS be
public?
NEFIS tables are available for free. On the other
hand, same information is part of an on-line
system to which you have a restricted access. How
should we handle this?
Clarify Data Rights

• within the NEFIS project
• beyond the time of NEFIS project

23
Data Rights
Which of the "drop down" options (if any) are
relevant to your data set?
Are these drop down options appropriate for
your data set?
95 Yes
But option Public needs further refinements
24
Data Rights
Who is responsible for data access management?
Which form of data storage do you prefer for your
data set?
25
Data Rights
Would you like to provide more data?
Are there any obstacles to provide additional
data?

• cost for the provider !
• cost for the user
• technical limitations
• unclear data rights management

26
Data Rights
Partners demands for further cooperation and data
provision

• Technical demands and demands on data management

• link data easily to the system without
  complicating and restricting data management and
  maintenance
• clear definitions and standards for data format
  and data content

• Financial demands

• capacities for time and personnel investments

27
User Needs and User Expectations - VTK
1. VTK Workshop
2. anonymous comments by partners regarding VTK
display and analysis options
3. Test CD
4. Questionnaire User Needs and User
Expectations
5. Expert Statement (UHH) EFIS - an electronic
system for periodic reporting
Evaluation VTK Information needs
28
User Needs and User Expectations - VTK
Questionnaire User Needs and User Expectations

• The respondet

• Usability

• Performances

• VTK features

• Tutorial and Help Functions

• Improvements

• Future Potential

29
User Needs and User Expectations - VTK
The respondet
A - familiar with the NEFIS project and the
provided dataset B - only familiar with the
provided dataset itself C - someone who is not
familiar with the NEFIS project and the provided
dataset, but has some broad knowledge in
using GIS applications and statistical programs
Questionnaire evaluated according

• Total
• GIS Knowledge
• Statistical knowledge

19 answered questionnaires
Expirences of useing the VTK average / less
than average
Respondent spent 7 hours using the VTK (5 hours
for one explicit dataset)
30
User Needs and User Expectations - VTK
How easy is it to find applicable and reasonable
functions to analyse and visualise data which are
of explicit interest?

• functions are atractive but it is not easy to
  find out how to use them and select the one that
  is needed
• VTK examples are an important addition to get a
  more transparant view which of the features are
  useful for which kind of analysis

31
User Needs and User Expectations - VTK
Respondent meaning about the VTK menu

• the menu should be developed more simple and
  informative
• some better getting started directions would
  be very helpful (essential even).

32
User Needs and User Expectations - VTK
The VTK incorporates automated map design

• the produced maps are interactive, including
  traditional GIS interactivity and visual
  manipulations
• outputs are available as data tables and
  thematic maps with different techniques for
  selected geographical area
• it allows production of statistical graphics
  (box plots, dot plots, scatter plots, etc.) that
  are dynamically linked to maps

This could imply that expert knowledge on
cartography or GIS is not required

• not necessariliy knowledge in GIS but some
  knowledge in cartography and data presentation
• the user should be familiar with the data itself

33
User Needs and User Expectations - VTK
Consideration of options to analyse available
datasets
Consideration of options to visualise available
datasets
34
User Needs and User Expectations - VTK
Ability to recall explicit information from
displayed formats Mapped data Data
plots Tabular data
high
Figures
satisfactory
Lack in the actual dataset itself !!!
Units
unsatisfactory
35
User Needs and User Expectations - VTK
VTK offers many features, tools and options to
analyse and to visualise data

• the feature should be applicable to the dataset
• the user should understand meaning and
  relevance of the feature
• the user should be able to recall relevant
  information after using a suitable feature
• the appearance of the outputs should be
  satisfactory for the user
• no essential functionality in that feature
  should be missing

36
User Needs and User Expectations - VTK
37
User Needs and User Expectations - VTK
Most respondents considered all listed
applications as important or as interesting
but the respondent was not always able to test
some explicit applications, mainly due to the
lack of help functions and tutorial documents
further tutorial documents and help functions can
be seen as a useful and important asset !
38
User Needs and User Expectations - VTK
Time investment to understand and use VTK features
high
Complexitiy to understand available VTK features
high
Suggestion
Develop a set of tutorial tailored at different
user groups and level of expertise
39
User Needs and User Expectations - VTK
Reduce the set of VTK features or Add new
features ???
Suggestion
Develop a VTK for two different groups of users

• Basic Version new users
• Advanced Version experienced users

Key User Friendliness
- improved interfaces - improved maneuvering
through features - improved guidance for getting
started

• Reporting
• reporting schemes
• reporting query options

40
User Needs and User Expectations - VTK
VTK functional and applicable for your purposes
as a data provider?
41
User Needs and User Expectations - VTK
User Backgrounds and Expectations are different

• Simplified self explanatory output features
• Clearly structured user guidelines
• Easy to follow help functions

• Define the purposes of the system Who are the
  potential users?
• Differentiate the use and applications
• Reporting most important purpose
• - guarantee that VTK outputs not ambiguous in
  their statement
• - contact data user data provider

Although some applications could be developed
further or improved, the VTK and its features
provide a solid basis to meet various tasks of a
full working EFIS.
42
Thank you