PRINCIPLES OF PSYCHOLOGICAL MEASUREMENT

1
PRINCIPLES OF PSYCHOLOGICAL MEASUREMENT 3090.03
Instructor Peter Papadogiannis, Ph.D. York
University
2
Why is Psychological Testing Important? 1. Allows
us to make important decisions about people.
e.g. Early School Placement, College Entrance
Decisions, Military Job Selections 2. Allows us
to describe understand behaviour 3. Measures
personal attributes 4. Measures performance 5.
Saves time 6. Most economical 7. Its Scientific
3
Psychological Test Definition
Is a measurement instrument that consists of a
sample of behavior obtained under standardized
conditions and evaluated using established
scoring rules.
4
Types of Reasoning Associated with Psychological
Testing
Inductive Reasoning - from Data to General
Theory. Deductive Reasoning - from General
Theory to Data.
5

• Characteristics of Psychological Instruments
• Behaviour Sampling
• Standardization
• Scoring Rules

6
Behaviour Sampling It is a sample of
behaviour. It is not an exhaustive measure - it
is too difficult to evaluate every behavior.
Attempts to approximate the exhaustive
procedure. Does not necessarily require the
respondent to engage in overt behavior. The
test must somehow be representative of behaviours
that would be observed outside of the testing
situation
7

• Standardization
• The behavior sample is obtained under
  standardized conditions.
• Each individual taking a psychological or
  educational test should be tested under
  essentially identical conditions.
• For example, SAT administration instructions
  pertain to
• Seating Arrangements, Lighting Conditions, Noise
  Levels Interruptions, Answering common questions
• Standardization is vital because many test
  results are referential in nature Your
  performance is measured relative to everybody
  elses performance.

8

• Standardization (cont.)
• Standardization reduces between subject
  variability due to extraneous variables.
• Standardization is easier to obtain with tests
  designed to be administered en masse.
• Tests such as the Wechsler Adult Intelligence
  Scale, which are administered individually, are
  less standardized.
• The individual giving the test is an important
  variable. They take special training to
  standardize the way they give the test.

9
Scoring Rules There are established scoring rules
for obtaining quantitative information from the
behavior sample. Objective Scoring Rules Most
mass produced tests fall into this category.
Different qualified examiners will all come to
the same score for an identical set of responses.
Subjective Scoring Rules When the judgement
of the examiner is an important part of the test,
different examiners can legitimately come to
different conclusions concerning the same sample
of behavior. There conclusions should be similar,
however. Good standardized psychological tests
all have a set of rules or procedures for scoring
responses to a test.
10
Types of Tests Most psychological tests can be
sorted into 3 general categories 1. Tests in
which the subject performs a task. 2. Tests that
involve observations of the subjects behaviour
within a particular context. 3. Self-report
measures
11
Tests of Performance Referred to as "Tests of
Maximal Performance" Subjects are given a
well-defined task that they try to perform
successfully. Participant must know what he/she
must do in response to the task. The subject
exerts maximal effort to succeed. Performance
tests are designed to uncover what an individual
can do, given the specific test conditions.
Examples - Intelligence Tests, language
proficiency - Biology test, flight simulator
12
Behaviour Observation Naturalistic
observation Involves observing the subjects
behaviour and responses in a particular
context. Differs from performance tests in that
the subject does not have a single, well defined
task. The observer can record duration
intensity Examples - Examiner might observe
children interacting or an individual having a
conversation or some other social interaction. -
Companies recruit observers to pose as
salespeople to observe employees behaviors.
Subjects may be unaware they are being tested.

13
Self Report Instruments

• - Participant is asked to report his or her
  feelings, attitudes, beliefs, values.
• When self-report makes sense
• Self-report relies upon the test takers
  awareness and honesty.
• It is the best method to measure internal states
  - things only the person themselves can be aware
  of and judge.
• People are not always good judges of their
  ability
• Provides an estimate

14
Self Report Instruments (cont.)

• Many personality inventories such as the MMPI and
  the 16PF measures are based on self-report.
• Clinicians include self-report measures as part
  of their initial examinations of presenting
  clients.
• Self-Report measures are frequently subject to
  self-censorship.
• People know their responses are being measured
  and wish to be seen in a favorable light.
  (self-serving bias)
• Items are frequently included to measure the
  extent to which people provide socially desirable
  responses.

15
History of Test Development circa 1000 BC.
Chinese introduced written tests to help fill
civil service positions Civil Laws, Military
Affairs, Agriculture, Geography 1850 The United
States begins civil service examinations. 1885
Germans tested people for brain damage 1890
James Cattell develops a "mental test" to assess
college students . Test includes measures of
strength, resistance to pain, and reaction time.
1905 Binet-Simon scale of mental development
used to classify mentally retarded children in
France. 1914 World War I produces need in U.S.
to quickly classify incoming recruits. Army Alpha
test and Army Beta test developed. Looked at
psychopathology. 1916 Terman develops Stanford
- Binet test and develops the idea of
Intelligence Quotient
16
History of Test Development (cont.) 1920 - 1940
factor analysis, projective tests, and
personality inventories first appear. 1941-1960
vocational interest measures developed
1961-1980 item response theory and
neuropsychological testing developed 1980 -
Present Wide spread adaptation of computerized
testing. "Smart" Tests which can give each
individual different test items develop
17

• Early Abuses of Tests in America
• Goddard (1906) began testing 378 residents and
  categorized them as
• Idiot (ma below 2), imbecile (3-7), feebleminded
  (8-12), moron (foolish)
• MA four years behind, were feebleminded
• Goddards desire was to separate people out
• Believed feeble minded people were the cause of
  most social problems (thievery, laziness,
  alcoholism, prostitution, immorality).
• Called for the colonization of morons to
  restrict their breeding. Further, he believed
  that many immigrants were feeble minded.
• Went to Ellis Island, administered tests
  translated from French to English to Yiddish,
  Hungarian, , Italian, Russian, to farmers,
  laborers, who had just crossed the Atlantic. Then
  interpreted results based on French norms.
• Favored deportation for low IQ immigrants but
  then also in a humanitarian gesture said we
  might be able to use moron laborer if only we
  are wise enough to train them properly.

18

• Early Abuses of Tests in America (cont.)
• Robert Yerkes, a Harvard psychology prof.
  Convinced the Department of War that it should
  test all of its 1.75 million recruits for
  intelligence tests, so they could be classified
  and given appropriate assignments (Goddard and
  Terman also chaired this committee).
• Army Alpha
• Army Beta Examinations
• Produced evidence that supported segregation.
  Sounded dire warnings that racial intermixture
  would inevitably cause a deterioration of
  American intelligence. Later recanted without
  foundation Probably the result of cultural and
  language differences.

19

• Application of Psychological Measurement
• Educational Testing
• Personnel Testing
• Clinical Testing

20

• Educational Testing
• Intelligence tests and achievement tests are
  used from an early age in the U.S and Canada.
  From kindergarten on, tests are used for
  placement and advancement.
• Educational institutions have to make admissions
  and advancement decisions regarding students.
  e.g, SAT, GRE, subject placement tests
• Used to assess students for special education
  programs. Also, used in diagnosing learning
  difficulties.
• Guidance counselors use instruments for advising
  students.
• Investigates school curriculum.

21

• Personnel Testing
• Following WW I, business began taking an active
  interest in testing job applicants. Most
  government jobs require some civil service
  examination.
• Tests are used to assess training needs,
  workers performance in training, success in
  training programs, management development,
  leadership training, and selection.
• For example, at the Lally School of Management,
  the Myers -Briggs type indicator is used
  extensively to assess managerial potential. Type
  testing is used to hopefully match the right
  person with the job they are most suited for.

22

• Clinical Testing
• Tests of Psychological Adjustment and tests
  which can classify and/or diagnose patients are
  used extensively.
• Psychologist generally use a number of objective
  and projective personality tests.
• Neuropsychological tests which examine basic
  mental function also fall into this category.
  Perceptual tests are used detecting and
  diagnosing brain damage.

23
Testing Activities of Psychologists Clinical
Psychologists - e.g. Assessment of Intelligence,
Assessment of Psychopathology Counseling
Psychologists e.g. Career Interest Inventories,
Skill Assessment School Psychologists e.g.
Assessment of Academic progress, Readiness for
School, Social Adjustment I/O Psychologists -
e.g. Managerial potential, Training Needs,
Leadership Potential Neuropsychologists - e.g.,
Assessment of Brain Damage, neurological
impairments. Forensic Psychology - intersection
between law and psychology --needed for legal
determinations e.g. Assessment for risk,
competency to stand trial, child custody
24
Information About Tests The Mental Measurement
Yearbook - A guide to all currently available
psychological tests. The MMY uses content
classifications do describe tests 1.
Acheivement 2. Behavior Assessment 3.
Developmental 4. Education 5. English
Language 6. Fine Arts 7. Foreign Languages 8.
Intelligence and Aptitude 9. Mathematics 10.
Neuropsychological 11. Personality 12.
Reading 13. Science 14. Sensory-Motor 15.
Social Studies 16. Speech and Hearing 17.
Vocations
25

• Ethics In Psychological Testing
• Given the widespread use of tests, there is
  considerable potential for abuse.
• A good deal of attention has therefore been
  devoted to the development and enforcement of
  professional and legal standards.
• The American Psychological Association (APA) has
  taken a leading role in the development of
  professional standards for testing.

26

• American Psychological Association Ethical
  Guidelines
• The investigator has the responsibility to make a
  careful evaluation of its ethical acceptability.
  
• The investigator is obliged to observe stringent
  safeguards to protect the rights of human
  participants.
• The researcher must evaluate whether participants
  are considered Subject at risk or Subject at
  minimal risk - No appreciable risk (physical
  risk, mental harm).
• The principal investigator always retains the
  responsibility for ensuring ethical practice in
  research. That is, the principal researcher is
  responsible for the ethical practices of
  collaborators, assistants, employees, etc. (all
  of whom are also responsible for their own
  ethical behavior).
• Except in minimal-risk research, the investigator
  establishes a clear and fair agreement with
  participants that clarifies the obligations and
  responsibilities of each. Must explain all
  aspects of the research that may influence the
  subjects decision to participate. Explains all
  other aspects that the participants inquire
  about.

27

• American Psychological Association Ethical
  Guidelines (cont.)
• In research involving concealment or deception,
  the research considers the special
  responsibilities involved.
• Individuals freedom to decline, and freedom to
  withdraw, is respected.
• Researcher is responsible for protecting
  participants from physical and mental discomfort,
  harm, and danger that may arise from research
  procedures. If there are risks, the participants
  must be aware of this fact.
• After the data are collected the investigator
  provides participants with information about the
  nature of the study and attempts to remove any
  misconceptions that may have arisen.
• The investigator has the responsibility to detect
  and remove any undesirable consequences to the
  participant that may occur due to the research.
• The information obtained from the participant
  should be treated confidentially unless otherwise
  agreed upon with the participant.

28

• Informed Consent
• Participants must be fully informed as to the
  purpose and nature of the research that they are
  going to be involved in.
• Participants must be fully informed about the
  procedures used in the research study.
• After getting this information, the participants
  must provide consent for their participation.
• Participants must be informed about their right
  to Confidentiality and their right to withdrawal
  without penalty.

29

• Debriefing
• Post-administration debriefing should
• Restate purpose of the research.
• Explain how the results will be used (usually.
  emphasize that the interest is in the group
  findings).
• Reiterate that findings will be treated
  confidentially.
• Answer all of the respondents questions fully.
• Thank the participant!

30

• Participant Feedback
• In clinical research, or research with
  interpretive instruments, there may be the need
  to provide more in-depth feedback about
  individuals responses (e.g., Research on
  Emotional Intelligence).
• In such cases, first and foremost, it is critical
  that this kind of detailed feedback be given by a
  qualified individual.

31
At Least 4 parties are involved in Professional
Test Use (1) Testing professionals the test
developer and publisher (2) Testing
professionals the individuals who administer the
testing procedure (3) The user the organization
or practice that will eventually use the
information to make certain decisions (4) The
test taker
32
DEVELOPING/SELECTING APPROPRIATE TESTS ? Define
what each test measures and what the test should
be used for. ? Describe the population(s) for
which the test is appropriate. ? Accurately
represent the characteristics, usefulness, and
limitations of tests for their intended
purposes. ? Describe the process of test
development. ? Provide evidence that the test
meets its intended purpose(s). ? Provide either
representative samples or complete copies of test
questions, directions, answer sheets, manuals,
and score reports to qualified users.


33
DEVELOPING/SELECTING APPROPRIATE TESTS (Cont.) ?
Indicate the nature of the evidence obtained
concerning the appropriateness of each test for
groups of different racial, ethnic, or linguistic
backgrounds who are likely to be tested. ?
Describe the population(s) represented by any
norms or comparison group(s), the dates the data
were gathered, and the process used to select the
samples of test takers. ? When feasible, make
appropriately modified forms of tests or
administration procedures available for test
takers with handicapping conditions. Warn test
users of potential problems in using standard
norms with modified tests or administration
procedures that result in non-comparable scores.


34
DEVELOPING/SELECTING APPROPRIATE TESTS (Cont.) ?
When a test is optional, provide test takers or
their parents/guardians with information to help
them judge whether the test should be taken, or
if an available alternative to the test should be
used. ? Provide test takers the information
they need to be familiar with the coverage of the
test, the types of question formats, the
directions, and appropriate test-taking
strategies. Strive to make such information
equally available to all test takers. ? Provide
test takers or their parents/guardians with
information about rights test takers may have to
obtain copies of tests and completed answer
sheets, retake tests, have tests rescored, or
cancel scores. ? Tell test takers or their
parents/guardians how long scores will be kept on
file and indicate to whom and under what
circumstances test scores will or will not be
released.

35
Responsibility of The Tester 1. Have competence
in test administration, interpretation and
feedback. 2. Have an understanding of basic
psychometrics and scoring procedures and be
competent in interpretation, and apply scientific
knowledge and professional judgment to the
results. 3. Take responsibility for the
selection, administration, and scoring, the
analysis, interpretation and communication of
test results. 4. Be familiar with the context of
use the situation, purpose, setting in which a
test is used.
36
Responsibility of The Tester (cont.) 5. Have
knowledge of legal and ethical issues related to
test use 6 . Awareness of ethnic or cultural
variables that could influence the results 7.
Have the ability to determine language
proficiency 8. Have knowledge of important
racial, ethnic, or cultural variables relevant
for individuals or groups to whom tests are
administered.
37

• Issues to Address with the Testee
• 1. Informed consent - Assuring confidentiality,
  freedom to withdraw, purpose of assessment, What
  kinds of attributes are being measured?
  
• 2. Who is the client?Individual, Group, Employer
• 3. What happens with results, who has access to
  it
• 4. Where will the data be stored, how, and for
  how long
• 5. Time frame in which results are to be
  considered valid
• 6. Who will be the payer, and how much
• 7. Where will the assessment take place
• 8. Are the facilities appropriate, conducive for
  testing
• 9. Will there be follow-up assessments or
  feedback

38

• Factors Not Under the Testers Control
• Some factors are not under the control of the
  administrator
• 1. How fatigued a test taker is.
• 2. Motivation level of the test taker.
• 3. Physical Discomfort
• 4. Test Anxiety

39
Ethnic and Cultural Variables

• Knowledge of attitudes of various racial, ethnic,
  or cultural groups toward testing.
• Ability to determine language proficiency.
• Ability to determine the potential effects of
  different test settings on different racial,
  ethnic, or cultural groups.
• Knowledge of specific biases that have been
  demonstrated for particular tests for individuals
  or groups of individuals from particular racial,
  ethnic, or cultural minority groups.

40
Test Fairness

• People with different values often disagree over
  the fairness of some testing practices.
• Factors that affect testing fairness 1.
  Obstacles that prevent people from performing
  well 2. Test may provide unfair advantage to some
  people 3. Some tests are not valid and used in
  wrong situations 4. Some tests are used for
  purposes that are inherently objectionable

41
Test Use Test Fairness

• A test is most likely to be seen as unfair
  when 1. It is the sole basis for the
  decision. 2. The consequences of doing poorly
  on the test is harsh
• Ways to reduce concerns over test
  unfairness 1. Multiple assessment
  procedures 2. Use more intensive screening
  procedures for those likely to be treated
  unfairly by a given test.

42
Types of Decisions

• Two distinctions are very useful for classifying
  decisions 1.
  Individual or Institutional
  2. Comparative or Absolute

43
Ethics References

• Websites www.apa.org and www.cpa.ca
• APA
• American Psychological Associations Code of
  Ethics for Psychologists
• The Ethical Practice of Psychology in
  Organizations
• CPA
• The Canadian Code of Ethics for Psychologists
• Companion Manual
• Guidelines for Non-discriminatory Practice
• Guidelines for Educational and Psychological
  Testing
• Practice Guidelines for Providers of
  Psychological Service

44
Ethics References (cont.)

• Websites for publications / reports
• Rights and Responsibilities of Test Takers
  Guidelines and Expectations (www.apa.org/science/t
  trr.html)
• Responsibilities in Providing Psychological Test
  Feedback to Clients (http//kspope.com/sci/feedbac
  k1.shtml)
• The Real World It is Better to Receive than to
  Give--Practical Tips for Giving and Receiving
  Performance Feedback (www.siop.org/tip/TipOct00/12
  waclawski.htm)
• Statement on the Disclosure of Test Data
  (www.apa.org/science/disclosu.html)
• Test Security (American Psychologist, Dec. 1999,
  Vol.54, No.12, p1078)
• Statement on the Use of Secure Psychological
  Tests in the Education of Graduate and
  Undergraduate Psychology Students