Measurement

1
Measurement
2
Measurement

• In Quantitative research, accurate measurement of
  the dependent variable is critical to the
  validity of the study

3
Tools of Measurement

• The data obtained from the measurement process is
  classified into levels of measurement
• The level of measurement of the data prescribes
  the type of statistical technique to be used

4
Levels of Measurement

• Nominal
• Ordinal
• Interval
• Ratio

5
Nominal

• Frequency data
• How many?
• I.e., the number of years failed in school

6
Ordinal

• Ranked data
• Where is a data point located on a rating scale?
• I.e, at what percentile is Johnnys SAT total
  battery score?

7
Interval

• Data set in which data points are equally spaced
• I.e., inches on a ruler

8
Ratio

• Data set which has a meaning zero point
• I.e., the Celsius scale (0 degrees is freezing)

9
The level of measurement of the data prescribes
the type of statistical technique to be used
10
Statistics to be used

• Nominal
• nonparametric statistics
• least powerful
• compares frequencies
• often Chi Square

11
Statistics to be used

• Ordinal
• nonparametric statistics
• compares ranks of data sets
• frequently used (Mann-Whitney U, Friedmans)

12
Statistics to be used

• Interval and Ratio
• parametric statistics
• most powerful
• all data is used in the analysis
• compares ? so SD of groups
• frequently used t-test, Pearson v Anova,
  Ancova, Manova

13
Instruments
14
Instruments used to measure the dependent
variable

• Paper and pencil tests can be surveys, attitude
  scales, personality scales
• Machines and other equipment

15
Instruments used to measure the dependent variable

• The purpose of educational and psychological
  testing is to draw an inference about an
  individual

16
Types of Instruments

• Paper and pencil
• In educational research, paper and pencil tests /
  surveys provide much of the measurements of the
  dependent variable

17
Issues related to paper and pencil measurements

• Validity
• Reliability

18
Validity

• Is the concept that
• a test instrument measures what it is supposed to
  measure
• a test is useful for the testers purpose

19
Three types of validity

• As described by most research texts
• content
• criterion
• construct

20
Content Related Validity

• The degree to which an item(s) measures or is
  related to the traits for which the test was
  designed.
• I.e., very important achievement tests but not
  very important in aptitude tests
• Assessment of content validity is often done by a
  panel of judges but no numerical way exists to
  describe

21
Criterion Related Validity

• A broad term that refers to two types of
• with two frames
• predictive - usefulness in using one test to
  predict future performance I.e., scores on the
  MCAT predict success as physicians
• concurrent - the extent to which a test is
  closely related to other measures I.e., grades,
  teacher ratings, etc.

22
Criterion Related Validity

• Can be expressed as the correlation between
• the set of two or more scores

23
Construct Related Validity

• The degree to which scores on a test can be
  accounted for by the explanatory constructs of a
  sound theory.
• A construct is a trait that cannot be observed
• Especially important for personality and aptitude
  tests
• I.e., IQ, fear

24
Self-test 1

• A reading test relies on reading and phonic
• skills and has good content validity. If the
• focus in reading skills in the school is on the
• whole language approach, is the test valid for
• that group?

25
Self-test 2

• A test is normed for use with students under
• the age of 12. A middle school teacher selects
• this test for use with a group of 13 year olds.
• Would the test be valid for use with this
• group?

26
Examples of Instruments

• Osgood The Measurement of Meaning
• Healthy 1 2 3 4 5 6 7 Sick
• Honest 1 2 3 4 5 6 7 Dishonest
• Fair 1 2 3 4 5 6 7 Unfair
• Hard 1 2 3 4 5 6 7 Soft

27
Types of tests and inventories

• Achievement tests
• Aptitude tests
• Interest inventories
• Personality inventories
• Projective devices

28
Checklists

• Rating scale
• Questionnaire - Likert methods (assigns scale
  values to)
• Semantic differentials - use of bipolar
  adjectives to measure attitudes or opinions
  between two extreme choices

29
Reliability

• Measures the test consistency
• An unreliable test is like a rubber yardstick
  measurement
• Reliability is usually expressed as a correlation
  coefficient

30
Types of Reliability

• Stability over time (test - retest)
• Stability over item samples (equivalent or
  parallel forms)
• Stability of items (internal consistency)

31
Types of Reliability

• Stability over scorers (inter-scorer)
• Standard error of measurement - permits the
  interpretation of individual scores on a test
• The SEM tells how much we can expect an obtained
  score to differ from the individuals true score.

32
Reliability

• A test may be reliable but not valid.
• However, for a test to be valid it must be
  reliable

33
Data Analysis

• Descriptive
• normal distribution
• standard scores - provide a method for expressing
  any score in a distribution in terms of its
  distance from the ? is SD units

34
Statistical Tools

• Nonparametric
• Parametric

35
Statistical Tools

• Correlation
• t-test
• Anova
• Chi Square

36
Statistical Tools

• Correlation
• relationship between two or more paired variables
• in the correlation coefficient, two factors are
  important the magnitude and the direction (,
  -) of the coefficient

37
Positively Correlated Variables

• The following variables are often positively
• correlated
• height and shoe size
• family income and value of family home
• productivity per acre and the value of farm land

38
Negatively Correlated Variables

• The following variables are usually negatively
• correlated
• academic achievement and hours spent per week
  watching television
• time spent in practice and the number of typing
  errors
• the year of a car and the trade in value

39
Variables Not Correlated

• Some variables which are not correlated
• include the following
• body weight and IQ
• shoe size and monthly salary

40
Interpretation of Correlation Coefficients

• Correlation coefficients range from -1.00 to
• 1.00
• Coefficient Relationship
• .00 to .20 negligible
• .20 to .40 low
• .40 to .60 moderate
• .60 to .80 substantial
• .80 to 1.00 high to very high

41
Parametric Tests

• Most powerful tests
• observations are independent
• samples have equal or nearly equal variances
• variables are at either interval or ratio scales

42
Level of Significance

• Criterion used for rejecting the null hypothesis

43
Comparison of Means

• Calculation based on means and standard
  deviations
• t-test - 2 independent samples, 2 dependent
  samples
• Anova

44
Nonparametric Tests

• Use when populations are non-normal
• Variables expressed in categories
• Variables expressed in ordinal from

45
Often Used Tests

• Chi Square
• Number of cigarettes smoked per day
• Major none 1-15 gt15
• Business 6 6 14
• Non- 24 80 16
• business

46
Test Your Knowledge

• Which statistic would be most appropriate?

47
Self-test 1

• Group one - rats were given a vitamin
• supplement and the other group received a
• conventional diet. Rats were randomly
• assigned. Test the null hypothesis that the
• vitamin supplement did not result in increased
• weight gain for the experimental group.

48
Self-test 2

• An achievement test in math was
• administered as both a pre- and posttest to two
• randomly selected fifth grade groups of
• students. Was there a significant difference in
• achievement at the time of the posttesting?

49
Issues

• Statistical significance - plt.01, plt.05
• Do the numbers indicate significance difference?
• Substantive (practice) - are the differences
  obtained meaningful

50
Distinctions in Educational Research

• Quantitative versus qualitative
• Descriptive versus Inferential
• True Experimental versus quasi-experimental
• Causal-comparative versus correlational
• Single-subject versus group

51
Causal-comparative (Ex-post Facto)

• Purpose
• Establish cause-effect relationship
• Problem
• Threats to the validity of the design because
  randomization is not possible.
• Group membership generally predetermined

52
Examples

• Does smoking cause cancer?
• Did the community mental health movement improve
  mental health?
• Is there a difference in self-esteem in juvenile
  delinquents and non-juvenile delinquents?

53
Which is a better example of causal-comparative
or correlational research
54

• Weight loss of Weight Watcher participants
• were compared based on amount of exercise.
• It was found that higher levels of exercise
• resulted in greater weight loss.

55

• Relationships with mothers of delinquents
• committing violent crimes were compared with
• that of mothers of delinquents not committee
  
• violent crimes. Results indicated that those
• committing violent crimes described themselves
• as having more positive relationships with their
• mothers than did those committing non-violent
• crimes.

56

• Students grades on their EPR 692 proposals
• were significantly related to the number of
• hours spent in preparation of the proposal.

57

• Physical strength at age 20 was found to be
• unrelated to physical strength at age 40.