Psychometric properties of online selfreport memory questionnaires: The EMQ and PMQ T. Buchanan1, T.

1
Psychometric properties of online self-report
memory questionnaires The EMQ and PMQ T.
Buchanan1, T. Ali1, T. M. Heffernan2, J. Ling3,
A. Parrott4, J. Rodgers5, A. B. Scholey2.
Poster presented at German Online Research '02,
10th October 2002, Stuttgart, Germany

• Introduction
• While objective techniques provide the most
  reliable assessments of memory, self-report
  questionnaires (e.g. Everyday Memory
  Questionnaire (EMQ), Sunderland et al, 1983
  Prospective Memory Questionnaire (PMQ), Hannon et
  al., 1995) can also provide valuable information
  about the extent to which people experience their
  memory as being impaired (e.g. by head injury).
  In the context of an online study exploring links
  between memory and recreational drug use (Rodgers
  et al, 2001), 763 participants completed online
  versions of the EMQ and PMQ.
• The PMQ measures failures to remember to do
  things at some future point in time (e.g. make an
  important phone call) and has four subscales.
• Long Term Episodic (LT) remembering to complete
  task hours or days after cue to perform it
  irregular schedule
• Short Term Episodic (ST) remembering to
  complete task within few minutes after cue
  occurs routinely
• Internally Cued (IC) task does not have clear
  specific external cue e.g. forgetting what one
  is about to say
• Techniques to Remember (TR) - the use of
  techniques to aid memory for tasks one has to do
  e.g. leaving notes
• The EMQ measures slips in "normal" memory
  function (e.g. forgetting where you have put
  something) and is a unitary scale. There are
  suggestions of disparate factors within it (e.g.
  Cornish, 2000) but its latent structure is not
  well established and a single total score is
  typically used.
• This sample represents an extension of that
  reported by Rodgers et al. A report on the
  extended dataset is currently being prepared for
  publication.

Results Factor Structure of PMQ To compare the
factor structure in our sample with that
predicted by Hannon et al, we performed a
principal components analysis, specifying
extraction of four factors, followed by Varimax
rotation. The structure obtained is shown in
Table 1. Of the four factors delineated by Hannon
et al, only two are clearly present in our data
the Long Term and Techniques to Remember
subscales. The highest loadings of the items
defining the other two scales are scattered
across three components in each
case. Reliabilities Cronbachs alpha for the EMQ
was .94 (Cornish, 2000, reports .899 for paper
version). Alphas for the PMQ LT and TR scales
were .85 and .89 respectively (Hannon et al,
1995, report alphas ranging between .78 and
.90). While these results were obtained with
the whole sampleincluding drug usersconclusions
with respect to reliability and factor structure
do not change when sample is restricted to non-
drug-users.
Extracted Component
Item Notional Subscale 1 2
3 4 1 LT .69 .01 .09 .04
2 LT .62 .08 .04 .28 3 LT .43 .12
.10 .15 4 LT .69 .09 .11 .14 5 LT
.56 .05 -.01 .05 6 LT .71 .05 .13 .07
7 LT .49 .32 .12 .15 8 LT .59 -.06
.24 .03 9 LT .60 .07 .16 .12 10 LT
.62 .15 .22 -.03 11 LT .67 .14 .11 .10
12 LT .50 .18 .08 .08 13 LT .53 .14
.27 -.01 14 LT .57 .14 .08 .09 15 ST
.19 .03 .28 .20 16 ST .12 .06 -.06 .22
17 ST -.10 .10 .24 .58 18 ST -.06 .08
.50 .39 19 ST .07 .08 .38 .31 20 ST
.17 .10 .75 -.10 21 ST .04 -.01 .82 .22
22 ST .23 .10 .28 .19 23 ST .21 .08
.33 .18 24 ST .21 .05 .17 .34 25 ST
.12 .13 .76 -.03 26 ST -.01 .09 .16 .67
27 ST .09 -.02 .56 -.06 28 ST .41 .07
.16 .20 29 IC .42 .16 .08 .62 30 IC
.51 .17 .08 .59 31 IC .44 .21 .10 .63
32 IC .41 .21 .04 .63 33 IC .45 .20
.10 .45 34 IC .57 .15 -.01 .35 35 IC
.21 .15 .61 .40 36 IC .35 -.04 .49 .27
37 IC .40 .06 .40 .13 38 IC .39 .03
.60 -.05 39 TR .13 .76 .00 -.04 40 TR
.11 .75 .00 .09 41 TR .09 .61 .24 -.06
42 TR .00 .78 -.02 -.06 43 TR .21 .67
.04 .24 44 TR .04 .39 .21 .27 45 TR
.25 .57 .05 .35 46 TR .06 .66 .03 .29
47 TR .05 .71 -.01 .05 48 TR .06 .52
.08 .32 49 TR .10 .64 .06 .24 50 TR
.29 .53 .12 .22 51 TR .13 .61 .09 .11
52 TR .09 .66 .01 -.09 Table 1
Factor structure of online PMQ. Highest loading
for each item shown in bold.

• Method
• Participants recruited through messages posted in
  Usenet discussion groups links from other
  experiments announcements to students personal
  contacts.
• Participants completed EMQ, PMQ, drug use
  questionnaire, demographic items and questions
  related to participation (e.g. Are you currently
  under the influence of any substance).
• 763 people (298 M, 465 F) accessing the
  questionnaire from unique IP addresses indicated
  that data could be used in analyses and they were
  not currently intoxicated.
• Modal age group was 21-25 yrs old (32 of
  sample). Modal location was Europe (71). Modal
  level of education was Some college or
  university (31).
• 309 were cannabis users (at least 1-4 times /
  month). 282 had taken Ecstasy on at least one
  occasion. 242 had never taken any drugs at all.

Discussion and Conclusions While the EMQ was
found to have satisfactory internal reliability,
the expected factor structure was not found for
the PMQ in an exploratory factor analysis only
two subscales (Long Term and Techniques to
Remember) were clearly identifiable. These two
subscales had acceptable reliability. The failure
to replicate Hannon et als factor structure
could be attributable to a number of causes (e.g.
differences in the samples usedHannon et als
development samples largely comprised college
students with a few participants from
brain-injured and alcoholic populations effect
of mode of administration insufficient
heterogeneity in scores on the ST and IC factors
in Rodgers et als sample). However, as the ST
and IT subscales do not seem to form coherent
constructs in the present sample, they are not
suitable for use in analyses of this dataset. The
EMQ and two of the PMQ's subscales are acceptable
(in terms of latent structure and reliability)
for use in the current online sample. It remains
to be seen whether they are valid measures of the
constructs they claim to address (although
correlations with drug-use variables observed by
Rodgers et al suggest they do have some degree of
validity). These data suggest the online use of
the PMQ Short Term and Internally Cued subscales
would be unwise, unless they are found to work
properly in other online datasets. Additionally,
the current findings provide support for the
notion (e.g. Buchanan, 2002) that the
psychometric properties of self-report
questionnaires may change subtly when converted
for use online, and reinforce the need to
establish the psychometric adequacy of any
questionnaire used on the web.

• References
• Buchanan, T. (2002). Online assessment Desirable
  or dangerous? Professional Psychology Research
  and Practice, 33, 148-154.
• Cornish, I. M. (2000) Factor Structure of the
  Everyday Memory Questionnaire. British Journal of
  Psychology, 91, 427-438.
• Hannon, R., Adams, P., Harrington, S.,
  Fries-Dias, C., Gibson, M. T. (1995). Effects
  of brain injury and age on prospective memory
  self-rating and performance. Rehabilitation
  Psychology, 40, 289-297.
• Rodgers, J., Buchanan, T., Scholey, A.B.,
  Heffernan, T.M., Ling, J., Parrott, A. (2001).
  Differential effects of Ecstasy and cannabis on
  self-reports of memory ability a web-based
  study. Human Psychopharmacology Clinical and
  Experimental, 16, 619-625.
• Sunderland, A., Harris, J. E., Baddely, A. D.
  (1983). Do laboratory tests predict everyday
  memory? Journal of Learning and Verbal Behaviour,
  22, 341-357.

Figure 1 The PMQ
Figure 2 The EMQ
1Psychology Department, University of
Westminster, London, W1B 2UW UK 2Human Cognitive
Neuroscience Unit, Northumbria University,
Newcastle upon Tyne NE1 8ST UK 3Psychology
Section, University of Teesside, Middlesbrough,
TS1 3BA UK 4Department of Psychology, University
of East London, E15 4LZ UK 5Doctorate in
Clinical Psychology, University of Newcastle,
Newcastle upon Tyne, NEI 7RU UK.