An Integrated Approach To Improving The Methodology Of Research On Workplace Behaviour

1
An Integrated Approach To Improving The
Methodology Of Research On Workplace Behaviour

• Ralph Fevre, Trevor Jones and Amanda Robinson,
  Martyn Rogers, Cardiff School of Social Sciences
  Duncan Lewis, Business School, University of
  Glamorgan
• Contact Fevre_at_cardiff.ac.uk

2
There are three types of problem with existing
survey data in the field

• Hardly any of these data can legitimately be
  claimed to have been produced by surveys using
  nationally-representative samples of employees.
• A self-administered questionnaire has been the
  research instrument of choice in the field.
• The majority of self-administered questionnaires
  have been designed using questions taken from a
  standard battery (the Negative Acts Questionnaire
  or NAQ,) without any attempt to test the validity
  or reliability of the items.

3

• Being ignored or facing a hostile reaction when
  you approach
• Persistent criticism of your work and effort
• Having your opinions and views ignored
• Practical jokes carried out by people you dont
  get on with
• Being given tasks with unreasonable or impossible
  targets or deadlines
• Having allegations made against you
• Excessive monitoring of your work
• Pressure not to claim something which by right
  you are entitled to (e.g. sick leave, holiday
  entitlement, travel expenses)
• Being the subject of excessive teasing and
  sarcasm
• Being exposed to an unmanageable workload
• Threats of violence or physical abuse or actual
  abuse

• Someone withholding information which affects
  your performance
• Being humiliated or ridiculed in connection with
  your work
• Being ordered to do work below your level of
  competence
• Having key areas of responsibility removed or
  replaced with more trivial or unpleasant tasks
• Spreading of gossip and rumours about you
• Being ignored, excluded or being sent to
  Coventry
• Having insulting or offensive remarks made about
  your person (i.e. habits and background), your
  attitudes or your private life
• Being shouted at or being the target of
  spontaneous anger (or rage)
• Intimidating behaviour such as finger-pointing,
  invasion of personal space, shoving,
  blocking/barring the way
• Hints or signals from others that you should quit
  your job
• Repeated reminders of your errors or mistakes

4

• April 2007 pilot study for Workplace Behaviour
  Survey (WBS)
• representative sample of 1024 current and recent
  (within the last two years) employees
• national omnibus survey conducted face-to-face
  with CAPI.
• 10.8 of Hoel and Cooper sample said that they
  had been bullied in the last six months.
• 25.7 had some experience of bullying in the
  previous five years (Hoel and Cooper 200012-13).
• In our survey, (all weighted results) 4.6 had
  been bullied at all in the last six months 5.6
  had been bullied at all in the last year 7.0
  had been bullied at all in the last two years.
• For the majority of NAQ items, we found less than
  half of the prevalence rates that Hoel and Cooper
  found.

5
The Prevalence Of Bullying And Harassment In The
UK And Ireland
6
Questionnaire Design

• Conceptualization and Operationalization
• Cognitive Testing
• cognitive testing in the course of sixty
  interviews conducted in batches of fifteen on
  four Saturdays in June and July 2007 in Cardiff,
  Ilford, Birmingham and Bristol.
• 3 interviewers were used for each test with each
  conducting five interviews, many taking as long
  as forty five minutes.
• interviewees were selected to meet quotas
  assigned according to age, gender, ethnicity and
  sector of employment (public/private) and
  received a small payment (20).

7

• major problems with providing a comprehensive yet
  clear definition of workplace bullying that was
  interpreted in similar ways by different groups
  of respondents.
• some bullying questions are of doubtful validity
  particularly in the case of questions with
  longer and more complex preambles, researchers
  should be wary of claiming that respondents have
  experienced the behaviour contained in the
  definition of bullying they are given.
• given the difficulties that many respondents had
  with the definitions of bullying and the time it
  took for definitions to be read again or
  explained, we came to the conclusion that a
  self-assessed bullying question and definition
  would add little to the study.

8

• The qualitative interviews raise questions about
  the validity and the reliability of the NAQ items
  that have formed the basis of much previous
  research.
• Many problems echo common limitations of survey
  research i.e. that questions assume a certain
  level of English vocabulary skills, as well as
  ability in comprehension.
• Some of the items were over long and included
  words that a number of respondents did not
  understand.
• Most people reported that they understood the
  question, and yet, from their responses, we
  judged many people misunderstood what they were
  being asked.

9

• Overlap several items concerned threats of
  different types, and respondents appeared to have
  difficulty distinguishing between them.
  Furthermore, some single NAQ items combined one
  or more very different kinds of behaviour.
• Difficulties in interpretation underlined when a
  substantial proportion of respondents reported
  that they had experienced particular NAQ items,
  but viewed these as unambiguously positive,
  rather than negative experiences.
• As a result of the qualitative interviews,
  individual items were revised, dropped, merged,
  separated, or confirmed and the NAQ was shortened
  to 21 items.

10

• Four completely new questions were added to cover
  the gaps which were identified in the cognitive
  testing process.
• Where changes were made, we subjected the revised
  questions to further testing.
• As in Levine et al (2005), few of the questions
  only four of the original twenty-two survived
  this process intact.
• The final instrument included thirteen questions
  which had major or minor revisions.

11

• Someone withholding information which affects
  your performance.
• Pressure from someone else to do work below your
  level of competence.
• Having your opinions and views ignored
• Someone continually checking up on you or your
  work when IT IS NOT NECESSARY
• Pressure from someone else NOT to claim something
  which by right you are entitled to (e.g. sick
  leave, holiday entitlement, travel expenses)
• Being given an unmanageable workload or
  impossible deadlines.
• Your employer not following proper procedures
• Being treated unfairly compared to others in your
  workplace
• Being humiliated or ridiculed in connection with
  your work
• Gossip and rumours being spread about you or
  having allegations made against you
• Being insulted or having offensive remarks made
  about you
• Being treated in a disrespectful or rude way
• People excluding you from their group
• Hints or signals from others that you should quit
  your job
• Persistent criticism of your work or performance
  which is unfair
• Teasing, mocking, sarcasm or jokes which go too
  far
• Being shouted at or someone losing their temper
  with you
• Intimidating behaviour from people at work
• Feeling threatened in any way while at work

12
Recommendations

• No point in conducting more studies with
  convenience samples and self-administered
  questionnaires
• If researchers are constrained to a convenience
  sample, such as employees from a particular type
  of organisation, it might be more fruitful to
  adopt a qualitative or ethnographic approach in
  order to yield more information about the
  contextual issues and explanations for workplace
  bullying in that particular setting.
• This is why we are including 4 organisational
  case studies as complement to the WBS.

13

• WBS results are informing the twenty in-depth
  interviews (plus 3 key informant interviews)
  taking place in each case study.
• We are not suggesting that other researchers
  simply adopt our revised instrument in place of
  the standard NAQ.
• Some of the work we have done on questionnaire
  design is transferable but other researchers will
  need to repeat much of the development work we
  describe above, and particularly the process of
  cognitive testing, in their own countries.