After all the coding is done '''

1
After all the coding is done ...

• Harry Ganzeboom
• Center for Survey Research Academia Sinica
• July 24-25 2008

2
Scaling occupations

• Detailed occupation codes have various uses, but
  for most applications they are condensed again
  into social status scales.
• There is a great variety of national and
  international social status scales and ways they
  are constructed.
• Main division
• Nominal categories EGP (Goldthorpe), Wright,
  Esping-Andersen.
• Continuous scales Prestige, Socio-economic Index
  SEI
• Each of these have their own theoretical
  backgrounds.
• The varieties of social status scales can only be
  compared when you have access to detailed
  occupations (and more).

3
Tools for ISCO-88

• http//home.fsw.vu.nl/HBG.Ganzeboom/ISMF
• This webpage contains several useful SPSS tools
  to work with ISCO-88 codes
• ADD VALUE LABELS for all occupations
• RECODE for EGP social classses
• RECODE for SIOPS Treimans prestige scale
• RECODE for ISEI Ganzeboom et al.s SEI scale
• Note that the tools will work (A) for multiple
  occupations, and (B) for all levels of detail of
  coding (providing you have used trailing zeroes).
• There are also tools for ISCO-68 and will be for
  ISCO-08.

4
ISEI (1)

• A SEI socio-economic index or Duncan score
  scales occupation by averaging status
  characteristics of job holders, most often their
  education and earnings.
• Often the criterion information is taken from
  census data.
• ISEI was created for ISCO-88 using criterium
  information for educational and earnings ranks on
  a world-wide sample of 70.000 men from 17
  countries.

5
ISEI (2)

• ISEI was constructed as an optimal scaling of
  (detailed) occupations as an intervening variable
  between education and earnings Occupation is
  what you do to convert your qualifications into
  income.
• Metric between 10-90, but this is entirely
  arbitrary.
• ISEI was originally developed for ISCO-68, but
  its second generation version (for ISCO-88) has
  become widely used, also outside sociology.

6
Prestige

• Prestige popular evalation of occupational
  status, i.e. you ask respondents to value
  occupations.
• Many local versions have been integrated by
  Treiman (1977) into the Standard International
  Occupational Prestige Score SIOPS, related to
  ISCO-68.
• The version on my website is a mapping of the
  original SIOPS to ISCO-88.

7
EGP

• EGP class typology combines detailed occupation
  codes with measures on self-employment and
  supervising status.
• This leads to a nominal (partly ordered) set of
  distinctions 12-10-7-5 categories.
• EGP has become the de facto standard for
  stratification research. Much used.

8
Relationships EGP, ISEI, SIOPS

• All these measures are strongly associated. You
  need a lot of data if you are going to argue
  about the differences.
• EGP and ISEI resemble each other more than SIOPS.
• SIOPS prestige is theoretically the best idea,
  but it does not work well in practice.
• I prefer to use ISEI for my further discussion
  here.

9
Checks to be run ...

• Use value labels to see whether the coders have
  indeed entered only valid codes.
• It is surprising to learn how often this check
  has not been run!
• It is even more surprising to learn how often
  this is the only check ever run!!!

10
MTMM-models

• Multi-Trait Multi-Method models were developed in
  psychometrics to estimate the reliability and
  validity of attitude items.
• The idea is that you can learn about reliability
  and validity (both!!) when you apply multiple
  methods (e.g. respons formats) to multiple
  related traits (e.g. personality
  characteristics.
• Remember
• Reliability lack of random errors
• Validity lack of systematic error

11
MTMM model
ROCC
FOCC
FISEI1
FISEI2
RISEI1
RISEI2
12
Estimating MTMM for two coders

• The elementary MTMM model for two traits
  (occupations) and two methods (coders) has 7
  parameters.
• The data generate only 6 degrees of freedom.
• However, by contraining (equalizing) the
  parameters, we can find the following interesting
  information
• How random error each coder has coded relative to
  the other.
• Whether FOCC and ROCC differ in the amount of
  random error.
• How much systematic bias each coder has added to
  their codes.
• Degree of attention brought about by the coding
  unreliability corrected (disattenuatud)
  correlation between FOCC and ROCC.

13
(No Transcript)
14
What are we learning by staring at these
correlations?

• Within-coder correlation at best 0.81. This means
  0.90 index of reliability.
• Coders agree slightly less on fathers occ than
  respondents. Loss is around 0.97.
• Within- and cross-coder intergenerational
  correlations are around 0.33 and fairly
  homogenous.
• Coder 1 has created slightly more consistency
  between father and respondent.

15
MTMM assumptions

• Coders are equally reliable for fathers and
  respondents.
• However, fathers occupations may be easier to
  code (less) reliably than respondents
  occupations.
• Systematic error is the same for all coders.

16
If estimated by SEM (Lisrel), we learn

• Reliability coder 1/coder 2 0.915 / 0.886 (NS).
• Reliability FOCC/ROCC 0.975 / 1.000 (NS).
• Coder unique consistency 0.015 (significant).
• Corrected intergenerational correlation 0.413.
• The interesting conclusion for this (Italian)
  example is clearly the corrected
  intergenerational correlation. Note that this is
  even so with high coder reliability!

17
Conclusions

• Even if coders do a decent and honest job, they
  introduce random and systematic error.
• These errors are in the coding process, not by
  the data collection!
• If coders introduce only 10 error, they bring
  down the intergenerational correlation by 20!

18
More sources of measurement problems .. and their
repairs

• It is important to see that coder errors are just
  one single source of bad measurement.
• It might be true that even bigger trouble is
  created by what the respondents say.
• If you want to assess measurement error at the
  respondents level, you need to ask the question
  twice
• Within the same interview
• From different sources (e.g. spouses about each
  other).
• At diffent interviews, e.g. in panel designs.

19
Another source of error the respondent.

• Note that all of the above is about errors
  generated in the coding proces.
• Occupational measures also contain other errors,
  most prominently generated by the respondent /
  interviewer.
• This type of error can only be estimated by
  asking the question again
• In the same interview.
• From a different source (e.g spouses about each
  other).
• In a different interview (panel).

20
Can you ask the occupation question again in the
same interview?

• Yes, an acceptable way for respondents is to ask
  an open question (see above) and a closed
  question.
• Closed questions may not be as valid and flexible
  as open questions, but they may be more reliable.
  At least they do not suffer from coding error...
• This type of multiple measurement has been tried
  in ISSP87 for four countries and six Dutch
  surveys. It will be replicated in ISSP09.

21
Main conclusions on double measurement

• Crude closed questions are slightly more reliable
  than detailed open question.
• Crude questions suffer slightly more from
  systematic error than detailed questions
• Correlated error (echo effects)
• Education bias.
• However, the main boost comes from using multiple
  indicators, that leads to disattenuation.
  Estimates from ISSP and Dutch data suggest
  measurement relationships of around 0.85. This
  would suggest that coding error is the major
  source of random error.

22
ISCO 2008

• ILO has recently revised the ISCO to ISCO-08.
• Current situation is that the new classification
  has been fixed and published.
• However, there are no definitions or manuals
  available yet.
• For previous versions it laster 1-2 years before
  these became available.

23
Stated goals of ISCO-08

• Bring occupational classification in line with
  changed technologies and division of labor (e.g.
  ICT/IT).
• Make ISCO applicable in a wider range of
  countries and economies.
• To mend often noted problems with the application
  of ISCO-88.
• To produce a minor revision, not a totally
  different classification.

24
Problems with ISCO-88 (1)

• Unlike its predecessor (ISCO-68), ISCO-88 is
  primarily skill oriented. However, in practice
  the major group differentiation does not closely
  correspond to major ISCED (education) levels.
• ISCO-68 was more sensitive to employment status
  (self-employment) and industry.

25
Problems with ISCO-88 (2)

• Despite its stated principles, it is hard to pay
  tribute to skill level differentiation in manual
  work. ISCO-88 differentiates between (7000) Craft
  workers, and (8000) Machine Operators, which is
  similar, but not the same as Skilled versus
  Semi-skilled Manual Workers.
• In addition, many occupations occur both in the
  7000 and 8000 categories.

26
Problems with ISCO-88 (3)

• ISCO-88 argued that occupation and employment
  status are different things and need to be
  measured separately.
• As a consequences some employers became
  classified with their employees, in particular
  there is no distinction between managing
  proprietors and managers, and not between working
  proprietors and their employees.

27
Problems with ISCO-88 (4)

• Managers were organized into three levels
• Corporate managers
• Department managers Production, Support
• General Small enterprise managers.
• The primary distinction here is the number of
  managers in an organisation, which is not often
  available in data.
• It is somewhat hard to classify work supervisors
  Foremen in ISCO-88.

28
Problems with ISCO-88 (5)

• Farmers are hard to classify in ISCO-88, because
  they appear in 5 places
• Operations Department Manager (1211)
• Small Establishment Manager (1311)
• Skilled Agricultural Worker (6100)
• Subsistence Farmer (6200)
• Farm Laborer (9200)
• None of this corresponds closely to distinctions
  made in farm work in national classifications.

29
Problems with ISCO-88 (6)

• ISCO-88 is overly broad in (5000) Service and
  Sales Occupations.
• In particular (5200) Sales Workers is very
  undifferentiated.

30
Problems with ISCO-88 (7)

• It is hard to find fitting codes for crude
  occupations factory worker, skilled worker,
  foreman, semi-skilled worker, apprentice.
• However, in some instances, there is no problem
  if one used major and sub-major groups codes
  e.g. (9000) for Unskilled Worker.

31
ISCO-08 versus ISCO-88

• ISCO-08 groups
• 10 major
• 34 sub-major
• 120 minor
• 403 unit
• Total 567 groups

• ISCO-88 groups
• 10 major
• 28 sub-major
• 115 minor
• 363 unit
• Total 516 groups

32
Mergers and Splits

• Mergers Many-to-one recodes.
• Splits One-to-one recodes.
• Mergers splits Many-to-many recodes.
• All of these occur when comparing ISCO88 to
  ISCO08.
• When we crosswalk from 88 to 08 (and have no
  further information), only mergers are relevant.
• When we have ISCO88 and further information (like
  original verbatim info of original source
  classification), we also need to consider splits.

33
Mergers
34
Splits
35
Major groups

• 10 major groups Essentially unchanged, with
  minor changes of titles.
• However If minor groups have been moved between
  major groups (see below), this de facto changes
  major groups too!
• The major group that is likely most affected by
  such shifts is (5000) and in particular (5200)
  Sales Workers, that now contains a number of
  Elementary Sales Occupations.

36
Sub-major groups (2 digits)

• 34 sub-major groups expanded from 28 major
  groups.
• Truly NEW
• (0100, 0200, 0300) Army ranks (3x)
• (9400) Food Preparation Workers
• Other new major groups are upgraded or
  merged minor groups. Roughly speaking, about
  half of the sub-major groups has remained the
  same, the other half has a different composition
  than in 1988.

37
ICT occupations

• Altogether, ISCO-08 distinguishes ca. 20 ICT
  occupations, that occur at several levels
• (2500) ICT Professionals (11x)
• (3500) ICT Technicians (5x)
• (1330) ICT Service Manager (1x)
• (2356) ICT Teachers (1x)
• (2434) ICT Sales Professionals (1x)
• Neither (2500) nor (3500) are new actually both
  existed already in ISCO-68!

38
Problem 1 Imperfect skill orientation

• Some ambiguities between (7000) Craft Workers,
  and (8000) Machine Operators have been removed.
• An NEW feature is the distinction between (8100)
  Stationary Machine Operators, and (3130) Process
  Control Technicians, which probably refers to the
  complexity of the process / machine controlled /
  operated.

39
Problem 2 Employment status

• Although somewhat indirect, ISCO-08 has better
  fitting codes for Large Entrepreneurs and
  Foreman.
• There is an ambiguous distinction between (1420)
  Retail and Wholesale Trade Managers, and (5221)
  Shop Keepers.

40
Problem 3 Managers

• The implicit reference to firm size (i.e. number
  of departments) has disappeared, the same things
  are now referred to by main activity.
• At the sub-major group level Corporate Managers
  are now longer grouped with department managers,
  but with (high) Government Officials.
• Major changes occur at the 3-digit and 4 digit
  level.
• (1330) ICT Services Managers
• (1340) Professional Services Managers (9x)

41
Problem 4 Farmers

• Self-employed farmers can still be coded in as
  (1310) Managers in Agriculture etc.
• However, it also remains possible to code them
  with (6100) Market-oriented Skilled Agricultural
  Workers.
• Interestingly, a NEW feature is that (6200)
  Subsistence Farmers has now four minor groups.

42
Problem 5 Crude Sales / Service

• Sales salespersons are split
• (5221) Shop Keepers
• (5222) Shop Supervisors
• (5223) Shop Sales Assistants
• This is an improvement.
• Also, more levels and locations of sales (market,
  stall, cashiers) have been regrouped in the
  sub-major group (5200).
• This has made the sub-major group even more
  heterogeneous than it was.

43
Interesting ..

• Cooks are now split up into
• (3434) Chef a Culinary Associate Professional
• (5120) Cooks
• (9400) Food Preparation Workers
• (9411) Fast Food Preparers
• (9412) Kitchen Helper
• I am very happy with this...

44
Problem 6 Crude occupations

• Some of the new features mend this problem
• Foreman can now be classified as (3120)
  Production Supervisor.
• Shop keeper can go in two places.
• Skilled Worked can be more conveniently coded
  as (7000).

45
Interesting ...

• Specialized Secretaries and Office Managers are
  now in (3000) Associate Professionals.
• Some new occupations
• (2230) Traditional and Complementary Health
  Professional
• (5245) Service Station Attendant
• (7234) Bicycle Repairman
• (9334) Shelf Filler
• (9412) Kitchen Helper
• Disappeared
• (2121) Mathematician, Statistician
• (6142) Charcoal Burner

46
How can we reclassify existing data?

• A simple conversions of ISCO-88 into ISCO-08 is
  not possible.
• Conversion tool will become available, that will
  do two things at the same time
• Straight recode of ISCO-88 into ISCO-08 (best
  fit). Truncate trailing decimals, if this is the
  only thing that you want or can do.
• Trailing decimals suggest the amount of
  alternatives (splits). You will have to consult a
  separate document to list these options. For this
  to be usefull you will need original strings or
  classifications.