Sharing best practices for the redesign of three business surveys

1
Sharing best practices for the redesign of three
business surveys

Charles Tardif, Business Survey Methods
Division,Statistics Canada presented at the
ICES-III, session 75 June 21st, 2007
2
Alternative title

Can we design a monthly Unified Enterprise
Survey?
3
Outline

• What are monthly business surveys? a simplified
  view
• Common objectives of monthly business surveys
• Statistics Canada (STC) monthly business surveys
• Monthly Survey of Manufacturers (MSM)
• Monthly Survey of the Food Services and Drinking
  Places (MFS)
• Monthly Wholesale and Retail Trade Survey (MWRTS)

4
Outline (contd)

• Commonalities between the surveys
• Comparing survey elements
• Summary A monthly unified enterprise survey?
• Conclusion

5
What are monthly business surveys? a simplified
view

• Production of monthly estimates mainly on
  financial information (sales, revenues, expenses,
  etc)
• By geographic level
• By industrial level, classified by the North
  American Industrial Classification System (NAICS)
• With a targetted quality, expressed in terms of
  Coefficients of Variation (CV).

6
Common objectives of monthly business surveys

• Measure trends and levels for key financial
  variables
• Establish best possible survey design to meet the
  surveys objectives
• Meet the 6 elements of the STC Quality Framework
• Relevance, accuracy, timeliness, accessibility,
  interpretability, coherence
• Maximise use of administrative data, mainly tax
  data, to reduce response burden

7
STC monthly business surveys
MSM Manufacturers MFS Restaurants MWRTS Retail MWRTS Wholesale
Variables of interest Shipments, inventories and orders Sales, Number of locations Sales, Number of locations Sales, Inventories
Geo level Provinces, territories Provinces, territories Provinces, territories Provinces, territories
Industrial level (NAICS) 311 to 339, at the 3rd to 6th digit 722, at the 4th digit 44 and 45, for 19 trade groups (TG) 41, for 15 trade groups (TG)
8
STC monthly business surveys (contd)
MSM Manufacturers MFS Restaurants MWRTS Retail MWRTS Wholesale
Population size (establishments or clusters of est.) 100,000 90,000 180,000 100,000
Collection units 11,000 2,100 12,000 8,000
Number of domains More than 1,000 52 247 possible domains (19 TGs) 195 possible domains (15 TGs)
Yearly revenues (in billions) 550 36 350 450
9
Commonalities between the surveys

• All surveys
• share common objectives
• undertook or completed a redesign or a
  restratification in the last 2 years
• are facing pressure to make a more extensive use
  of Tax data
• have skewed populations
• have an annual counterpart, all integrated in the
  Unified Enterprise Survey (UES)
• are managed by different subject matter
  divisions, although centralized methodological
  support.

10
Commonalities between the surveys

• One monthly unified survey?
• Well see what has been done so far to harmonize
  the different surveys.

11
Comparing survey elements
MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail
Frame Before BR, no exclusions Business Register (BR), with the exclusions of the non-employing establishments Business Register (BR), with the exclusions of the non-employing establishments
Now BR, no exclusions, using a Survey specific Universe Frame BR, no exclusions, using a Survey specific Universe Frame BR, no exclusions, using a Survey specific Universe Frame
Now BR, no exclusions, using a Survey specific Universe Frame BR, no exclusions, using a Survey specific Universe Frame BR, no exclusions, using a Survey specific Universe Frame
Sampling Unit Before Establishment level Company level Company level
Now Establishment level or cluster of establishments Establishment level or cluster of establishments Establishment level or cluster of establishments
12
Comparing survey elements
MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail
Stratification variables Before Province/territory, NAICS and a size measure Gross Business Income (GBI) from the BR Province/territory, NAICS and a size measure Gross Business Income (GBI) from the BR Province/territory, NAICS and a size measure Gross Business Income (GBI) from the BR
Stratification variables Now Province/territory, NAICS and a size measure annualized monthly data, data from annual survey, tax data, GBI Province/territory, NAICS and a size measure annualized monthly data, data from annual survey, tax data, GBI Province/territory, NAICS and a size measure annualized monthly data, data from annual survey, tax data, GBI
Take-none stratum Before Bottom 2 by province No take-none stratum 5 by geo and TGs
Take-none stratum Now Bottom 10 (MSM, MFS) or 5 (MWRTS) by province and stratification NAICS, subject to a cap. Bottom 10 (MSM, MFS) or 5 (MWRTS) by province and stratification NAICS, subject to a cap. Bottom 10 (MSM, MFS) or 5 (MWRTS) by province and stratification NAICS, subject to a cap.
13
Comparing survey elements

• Stratification Now
• Differences between surveys, but all
• are using the Lavallée-Hidiroglou algorithm
  (efficient with skewed population) to stratify
  their population with the following
  characteristics
• 1 Take-all / must-take stratum by prov/NAICS
• 1 or a few take-some strata by prov/NAICS
• Minimum sample size per stratum
• Capped design weight
• Oversampling for out-of-scopes, deaths and
  non-response

14
Comparing survey elements

• Use of GST - Now
• All surveys are making use of GST data, although
  in a different way
• MSM and MWRTS
• Micro approach, with selected units replaced by
  GST data through modeling. Units have a design
  weight gt 1.
• Use of GST data varies by NAICS for MSM
• Standard stratification design explained in the
  previous slide is used

15
Comparing survey elements

• GST micro approach, selected units

Take-alls
Take- somes 1
S1
S2
Take- somes 2
Take-nones
16
Comparing survey elements

• Use of GST - Now
• All surveys are making use of GST data, although
  in a different way
• MFS
• Micro approach, where a sample of units known as
  simples are selected to model the value of all
  the other simple units. All units have a weight
  of 1.
• Independent stratification design for these
  simple units.
• Applied for selected combinations of provinces
  and NAICS.

17
Comparing survey elements

• GST micro approach, all units modelled

Take-alls
Simples
Complex units
Take-nones
18
Comparing survey elements
MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail
Sample selection STC Generalized Sampling (GSAM) system for sample selection STC Generalized Sampling (GSAM) system for sample selection STC Generalized Sampling (GSAM) system for sample selection
Sample selection Random sampling within each stratum Random sampling within each stratum and for the GST modelling Systematic sampling within each stratum
Sample selection Same sample, sampling births every month Same sample, sampling births every month Same sample, sampling births every month
19
Comparing survey elements
MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail
Edit and imputation Before MSM in-house EI program MFS in-house EI program MWRTS in-house EI program
Edit and imputation Now BANFF, a STC generalized system for EI BANFF, a STC generalized system for EI MWRTS in-house EI program
Edit and imputation Now All are doing outlier detection, historical edits, trend and mean imputation All are doing outlier detection, historical edits, trend and mean imputation All are doing outlier detection, historical edits, trend and mean imputation
20
Comparing survey elements
MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail
Estimation STC Generalized Estimation System (GES) used to produce the estimates. STC Generalized Estimation System (GES) used to produce the estimates. STC Generalized Estimation System (GES) used to produce the estimates.
Variance Sampling variance computed using GES Computing variability from other sources (imputation, use of admin data) being considered. Sampling variance computed using GES Computing variability from other sources (imputation, use of admin data) being considered. Sampling variance computed using GES Computing variability from other sources (imputation, use of admin data) being considered.
Publication Estimates published in The Daily, a STC publication. Estimates available in CANSIM. Estimates published in The Daily, a STC publication. Estimates available in CANSIM. Estimates published in The Daily, a STC publication. Estimates available in CANSIM.
21
Summary - A monthly unified enterprise survey?

• Advantages
• Taking advantage of the best practices of each
  survey to integrate all monthly surveys
• Easier to ensure coherence between the monthly
  surveys and their annual counterpart, for
  comparison purposes
• Annual surveys are already integrated

22
Summary - A monthly unified enterprise survey?

• Advantages (contd)
• Would facilitate the implementation of changes to
  the surveys (no need for a guinea pig)
• For example, introduction of the GST was done at
  different times for the 3 surveys
• Implementation of better measures of variability
  could be done more efficiently
• Integrating more components of variability
• Variance of the differences between the estimates

23
Summary - A monthly unified enterprise survey?

• Issues - Methodological
• As seen, there are conceptual differences between
  the surveys
• Different data elements collected
• Level at which the information is collected is
  different
• Use of administrative data is different
• Although these differences could be factored in
  the sample design

24
Summary - A monthly unified enterprise survey?

• Issues - Operational
• Currently managed by three different subject
  matter divisions, but centralized methodological
  support
• Impact on the field work

25
Conclusion

• Past
• Used to be important differences between the
  monthly surveys
• Present
• Important steps have been made to share best
  practices between the monthly surveys to
  harmonize them
• Future
• Should we go one step further and integrate them
  in a Monthly Unified Enterprise Survey?

26

• For more information, please contact

• Pour plus dinformation, veuillez svp contacter

• Charles.Tardif_at_statcan.ca