Data collection methods in official statistics are the techniques that government statistical agencies use to gather the raw information behind published figures on prices, employment, output, income and population. The main sources are sample surveys of households and businesses, full censuses, and administrative records created by other parts of government. Each source has its own strengths, costs and sources of error, and modern statistical systems increasingly combine them.
The terms below cover who collects the data, how samples are designed and why some fail, and two of the most important data sources for economic statistics. They are linked closely: the national statistical office designs a survey, chooses a sampling frame and a sampling method, adjusts for those who do not respond, and increasingly supplements or replaces survey answers with administrative data.
National Statistical Office (NSO)
A national statistical office (NSO) is the government agency with lead responsibility for producing a country's official statistics. Examples include the Office for National Statistics (ONS) in the United Kingdom, Statistics Canada and the Australian Bureau of Statistics (ABS). Its typical tasks are running the population census, compiling national accounts, measuring prices and the labour market, and coordinating the wider national statistical system, which also includes central banks and ministries that produce statistics.
Some countries are centralised, with one agency doing most of the work, while others are decentralised. The United States is the classic decentralised case: the Bureau of Labor Statistics (BLS), the Census Bureau and the Bureau of Economic Analysis each produce major series. The United Nations Fundamental Principles of Official Statistics stress professional independence, impartiality and equal access, so that figures are trusted regardless of the government in power.
Probability Sampling
Probability sampling is any method of selecting a sample in which every unit in the target population has a known, non-zero chance of selection. Because the selection probabilities are known, statisticians can produce unbiased estimates and calculate the sampling error, the uncertainty that arises because only part of the population was observed. It is the standard approach for household surveys such as the Labour Force Survey (LFS).
The main designs are:
- Simple random sampling, where every unit has the same chance of selection.
- Stratified sampling, where the population is divided into groups such as regions or firm sizes and a sample is drawn from each.
- Cluster sampling, where whole areas or groups are selected, reducing interview travel costs.
- Systematic sampling, which takes every k-th unit from an ordered list after a random start.
- Probability proportional to size, often used in business surveys so that large firms are more likely to be included.
Non-Probability Sampling
Non-probability sampling selects units without known selection probabilities. Common forms are quota sampling, where interviewers fill set numbers of people with given characteristics; convenience sampling, which uses whoever is easiest to reach; judgement or purposive sampling, where experts choose representative units; and cut-off sampling, which covers only the largest units above a threshold.
These methods are cheaper and faster, and in some areas of official statistics they are normal practice. Price collectors for a Consumer Price Index (CPI), for example, often choose representative products and outlets by judgement within a probability-selected set of locations, and business surveys may cover only the largest firms that account for most of an industry's output. The drawback is that sampling error cannot be measured validly, and estimates may be biased if the chosen units differ from the rest of the population in ways that matter.
Non-Response Bias
Non-response bias is the error that arises when units selected for a survey do not respond, or skip some questions, and those non-respondents differ systematically from those who do respond. If, for example, people who work long hours are harder to reach, a labour survey may understate employment. A low response rate does not automatically cause bias, but it raises the risk and leaves estimates resting on fewer observations.
Statisticians reduce the problem by following up non-respondents, then correct for what remains through weighting, which gives more weight to respondents who resemble the missing units, and imputation, which fills missing answers using models or similar units. Falling response rates in many countries have made this a pressing issue. In the United Kingdom, declining responses to the Labour Force Survey led the ONS to warn about the reliability of labour market estimates and to keep improving the LFS while repeatedly delaying the switch to its Transformed Labour Force Survey (TLFS).
Household Budget Survey
A household budget survey collects detailed information on what households spend, and usually on their income, over a defined period. Respondents typically complete an interview and keep a diary of purchases for one or two weeks. The results show how spending is split between food, housing, transport, recreation and other categories, and how patterns differ by income, age and region.
These surveys are a key source of expenditure weights for consumer price indices, alongside national accounts data on household consumption, and they are used to study living standards, poverty and the distributional effects of taxes. In the United Kingdom the Living Costs and Food Survey fills this role; in the United States it is the Consumer Expenditure Survey, sponsored by the BLS. Many developing countries run similar surveys, which also inform the measurement of poverty lines.
Administrative Data
Administrative data are records that government bodies and other organisations collect to run their programmes, such as tax returns, social security contributions, benefit claims, company registers, customs declarations and vehicle registrations. Statistical offices reuse them to produce statistics; customs records, for instance, are the basis of trade statistics, and records held by HM Revenue and Customs (HMRC) feed UK earnings and business data.
Compared with surveys, administrative data are cheap to obtain, cover whole populations and impose no extra burden on respondents, which avoids sampling error and non-response. Their weaknesses are that definitions follow administrative rather than statistical needs, coverage can change when laws change, and records may arrive late or contain errors that nobody checks. Access also depends on legal agreements and privacy safeguards. Many NSOs now combine administrative sources with smaller surveys and use the records as sampling frames.
Comparing the main data sources
| Source | Coverage | Main strength | Main weakness |
|---|---|---|---|
| Probability sample survey | Sample of the population | Measurable sampling error, statistical definitions | Cost and non-response |
| Non-probability sample | Selected units | Cheap and quick | Error cannot be measured |
| Administrative data | Everyone in the system | Full coverage, low cost | Definitions set for other purposes |
International guidance on these methods is produced by bodies such as Eurostat and the UN Statistical Commission, and the standards that make data comparable are described in international statistical data standards and statistical units in economic statistics.
Ref 1270000041