Index number bias and tests is the part of index-number theory that explains why no single price index can be exactly right and how statisticians judge the formulas they use. The underlying difficulty, the index number problem, is that many separate price and quantity changes must be summarised in one number, and different reasonable formulas give different answers. Economists define the ideal target as a cost-of-living index and measure the errors of practical indices against it, chiefly substitution bias from fixed baskets and chain drift from linking indices period by period.
A second, axiomatic approach sets out tests that a good formula should pass, such as the time-reversal test and the factor reversal test. These ideas shape real decisions: pensions, tax brackets and wages are often raised through a cost of living adjustment based on measured inflation, so small biases add up to large sums over time. The general difference between price and volume measures is covered under nominal and real values.
Index Number Problem
The index number problem is the impossibility of combining a set of individual price and quantity changes into one price index and one volume index that are correct in every sense. When some prices rise, others fall and buyers change what they purchase, there is no unique answer to the question of how much prices rose on average. The Laspeyres price index weights price changes by base-period quantities, the Paasche index by current-period quantities, and the Fisher ideal index takes the geometric mean of the two:
Here is the Laspeyres index and the Paasche index. Because buyers tend to buy less of goods whose relative price has risen, Laspeyres usually exceeds Paasche, and the gap widens as price changes become more dispersed and as the two periods move further apart. Index-number theory narrows the choice using economic and axiomatic arguments, and national statistical offices document their chosen price index formulas for that reason.
Cost-of-Living Index
A cost-of-living index (COLI) is the theoretical index that measures the change in the minimum cost of reaching a given standard of living, or utility level, when prices change. Unlike a fixed-basket index, it lets the basket change: if beef becomes dearer and chicken cheaper, a household can switch towards chicken and stay as well off for less than the cost of its old basket. The COLI cannot be observed directly, because utility is not measured, but it is the benchmark against which practical indices are judged, drawing on the theory of utility and preferences. Under standard assumptions the Laspeyres index is an upper bound for a COLI based on the base-period standard of living, and the Paasche index a lower bound for one based on the current standard. Superlative indices such as Fisher and Törnqvist approximate the COLI closely. The gap between a practical index such as the consumer price index and the true COLI is called cost-of-living bias; the cost of living itself refers to the level of prices faced by households.
Substitution Bias
Substitution bias is the overstatement of the rise in the cost of living by a fixed-basket price index, because the index ignores consumers switching from goods that have become relatively dearer to those that have become relatively cheaper. It arises at two levels: between broad categories, in the weighted upper level of most CPIs, and between varieties within an elementary aggregate, where the choice of elementary index formula matters. The best-known estimate came from the Boskin Commission, an advisory panel to the US Senate, which reported in 1996 that the US CPI overstated inflation by about 1.1 percentage points a year; a few tenths of this came from substitution and the rest from quality change, new goods and shifts between shops. In response the Bureau of Labor Statistics adopted geometric means at the elementary level in 1999 and began publishing a chained CPI in 2002. Updating expenditure weights more often also reduces the bias.
Chain Drift
Chain drift is the tendency of a chained index to record a change over a period in which prices and quantities have returned to their starting values. A chained index multiplies together period-to-period links so that weights can be updated often, as described under base periods and chaining. If prices bounce, for example because of temporary sales, the links may not cancel. A chained Carli index of two items shows the problem: if item A costs 1, 2 and then 1 over three months while item B costs 2, 1 and then 2, the first link is and the second link , so the chained index reads , a 56% rise, although both prices are back where they began. A chained Jevons index gives 1 for each link and shows no drift. Weighted chained indices can also drift when quantities react strongly to short-lived price cuts, a common issue with scanner data. Statisticians limit drift by chaining annually rather than monthly and by using multilateral methods for high-frequency data.
Factor Reversal Test
The factor reversal test, proposed by the American economist Irving Fisher, requires that a price index multiplied by the matching quantity index, built with the same formula but with prices and quantities swapped, equals the change in total value:
- and are the price and quantity indices.
- and are prices and quantities in the base and current periods.
- The right-hand side is the ratio of total spending in the two periods.
Take two goods: A's price rises from 1 to 2 while its quantity falls from 10 to 8, and B's price stays at 2 while its quantity rises from 5 to 6. Spending goes from 20 to 28, a value ratio of 1.4. The Laspeyres price index is and the Laspeyres quantity index , whose product of 1.5 fails the test. The Fisher price index is and the Fisher quantity index , whose product is exactly 1.4. The Fisher index passes, one reason Fisher called it ideal and the US Bureau of Economic Analysis uses it for GDP volume measures.
Which formulas pass which tests
| Index | Time-reversal test | Factor reversal test | Substitution bias |
|---|---|---|---|
| Laspeyres | Fails | Fails | Upward |
| Paasche | Fails | Fails | Downward |
| Fisher | Passes | Passes | Small |
| Carli (unweighted) | Fails | Not applicable | Upward |
| Jevons (unweighted) | Passes | Not applicable | Smaller |
Quality change is a further source of bias, handled with techniques such as hedonic adjustment.
Ref 1100001800