Rising prices - the consumer’s view

Inflation has become a news headline in Europe, with Eurostat reporting that the average increase in prices from last year was 8.1%. At the same time, consumers see price increases for some everyday items increasing by substantially more. Petrol prices in Germany, for example, have increased by more than 20% since last year. In the Netherlands, the price for meat has increased by more than 10% from last year, surpassing price increases in other categories. Inflation captures the average person buying an average number of goods and services—but a meat-loving driver will experience inflation much differently than a vegetarian using public transport.

Not only will groups experience inflation in a different way, but inflation also causes people to change their behaviour to adjust for rising prices. Higher petrol prices cause people to drive less or buy more fuel-efficient vehicles or, in some cases, to take public transit. Some expect higher fuel prices to further drive the electrification of personal automobiles, whether hybrid or fully electric. But even here, different groups can substitute in different ways. People living in rural areas have fewer opportunities to reduce driving or take public transportation, meaning that they cannot compensate for higher fuel prices in a way that others living in dense city centres can.

Rising prices - the economist’s view

These habits are important because inflation is measured based on an average “basket” of goods and services that people buy. This basket contains a wide range of goods, such as food, appliances, and footwear, as well as services, such as maintenance costs. When people substitute one good for another, then the content of that basket should change to properly reflect inflation, as the think tank Bruegel points out. If apples become prohibitively expensive, but pears are a ready substitute that people are buying, it does not necessarily make sense to continue to measure price increases in apples.

These substitutions are why many countries, including those in the eurozone, still rely on indexes of consumer prices to measure inflation over the long-term rather than looking at constantly changing consumer habits. The Euro area’s Harmonised Index of Consumer Prices (HICP), for instance, tracks the price of 295 goods and services from 19 Euro area countries, broken down by categories like food, transport, health, and housing.[1] Price changes are recorded monthly or annually, but what goods and services go into HICP’s basket changes only every February.  Changing the basket only once a year gives officials time to try to understand changing consumption patterns. While these updates are necessary, they are by definition slow and fail to capture immediate reactions of inflation to sudden shocks, like the global pandemic and war.

New datasets, including open ones, offer economists and policy-makers a chance to understand better how people are consuming goods and services in real time. The greater the variety of data—and the more frequently it is updated—the more accurate the picture can be for those looking to understand how inflation is being experienced.

Statistics offices on the forefront of new inflation data

New Zealand has been studying how to collect and use price information from new sources and is developing a new framework to integrate and calculate inflation. They point out that these new data sources can clearly give a close-to full coverage of products compared to more traditional frameworks. Supermarket scanner data, collected in co-operation with New Zealand businesses, also provides an almost real-time picture of substitutions that consumers are making on a daily basis.

Similarly, the UK Office for National Statistics (ONS) has ambitious plans to leverage new sources of data to produce aggregate measures of consumer-price statistics by January 2023, also looking to source web-scrapped pricing and scanner data from retailers. For web-scraped data, the ONS plans to work with pricing data scraped by a third-party data collector, MySupermarket, and has developed relationships with various retailers to collect data on consumer purchases.

The role of open data and “data spaces”

Many of the datasets, particularly on consumer-consumption patterns, are not traditionally associated with open-data portals. Companies that rely on sales are reticent about sharing information about their customers, given concerns around data privacy and commercial confidentiality. This is not to say, however, that open-data portals do not already have access to valuable data sets that can help economists and policymakers to better understand inflation.

One important component of the basket of goods are government-provided goods and services. This can include, for example, information on public transportation. Here, Bordeaux Métropole in France collects point of sale information on transport tickets. At the same time, web scraped pricing data could also be provided online as an open-data source, and already exists on data.europa.eu as indices using web scraped price data in the United Kingdom.

Getting a complete picture of inflation for statistics offices will likely require co-operation between data providers of both open and closed data. This is where the European Union’s framework for data spaces—a topic addressed in a recent webinar on data.europa.eu—can help facilitate co-operation.

A potential way forward for the eurozone?

Assuming that data collectors can co-operate to provide the right data ecosystem to bring a more accurate picture of inflation, economists and policy-makers still face challenges dealing with a more nuanced and much larger dataset. New measuring methods need to account for large volumes of data and high product churn. New index calculation methods need to understand how to automatically adapt to changes of quality, the appearance of new services, or the rapid swinging of seasonal product prices such as vegetables and fruits.

Ultimately, these new data sources can lead to an unlimited number of baskets for goods and services, covering all kinds of products for all kinds of consumers. The wider coverage of this data also implies the inclusion of information conceptually out of the scope of normal inflation indexes (such as businesses expenditures).

Nonetheless, these methodological challenges, once resolved, can lead to potentially huge benefits for a better understanding of inflation and open-data portals have an important voice in the development of this new data ecosystem that could better measure inflation.

[1] Under “food”, for example, one can find the sub-category “meat” and then the price of different types of meat: from beef to lamb, to edible offal.