Statistical Journal of the IAOS

Many National Statistical Offices are modernizing the systems and processes underpinning the production of official agricultural statistics. Moving data and processes to the cloud, collecting survey data via the web, automating editing and imputation, incorporating more administrative, remotely sensed and other non-survey data in the estimation process, and more flexible dissemination of information are only some of the areas of current efforts. Although specific modernization efforts have been described, less discussion has been focused on exactly what the future of official agricultural statistics will be. During the 9^th International Conference on Agricultural Statistics, which was held May 17–19, 2023, at the World Bank in Washington DC USA, four statistical leaders with diverse perspectives envision the not-too-distant future of official agricultural statistics in 2040.

A country’s statistical capacity takes an indispensable part in its development. We offer a comprehensive comparison between the World Bank’s Statistical Performance Indicators and Index (SPI) and its predecessor, the Statistical Capacity Index (SCI) regarding different conceptual and empirical aspects. We further examine the relationships of the two indexes with some agriculture development indicators such as food security, food sustainability and productivity as well as other key indicators including headcount poverty, GDP per capita, and an SDG progress index. Our analysis employs the latest SPI data update in 2022, which were not available in previous studies. We also propose clear guidelines on how the SPI can be maintained and updated in the future to ensure that this process is transparent, replicable, safeguarded with high quality, and provides comparable data over time.

As is the case for many National Statistics Institutes, the United States Department of Agriculture’s (USDA’s) National Agricultural Statistics Service (NASS) has observed dwindling survey response rates, and the requests for more information at finer temporal and spatial scales have led to increased response burdens. Non-survey data are becoming increasingly abundant and accessible. Consequently, NASS is exploring the potential to complete some or all of a survey record using non-survey data, which would reduce respondent burden and potentially lead to increased response rates. In this paper, the focus is on a large set of records associated with potential farms, which are operations with undetermined farm status (farm/non-farm) and are referred to here as operations with unknown status (OUS). Although they usually have some agriculture, most OUS records are eventually classified as non-farms. Those OUS that are classified as farms tend to have higher proportions of producers from under-represented groups compared to other records. Determining the probability that an OUS record is a farm is an important step in the imputation process. The OUS records that responded to the 2017 U.S. Census of Agriculture were used to develop models to predict farm status using multiple data sources. Evaluated models include bootstrap random forest (RF), logistic regression (LR), neural network (NN), and support vector machine (SVM). Although the SVM had the best outcomes for three of the five metrics, the sensitivity for identifying farms was the lowest (13.8%). The NN model had a sensitivity of 80.5%, which was substantially higher than the other models, and its specificity of 45.3% was the lowest of all models. Because sensitivity was the primary metric of interest and the NN performed reasonably well on the other metrics, the NN was selected as the preferred model.

Gridded landcover datasets like the NASS Cropland Data Layer (CDL) provide a useful resource for analyses of cropland management. However, many farm operation decisions are made at the field level, not the pixel level. To capture relationships between land cover and field characteristics – size, contiguity, etc. – some method is needed to aggregate gridded data into crop fields. To provide a uniform and consistent approach for aggregation of gridded data at the field level over a series of years, this research project developed a set of Crop Sequence Boundaries (CSBs), which are polygons that delineate areas of homogeneous cropping sequences for the contiguous US. The CSBs are open-sourced algorithm-based, geospatial polygons derived using historic CDLs together with road and rail networks to capture areas with common cropping sequences. The CSB approach used geospatial functions in Google Earth Engine (GEE) and in the ArcGIS Pro application. These geospatial functions are run in parallel by sub-dividing the contiguous US into smaller regions based on road and rail boundaries to prevent overlaps or gaps in the data. As a new set of algorithmically delineated field polygons, the CSBs enhance applications requiring large-scale crop mapping with vector-based data.

Over the past decade, national statistical offices in low- and middle-income countries have increasingly transitioned to computer-assisted personal interviewing and computer-assisted telephone interviewing for the implementation of household surveys. The byproducts of these types of data collection are survey paradata, which can unlock objective, module- and question-specific, actionable insights on survey respondent burden, survey costs, and interviewer effects – all of which have been understudied in low- and middle-income contexts. This study uses paradata generated by Survey Solutions, a computer-assisted personal interviewing platform used in recent national household surveys implemented by the national statistical offices of Cambodia, Ethiopia, and Tanzania. Across countries, the average household interview, based on a socioeconomic household questionnaire, ranges from 82 to 120 minutes, while the average interview with an adult household member, based on a multi-topic individual questionnaire, takes between 13 to 25 minutes. The paper further provides guidelines on the use of paradata for module-level analysis to aid in operational survey decisions, such as using interview length to estimate unit cost for budgeting purposes as well as understanding interviewer effects using a multilevel model. Our findings, particularly by module, point to where additional interviewer training, fieldwork supervision, and data quality monitoring may be needed in future surveys.

Household Consumption and Expenditure Surveys (HCES) collect comprehensive information on households’ consumption and can provide a range of analyses on access to food. They are key to estimating poverty (SDG 1.2.1) and Prevalence of Undernourishment (SDG 2.1.1).

Before the food data becomes meaningful for analysis, it needs extensive preparation. While NSOs are responsible for poverty statistics and typically prepare the data for this purpose, it is often organizations or researchers that use the data for food security. Although the preparation of the data for these two purposes has a lot in common, they rely on different traditions and guidelines.

This paper presents results from an ongoing project that aims to bridge the gap between these two processes. The project’s goal is that NSOs take the lead in preparing the HCES food data for all uses. An expected result is that the food security statistics will be available at the same time as the other main outputs from the survey and can be used for planning for improved food security. The project includes preparing a guideline for NSOs and others (endorsed by the United Nations’ Statistical Commission in 2024), building capacity in NSOs, and using results in a regional context.

This paper presents an approach to estimate the between-subject variability in nutrient intake (through the coefficient of variation [CV]) and a method to estimate the prevalence of nutrient inadequacy (PoNI) (for eight micronutrients) using household consumption and expenditure survey (HCES) data. Prevalence values are compared to individual-level estimates derived using the National-Cancer-Institute method. Data come from the 2015 Bangladesh Integrated-Household-Survey, which conducted a household-level 7-day recall (7DR) and two rounds of individual-level 24-hour recall (24HR), filled by one respondent on behalf of all members, for the same rural households. The PoNI values based on 7DR are lower than those calculated from 24HR data, due to the larger average intake estimates from 7DR data. After controlling for differences in average intake estimates and adjusting household-level data for random measurement errors, the PoNI values from 7DR and 24HR data are remarkably close. This highlights the potential use of HCES data (conducted according to international agreed standards) for estimating the level of between-subject variability in usual nutrient intake in a population. The CVs from HCES could be used to compute the PoNI using average intake estimates from individual-level data; and the inadequacy of global nutrient supply using Supply and Utilization Accounts data.

The question “What is a small-scale producer?” keeps receiving different answers depending on the context in which is posed. Alternative ways of defining smallholders reflect heterogeneous historical and institutional eco-systemic contexts and depend upon what is the role of small-scale agriculture in the rural economy. This has become a pressing issue given the need to monitor the Sustainable Development Goals (SDGs), which refers to “small” farmers. Two important related issues are: 1) the adoption of a specific and robust definition of small-scale food producer (SSFP) and 2) the empirical implementation of this definition to determine the SSFPs. The calculations require suitable databases with microdata at the level of individual farms. Based on the 2020 agricultural census results, we identified the small food producers in Italy. We also proposed and compared other approaches to identify SSFPs, that are simpler than that proposed by the FAO and could also be calculated for other census years. Since revenues are not available for every farm – even the census did not collect this information – the standard indicator of production was used instead of revenues to identify SSFPs.

The recent increasing attention to the economic and policy analysis of the food systems from international fora, public institutions and academia calls for the availability of information and data capable of informing about the interrelations across economic sectors and within value chains. The international policy agenda is pushing for a more effective application of measures at country and regional level in line with the recommendations of the 2030 Agenda and its Sustainable Development Goals, for which more systematic and integrated data about economic, social and environmental impacts of policies are requested. The Food Value Chain Domain recently published in FAOSTAT responds to this call. Its data and information shed light on the distribution of final domestic food expenditures across industries (Agriculture, Food Processing, Wholesale, Retail, Accommodations and Food Services) and primary factors (e.g.: Labour, Gross Operating Surplus) on the relative food value chain. The FAOSTAT Domain offers therefore robust and granular information on both the farm and the post-farm gate component of the Food Value Chain.

The applied Global Food Dollar methodology, that FAO is contributing to upscale at global level, is based on Leontief decomposition approach on the Input-Output tables. Moreover, whenever the Input-Output table are not available, it is now possible to impute them from Supply-Use tables by applying a conversion methodology, developed by FAO in compliance with European (EUROSTAT), United Nations (UNSD) and international statistical standards as the System of National Accounts. This allows to extend the analysis to several African, Asian, and Latin American countries that produce on regular basis only Supply and Use Tables, and not Industry by industry Input Output Tables. The potential time and data coverage of the methodology is therefore significantly expanded.

The aim of this paper is to describe the conceptual framework of the conversion methodology of Supply-Use Tables into Input-Output Tables of the Global Food Dollar methodology, and the potential implementation scope of these methodologies. Preliminary analytical findings of the applied methodologies are presented as well.

The new methods and data presented in this paper, being based on data compliant with the International Statistical Standards, as the System of National Accounts, and therefore comparable across countries, associated to larger data availability, have the potential to effectively support food policies at international, regional and national level, as well as contribute to a decision making in line with the 2030 Agenda.

This paper proposes a conceptual and empirical framework to develop rural transformation strategies tailored to the agroecological potential and market access of rural areas in Pakistan. Such a framework allows to move away from stereotypical countrywide policies as in use in Pakistan and many other countries. Using publicly available geospatial measures of vegetation greenness and an urban gravity model to proxy the agricultural market demand, we classify Pakistan’s rural districts into categories with similar comparative advantages and describe dominant livelihood activities. The framework recommends market-based approaches to support commercial agriculture or non-agriculture business development in well-connected areas and where households have accumulated human and physical capital. In areas with less developed agricultural potential or market access, households will benefit from area-based and community-driven development, skill development, and labor programs. Since data collection is often challenging in rural areas, statistical agencies can use such an empirical framework to advise policymakers on prioritizing public investments and tailoring rural transformation pathways. In addition, statistical agencies can also extend the framework at different levels of resolution, from national to local level, and complement it with primary data sources to validate the usefulness of the approach.

Rising food prices may rapidly push vulnerable populations into food insecurity, especially in developing economies and in low-income countries, where a substantial share of the financial resources available to the poorest households is spent on food. To capture soaring food prices and help in designing mitigating measures, we developed two complementary products: a nowcasting model that estimates official food consumer price inflation up to the current month and a daily food price monitor that checks whether the growth rate of a few basic food commodities exceeds a statistical threshold. Both products were designed with the consideration that the rapid acquisition of data and the automated extraction of insights are indispensable tools for policymakers, particularly in times of crisis. Our framework is characterized by three key aspects. Firstly, we leverage two non-traditional data sources to emphasize the importance of real-time information: a crowdsourced repository of daily food prices and textual insights obtained from newspapers articles. Secondly, our framework offers a global perspective, encompassing 225 countries and territories, which enables the monitoring of food prices dynamics on a global scale. Thirdly, results are made accessible daily via an intuitive and user-friendly interactive dashboard.

This study focuses on the statistical downscaling of ERA5-Land reanalysis data using the Statistical DownScaling Model (SDSM) to generate climate change scenarios for the Spree catchment. Linear scaling was used to reduce the biases of the Global Climate Model for precipitation and temperature. The statistical analyses demonstrated that this method is a promising and straightforward way of correcting biases in climate data. SDSM was used to generate climate change scenarios, which considered three emission scenarios: RCP 2.6, RCP 4.5, and RCP 8.5. The results indicated that higher precipitation is expected under higher emission scenarios. Specifically, the summer and autumn seasons were projected to experience up to 50 mm more rainfall in the next 80 years, and the temperature was projected to increase by up to 1^∘C by 2100. These projections of climate data for different scenarios are useful for assessing water management studies for agricultural and hydrologic applications considering changing climate conditions. This study highlights the importance of statistical downscaling and scenario generation in understanding the potential impacts of climate change on water resources. The results of this study can provide valuable insights into water resource management, especially on adapting to changing climate conditions.

Statistics Netherlands’ social surveys are based on a sequential mixed-mode data collection approach using web, telephone, and face-to-face interviewing. This article illustrates how Statistics Netherlands addressed the sudden, unforeseen loss of face-to-face interviews in social surveys amidst the COVID-19 pandemic. At the beginning of the pandemic, survey processes were immediately adjusted in several ways to mitigate the negative effects of respondent attrition. Where possible, sampled people initially assigned to face-to-face interviewing were motivated to respond through web or telephone to minimize the loss of response. At the same time regression analysis and simulation were conducted to obtain quantitative insight into the effects of losing face-to-face responses in the sequential mixed-mode designs. Furthermore, alternative model-based estimation procedures based on structural time series models were implemented to compensate for the bias that is a result of the loss of face-to-face responses. These initiatives are illustrated with applications to the Dutch Labor Force Survey, the Housing Survey, and the Health Survey.

Taking a representative sample to determine prevalence of variables such as disease or vaccination in a population presents challenges, especially when little is known about the population. Several methods have been proposed for second stage cluster sampling. They include random sampling in small areas (the approach used in several international surveys), random walks within a specified geographic area, and using a grid superimposed on a map. We constructed 50 virtual populations with varying characteristics, such as overall prevalence of disease and variability of population density across towns. Each population comprised about a million people spread over 300 towns. We applied ten sampling methods to each. In 1,000 simulations, with different sample sizes per cluster, we estimated the prevalence of disease and the relative risk of disease given an exposure and calculated the Root Mean Squared Error (RMSE) of these estimates. We compared the sampling methods using the RMSEs. In our simulations a grid method was the best statistically in the great majority of circumstances. It showed less susceptibility to clustering effects, likely because it sampled over a much wider area than the other methods. We discuss the findings in relation to practical sampling issues.

Identification and replacement of erroneous data is of fundamental importance for the quality of statistical surveys. If statistical units are continuously sampled over an extended period, time series methods can facilitate this task. Numerous outlier identification and replacement procedures are accessible for this particular purpose, like RegArima Approaches within the seasonal adjustment procedures in X13-Arima or Tramo/Seats. These algorithms can be used to identify different types of outliers, like additive outliers, level shifts or transitory changes. In this paper an alternative outlier identification procedure is proposed which is based on a nonlinear model estimated with support vector regressions. The focus of this procedure is on the identification of additive outliers and on the applicability for short time series with less than 3 years of observations.

Predictors of macroeconomic indicators rely primarily on traditional data sourced from National Statistical Offices. However, new data sources made available from recent technological advancements, namely data from online activities, have the potential to bring about fresh perspectives on monitoring economic activities and enhance the accuracy of forecasting. This paper reviews the literature on predicting macroeconomic indicators, such as the gross domestic product, unemployment rate, consumer price index or private consumption, based on online activity data sourced from Google Trends, Twitter (rebranded to X) and mobile devices. Based on a systematic search of publications indexed on the Web of Science and Scopus databases, the analysis of a final set of 56 publications covers the publication history of the data sources, the methods used to model the data and the predictive accuracy of information from such data sources. The paper also discusses the limitations and challenges of using online activity data for macroeconomic predictions. The review concludes that online activity data can be a valuable source of information for predicting macroeconomic indicators. However, one must consider certain limitations and challenges to improve the models’ accuracy and reliability.

Today, there is a greater demand to produce more timely official statistics at a more granular level. National Statistical Institutes (NSIs) are more and more looking to novel data sources to meet this demand. This paper focuses on the use of one such source to compile more timely and detailed official statistics on port visits. The data source used is sourced from the Automatic Identification System (AIS) used by ships to transmit their position at sea. The primary purpose of AIS is maritime safety. While some experimental statistics have been compiled using this data, this paper evaluates the potential of AIS as a data source to compile official statistics with respect to port visits. The paper presents a novel method called “Stationary Marine Broadcast Method” (SMBM) to estimate the number of port visits using AIS data. The paper also describes how the H3 Index, a spatial index originally developed by Uber, is added to each transmission in the data source. While the paper concludes that the AIS based estimates won’t immediately replace the official statistics, it does recommend a pathway to using AIS-based estimates as the basis for official port statistics in the future.

In a rapidly globalizing world, understanding the relationships between major stock markets is of paramount importance for investors and financial analysts. This study explores the interdependence and cointegration of stock markets in Japan, India, and the USA, and explores the dynamics of global financial markets as well as the survival of a long-term and short-term link between these three indices. These leading stock markets were selected because of the researchers’ desire to learn more about the connections between them. From April 2012 through March 2022, we used monthly data from three major stock market indices: the NIKKEI (Japan), theBSE SENSEX (India), and the NASDAQ (USA). Stock market performance in both the United States and India tend to move together. Additionally, the GC test is utilized in an effort to ascertain if the markets have any form of forecasting ability. Based on the results of the tests conducted, it was determined that the NASDAQ index can accurately predict the SENSEX index, but the NIKKEI index. The United States and the Indian stock markets are highly correlated. To further investigate the markets’ potential for foresight, the Granger causality test is applied. Tests showed that while the NASDAQ index predicted the SENSEX index with high precision, the NIKKEI index did not. After a causal relationship has been established, we then look for evidence of a short- and long-term connection.

This study offers a comprehensive examination of economic growth and convergence in the Indian States and Union Territories (U.T.) over the period from 1991 to 2020. It investigates absolute, sigma, and conditional convergence within this diverse set of states and utilizes the augmented Solow and extended Solow models to explore conditional convergence dynamics. The empirical findings reveal several significant insights. First, there is no substantial negative correlation between the initial per capita GDP ratio and the average annual growth rate, indicating the absence of absolute convergence across the Indian States and U.T. economies during the study period. These results align with those obtained from sigma convergence analysis, reinforcing the absence of widespread convergence. However, conditional convergence is observed, as evidenced by the rate of conditional convergence (coefficient of initial GDP per capita) estimated at 0.038 among the Indian States and U.T. The presence of conditional convergence implies that while initial conditions matter, other factors, including physical and human capital, population growth, and additional variables, significantly contribute to the growth and convergence of Indian regions. The study concludes that policies aimed at promoting economic growth in the Indian States should prioritize the expansion of the labor force, investments in physical and human capital, and prudent government consumption. Furthermore, fostering an environment that encourages access to new technologies and ideas, maintaining sound macroeconomic management, and increasing investments in human capital formation are essential for sustained growth. Effective resource allocation through prudent budgetary policies and heightened investments in the health sector are recommended. Incentives to reduce fertility rates and adept monetary policy management are also identified as crucial elements for ensuring stability and sustainable growth. In summary, this research underscores the importance of adopting a holistic approach to foster economic growth and convergence in the Indian States. The suggested policy measures create a conducive environment for sustained development and prosperity in this diverse and dynamic region.