Machine learning of stochastic automata and evolutionary games

Abstract

A large categories of time series fluctuate dramatically, for example, prices of agriculture produce. Traditional methods in time series and stochastic prediction may not capture such dynamics. This paper tries to use machine learning to tune the model for a real situation by establishing a price determination mechanism on the model of stochastic automata (SA) and evolutionary game (EG). Time series volatility attributed to the chaotic process can be obtained through the learning algorithm of Markov Chain Monte Carlo (MCMC). Using machine learning through the chaotic analysis of stochastic automata and evolutionary games, we find that a more spatially aggregated distribution (smaller entropy) leads to larger time series fluctuations, regardless of the initial distribution of crops. By integrating the factors discovered in this study, we can develop a better learning algorithm in a highly volatile time series in agriculture prices.

Keywords

Distribution entropy spatial diffusion stochastic automata (SA)evolutionary game (EG)machine learning

1 Introduction

The extra-ordinary fluctuation of certain time series, such as agriculture produce prices, is always difficult to be explained and predicted. The reasons behind the huge variations in prices are multi-folds. The variations in demand or production in different parts of the market should be considered in the modeling [1, 2]. The extent of price fluctuations also varies in different kinds of markets [3, 4]. For instance, while the prices of pork only change periodically, the prices of vegetables and fruits change every season. Previous studies suggest using static regression methods for the prediction and ignored the accumulated effects of interactions among individual participants, such as words-of-mouth or spatial aggregations [2 , 5].

Complementary to the conventional economic studies with linear regression analysis under an equilibrium market condition, we propose a novel approach by performing agent-based Monte Carlo simulation in a complicated yet volatile artificial market [6]. Through the help of machine learning, we can tune the parameters of SA/EG models and conclude the interactive behaviors among farmers.

In the literature, [5] argues that interactive behaviors in a market are important because the decision of one farmer affects that of another and subsequently influences the supply of agricultural products in the entire market [4, 7]. Suggests that the market supply varies because farmers have heterogeneous expectations regarding product prices and update their beliefs based on the common climate estimations and the present market conditions. This study specifically focuses on two factors in the price formation process, namely, the extent of spatial aggregation and price elasticity.

Many assumptions related to the production and decision making of farmers have also been proposed in the literature. The simplest of these assumptions is called the naïve or static expectation, which predicts the current state as the future state. That is, a farmer expects that the price of the next period is equal to that of the current period. In [8], the behaviors of farmers are assumed to be adaptive or error-learning expectations, which shape future-oriented expectations by observing the previous behavior of a certain variable. In other words, the predicted price for the previous period versus that for the current period is assumed to be adjusted from the error price for the previous period [9]. Examines the influence of extrapolative expectations on a stable market, in which the price for the next period is determined under the assumption that the prices for two consecutive periods are the same.

By combining SA and EG, we build a model for an artificial market with period-doubling chaotic behavior in price elasticity. The behaviors of individuals in this market are to be examined from a micro perspective. When these individuals gather and engage in complex interactions, the model engages superior prediction ability that overcomes extra-ordinary fluctuation in time series.

2 Machine learning model

2.1 Market price determination modeling

Differing from the classical cobweb theory for investigating the macroscopic behavior of price fluctuations in total supply and demand amount, we use SA/EG models to investigate the short-term behavior in price formation. We emphasize on the effect of individual decisions determined by their interactive effects of competition and cooperation in a market.

The SA model comprises a large number of cells, which mimic an artificial market. The state of a cell represents its decision, which is to be made based on external (e.g., climate and expected price) and internal factors (e.g., cultivation decisions of neighboring farmers). Each cell in this study represents a crop to be cultivated. We denote the states as 1 ∼ 3 for different decisions of crop types, depending on internal and external factors. The influence factor of climate is global. For example, we can formulate climatic factors, C_t, as $C_{t} = {\begin{matrix} 1, if 2 ⩽ month < 3, \\ 2, if 4 ⩽ month < 9, \\ 3, if 9 ⩽ month < 11, \end{matrix}$ (1) where C_t is the propensity of climate.

The expected price is an important external factor for each decision cell. According to cobweb theory, the expected price pattern can be formulated as in [10]. With p₀, the price at period 0, the farmers start cultivating and sending their crops Q to the market until period 1. The price p₁ for period 1 is determined by the supply and demand during this period. Afterward, farmers will make predictions and make decisions regarding their inputs for period 2. The price p₂ for period 2 is determined by the supply and demand in this period, and so on. We employ the extrapolative model (2) to evaluate the expected price of a farmer: $p_{k, t}^{e} = α_{k} p_{t - 1} + (1 - α_{k}) p_{t - 2},$ (2) where $p_{k, t}^{e}$ represents the expected price for the current period, p_t-1 denotes the actual price for the previous period, and p_t-2 denotes the actual price for the two previous periods. The adjustment coefficient α_k satisfies 0 ⩽ α_k ⩽ 2. After identifying the expected price, the propensity of market trend, E_k, can be defined as $E_{t} = {\begin{matrix} + 1, if (p_{k, t}^{e} - p_{t - 1}) > 0, \\ - 1, if (p_{k, t}^{e} - p_{t - 1}) ⩽ 0 . \end{matrix}$ (3)

If $(p_{k, t}^{e} - p_{t - 1}) > 0$ or ⩽0, the trend of the market price is expected to increase or decrease in the future, respectively.

The influence of internal factors is often difficult to be captured in traditional methods. In this study, we only consider one internal factor: the influence of neighbors. The decision of each cell in a SA will be influenced by its neighbors, and meanwhile, the decision of neighbors will also be influenced by this cell. When most neighbors are planting the same crop, those are not planting that crop will be converted to the same one. In the contrast, if each neighbor plants different crops, the influence of crowding will be weak. We adopt the neighborhood configuration in Von Neumann SA as defined in Fig. 1. The region from the central lattice grid is to be influenced by the four surrounding neighbor cells. Furthermore, the influences are mutual, because the relation of the neighborhood is commutative. The propensity of the k-th cell at period t + 1 can be defined by the rule of EG:

Fig. 1

The four neighbors in the Von Neumann SA.

$S_{k, t + 1} = sign (β {NS}_{k, t} + (1 - β) (E_{k, t} + C_{t})),$ (4) where NS_k,t denotes the propensity of the evolutionary influence of neighbors for the k-th cell at period t.

Finally, the price formation process is modeled by the change in supply $Q_{t}^{s}$ and demand $Q_{t}^{d}$ in period t. The total market supply is the sum of all cells, that is, $Q_{t}^{s} = \sum_{k = 1}^{N} S_{k, t}$ . Determining the price of crops is conditioned upon balancing supply with demand while considering the dynamic changes in their prices. We also assume a direct proportion between a price change and supply and demand $(Q_{t}^{d} - Q_{t}^{s})$ [9, 11]), the relationship of which can be expressed as $\frac{{dp}_{t}}{dt} = γ (Q_{t}^{d} - Q_{t}^{s}) for γ > 0 .$ (5)

The optimal production quantity is computed as $x_{k, t} = \frac{p_{k, t}^{e} - b_{k}}{{nB}_{k}}$ (6) while the price is defined as $p_{t} = - \frac{d}{D} \sum_{k = 1} N \frac{x_{k, t - Δ}}{D}$ (7) where Δ is a time lag with an assumed delay of Δ = 15.

2.2 Model training by MCMC

Estimating the model parameters in the previous section is highly complicated and nonlinear. This study employee MCMC approach with Rstan implementation to build a SA/EG-based production and marketing economics. Recent advances in the MCMC method offer an excellent means to perform the task.

The fitting between data and parameters is approximated by a multidimensional integral with randomly generated samples through an ergodic Markov chain. The efficient algorithm helps the generated samples converged in a large dimension of parameter space. The advanced MCMC techniques can more effectively overcome the limitations of empirical data in the distribution of long memories and extreme values, and try to identify predictable parts of uncertainty. By the sampling limitation of empirical data, it was difficult to reflect the various supplies and demand from the volume of trading. We therefore use the assimilation method, combined with resampling, time difference, valence adjustment, independent events, and other conversion functions and exogenous variables.

The simulated time series are generated based on the identified set of parameters on the price formation process of SA/EG. The key component in the assimilation process is to know the relation between causes and effects. We concentrate on the mechanism-based model and call the relation system model, which serves the key to correctly perform forward and reverse inference. We have the system dynamics $o_{t} = {\tilde{Bq}}_{t}$ (8) $q_{t + 1} = {\tilde{Aq}}_{t} + {\tilde{Ev}}_{t} .$ (9)

2.3 Experiment design

The experimental platform is built on the R environment. The SA is designed in a 100×100 lattice for 10,000 farmers. The probability of the initial input from period 0 is assumed to be p = 0.1. Learning and simulation are conducted based on the aforementioned definitions.

As a basis for the data assimilation of agriculture prices, we test on time-series data obtained in a real situation. Drawing on the market information of sweet potato transactions from the beginning of 2016 to the beginning of 2020, we can see the price fluctuates significantly for this easy-to-store produce in Fig. 2, where local volatility intense. According to the price volatility of crops analyzed by the Council of Agriculture, the short-term price decline was attributable to sufficient sources of goods, or temperature rise and speeding growth rate; and the overall price decline was attributable to the absence of disastrous climate, smooth production, and supply of vegetables. As pointed out in the 2017 annual price report of the Taipei crops market, among 20 types of vegetables, carrot and sweet potato have the most stable mean annual price fluctuation. The local volatility increased markedly if observed from the statistical chart of the daily average price trend. Most of the influential factors to local volatility are due to climatic factors, according to the statistical analysis of the Council of Agriculture.

Fig. 2

Price and quantity trend of sweet potatoes between 2016–2020.

To observe the phenomena caused by the self-evolution of cells from their initial conditions, we design scenarios for the learning experiments. Each individual has a different initial state and, collectively, all states form a spatial distribution at the initial stage. The major purpose of this test is to identify the initial distribution states and those factors that greatly influence the decisions of farmers. We assume two initial distribution states and compare them quantitatively. We substitute the external (p = 0.01) and internal factors (q = 0.1) into the Equation (8), whereby the probability distribution values are listed in Table 1:

Table 1

The probability distribution for state changes

NS _k,t	0	1	2	3	4	5	6	7	8
Pr _k,t	0.01	0.02	0.03	0.04	0.05	0.06	0.07	0.08	0.09

$\Pr_{k, t} = 1 - (1 - p) {(1 - q)}^{{NS}_{k, t}}$ (8) where NS_k,t represents the overall neighborhood state.

3 Analysis

3.1 Price formation with assimilation in machine learning

Based on our framework in Equations (1)∼(7), learning in assimilation is conducted. Taking 5-year’s pineapple data in Taipei produce auction market from 2015 to 2019, as shown in Fig. 3, we apply the MCMC assimilation to our SA/EG model. We ran MCMC iterations for 16 chains and 5000 iterations. Based on observed supply and price time-series (in blue and red line), we estimated the market demand, shown in the third graph (black line) of Fig. 4. The original demand was shown in the last subgraph (green line). From the graph, we can see that our estimated demand is akin to the original demands.

Fig. 3

Market Clearing Assimilation. In the SA/EG artificial market, the estimated demand (black) is closed to the original demand (green), based on observed supply (blue) and price (red).

Fig. 4

Convergence plot of the MCMC iterations.

The status of convergence for the parameters in each chain is shown in Fig. 5 All chains converge boundedly and some chains outperform others.

Fig. 5

The phase transition diagram for the three strategies of the evolutionary game. The analysis shows the dynamics in (4) makes the state choice transiting from 1 to 3 periodically.

To ensure the SA lattice will not freeze due to the improper choice of EG, we analyze the phase transition for our EG model. Figure 5 shows that the state choice will gradually go into periodical transition, alternating from 1 to 2, from 2 to 3, and back to 1, in accelerated speed.

3.2 Entropy measures

To investigate the effect of spatial distribution, two initial conditions in spatial distribution are arranged. In the first condition (farmers in clustering), the model follows a time-critical circulating rule and considers the changing market price as an indicator of the overall phenomenon. In the second condition (farmers in scattering), the model follows the same rule to produce results. Both of the simulations are influenced by the extent of clustering for the initial distribution. Different initial distribution states are measured and compared with the entropy measure, which is defined as $E (P_{x}) = \sum_{x} x \log x .$ (9)

In case a cluster is misjudged, or a discrepancy is observed between the next market price and the viewpoint of a cluster, the market tends to face divergence and chaos. This analysis aims to measure the spatial difference by using the method defined by the entropy function.

The entropy diagram in Fig. 6 shows that if the initial distributions are assumed to be excessively concentrated and if a clustering phenomenon occurs, then the entropy functions will fluctuate to a great extent. In such a case, the prices of crops will demonstrate great volatility. If natural disasters are not taken into consideration, then different spatial distributions may lead to varying degrees of fluctuation.

Fig. 6

Comparison of entropy changes along with time in the cases of clustered and scattered spatial distribution. The clustered distribution has low entropy in the process of evolution.

3.3 Discussion

The findings indicate that the classical prediction is not always consistent because the future trends are sensitive to initial conditions. Given the extremely high sensibility of initial conditions, the system will face chaos even if the propensity to external factors slightly increases. In other words, increasing or decreasing the propensity to internal or external factors may lead to a phase transition, a chaotic state, or push the system back to its initial state with a long-lasting evolution. Therefore, those factors that influence the local volatility in the prices of crops may also include decisive chaos apart from climatic factors.

The price model examined in this study analyzes the influence of dynamic supply and demand. We determine the prices of crops based on the balance between supply and demand and perform simulation by combining the cultivation decision and price expectation rules of farmers. Market supply can be considered a dynamic process of dictating the prices of crops under a condition of demand changes. In other words, the market price of crops depends on the accumulative supply of all farmers in a perfect competition market. After each production period, farmers must make decisions on the type of crops for the next period. They can either continue cultivating their existing crops or switch to other crops depending on their predictions of future price, climate conditions, and the influence of their neighbors.

The external and internal factors that affect the decisions of farmers are also identified to simulate the comprehensive phenomenon resulting from the interactions among individuals in the market [12]. We show that our machine learning method is compatible with existing existing methods [13, 14]. We therefore suggest that a high propensity to external and internal factors results in a high and low degree of fluctuation in both price volatility and supply change, respectively, providing the propensity to internal factors remain constant.

4 Conclusions

The prices of crops play an important role in stabilizing the supply and demand for produce. This paper applies machine learning to the model of SA/EG to study the decision-making behavior of farmers and the local volatility of crop prices to provide a reference for policymakers and to achieve a balanced supply and demand for crops in the market.

Farmers can assess their need to invest in production via a mutual influencing process. The fluctuation in the prices of crops is affected not solely by climatic factors but also by internal factors. The future proliferation can thus be learned according to different initial spatial distributions, thus providing a new approach to early warning.

This study contributes three key findings to the literature. First, the market price of crops demonstrates local volatility because of external and internal factors (e.g., price expectations, climate, and influence of neighbors) instead of one single factor (e.g., typhoons). In a long-lasting evolution, any change in the above factors may lead to considerable fluctuations in prices through positive feedback. In other words, given that the factors that influence the local volatility in prices may indicate decisive chaos, a “long-term” fluctuation in prices cannot be predicted. Therefore, a real economic market is in a dynamic state without a long-lasting balance.

Footnotes

Acknowledgments

This work was supported in part by grants from the Ministry of Science and Technology, Taiwan (MOST 109-2410-H-992-018-MY2).

References

Nerlove

, Adaptive expectations and Cobweb phenomena, The Quarterly Journal of Economics 72 (1958), 227–240.

Steven

, Paul

and Colin

, Rapid agricultural supply chain risk assessment, Agricultural and rural development department, World Bank, Conceptual Framework and Guidelines for Application 2008.

Gohin

and Cordier

, Agricultural price volatility and speculation by commodity index funds: a theoretical analysis, Agricultural Finance Review 77 (2017), 429–444.

Yan

H.S.

, Jiang

N.Y.

, Shi

W.W.

, Meng

X.G.

and Jiang

T.H.

, Product price forecasting based on correlative price net and neural networks, International Journal of Industrial Engineering: Theory, Applications and Practice 24 (2017), 306–327.

Vollmer

and Von Cramon-Taubadel

, Dynamic price discovery in the European wheat market based on the concept of partial cointegration. In 2018 Conference, July 28-August 2, 2018, Vancouver, British Columbia (No. 276031). International Association of Agricultural Economists.

Pemberton

, Benjamin

, Davis

and Stephens

, Models for short-term stock price forecasting, International Journal of Industrial Engineering: Theory, Applications and Practice 12 (2005), 172–179.

Mahajan

, Muller

and Bass

F.M.

, New product diffusion models in marketing: A review and directions for research, Journal of Marketing 54 (1990), 1–26.

Nerlove

, Adaptive expectations and Cobweb phenomena, The Quarterly Journal of Economics 72 (1958), 227–240.

Gaunersdorfer

, Endogenous fluctuations in a simple asset pricing model with heterogeneous agents, Journal of Economic Dynamics and Control 24 (2000), 799–831.

10.

Bacsi

and Vizvari

, Modeling chaotic behaviour in agricultural prices using a discrete deterministic nonlinear price model, Annals of Operations Research 89 (1999), 125–148.

11.

Duffie

, Dynamic asset pricing theory, Princeton University Press Princeton, NJ. 2001.

12.

Mathevet

, Bousquet

, Le Page

and Antona

, Agent-based simulations of interactions between duck population, farming decisions and leasing of hunting rights in the Camargue (southern France), Ecological Modelling 165 (2003), 107–126.

13.

C.-C.

, Su

, Li

P.-C.

, Bolger

M.J.

and Pan

H.-N.

, Machine Vision and Deep Learning Based Rubber Gasket Defect Detection, Advances in Technology Innovation 5 (2020), 76–83.

14.

Rana

and Garg

, Slow Learner Prediction Using Multi-Variate Naïve Bayes Classification Algorithm, International Journal of Engineering and Technology Innovation 7 (2017), 11–23.