Extended optimal stochastic production control model with application to economics

Abstract

This study considers an inventory control system meeting uncertain demand in continuous time. The demand is a function of both time and price, with the price evolves as a Wiener process with no drift. The goal is to use the stochastic optimal control principle to completely solve a production planning model for the demand rate. A stochastic optimal control problem is formulated in which the stochastic differential equations of a type known as Ito’s equations are considered which are perturbed by a Markov diffusion process and analyzed by the optimal control of a single dimension stochastic production planning model. The existence of a complete solution to the associated HJB equation is established and the optimal policy is characterized. Numerical examples and solutions of this optimal control model are then presented.

Keywords

Markov process stochastic Ito differential equation optimal control diffusion process stochastic demand

1 Introduction

In production planning, one of the most unstable variables is the inventory level. This is influenced by certain unavoidable environmental uncertainties such as sudden random demand fluctuation, sales return, inventory spoilage, etc. They make ideal production policy for “wide” class of cost functional impossible [33]. To take care of these various sources of environmental randomness, the uncertainty is represented by a filtered probability by n-dimensional Brownian motion w, defined on $(Ω, F,)$ and satisfying the usual condition [5]. The deterministic problem requires moving to a stochastic one by considering the “noisy” environment in order to model their behavior fairly accurately by adding an additive noise term in the state dynamics [17].

The general form of production planning is formulated by representing the inventory level by a stochastic process {x (t) , t≥ 0 }, defined on the probability space and generated by $F t$ with an overall noise rate that is distributed like white noise, σdw_t and whose dynamics is governed by the Ito stochastic differential equation $dx (t) = [u (t) - y (t)] dt + σ {dw}_{t}, x (0) = x_{0},$ where σ is the intensity of the noise, y (t) denotes the demand rate at time t, and u (t), the production rate at time t, is a non-stochastic parameter controlled by the investor. A considerable amount of attention has been received by the problem of production-inventory planning which has wide applications to the Operations Research and Management Science. Applications on the optimal production-inventory planning are also widely reported in literature; see for example, Kenné and Gharbi [26] and Yang, Yan and Sethi [39]. The stochastic description of the production planning model is more realistic than deterministic description [35]. Such a model can be applied to a system subjected to random disturbances. Among the models with stochastic elements, the least complexes are those with a single noise terms.More complicated models which have uncertain parameters such as measurement errors, uncertain initial state vectors, and time-varying parameters can be found in Kendrick [25].

During the last two decades, a large number of researchers have applied the theory of stochastic optimal control and stochastic approaches in different fields of engineering, economics, operations research, production planning, advertising and medicine. Inventory and production planning models have a relatively long tradition in the dynamic optimization theory. The first major contribution in this field was of Holt et al. [24] who used the calculus of variation principle to solve a production inventory model. Sethi and Thompson [33] provided a nice introduction in the applications of the optimal control theory to inventory and production models. Also, a very good introduction about the dynamics of economics and management models, as well as their stochastic version, can be found in the book of Sethi and Thompson [34, 35]. Pekelman [29] was probably the first researcher who introduced the price as an additional decision variable in a production-inventory model.

Production planning has attracted a growing attention due to its increasing importance in today’s highly competitive environment; e.g., see [1 , 38]. El-Gohary, Tadj and Al-Rahmah [14] have used the stochastic optimal control to study the problem of optimal control of a stochastic production planning-inventory model. A stochastic optimal production control problem is considered with random demand in Fleming, Sethi and Soner [16]. The standard re-orders point/batch size inventory model is dealt with continuous stochastic demand in Browne and Zipkin [10]. The problem of dynamic pricing of inventories is investigated when demand is price sensitive and stochastic in Gallego and van Ryzin [20]. The optimal production planning is considered when the demand is a Markov diffusion process in Dohi, Kato and Osaki [13]. The optimal inventory policies are considered when the demand is a Poisson process and the commodity price fluctuates as a geometric Brownian motion or as an Ornstein-Uhlenbeck process in Berling and Martínez-de-Albéniz [8]. The controlled diffusion models to approximate stochastic manufacturing systems can often be solved; see [18 , 32].

Two main approaches are observed in the literature regarding stochastic control models of inventory management. One approach, which has been rarely used, is to analytically investigate the structure of optimal control policies using optimality principle approaches (e.g., [9]). The second approach is to find the optimal values for the parameters of a predetermined reasonable, e.g., control policy structure benefiting from the results of queuing theory, Markov decision processes, enumeration techniques, simulation or heuristics [28, 37]. However, it has the drawback of considering a pre-determined policy that is not guaranteed to be optimal. In this study, the former approach is applied. This study focuses on stochastic optimal control problems related to demand management and production control decisions for a single product under capacity limitations and demand uncertainties.

The objective of this study is to formulate the stochastic production planning model where the state variables are assumed to be observable. The Hamilton Jacobi Bellman (HJB) framework is used rather than stochastic maximum principles and the model is analyzed using the optimality principle. It is also of interest to show the existence of the solution to the associated HJB equation for the model and finally to characterize the optimal policy.

2 Production-inventory planning model inventory model

Let x (t) be the stochastic inventory at time t (state variable), u (t) be the stochastic production rate at time t (control variable), θ is the deterioration coefficient. The inventory evolves according to the state equation: $dx (t) = [u (t) - y (t) - θ x (t)] dt, x (0) = x_{0}$ (1)

The demand rate is assumed to be described by the following time dependent model [27]: $y (t) = μ_{y} (t) + σ_{y} ɛ (t)$ where the μ_y (t) is a deterministic known parameter and σ_y is also a known non-negative parameter. It is assumed that ɛ (t) is the white noise, a normally distributed random variable in time t with zero mean and unity variance: ɛ (t) ∼ N (0, 1) and E [ɛ (t₁) , ɛ (t₂)] = 0 for t₁ ≠ t₂ . Therefore the demand rate follows a normally distributed function $y (t) \sim N (μ_{y} (t), σ_{y}^{2}) .$

3 The demand growth dynamics

The demand rate obeys the stochastic differential equation (SDE) by using Ito’s formula $dy (t) = μ_{y} dt + σ_{y} gdw (t), y (0) = y_{0}$ (2) which is a time dependent Ornstein-Uhlenbeck process; g is a diffusion coefficient and w (t) is a white noise process.

The Lipschiz and growth conditions: $\begin{matrix} | μ_{y} (y_{1} - y_{2}) | \\ \leq K | y_{1} - y_{2} |, σ_{y}^{2} {| y |}^{2} \leq K^{2} (1 + {| y |}^{2}) \end{matrix}$ ensures the existence and uniqueness of the solution of (2) provided that the growth parameter is bounded, |μ_y| ≤ K, for some arbitrary constant K.

4 The optimal control problem

The problem is to find the optimal production rate u (t) that minimizes the expected total cost. The overall cost to be minimized is of the form

$\begin{matrix} J (0, x_{0}) \\ = \underset{u (t)}{E} {\int_{0}^{T} [\begin{matrix} h {(x (t) - \bar{x} (t))}^{2} \\ + c {(u (t) - \bar{u} (t))}^{2} \end{matrix}] dt + {Bx}_{T}} \end{matrix}$ (3) where h is an inventory cost coefficient, c is the production cost coefficient, $\bar{x}$ is a factory inventory goal level, $\bar{u}$ is a factory production goal rate, T is the length of the planning period, and B is the salvage value per unit of the inventory at time T.

An optimal production control policy u^* (x, y, t) is sought to maintain the inventory near the demand level whilst keeping the order costs as low as possible. We assumed that v (x, y, t), known as the value function to the objective function (3) and the linear stochastic optimal control problem is given by $v (x_{0}, u^{*}) = inf_{u (t)} J (0, x_{0})$ subject to two state Equations (1) and (2) for inventory and demand.

5 Solution of the optimal control problem

The Bellman principle of optimality holds that the optimal trajectory over the interval t ∈ [0, T] contains the optimal trajectory over the interval [t, T] with x_t = x and y_t = y which is of the form: $v (x, y, t) = \underset{u (t)}{min E} {\begin{matrix} \int_{0}^{T} [\begin{matrix} h {(x (t) - \bar{x} (t))}^{2} \\ + c {(u (t) - \bar{u} (t))}^{2} \end{matrix}] dt \\ + v (x + dx, y + dy, t + dt) \end{matrix}} .$

Applying the Taylor expansion $\begin{matrix} v (x + dx, y + dy, t + dt) \\ = v (x, y, t) + v_{t} dt + v_{x} dx + v_{y} dy + \frac{1}{2} v_{tt} {(dt)}^{2} \\ + \frac{1}{2} v_{yy} {(dy)}^{2} + \frac{1}{2} v_{xx} {(dx)}^{2} \\ + v_{xt} dxdt + v_{xy} dxdy + v_{yt} dydt + o (dt) . \end{matrix}$

After simplification, neglecting the higher order terms [12] such as (dt) ², dwdt and replacing (dw) ² with dt, and given that the variables dw and x are independent, we can write $\begin{matrix} E [v_{x} gdw] = E [v_{x} g] E [dw] = 0, \\ E [v_{xt} (dxdt)] = E [v_{yt} (dydt)] = 0, \\ E [dwdt] = 0, E [dtdt] = 0, \\ E {(dw)}^{2} = dt . \end{matrix}$

Using Ito’s formula, we obtain $\begin{matrix} v (x, y, t) \\ = \underset{u (t)}{min E} {\begin{matrix} [h {(x - \bar{x})}^{2} + c {(u - \bar{u})}^{2}] dt + v (x, y, t) \\ + v_{t} dt + v_{x} (u - y - θ x) dt + v_{y} μ_{y} dt \\ + \frac{1}{2} g^{2} σ_{y}^{2} v_{yy} dt + o (dt) \end{matrix}} . \end{matrix}$

Hence, the HJB equation is satisfied by the value function [11, 35]; $\underset{u (t)}{min E} {\begin{matrix} h {(x - \bar{x})}^{2} + c {(u - \bar{u})}^{2} \\ + \frac{\partial v}{\partial t} + \frac{\partial v}{\partial x} (u - y - θ x) \\ + μ_{y} \frac{\partial v}{\partial y} + \frac{1}{2} g^{2} σ_{y}^{2} \frac{\partial^{2} v}{\partial y^{2}} \end{matrix}} = 0$ (4) $with boundary condition v (x, T) = Bx$ (5)

It is possible to minimize Equation (4) by differentiating the expression in Equation (4) partially with respect to u and setting it equals to zero, whichyields $\frac{\partial v}{\partial x} + 2 c (u - \bar{u}) = 0 .$

Therefore the optimal production rate that minimizes the total cost can be expressed as a function of the current value function in the form: $u (x, t) = \bar{u} - \frac{1}{2 c} \frac{\partial v}{\partial x}$ (6)

Substituting (6) into (4) yields the following equation

$\begin{matrix} h {(x - \bar{x})}^{2} + \frac{\partial v}{\partial t} + \frac{\partial v}{\partial x} (\bar{u} - y - θ x) \\ + μ_{y} \frac{\partial v}{\partial y} + \frac{1}{2} g^{2} σ_{y}^{2} \frac{\partial^{2} v}{\partial y^{2}} - \frac{1}{4 c} {(\frac{\partial v}{\partial x})}^{2} = 0 \end{matrix}$ (7)

This is a HJB or partial nonlinear differential equation that must be satisfied by the current value function v (x, t) with the boundary condition (5).

The optimal production policy is active when the value function declines with the inventory, that is, when $\frac{\partial V}{\partial x} < 0$ otherwise u^* = 0 provided that $\bar{u} = \frac{1}{2 c} \frac{\partial v}{\partial x},$ so the optimal production control can be captured by the law $u^{*} = max (0, \bar{u} - \frac{1}{2 c} \frac{\partial v}{\partial x})$ (8)

The solution to the HJB equation has the form [40]: $\begin{matrix} v (x, y, t) & = & (x, y) Q_{1} (t) (\begin{matrix} x \\ y \end{matrix}) \\ + Q_{2} (t) (\begin{matrix} x \\ y \end{matrix}) + s (t) \end{matrix}$ where $Q_{1} (t) = (\begin{matrix} q_{1} & q_{2} \\ q_{2} & q_{3} \end{matrix})$ is a symmetric positive semidefinite 2×2 matrix forming the certainty equivalent part, $Q_{2} (t) = (\begin{matrix} q_{4} & q_{5} \end{matrix})$ and s (t) is a continuous differentiable scalar function of time accounting for the stochasticity of the process [15] so that

$\begin{matrix} v (x, y, t) & = & q_{1} (t) x^{2} + 2 q_{2} (t) xy + q_{3} (t) y^{2} \\ + q_{4} (t) x + q_{5} (t) y + s (t) \end{matrix}$ (9)

Substituting (8) into (7) and equating the coefficients with (9), we obtain the following ordinary coupled differential equations for Q₁ (t), Q₂ (t) and s (t): ${dq}_{1} = [2 q_{1} θ + \frac{{(q_{1} + q_{2})}^{2}}{c} - h] dt$ (10) ${dq}_{2} = [q_{1} + q_{2}] dt$ (11) ${dq}_{3} = [2 q_{2}] dt$ (12) $\begin{matrix} {dq}_{4} & = & [q_{4} θ + 2 h \bar{x} - 2 q_{1} \bar{u} - 2 q_{2} μ_{y} \\ - 4 q_{4} (q_{1} + q_{2})] dt \end{matrix}$ (13) ${dq}_{5} = [q_{4} - 2 q_{2} \bar{u} - 2 q_{3} μ_{y}] dt$ (14) $ds = [q_{4} \bar{u} + μ_{y} q_{5} + σ_{y}^{2} g^{2} q_{3} + q_{4}^{2} + h {\bar{x}}^{2}] dt$ (15) with the boundary conditions q₁ (T) = q₂ (T) = q₃ (T) = q₄ (T) = q₅ (T) = s (T) = 0 .

6 Numerical examples and solutions of the optimal control models

To illustrate the numerical example of the stochastic inventory model, the following parameters are considered: deterioration coefficient, θ = 0.1, inventory holding cost coefficient, h = 1, production cost coefficient, c = 1.5, inventory goal, $\bar{x} = 20$ , production goal, $\bar{u} = 20$ , stochastic production rate, μ_y = 4, standard deviation of demand rate, σ_y = 1, diffusion coefficient, g = 0.5, and T = 1.

Matlab [2, 21] is used in order to solve the solution of the inventory model numerically. The graph of the demand rate at time t of the stochastic differential equation in Equation (3) is shown by Fig. 1.

The boundary value problems of the system of differential equations from Equation (10) to Equation (15) are solved numerically here. Figure 2 shows the graph of the differential equation q₁ versus over time t. It shows that the value of q₁ starting with 0.8278 decreases with time t until it reaches 0 at the boundary value, T = 1. Figure 3 shows the differential equation q₂ that the value of q₂ starting with -0.3229 increases with time t until it reaches 0 at the boundary value. Figure 4 shows the differential equation q₃ that the value of q₃ starting with 0.2414 decreases with time t until it reaches 0 at the boundary value. Figure 5 shows the differential equation q₄ that the value of q₄ starting with –121.5 increases with time t until it reaches 0 at the boundary value. Figure 6 shows the differential equation q₅ that the value q₅ starting with 41.74 decreases with time t until it reaches 0 at the boundary value. Figure 7 shows the differential equation s that the value of s starting with –3523 increases with time t until it reaches 0 at the boundary value.

7 Conclusion

This study puts forward an inventory control system operating under stochastic demand and pricing subject to random fluctuations. The price is assumed to be a Wiener process with no drift. The method of solution adopted here can also be used when price is an arithmetic Brownian motion, y (t) = μ_y (t) + σ_yɛ (t), and consequently the demand rate follows the process, dy (t) = μ_ydt + σ_ygdw (t), y (0) = y₀ The stochastic optimal control model in the context of an inventory-production is developed and analyzed with stochastic demand. The solution of this model is carried out via the development of the Hamilton-Jacobi-Bellman equation satisfied by a certain value function. The existence of the solutions to the associated Hamilton Jacobi Bellman equation for this model is established and the optimal policies are characterized. The numerical solution of the stochastic control system for particular values of the parameters is presented. Further, the research can be done in the optimal control of stochastic production planning model with deterioration of the inventory items, where the demand rate is constant or is a function of time.

References

Applequist

, Samikoglu

, Pekny

and Reklaitis

, Issues in the use, design and evolution of process scheduling and planning system, ISA Transactions 36(2) (1997), 81–121.

Attaway

, Matlab: A Practical Introduction to Programming and Problem Solving, Elsevier, Oxford.

Balasubramanian

and Grossmann

I.E.

, Scheduling to minimize expected completion time in flow shop with uncertain processing times, Computers & Chemical Engineering 26 (2002), 41–57.

Bassett

M.H.

, Perkny

J.F.

and Reklaitis

G.V.

, Using detailed scheduling to obtain realistic operating policies for a batch processing facility, Industrial Engineering and Chemical Research 36(5) (1997), 1717–1726.

Barles

, Solution de Viscosité des Equations de Hamilton Jacobi, Springer, Berlin, 1994.

Baten

M.A.

and Khalid

, Optimal control of a continuous review production inventory system with Gamma distributed deterioration, Journal of Intelligent & Fuzzy Systems 27(6) (2014), 2937–2948.

Baten

M.A.

and Khalid

, Optimal production cycle time for inventory model with linear time dependent exponential distributed deterioration, Journal of Intelligent & Fuzzy Systems 30(2) (2016), 1243–1248.

Berling

and Martínez-de-Albéniz

, Optimal inventory policies when purchase price and demand are stochastic, Operations Research 59(1) (2011), 109–124.

Bellman

, Dynamic Programming, Princeton University Press, New Jersey, 1957.

10.

Browne

and Zipkin

, Inventory models with continuous, stochastic demands, The Annals of Applied Probability 1(3) (1991), 419–435.

11.

Davis

M.H.A.

, Panas

V.G.

and Zariphopoulou

, European option pricing with transaction costs, SIAM Journal of Control and Optimization 31 (1993), 470–493.

12.

Davis

M.H.

, Linear Estimation and Stochastic Control, John Wiley and Sons, New York, 1977.

13.

Dohi

, Kato

and Osaki

, Optimal production planning under diffusion demand pattern, Mathematical and Computer Modelling 21(11) (1995), 35–46.

14.

El-Gohary

, Tadj

and Al-Rahmah

, Optimal control of a stochastic production planning model with different demand rates, International Journal of Applied Mathematics 20(3) (2007), 333–349.

15.

Fleming

W.H.

and Rishel

R.W.

, Deterministic and Stochastic Optimal Control, Springer Verlag, New York, 1975.

16.

Fleming

W.H.

, Sethi

and Soner

H.M.

, An optimal stochastic production planning problem with randomly fluctuating demand, SIAM Journal of Control and Optimization 25(6) (1987), 1494–1502.

17.

Fleming

W.H.

and Soner

, Controlled Markov process and Viscosity Solutions, Springer Verlag, New York, 1993.

18.

Ghosh

M.K.

, Arapostathis

and Marcus

S.I.

, Optimal control of switching diffusions with applications to flexible manufacturing systems, SIAM Journal on Control Optimization 31 (1993), 1183–1204.

19.

Ghosh

M.K.

, Arapostathis

and Marcus

S.I.

, Ergodic control of switching diffusions, SIAM Journal on Control Optimization 35 (1997), 1952–1988.

20.

Gallego

and van Ryzin

, Optimal dynamic pricing of inventories with stochastic demand over finite horizons, Management Science 40(8) (1994), 999–1020.

21.

Hahn

and Valentine

, Essential MATLAB for Engineers and Scientists, Elsevier, Oxford, 2013.

22.

Harrison

J.M.

and Taksar

M.I.

, Instantaneous control of Brownian Motion, Mathematics of Operations Research 8 (1983), 439–453.

23.

Harrison

J.M.

, Sellke

T.M.

and Taylor

A.J.

, Impulse control of Brownian Motion, Mathematics of Operations Research 8 (1983), 454–466.

24.

Holt

C.C.

, Modigliani

, Muth

J.F.

and Simon

H.A.

, Planning Production, Inventories, and Workforce, Prentice-Hall, New Jersey, 1960.

25.

Kendrick

D.A.

, Stochastic control for economic models: Past, present and the paths ahead, Journal of Economic Dynamics & Control 29 (2005), 3–30.

26.

Kenné

J.P.

and Gharbi

, Stochastic optimal production control problem with corrective maintenance, Computers and Industrial Engineering 46 (2004), 865–875.

27.

Kiesmüller

G.P.

, Optimal control of a one product recovery system with lead times, International Journal of Production Economics 81-82 (2003), 333–340.

28.

Mahadevan

, Pyke

D.F.

and Fleischmann

, Periodic review, push inventory policies for remanufacturing, European Journal of Operational Research 151(3) (2003), 536–551.

29.

Pekelman

, Simultaneous price-production decisions, Operations Research 22 (1974), 788–794.

30.

Petkov

S.B.

and Mararas

C.D.

, Multiparied planning and scheduling of multiproduct batch plants under demand uncertainty, Industrial Engineering and Chemistry Research 36(1) (1997), 4864–4881.

31.

Rogers

D.F.

, Evans

J.R.

, Plante

R.D.

and Wong

R.T.

, Aggregation and disaggregation techniques and methodology in optimization, Operations Research 39 (1991), 553–582.

32.

Sethi

S.P.

, Suo

, Taksar

M.I.

and Zhang

, Optimal production planning in a stochastic manufacturing system with long-average cost, Journal of Optimization Theory and Applications 92(1) (1997), 161–188.

33.

Sethi

S.P.

and Thompson

G.L.

, Simple Models in Stochastic Production Planning, In Applied Stochastic Control in Econometrics and Management Science, Bensoussan

, Kleindorfer

and Tapiero

, (eds.), North-Holland Publisher, Amsterdam, 1981, pp. 295–304.

34.

Sethi

S.P.

and Thompson

G.L.

, Optimal Control Theory: Applications to Management Science, Martinus Publishing Nijhoff, Boston, 1981.

35.

Sethi

S.P.

and Thompson

G.L.

, Optimal Control Theory: Applications to Management Science and Economics, Kluwer Academic Publishers, Dordrecht, 2000.

36.

Shekarian

, Glock

C.H.

, Amiri

S.M.P.

and Schwindl

, Optimal manufacturing lot size for a single-stage production system with rework in a fuzzy environment, Journal of Intelligent & Fuzzy Systems 27(6) (2014), 3067–3080.

37.

Van der Laan

E.A.

and Teunter

R.H.

, Simple heuristics for push and pull remanufacturing policies, European Journal of Operational Research 175 (2006), 1084–1102.

38.

Yan

, Jiang

, Shi

and Li

, An iterative learning method for multi-cycle flexible production/inventory control under random demands, Journal of Intelligent & Fuzzy Systems 26(6) (2014), 2591–2607.

39.

Yang

, Yan

and Sethi

S.P.

, Optimal production planning in pull flow lines with multiple products, European Journal of Operational Research 119 (1999), 582–604.

40.

Arnold

, Stochastic Differential Equation, Wiley, New York, 1974.