Adaptive neurofuzzy control of engine idle speed

Abstract

Engine idle speed control problem has significant effects on fuel consumption, combustion stability and pollutions produced in urban traffics. In this paper, a Multi-Agent adaptive Critic based NeuroFuzzy Controller (MACNFC) for engine idle speed is presented to improve fuel efficiency, reduce emissions, and increase disturbance rejection capability while preventing the engine from stalling. The proposed MACNFC consists of a neurofuzzy controller, a learning mechanism and a set of fuzzy critic agents. It is assumed that each critic has been well designed to serve a particular objective. The role of the critics is to evaluate the controller performance in terms of satisfactory achievements of the control objectives through evaluation of plant output and provide appropriate continues reinforcement signals. If these signals become zero, it means that the critics are satisfied by the performance of the controller from their own point of view. If deviation of signals from zero become larger, it shows the more stress and more dissatisfaction. The reinforcement signals provided by critics contribute collaboratively to update neurofuzzy controller parameters via the learning mechanism for achieving predefined criteria and goals. The controller should modify its characteristics so that all reinforcement signals are decreased and consequently all critics are satisfied. To illustrate the effectiveness of the proposed method some simulations are given. Obtained results show that proposed method not only maintains the engine speed as close as possible to a desired value in the presence of various disturbances, but also improves fuel consumption for large production volumes of engine operating in different idle speed regimes.

Keywords

Idle speed control engine model fuel consumption disturbance rejection neurofuzzy controller critic agent

Notation

The following notation is used in this paper.

θ: air-bypass valve, throttle angle, (first input)

τ: Induction of Power Stroke (IPS) delay

α_i: model parameter (engine pumping)

β_i: model parameter (throttle-plate)

φ_i: model parameter (engine torque)

J_e: effective engine rotational inertia

$\dot{m}$ : air-bypass valve mass flow rate

T: intake manifold air temperature

δ: Spark-advance, (second input)

K_i: regular torque load parameter

P_a: atmospheric air temperature

$\dot{M}$ : cylinder air mass flow rate

V_m: intake manifold volume

P: intake manifold pressure

T_l: engine load torque

T_d: disturbance torque

T_e: net engine torque

K_τ: delay parameter

R: air gas constant

N: engine speed

1 Introduction

The increasingly stringent legislation on emission levels and fuel economy together with an increasing market competition is forcing the automotive industry to improve the performances of internal combustion (IC) engines in terms of fuel economy, reduced pollutant emissions and improved drivability. For IC engines, an improvement of a few percentage points in fuel economy and emission level will be translated into significant saving of fuel and running costs over the engine life as well as vast environmental benefits. On average, about 30% of vehicles’ fuel consumption in a city driving is spent at idle [1], and this percentage continues to increase with the increasing trend of traffic loads. It is therefore very important to optimize vehicle and powertrain operations at idle. Automotive idle speed control (ISC) is one of the most challenging aspects in engine control fields. The ISC problem is typically addressed as a disturbance rejection problem in which the primary ISC goal is to maintain the engine speed as close as possible to a desired value that minimizes fuel consumption for large production volumes of engine operating in different idle speed regimes (loads, temperature), with different aging history, and being driven under vastly different environment conditions [2]. As the quality of the ISC has a significant impact on fuel economy, emission levels, combustion stability, transient response, and NVH (Noise, Vibration, and Harshness) characteristics, the selection of target idle speed corresponds to a trade-off among fuel consumption, idle operation stability, and emission levels. Some basic requirements for ISC systems are listed as [3]:

Low idle speed set point is needed for good fuel economy.

Low idle speed set point is needed for the reduction of emissions.

The ISC stability is needed to be guaranteed with regard to the target idle speed, where if it is too low, stalling may occur and if it is too high, harsh gear engagement may occur.

ISC nonlinear delays need to be compensated.

To achieve these requirements, many different closed-loop designs for ISC have been proposed in the literature including Proportional-Integral-Derivative (PID) control with a tuning rule [4], Linear Quadratic control (LQ) [5], H_∞ control [6, 7], Model Predictive Control (MPC) [8], Adaptive control [9, 10], Sliding Mode Control (SMC) [11], fuzzy control [12], neural network control [13], and robust control [14, 15], etc. The actuator saturation, time delays, and digital controller implementation for ISC are studied in [11 , 16–18], respectively. A former comprehensive survey of engine models and control strategies developed for ISC can be found in [2] and a recent survey was made in [3]. Although several intelligent and classical control strategies have been designed for automotive ISC, as mentioned earlier, but due to its multivariable, nonlinear and time-varying behavior, none of them provide all goals such as adaptive set point tracking, disturbance rejection or improving fuel consumption. This paper focuses on solving these complex control problems via multi-agent adaptive critic based neurofuzzy controller that may improve transient responses, rejection of disturbance and robustness compared to other control methodologies. During the past few years biologically motivated intelligent computing has been successfully employed for solving different types of problems. Based on successful implementation of this model for decision making and controlling of uncertain nonlinear systems [19, 20], delayed systems [21, 22], simple linear systems [23, 24], as well as more complex nonlinear systems in [19 , 25–28], this study concentrates on designing of a multi-agent adaptive critic-based neurofuzzy controller for the idle speed control system.

The rest of this paper is organized as follows. Section 2 gives the nonlinear model of engine idle speed system. In Section 3, the whole structure of the proposed controller is formulated. Section 4 describes the structure of proposed controller and its application to ISC problem. The performance of the proposed method under the whole range of operation conditions of engine is then shown in Section 5. Finally some conclusion and remarks are discussed in Section 6.

2 Problem statement

The control of internal combustion engine in the idle regime is still one of the most challenging problems for automotive industry, where improvements in performance and robustness can directly translate into better fuel economy and emissions. The goal of ISC is to maintain the engine speed as low as possible, since by lowering idle speed, fuel consumption is reduced [29]. However, lowering idle speed increases the possibility of engine stall, noise, vibration, and harshness [30]. Hence, a reasonable target idle speed needs to be maintained in order to overcome mechanical frictions, misfiring, and load disturbances to prevent engines oscillation, vibration, hunting, and stalling under a variety of circumstances.

For simulation of ISC systems, many different models have been proposed and considered in literature. For example, in [31, 32] a linear model of the system with a time delay is put forward, while the nonlinearity of the ISC system is considered in [33 –35]. The model developed in [33] is a comprehensive model that incorporates the static air-bypass valve characteristics, manifold filling dynamics, engine pumping characteristics, intake-to-power stroke delay, quasistatic torque generation characteristics, and engine rotational dynamics. So in this paper, we used this model. The modeling aspects of each subsystem are as follows [33]:

The air-bypass mass flow rate $\dot{m}$ is characterized as a function of the air-bypass valve position and intake manifold pressure which is determined by the valve input command θ, i.e. $\dot{m} = f_{1} (θ, P) = f_{θ} (θ) f_{P} (P)$ (1) where $f_{θ} (θ) = β_{0} + β_{1} θ + β_{2} θ^{2}$ and $f_{P} (P) = {\begin{matrix} 1 & P \leq P_{a} / - 2 \\ \frac{2}{P_{a}} \sqrt{{PP}_{a} - P^{2}} & P > P_{a} / - 2 \end{matrix}$

A constant intake manifold temperature and no intake manifold leaks are assumed in this work. Thus for the manifold air-mass-balance P we have: $\dot{P} = K (\dot{m} - \dot{M}) where K = RT / - V_{m}$ (2)

The engine pumping behavior is a function of engine speed N and manifold pressure P in the form of (3) $\dot{M} = f_{2} (N, P) = α_{0} NP + α_{1} {NP}^{2}$ (3)

The transport delay τ between the end of induction and the beginning of the torque production is: $τ = K_{τ} / N$ (4)

The net engine torque T_e is obtained as a function of cylinder air mass flow rate $\dot{M}$ , engine speed N, and spark advance δ, and air-fuel ratio. But in modern engines the air-fuel ratio is controlled even at idle, hence this parameter is assumed to be constant here. So T_e can be described as:

$\begin{matrix} T_{e} & = & f_{3} (\dot{M} (t - τ), N, δ) = φ_{0} + φ_{1} \dot{M} (t - τ) \\ + φ_{2} δ + φ_{3} δ^{2} + φ_{4} δ N + φ_{5} N + φ_{6} N^{2} \end{matrix}$ (5)

For the load torque $T_{l} = N^{2} / K_{i}^{2} + T_{d}$ (6) and the rotational dynamics are captured by $\dot{N} = \frac{1}{J_{e}} (T_{e} - T_{l})$ (7) Where J_e is the effective rotational inertia of the engine. Equations (1–7) are the mathematical description of the nonlinear engine model. The numerical values of the model parameters are given in Appendix.

As can be seen from above equations, the ISC is a highly nonlinear, time varying with variable time delays, complicated and uncertain dynamic control problem. The schematic diagram of the engine is shown if Fig. 1. As can be seen from this figure, the model is a two-input one-output system where the system output is engine speed N in rpm. The control inputs of the model are throttle angle θ and spark advance δ in degrees. Disturbances act on the engine in the form of load torque disturbance T_l in Nm.

In order to design an ISC system which able to substantially reduce emission and fuel consumption while maintaining idle regime stability, the idle speed controller with two actuators (throttle and spark advance) is considered here. The effect of spark advance on engine speed is faster than the effect of throttle therefore here spark advance is used to effectively reject transient disturbances. Fuel efficiency can also be increased by operating the engine with advanced spark angle. But it is not desirable to use large offsets in spark advance for long term control as this will have a detrimental effect on emissions and engine vibration [33]. Hence, the throttle angle is used to offset steady state load disturbances and to force the spark advance to return to its nominal value.

There are two types of controllers for engine speed regulation at idle. The first controller actuates both throttle and spark advance in a single control function, as shown Fig. 2(a), while in the second controller separated loops control the throttle and spark without any coordination as shown in Fig. 2(b). The advantage of the second type is to avoid severe interaction between the inner and outer control loops of the first type [35]. In this way, both control loops are governed by the same rpm error without control interactions, improving the advantages about the authority, quickness and steady state performance. Hence, to evaluate the performance and robustness of the proposed method in achievement of disturbance rejection, reducing the fuel consumption and low emissions at idle the second type is considered here.

3 Multi agent adaptive critic based neurofuzzy controller

The multi agent adaptive critic based neurofuzzy controller is a psychologically motivated algorithm which is developed to reduce the complexity of computations in practical problems. In this approach, which in a way is an extension to reinforcement learning, there exists an element in the control system called critic whose task is to assess the present situation of the system which has resulted from the applied control action in terms of satisfactory achievement of the control goals and to provide the so called reinforcement signal (the stress). The neurofuzzy controller should modify its characteristics so that the critic’s stress is decreased. The block diagram of the proposed system is shown in Fig. 3. As can be seen from this figure, it contains three main items as neurofuzzy controller, critic agents and learning mechanism. In the subsequent sections, a brief discussion of the above elements is presented.

3.1 Neurofuzzy controller

In this subsection, the principles of fuzzy control are explained and then the design of a neurofuzzy controller which has adaptation features against the variation of plant parameters is introduced.

In general, a fuzzy logic system provides a nonlinear mapping from a set of crisp inputs to a set of crisp outputs, using both intuition and mathematics [36]. In order to do that, each fuzzy logic system is associated with a set of rules, which heuristically define the dynamics of the plant to be controlled. Using a fuzzifier method any set of crisp inputs is mapped to a fuzzy set. Various rules in the rule base are applied to the fuzzy input data, in order to create a fuzzy output. This output is in turn defuzzified to generate a crisp output value. A defuzzifier is a mapping that, given a fuzzy set, determines the best crisp representative of that set. There are two types of fuzzy systems that are used in control engineering; Takagi-Sugeno-Kang (TSK) and Mamdani method. The second type is used for static systems with slow changing dynamics while the TSK method is more efficient for systems with fast changing dynamics. Due to the above mentioned issues and the fact that the automotive engines have fast changes in terms of system parameters and dynamics, the TSK method is used in this paper.

The TSK fuzzy inference system is based on fuzzy rules which use linguistic If-Then sentences to describe the relationship between inputs and outputs. In this paper we consider a Multi-Input Single-Output (MISO) fuzzy system consisting of M rules with the following type:

$\begin{matrix} {Rule}_{i} : {Ifx}_{1} = A_{i 1} And x_{2} = A_{i 2}, \dots And x_{p} = A_{ip} \\ then {\tilde{u}}_{i} = f_{i} (x_{1}, x_{2}, \dots, x_{p}), i = 1, 2, \dots, M \end{matrix}$ (8)

Where M is the number of fuzzy rules. x₁, x₂, …, x_p are the input variables of the fuzzy systems, A_ij is characterized by its corresponding membership function μA_ij (x_j) in ith rule and ${\tilde{u}}_{i}$ is the consequence of the ith rule and f_i (.) : R^p → R. The output of the model can be calculated using defuzzification as Equation (9), where $μ_{i} (\underline{x})$ is the firing strength of the ith rule and is given by Equation (10). $u = \frac{\sum_{i = 1}^{M} f_{i} (\underline{x}) μ_{i} (\underline{x})}{\sum_{i = 1}^{M} μ_{i} (\underline{x})}$ (9) $\begin{matrix} μ_{i} (\underline{x}) & = & \prod_{j = 1}^{p} μ A_{ij} (x_{j}), \\ \underline{x} = {[x_{1}, x_{2}, \dots, x_{p}]}^{T} \in R^{p} \end{matrix}$ (10) μA_ij (x_j) is the membership function of jth input in the ith rule and $μ_{i} (\underline{x})$ is the degree of validity of the ith rule. In the most applications of TSK fuzzy system, the crisp function is defined as a linear combination of the input variables plus a constant term, i.e.: ${\tilde{u}}_{i} = f_{i} (x_{1}, x_{2}, \dots, x_{p}) = ω_{i 0} + \sum_{j = 1}^{p} ω_{ij} x_{j}$ (11)

The TSK neurofuzzy model has two sets of adjustable parameters: one the antecedent parameters, which are belong to the input membership functions such as centers and deviations of Gaussians and another, the rule consequent parameters such as the linear weights of output in Equation (11). Tuning the parameters of antecedent parts will have some demerits such as high computational cost, falling into a local optimum, high sensitivity in the final result to the initial setting of the learning rate and more noteworthy of all, departure from the initial linguistic nature of the fuzzy model, the strongest point of the fuzzy model itself [37]. Hence, the parameters of the membership functions were set on the basis of the linguistic knowledge of the engine behavior and only the rule consequent parameters ω_ij were considered to be updated here.

The neurofuzzy controller applied in this paper is a standard TSK controller composed of six layers. Figure 4 shows a sample neurofuzzy system with two-input and one-output TSK fuzzy inference system. The node functions in same layer are of the same function family as described as follows.

The task of the first layer, specified by I, is the assignment of inputs’ scaling factors in order to map them to the range of [–1, +1]. Each nod in the second layer, denoted by M, specifies the degree to which the given input x₁ and x₂ satisfies the linguistic labels. Third layer nodes, denoted by T, multiply the incoming signals and constitute the antecedent parts of fuzzy rules, $\prod_{j = 1}^{p} μ A_{ij} (x_{j})$ . Each node in the fourth layer specified by N calculates the ratio of corresponding firing strength to the sum of all rules firing strengths, hence the term $\frac{μ_{i}}{\sum_{i = 1}^{M} μ_{i}}$ . The nodes function of the fifth layer is performing a linear combination of input variables plus a constant value, thus calculating the corresponding rule’s consequent part ${\tilde{u}}_{i}$ . As we stated earlier, the coefficients of these linear combinations and that of constant values will be adapted during the learning stage. Finally defuzzification is carried out in the sixth layer in order to calculate the proper control signal according to (9).

3.2 Critic agent

Critic agent is the main part of any learning system. The performance of the critic can be compared with the performance of emotional cues in humans. In absence of an exact evaluation of the present state in term of the objective value function, emotional cues like stress, satisfaction etc. can guide our control action into changing in the right direction so as to produce desired response. Similarity, in multi-agent adaptive critic based neurofuzzy controller each critic agent assesses the controller performance through evaluation of plant output and provides appropriate reinforcement signal, namely r. This signal is allowed to have a real value in [–1, +1] range and shows the performance of the system. If this signal becomes zero, it means that the critic is satisfied by the performance of the controller from its own point of view. If the signal becomes larger, it shows the more stress and more dissatisfaction. The signal produced, contributes collaboratively for updating parameters of the neurofuzzy controller. In this paper, each neurofuzzy controller consists of two critics. One critic is satisfied when the actual engine speed tracks the desired idle speed and another one is satisfied by low control cost and action. These reinforcement signals are used to train and fine tune their corresponding neurofuzzy controller. Basically, these critics act as intelligent guides for the controller. The learning mechanism will adapt the controller parameters in order to satisfy all critics and reduce the stresses of them. All critics are defined in fuzzy forms here. Fuzzy systems are very useful for critic modeling because the critic just gives us an approximate evaluation of current states of system.

3.3 Learning mechanism

The main objective of learning mechanism is to satisfy all critics and reduces total stress. This aim can be extracted through minimization of bellow energy function: $E = E_{1} + E_{2} = \frac{1}{2} k_{1} r_{1}^{2} + \frac{1}{2} k_{2} r_{2}^{2}$ (12)

Where r_j is the output signal (stress) of the jth critic and k_j are importance coefficients of reinforcement signals and indicate which critics are more important. For the ISC problem, the energy function E₁ is minimized when the actual engine speed tracks the desired idle speed while another one, E₂, is minimized by low control cost and action. By minimizing the energy function E, we can reduce the total stress of the system and satisfy all critics. Learning mechanism is adjusting the weights of model by means of a nonlinear optimization method, e.g. the steepest descent or conjugate gradient. With steepest descent, the weights will be adjusted by the following variations:

$\begin{matrix} Δ ω_{i} & = & Δ ω_{i, 1} + Δ ω_{i, 2} = - η \frac{\partial E}{\partial ω_{i}} \\ = & - η (\frac{\partial E_{1}}{\partial ω_{i, 1}} + \frac{\partial E_{2}}{\partial ω_{i, 2}}) \\ i = 1, 2, \dots, p \end{matrix}$ (13) where η is the learning rate of the neurofuzzy controller. The right hand side of (13) can be calculated by chain rule: $\frac{\partial E_{1}}{\partial ω_{i, 1}} = \frac{\partial E_{1}}{\partial r_{1}} \cdot \frac{\partial r_{1}}{\partial y} \cdot \frac{\partial y}{\partial u} \cdot \frac{\partial u}{\partial ω_{i, 1}} i = 1, 2, \dots, p$ (14) where u is the control signal (the output of neurofuzzy controller), y is the actual outputs of the plant under control. According to (12): $\frac{\partial E_{1}}{\partial r_{1}} = k_{1} r_{1}$ (15)

The term $\frac{\partial y}{\partial u}$ is the gradient of the system and shows the long term variations of the plant output to the control signal. As in most cases, the system is designed in such a way that this variation is a positive constant, the sign of this value, i.e., positive, is sufficient for adaptation rule. For non-minimum phase systems although this term becomes negative for the time being due to the transient inverse response behavior, if reflected in (13), it will have a misleading effect on the tuning parameters. In most cases calculating the remaining part $\frac{\partial r_{1}}{\partial y}$ is not straightforward, however, it can be approximated via simplifying assumptions. If, for example error is defined by $e = y_{ref} - y$ (16) where y_ref is the desired output (reference input), then $\frac{\partial r_{1}}{\partial y} = - \frac{\partial r_{1}}{\partial e}$ (17)

Since with the increasing or decreasing of the error (deviation of engine speed from a set point), reinforcement signal r will be also incremented or decremented respectively, and on the other hand, online calculation of $\frac{\partial r_{1}}{\partial e}$ is accompanied with measurement error, thus it can be replaced by its sign, –1, in (14). Using (11) and (14), adaptation rule of the tunable parameter will be as follows: $Δ ω_{i, 1} = η k_{1} r_{1} \cdot \frac{\partial u}{\partial ω_{i, 1}}$ (18)

To follow up Equations (14) to (18), the weight Δω_i,2 can be calculated as (19). $Δ ω_{i, 2} = η k_{2} r_{2} \cdot \frac{\partial u}{\partial ω_{i, 2}}$ (19)

From Equations (13), (18) and (19), we have: $Δ ω_{i} = η (k_{1} r_{1} \frac{\partial u}{\partial ω_{i}} + k_{2} r_{2} \frac{\partial u}{\partial ω_{i}})$ (20)

For the neurofuzzy controller introduced in the subsection A, the control signal using (9) and (11) has the form as in (21). $u = \frac{\sum_{i = 1}^{M} (ω_{i 0} + \sum_{j = 1}^{p} ω_{ij} x_{j}) μ_{i}}{\sum_{i = 1}^{M} μ_{i}}$ (21)

Hence, in according to (20) the update rules for the parameters of the neurofuzzy controller will be given as (22) and (23) $\begin{matrix} Δ ω_{i 0} & = & η (k_{1} r_{1} + k_{2} r_{2}) \frac{\partial u}{\partial ω_{i 0}} \\ = & η (k_{1} r_{1} + k_{2} r_{2}) \frac{μ_{i}}{\sum_{i = 1}^{M} μ_{i}} \end{matrix}$ (22) $\begin{matrix} Δ ω_{ij} & = & η (k_{1} r_{1} x_{1} + k_{2} r_{2} x_{2}) \frac{\partial u}{\partial ω_{ij}} \\ = & η (k_{1} r_{1} x_{1} + k_{2} r_{2} x_{2}) \frac{μ_{i}}{\sum_{i = 1}^{M} μ_{i}} \end{matrix}$ (23)

4 Controller design for the ISC system

The primary goal of ISC system is to maintain idle speed operation at a feasible minimum speed without affecting the ISC quality and emissions. Moreover, it is desirable to maximize fuel efficiency and disturbance rejection while simultaneously minimizing vehicle vibration. Hence, a reasonable target idle speed needs to be maintained in order to overcome mechanical frictions, misfiring, and load disturbances to prevent engines oscillation, vibration, hunting, and stalling under a variety of circumstances. Figure 5 shows the controller configuration for idle speed control with above criteria. In this figure, inputs of the first critic, denoted by critic 1, are engine speed error and its derivative with five input labels {NL, NS, ZE, PS, PL}, while its output which is used to evaluate the engine performance has seven labels, {NL, NM, NS, ZE, PS, PM, PL}. Figure 6 shows the membership function of the inputs linguistic variables. If the speed error is high but its derivative shows a decreasing trend then the performance is not too bad and we can hope to have a better performance if we carry on. Also, if the speed error is low but its derivative has a large positive value, the critic should not be satisfied with the behavior. On the basis of these linguistic descriptions, we designed a fuzzy critic with fuzzy sets and rules base shown in Table 1.

Both of the first critics, denoted by critic 1, in throttle and spark advance controllers, are same and have the same membership functions, rules, fuzzifier and difuzzifier. For the first critic deduction is performed by max-product law, and centroid law is used for defuzzification.

For the engine model described here, the spark advance that results in maximum brake torque for idle speed 750 rpm is about 31 degrees. However, If the engine is operating at or near idle speed, especially in traffic load, the amounts of cylinder air intake, compression pressure, and compression temperature are increased too much, in this situation the spark advance must be retarded relative to this value to prevent engine detonation. Hence in designing the structure of third critic, we assume that the critic will be satisfied when the advanced spark angle close to 23 degrees. This spark advance value is to allow rapid system response to torque disturbances while maintaining a reasonable level of fuel efficiency and reducing pollution produced in traffic load. In designing of the second and third critics, specified by critic 2 and critic 3 in Fig. 5 respectively, we consider the throttle and spark advance control signals are in range θ ∈ [0, 30] degrees and δ ∈ [10, 40] degrees respectively. The desired idle speed (reference input) is also chosen 750 rpm. The fuzzy sets and rules base of second and third critics are shown in Fig. 7. As each neurofuzzy controller in Fig. 5 has error and derivative of error of the engine speed as their inputs and there are twenty five rules in the fuzzy rule base, using (21), (22) and (23), the parameters of each neurofuzzy controller will be updated as follows:

\allowdisplaybreaks

For throttle angle control system: $θ = \frac{\sum_{i = 1}^{25} (a_{i 0} + a_{i 1} e + a_{i 2} \dot{e}) μ_{i}}{\sum_{i = 1}^{25} μ_{i}}$ (24) $\begin{matrix} Δ a_{i 0} & = & η_{θ} (k_{1} r_{1} + k_{2} r_{2}) \frac{μ_{i}}{\sum_{i = 1}^{25} μ_{i}}, \\ i = 1, 2, \dots, 25 \end{matrix}$ (25) $\begin{matrix} Δ a_{i 1} & = & η_{θ} (k_{1} r_{1} + k_{2} r_{2}) e \frac{μ_{i}}{\sum_{i = 1}^{25} μ_{i}}, \\ i = 1, 2, \dots, 25 \end{matrix}$ (26) $\begin{matrix} Δ a_{i 2} & = & η_{θ} (k_{1} r_{1} + k_{2} r_{2}) \dot{e} \frac{μ_{i}}{\sum_{i = 1}^{25} μ_{i}}, \\ i = 1, 2, \dots, 25 \end{matrix}$ (27)

For spark advance control system: $δ = \frac{\sum_{i = 1}^{25} (b_{i 0} + b_{i 1} e + b_{i 2} \dot{e}) μ_{i}}{\sum_{i = 1}^{25} μ_{i}}$ (28) $\begin{matrix} Δ b_{i 0} & = & η_{δ} (k_{1} r_{1} + k_{3} r_{3}) \frac{μ_{i}}{\sum_{i = 1}^{25} μ_{i}}, \\ i = 1, 2, \dots, 25 \end{matrix}$ (29) $\begin{matrix} Δ b_{i 1} & = & η_{δ} (k_{1} r_{1} + k_{3} r_{3}) e \frac{μ_{i}}{\sum_{i = 1}^{25} μ_{i}}, \\ i = 1, 2, \dots, 25 \end{matrix}$ (30) $\begin{matrix} Δ b_{i 2} & = & η_{δ} (k_{1} r_{1} + k_{3} r_{3}) \dot{e} \frac{μ_{i}}{\sum_{i = 1}^{25} μ_{i}}, \\ i = 1, 2, \dots, 25 \end{matrix}$ (31)

Where η_θ and η_δ are the learning rates of throttle angle control system and park advance control system respectively.

In general, the performance of the algorithm is very sensitive to the proper setting of the learning rate. If the learning rate is set too high, the algorithm may oscillate and become unstable. If the learning rate is too small, the algorithm will take too long to converge. It is not practical to determine the optimal setting for the learning rate before training, and, in fact, the optimal learning rate changes during the training process, as the algorithm moves across the performance surface. The performance of the algorithm can be improved if we allow the learning rate to change during the training process. An adaptive learning rate will attempt to keep the learning step size as large as possible while keeping learning stable. In this paper we consider an exponential learning rate as η_θ = η_δ = γ₀e^-γ₁t + γ₂, where t is time and γ₀, γ₁ and γ₂ are constant. In this case γ₀, γ₁ and γ₂ are chosen by trial and error as 0.2, 3 and 0.36 respectively. We also choose k₁ = k₂ = k₃ = 6.

5 Simulation results

In this section the efficiency and the robustness properties of the proposed control approach is illustrated and compared with PID controller. At first, the response of the nonlinear engine model discussed in section II, to step changes the idle speed set-point is demonstrated. Figure 8 shows the set-point tracking performance for both PID controller and proposed method. The transient performance indices of the time response of the system with two controllers are also outlined in Table 2. As can be seen, although both controllers perform well, but the performance of the proposed method is much better than PID controller in terms of rise-time, settling-time and overshoot.

The next analysis is on evaluating the robustness of the controllers with respect to the disturbances. For this purpose the additional load torque of 20 Nm is applied to the system at t = 15 s and removed at t = 25 s. The main performance measure of this test is the maximum deviation of the engine speed from idle. In particular, a large dip following an increased load may cause the engine to stall while a large flare, in the case of a reduction in the load, is undesirable for both fuel economy and noise. Figure 9 shows the deviation from the idle speed (750 rpm) when power steering load is applied to two controllers. The dips correspond to the introduction of the disturbance and flares correspond to the release. As can be seen from this figure the control system is forced to spend a considerable amount of time reaching the target speed again when PID is used but the proposed method not only achieves a very good disturbance rejection in terms of maximum deviation from the set-point but also improves the transient response indices and reduces the steady state engine idle speed error.

In real driving, idle speed set-point may change as required to accommodate the states of accessories, namely in cold start conditions or changes in battery voltage. Hence in the next scenario the behavior of proposed controller when engine is operating in non-nominal conditions is considered. In this case the idle speed set-point starts from 1000 rpm and decreases smoothly. The power steering disturbance sequences is also engaged and maintained for several seconds, then released during the simulation. Figure 10 shows the proposed controller can perfectly manage the system with the maximal disturbance in various non-nominal conditions. In this simulation the dips correspond to the introduction of the disturbance and flares correspond to the release.

Because of the fast adaptation capability of the proposed controller, it is intriguing to evaluate the robustness of the closed loop system with respect model parameter variations. In this evaluation up to 19 different parameters of the nonlinear engine model are varied by±20% around their nominal values. For all these variations the engine idle speed N is considered 750 rpm but an additional load torque of 10Nm is applied to the engine at t = 5 s. Figure 11 shows the performance of the proposed controller. It is clear that the maximum absolute engine speed error is about 1.8% which confirms the efficiency and the robustness of the controller against system parameter uncertainties and variations.

6 Conclusion

In this paper, a multi agent adaptive critic based neurofuzzy controller to engine idle speed control problem was investigated. The control framework consisted of two critics and a neurofuzzy controller whose parameters were adapted online according to reinforcement signals provided by critics with the back-propagation of error algorithm. The tasks of critics were to assess the present situation resulted from the applied control action in terms of satisfactory achievement of the control goals and provided the reinforcement signal. The efficiency and the robustness of the proposed method against system parameter uncertainties and load torque disturbances and also in idle set-point tracking were shown by nonlinear simulation results. Obtained results showed that the proposed controller is able to handle a wide operation range although it is only designed for the nominal operating condition and at idle. The advantages of the controller such as Online learning, fast convergence, robustness and relative independency to plant model makes it possible to use this controller in different type of engines.

\textwidth .

Footnotes

Appendix

The engine parameters used for simulations are taken from [33]

β₀ = 1	(g/s)	β₁ = 0.907	((g/(s deg ²))
β₂ = 0.0998	((g/(s deg ²))	α₀ = 0.020	(g/(kPa))
α₁ = 1.054 · 10^-4	(g/(kPa²))	K_τ = 0.75	(–)
φ₀ = 3.992	(Nm)	φ₁ = 0.387	(Nms/g)
α₂ = 6.350 · 10^-2	(Nm/deg CA)	φ₃ = -1.120 · 10^-3	(Nm/((deg CA) ²)
α₄ = 4.241 · 10^-4	(Nms/degCA)	φ₅ = 1.257 · 10^-2	(Nms)
α₆ = -4.027 · 10^-4	(Nms²)	${\tilde{φ}}_{0} = 4.623$	(Nm)
${\tilde{φ}}_{1} = 0.387$	(Nms/g)	${\tilde{φ}}_{2} = 0.020$	(Nms)
${\tilde{φ}}_{3} = - 4.027 \cdot 10^{- 4}$	(Nms²)	P_a = 101.325	(Kps)
K_i = 4.386	$(1 / (m \sqrt{kg}))$	J_e = 4.386	(m²kg)

References

Jurgen

, Automotive Electronics Handbook, McGraw-Hill Inc., New York, USA, 1995.

Hrovat

and Sun

, Models and control methodologies for IC engine idle speed control design, Control Eng Pract 5(8) (1997), 1093–1100.

, Modeling, Identification, Design, and implementation of nonlinear automotive idle speed control systems-an overview, IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews 37(6) (2007), 1137–1151.

Howell

and Best

, On-line PID tuning for engine idle-speed control using continuous action reinforcement learning automata, Control Eng Pract 8(2) (2000), 147–154.

Powell

B.K.

and Powers

W.F.

, Linear quadratic control design for nonlinear IC engine systems, 10th Anniversary Int Symp Autom Technol Autom vol. 1, 1981, pp. 1–18.

Williams

S.J.

, et al., Idle speed control design using an H∞ approach, IEEE American Control Conference 1989, pp. 1950–1956.

Ford

and Glover

, Spark ignition engine idle speed control using a novel framework and enabling control of the tradeoff between fuel consumption and load rejection capability, Vehicle System Dynamics 36(2-3) (2001), 225–251.

Manzie

and Watson

, A novel approach to disturbance rejection in idle speed control towards reduced idle fuel consumption, Journal of Automobile Engineering 217 (2003), 677–690.

Yin

, Syu

, Chen

, Wu

, et al., Application of adaptive idle speed control on V2 engine, SAE Int J Engines 9(1) (2016).

10.

Kim

and Park

, Application of adaptive control to the fluctuation of engine speed at idle, Information Sciences 177(16) (2007), 3341–3355.

11.

and Yurkovich

, Sliding model control of delayed systems with application to engine idle speed control, IEEE Transactions on Control Systems Technology 9(6) (2001), 802–810.

12.

Thornhill

and Thompson

, Adaptive fuzzy logic control of engine idle speed, Journal of Systems and Control Engineering 213(2) (1999), 145–155.

13.

Puskorius

and Feldkamp

, Automotive engine idle speed control with recurrent neural networks, IEEE American Control Conference 1 (1993), 311–316.

14.

Petridis

A.P.

and Shenton

A.T.

, Inverse-NARMA: A robust control method applied to SI engine idle-speed regulation, Control Engineering Practice 11 (2003), 279–290.

15.

Hamilton

G.K.

and Franchek

M.A.

, Robust controller design and experimental verification of I.C. engine speed control, International Journal of Robust and Nonlinear Control 7 (1997), 609–627.

16.

X.J.

, Hong

and Su

, Idle speed control for GDI engines with interval time-delay, Advanced Materials Research 588-589 (2012), 1598–1601.

17.

Bengea

S.C.

, Li

and DeCarlo

R.A.

, Combined controller-observer design for uncertain time-delay systems with application to engine idle speed control, Journal of Dynamic Systems, Measurement, and Control 126 (2004), 772–780.

18.

De Santis

, Di Benedetto

and Pola

, Digital idle speed control of automotive engines: A safety problem for hybrid systems, Nonlinear Analysis 65 (2006), 1705–1724.

19.

Fordoni

and Melit

, Multiple selections of kashmir tunnel risk by fuzzy multiple-criteria decision analysis, J Bioinf Intell Control 4 (2015), 67–74.

20.

Rashidi

, Lucas

and Khaki Sedigh

, Design of a robust and adaptive Emotional learning Based Intelligent Controller for multiobjective nonlinear systems, 11th Iranian Conference on Electrical Engineering, Vol. 3, 2003, 158–165.

21.

Rashidi

and Rashidi

, Multi agent adaptive critic based neuro-fuzzy controller for multi-objective nonlinear systems, WSEAS Trans on Systems 5(3) (2006), 572–578.

22.

Esfandiari

and Khaloozadeh

, Modeling of parliament elections using artificial neural networks, J Bioinf Intell Control 3 (2014), 134–139.

23.

W.-X.

, An approach to multiple attributes decision making with hesitant interval-valued fuzzy information and its application, Journal of Intelligent & Fuzzy Systems 28(2) (2015), 495–503.

24.

Seyyedbarzegrar

and Mirzaie

, ANFIS based model for power loss estimation of metal oxide surge arrester, Journal of Intelligent & Fuzzy Systems 29(5) (2015), 1779–1786.

25.

Iranmanesh

S.H.

, Mirseraji

G.H.

and Shahmiri

, An Emotional Learning based Fuzzy Inference System (ELFIS) for improvement of the completion time of projects estimation, IEEE International Conference on Computers & Industrial Engineering (2009), pp. 470–475.

26.

, An ANFIS-based transformer insulation fault diagnosis method using emotional learning, IEEE International Conference on Natural Computation 1 (2007), 74–78.

27.

Hashemi

, Kazemi

and Soleymani

, Assessment of an adaptive neuro fuzzy inference system for islanding detection in distributed generation, Journal of Intelligent & Fuzzy Systems 26(1) (2014), 19–31.

28.

Rashidi

and Rashidi

, Firing angle control of TCSC using Emotional Learning Based Fuzzy Controller, IEEE International Conference on Systems, Man and Cybernetics, Vol. 2, 2003, pp. 1989–1994.

29.

Nagashima

and Levine

W.S.

, Development of an engine idle speed and emission controller, pp, IEEE American Control Conference (2006), 3278–3283.

30.

Nielsen

, Kjergaard

, Vesterholm

and Hendricks

, Advanced Nonlinear Engine Idle Speed Control Systems, SAE Technical Paper No. 940974, 1994.

31.

Di Cairano

, Yanakiev

, Bemporad

, Kolmanovsky

I.V.

and Hrovat

, Model predictive idle speed control: Design, analysis, and experimental evaluation, IEEE Transactions on Control Systems Technology 1(99) (2011), 1–14.

32.

, Chen

and Yu

, Nonlinear model predictive control for idle speed control of SI engine, IEEE International Conference on Decision and Control 2009, pp. 6590–6598.

33.

Pfiffner

and Guzzella

, Feedback linearization idle-speed control: Design and experiments, Kybernetika 35(4) (1999), 441–458.

34.

Alt

, Svaricek

, Blath

J.P.

and Schultalbers

, Multiple sliding surface control of idle engine speed and torque reserve with load torque estimation, International Workshop on Variable Structure Systems 2008, pp. 47–54.

35.

Palma

and Cristofaro

F.D.

, A Simulation Based Investigation of Interactions between VVA and Idle Control for SI Engines, Virtual Control Conference 2010.

36.

Mohagheghi

, Harley

R.G.

and Venayagamoorthy

G.K.

, Modified Takagi-Sugeno fuzzy logic based controllers for a static compensator in a multimachine power system, IEEE International conference on Industrial Applications 4 (2004), 2637–2642.

37.

Fakhrazari

and Boroushaki

, Adaptive critic-based neurofuzzy controller for the steam generator water level, IEEE Transactions on Nuclear Science 55(3) (2008), 1678–1685.