Bench calibration method for automotive electric motors based on deep reinforcement learning

Abstract

The efficiency and control accuracy of Interior Permanent Magnet Synchronous Motor (IPMSM) are the main factors affecting performance. Manual calibration has the disadvantage of high work intensity, long calibration period and high technical requirement, which leads to low calibration accuracy and motor efficiency. Thus, a novel calibration method based on Deep Deterministic Policy Gradient (DDPG) and Long Short-Term Memory (LSTM) is proposed. By constructing a deep reinforcement learning network, the self-optimization of the optimal working point under any working condition is realized, and the MAP for IPMSM in full speed-torque range is obtained. The method can be used to quickly realize the optimal matching of d-q axis current with arbitrary stator current. It focuses on solving the problem of motor overheating caused by long adjustment time of manually calibrated MAP when the motor is overloaded, to realize fast calibration in overload area. Moreover, the method reduces the dependence on the motor parameters and increases the adaptability of the calibration MAP data to the operating conditions. The simulation and bench test indicate that the method can meet the response requirements of motor torque, and results reveal that the motor efficiency is greatly improved.

Keywords

Interior permanent magnet synchronous motor deep reinforcement learning bench calibration optimal control optimal efficiency

1 Introduction

With the development of new energy vehicles, interior permanent magnet synchronous motors (IPMSM) have been widely used due to its advantages of high efficiency, high power density and good torque characteristics. The motor controller for the electric vehicles requires high-precision torque control [1]. The inner loop of the control system is a current loop, which receives target torque from the accelerator pedal, and then the target torque is translated into current command through the synchronous rotating coordinate system. The core of the drive motor control is to output greater torque and reach higher speed within the constraints of DC voltage and voltage utilization [2, 3]. Therefore, it is necessary to match the d-q axis current reasonably [4]. The commonly used analytical principle is to minimize the stator current at a certain torque, and the current command corresponding to each torque command at different speeds is obtained, realizing the maximum torque per ampere (MTPA) control [5, 6].

Methods for realizing MTPA mainly included: model-based online calculation method [7, 8], curve fitting method based on experimental data [9] or the finite element analysis (FEA) simulation results [10] and manual calibration method based on bench experiment. The traditional MTPA calibration method is to construct a mathematical model of the motor based on motor parameters, such as the maximum current, the maximum output voltage, the maximum torque of the motor, and so on. Then the current values corresponding to the respective torques at different speeds are generated according to the mathematical model. This calibration method has high requirements on the accuracy of the mathematical model established, so it is necessary to comprehensively consider the influence of parameter changes on model accuracy to obtain the optimal MTPA curve [10 –12]. However, the uncertainty of motor parameters and inverter nonlinearity [13, 14] increases the difficulty of the model establishment. In order to obtain a high-precision mathematical model, some parameters need to be calibrated and then repeatedly calibrated through tests. The accuracy does not be improved after a long time trial, which results in limitations in practical engineering applications. Moreover, IPMSM has strong coupling and nonlinearity, which makes it difficult to achieve accurate modeling. Therefore, the model-based calibration algorithm is very complex, and it is difficult to meet the real-time needs of automotive electric motors [15]. Curve fitting based on test data requires a certain amount of data to establish a functional relationship between input variables and responses results, which has a higher requirement on the source and accuracy of experimental data [16]. Otherwise, the fitting results cannot be applied to all working conditions.

The manual calibration based on bench experiment is a method of testing the motor and its adaptive controller before the product is put into service to obtain the relationship of torque-speed-current. Through matching calibration, some control parameters are solidified to form a MAP. Motor calibration is of great significance to improve torque control accuracy and efficiency of motor. The traditional manual calibration has high work intensity and long calibration cycle, which seriously affects the development efficiency and project progress [17, 18]. At the same time, the manual calibration of the motor is likely to cause a large calibration offset error. Even the calibration results of different operators are not the same, so the accuracy of the results cannot be guaranteed; the traditional calibration table method adopts the static method, that is, the principle of “one-time calibration and permanent use”. However, the parameters of the IPMSM vary greatly depending on the working conditions and the environment. The uncertainty of parameters is an important factor affecting the external characteristic output of the motor. When the motor parameters change with the operating conditions [19], the difference between the curve obtained by the static calibration method and the actual curve will become larger and larger, which will affect the overall efficiency of the motor. More importantly, due to the uncertainty of parameters, the use of manually calibrated MAP in the overload area of the motor will cause the phenomenon of motor overheating if the adjustment time is too long, which will affect the normal performance of the motor and even damage the motor. Since only one or several prototypes are calibrated in the calibration process, the traditional calibration method requires high consistency of mass production motor. When the consistency is lack, there will be a big gap between the calibration curve and the actual curve, which will affect the improvement of motor efficiency in the actual application process.

It is well known that intelligent control has the advantages of being without need of an exact system dynamic model, simplicity, less intensive mathematical design, and is suitable for dealing with the nonlinearities and uncertainties [20 –22], such as Fuzzy logic control (FLC), artificial neural network control(ANNC), and neuro fuzzy control(NFC), etc. [23, 24] Theoretically, they can handle any nonlinear model for the power control system [25, 26], and more robust than the conventional controllers [27]. Some intelligent controller with the thoughts of human logic for IPMSM process control and obtain good performance. In recent years, researchers have tried to apply the intelligent controller in the IPMSM drives [28, 29]. Compared to many types of intelligent controllers, FLC is the one of the easiest to implement for high performance IPMSM drive.

Contrary to the conventional FLC of the IPMSM drive with zero d-axis current, a simplified fuzzy speed controller with MTPA incorporated for the IPMSM drive will obtain better performance. It through simplified the d-axis current around some operating point to get the electromagnetic torque. An online loss-minimization MTPA algorithm is further integrated with a FLC-based IPMSM drive to yield high efficiency and high dynamic performance over a wide speed range [30 –32].

However, each of these controllers has its own performance or implementation shortcomings. On the one hand, FLC needs much more manual adjustment by trial and error to obtain high performance and for the most three-phase IPMSM, the FLC controller will be invalid because L_d is not equal to L_q and this will adding the coupling nonlinearity of the IPMSM, thus make the control design difficult [33]. On the other hand, it is extremely tough to create a series of training data for ANNC that can handle all the operating modes, and the same to other intelligent control algorithm. To the best of the authors’ knowledge, no work has been reported on the stand-alone online adaptive intelligent controller of IPMSM drives.

With the development of artificial intelligence technology, reinforcement learning (RL) has been widely used in the field of industry control [34 –36]. RL is a trial-and-error learning method based on Markov decision process. By constructing the optimized bellman equation, the RL task with Markov decision process is completed. A RL system with Markov property includes at least state set, action set, strategy set and reward and punishment function [37]. RL does not need to rely too much on the mathematical model and ontology parameters of the controlled object. Moreover, this control method can ensure that the current optimal output characteristics can be truly and accurately reflected in each state [38, 39]. The control method based on RL has the advantage of not relying on accurate motor model to estimate the relationship between target torque and d-q axis current in advance and obtaining the best torque-speed-current MAP only through continuous trial and error learning. In this text, the method based RL is adopted to realize the automatic calibration for IPMSM, which overcomes the problems existing in manual calibration, such as long construction period, low efficiency, poor accuracy and poor robustness. This method transforms the static calibration mode of manual calibration into a dynamic search mode with self-learning and self-optimization, optimize the d-q axis current with maximum torque of arbitrary stator current in real time, and the search results are formed into a MAP.

The main contents of this text are organized as follows: In Section 2, the principle of IPMSM calibration based on constant magnetic field control is analyzed theoretically, and the importance of calibration work for improving motor control is clarified; In Section 3, an automatic calibration method for automotive electric motors based on Deep Deterministic Policy Gradient and Long Short-Term Memory (DDPG-LSTM) is proposed to automatically obtain the MAP in full speed-torque range of IPMSM. In Section 4 and 5, the effectiveness and feasibility of the method are verified by simulation analysis and experimental verification.

2 Analysis of calibration control for IPMSM

The mathematical model of IPMSM in the d-q reference frame can be expressed as follows [40]: $u_{d} = p ψ_{d} + R_{S} i_{d} - ω_{e} ψ_{q}$ (1) $u_{q} = p ψ_{q} + R_{S} i_{q} + ω_{e} ψ_{d}$ (2) $ψ_{d} = L_{d} i_{d} + ψ_{f}$ (3) $ψ_{q} = L_{q} i_{q}$ (4) $T_{e} = p [ψ_{f} i_{d} + (L_{d} - L_{q}) i_{d} i_{q}]$ (5) where u_d and u_q, i_d and i _q, ψ_d and ψ_q, L_d and L_q are the transformed d-q axis voltages, currents, stator flux-linkages, stator inductances, respectively; R_s is the stator resistance; ψ_f is the permanent magnet flux-linage; ω_e is the electrical angular speed; p is the differential partial; T_e is the torque. As a result of the dynamic model is for IPMSM, the stator inductances L_d is not equal to L_q.

By using Equations (1)–(5), the time phase voltage equation can be expressed as: ${\dot{U}}_{s} = R_{s} {\dot{I}}_{s} + j ω_{s} L_{d} {\dot{I}}_{d} + j ω_{s} L_{q} {\dot{I}}_{q} + j {\dot{E}}_{0}$ (6)

Then the phase diagram according to Equation (6) is shown in Fig. 1. Generally, we control motor torque by adjusting the current with a closed loop. Fig. 1 depicts the relationship between the stator voltage U_s and the current I_s with the phase difference φ and the relationship between the stator current I_s and the permanent magnet flux-linage Ψ_f with the phase difference β. From Fig. 1, we can obtained using different β as well as different d-q current pairs (i.e. I_d1, I_q1, I_d2, I_q2, ...) under the conditions of constant stator current I_s(i.e. I_s1 = I_s2 = ... .). Therefore, we can directly adjust the power angle β ₁ or β ₂ to control the motor torque.

Fig. 1

The phasor diagram of composite voltage.

Fig. 1 show the phase difference β of the stator current phasor and the permanent magnet flux phasor, the phase difference φ between the stator voltage and the stator current phasor, and the phase difference γ =β–90° between the stator current phasor and the electromotive force phasor. Meanwhile, the correspondence between stator current phasor and d-q axis current with respect to β or γ can also be obtained [41], as shown in Equation (7). ${\begin{matrix} i_{d} = i_{s} \cos β = - \cos γ \\ i_{q} = i_{s} \sin β = \sin γ \\ β = {tan}^{- 1} (\frac{i_{q}}{i_{d}}) or γ = - {tan}^{- 1} (\frac{i_{q}}{i_{d}}) \end{matrix}$ (7)

Under the condition that the stator current is constant, with the change of phase difference β, countless pairs of d-q axis current combinations and torque values will be obtained. In all combinations, the maximum torque point and the minimum torque point will appear, respectively, as shown in Fig. 2. The power angle β corresponding to the maximum torque and minimum torque points are unique in the Fig. 2(b). Combined with Equation (7), the unique d-q axis current pair can be obtained [42 –44]. It can be seen from Equation (7) that the optimal β is closely related to the parameters of the motor [45]. $β = \frac{π}{2} + \arcsin [\frac{- ψ_{f} + \sqrt{ψ_{f}^{2} + 8 {(L_{d} - L_{q})}^{2} i_{s}^{2}}}{4 (L_{d} - L_{q}) i_{s}}]$ (8)

Fig. 2

The relationship between the power angle and the combined torque.

However, the strong coupling characteristic of the IPMSM makes the d-q axis current coupling with each other, when the d-q axis voltage is changed, the d-q axis current will be affected simultaneously, and the degree of change is obviously different with the change of velocity. Moreover, with the change of working conditions, leading to a large error in the optimal working point of the theoretical calculation, resulting in poor control effect [46, 47].

On the one hand, experiments are performed to measure the d-axis and q-axis inductance and the result is shown in Fig. 3. It shows that L_d and L_q variations with I_d and I_q, L_q and L_d present different variation patterns. Furthermore, we can also obtain that L_q decreases as I_q increases from Fig. 3. However, L_d exhibits a more sophisticated variation pattern. On the other hand, these nonlinear parameters may also be obtained via finite element studies and then experimentally validated [5]. As seen in Fig. 3, the tested IPMSM parameters become extreme nonlinearities in the discontinuous operation subregion. It is also shown that the L_q is much more varied than L_d if the system does not consider current phase. It is important that the L_d is far from a correct value. Therefore, it is necessary to consider that the variations of L_d and L_q make a great impact on IPMSM control performance [48, 49].

Fig. 3

Motor parameters (L_d and L_q) change with the working conditions.

To sum up, the optimal operating point of motor based on traditional vector control calculation cannot overcome the influence of parameters uncertainty and the strong coupling characteristics, so the optimal d-q axis current calculated cannot truly reflect all the actual requirements. Therefore, in order to improve the control accuracy of motor torque, it is necessary to conduct a series of tests between the motor and the matching motor controller before the product is put into service. Accurate calibration of the corresponding relationship between d-q axis current and torque is of great significance to improve the working efficiency of the motor [50].

3 DDPG-LSTM based rapid calibration method for automotive electric motors

In order to obtain the optimal d-q axis current matching for arbitrary stator current and improve the working efficiency of motor, this text proposes a bench calibration method based on deep reinforcement learning method to quickly obtain MAP in full speed-torque range of automotive electric motors. Black control diagram of novel d-q axis current optimization approach based on reinforcement learning theory is shown in Fig. 4. The diagram mainly contains three parts:

Agent: generate reference d-q axis voltage;

Environment: generate series of target signal, such as torque, d-q axis current and so on;

Reward: according to the feedback signal from Environment generate immediate reward;

Fig. 4

Black diagram of d-q axis current optimization control based on DDPG-LSTM.

A: State set

A complete state space should contains all possible states of the controlled object. Generally, the key of motor control is to contorl torque, and the torque is determined by I_d and I_q. In any conditions, there are four states in the relationship between target value and reference value, as shown in Table 1, such as S1, S2, S3 and S4.

Table 1

Four states of d-q axis current and corresponding control method (CM)

I_dref - I_d	I_qref - I_q	State	CM
I_dref> I_d	I_qref> I_q	S1	CM_S1
I_dref< I_d	I_qref < I_q	S2	CM_S2
I_dref< I_d	I_qref> I_q	S3	CM_S3
I_dref> I_d	I_qref< I_q	S4	CM_S4

There are corresponding adjustment measures for the four different working states, as shown in Table 1. In order to obtain high performance control effect, the best control method should be applied to the motor under different working states. By establishing a regular control scheme based on vector control and expert experience, the efficient control of the motor can be realized by establishing corresponding control laws for four working states, such as CM_S1, CM_S2, CM_S3 and CM_S4 in the Table 1. The detailed control flow of CM_S1 is shown in Fig. 5, which briefly describes the idea of establishing the control law. In Fig. 5, ω indicates the real-time electrical angular velocity, and ω_L, ω_M, and ω_H respectively indicate the low speed interval, intermediate speed, and high speed interval set according to the actual control requirements. u_dmax and u_qmax are the maximum of the d-q axis voltages. ɛ represents the minimum amount of d-q axis voltage adjustment. The setting of this value is related to the standard value range of the overall variable when the motor control algorithm is designed.

Fig. 5

The control flowchart of CM_S1 control method.

B: Control strategy π

As shown in Fig. 3, agent generate the action of u_d and u_q, the limition of u_d and u_q is expressed as Equation (9): $\sqrt{u_{d}^{2} + u_{q}^{2}} ⩽ u_{\max}^{phase}$ (9) where $u_{\max}^{phase}$ is the peak of phase voltage [3], when the stator phase voltage is a three-phase balanced sinusoidal voltage, the maximum amplitude of the fundamental phase voltage is 0.577u_dc. Therefore, the action set π is a series of strategy pairs of u_d and u_q. However, in order to obtain the optimal u_d and u_q, it is necessary to consider the factors, such as d-q axis current error, torque, speed and motor parameters and so on.

C: Reward and punishments function

Considering the signal that the motor control needs to observe from the environment is not a single variable signal, it is necessary to consider the information such as I_q, I_d, and T_e to formulate the next action simultaneously. The d-q axis immediately reward signal is defined as: ${\begin{matrix} R_{d} = 10 (case 1 d) - 100 (case 2 d) + 5 (case 3 d) \\ R_{q} = 10 (case 1 q) - 100 (case 2 q) + 5 (case 3 q) \end{matrix}$ (10) where variations of case1d, case2d, case3d, case1q, case2q, and case3q are calculated in Figs. 6 and 7, respectively:

Fig. 6

Calculation processes of case1d, case2d, and case3d.

Fig. 7

Calculation processes of case1q, case2q, and case3q.

where errorLimit is the limit value of d-q axis current error, it can be set according to actual control requirements.

D: Improved DDPG-LSTM algorithm and parameter updates

Traditional DDPG structure contains the actor and the critic (AC) network, the actor network is used to generate the control strategy π (s), the critic network is used to evaluate and correct future performance of the control strategy. In order to further improve the effectiveness of the data when the actor and the environment interact and optimize the control strategy output, the LSTM layer is respectively added into the output of the actor network and the critic network to reduce the deviation produced by value function estimation, and thus solve the problem of data correlation and non-static distribution. So, a novel optimal control approach based on DDPG-LSTM is proposed as shown in Fig. 8.

Fig. 8

Black diagram of IPMSM control based on DDPG-LSTM.

The relationship of the control variables u_d and u_q at two times before and after could shown as the following equation. ${\begin{matrix} u_{d} (k) = {\overset{︷}{f (R_{d} (Δ I_{d}^{k}, Δ T_{e}^{k}), u_{d} (k - 1), ω_{e} (k))}}^{agent \to action} \\ u_{q} (k) = {\overset{︷}{f (R_{q} (Δ I_{q}^{k}, Δ T_{e}^{k}), u_{q} (k - 1), ω_{e} (k))}}^{agent \to action} \end{matrix}$ where $R_{d} (Δ I_{d}^{k}, Δ T_{e}^{k})$ and $R_{q} (Δ I_{q}^{k}, Δ T_{e}^{k})$ are the immediately reward of d-q axis action, $Δ I_{d}^{k}$ , $Δ I_{q}^{k}$ and $Δ T_{e}^{k}$ are the d-axis current error, q-axis current error, and torque error, respectively; f(.) is respresent the function of agent; R(.) is respresent the function of reward.

(1) Basic DDPG network

The basic framework of DDPG algorithm is AC network framework. In the process of building policy function π and value function Q, this text adopts a single neural network model to build actor network and critic network, respectively.

1) Objective function construction of DDPG algorithm

In DDPG algorithm, the input is the current state, the output is the deterministic action value.

Combining Equations (1)-(4), we can obtain the Equation (11): $\dot{x} = Ax + By + C$ (11) where, $\begin{matrix} A = [\begin{matrix} \frac{- R_{s}}{L_{d}} & \frac{L_{q}}{L_{d}} ω_{e} \\ \frac{- L_{d}}{L_{q}} ω_{e} & \frac{- R_{s}}{L_{q}} \end{matrix}]; B = [\begin{matrix} \frac{1}{L_{d}} & 0 \\ 0 & \frac{1}{L_{q}} \end{matrix}]; \\ C = [\begin{matrix} 0 \\ \frac{- ψ_{f}}{L_{q}} \end{matrix}] x = [\begin{matrix} i_{d} \\ i_{q} \end{matrix}], y = [\begin{matrix} u_{d} \\ u_{q} \end{matrix}] \end{matrix}$

As we all known, in discrete-time systems we can define $\dot{x} \tilde{=} (x_{k + 1} - x_{k}) / T_{s}$ . Therefore, the Equation (11) can be simplied expressed as Equation (12) in kth time:

$\begin{matrix} x_{k + 1} = (T_{s} A + 1) x_{k} + T_{s} {By}_{k} + T_{s} C = A^{'} x_{k} \\ + {B^{'} y}_{k} + C^{'} \end{matrix}$ (12) where x_k represents the current states vector of the system, y_k represents the control vector.

The control objective is to find the sequence of optimal control signal, denoted with y_k, k = 1, 2, . . . , ∞. Suppose that the objective function associated with this system is: $J (x_{k}, y_{k}) = \sum_{k = 0}^{\infty} γ^{k} (x_{k}^{T} {Qx}_{k} + y_{k}^{T} {Ry}_{k})$ (13) where γ = [0, 1] is the discount factor, which is used to ensure the boundedness of the cost; Q and R are the symmetric positive definite matrices. Note that the functional J is dependent on the initial time k and the initial state x_k, and it is referred to as the cost-to-go of state x_k; so that the functional J is minimized, according to Bellman, and the optimal value function is equal to: $J^{*} (x_{k}, y_{k}) = \min {x_{k}^{T} {Qx}_{k} + y_{k}^{T} {Ry}_{k} + γ J^{*} (x_{k + 1}, y_{k + 1})}$ (14)

The optimal control $y_{k}^{*}$ at time k is the y_k which achieves this minimum. $y_{k}^{*} = arg min {x_{k}^{T} {Qx}_{k} + y_{k}^{T} {Ry}_{k} + γ J^{*} (x_{k + 1}, y_{k + 1})}$ (15)

It is possible to write Equation (12) in iteration form, as shown in Equation (16). ${\begin{matrix} J_{i + 1} (x_{k}, y_{k}) = x_{k}^{T} {Qx}_{k} + y_{ik}^{* T} {Ry}_{ik}^{*} + γ J_{i}^{*} (x_{k + 1}, y_{k + 1}) \\ y_{ik}^{*} = arg min {x_{k}^{T} {Qx}_{k} + y_{k}^{T} {Ry}_{k} + γ J_{i}^{*} (x_{k + 1}, y_{k + 1})} \end{matrix}$ (16)

In summary, in this iterative algorithm, the value function sequence J_i +1 and control law sequence y_ik are updated by implementing the recurrent iteration equation in (16) with the iteration number i increasing from 0 to ∞.

2) Convergence analysis

According to (16), we can derive ${\begin{matrix} J_{i + 1} (x_{k}, y_{k}) = x_{k}^{T} {Qx}_{k} + y_{ik}^{* T} {Ry}_{ik}^{*} + γ x_{k + 1}^{T} {Qx}_{k + 1} + γ y_{ik + 1}^{* T} \\ \dots + γ^{i} x_{k + i}^{T} {Qx}_{k + i} + γ^{i} y_{ik + i}^{* T} {Ry}_{ik + i}^{*} + γ^{i + 1} J_{0}^{*} (x_{k + i + 1}, y_{k + i + 1}) \\ x_{k + 2} = A^{'} x_{k + 1} + {B^{'} y}_{k + 1} + C^{'} \\ ⋮ \\ x_{k + i + 1} = A^{'} x_{k + i} + {B^{'} y}_{k + i} + C^{'} \end{matrix}$ (17)

The initial value function $J_{0}^{*} (x_{k + i + 1}, y_{k + i + 1}) = 0$ , which is not necessary for the optimal value function. Equation (17) can be further simplied as: $J_{i + 1} (x_{k}, y_{k}) = \sum_{j = 0}^{i} γ (x_{k + j}^{T} {Qx}_{k + j} + y_{ik + j}^{* T} {Ry}_{ik + j}^{*})$ (18)

It can be seen that during the iteration process, the control actions for different control steps obey different control laws. After the iteration number i + 1, the obtained control law sequence is (y_i, y_i_–1,..., y₀). With the iteration number i increasing to ∞, the obtained control law sequence has a length of ∞. For the infinite-horizon problem, both the optimal value function and the optimal control law are unique. Therefore, it is desired that the control law sequence will converge when the iteration number i⟶ ∞.

3) Actor network frame and parameter update

The input state variable of the action network is defined as s, and the role of the actor network is to generate the controller’s control strategy π. During the process of motor control, the control strategy is the d-q axis voltage. Therefore, the actual output of the actor network can be described as Equation (19): $u (k) = ω_{oa}^{T} (k) ψ_{a} (ω_{ia}^{T} s (k))$ (19) where $ω_{oa}^{T} (k)$ is the output weight matrix, $ω_{ia}^{T}$ is the input layer weight matrix, $ψ_{a} (ω_{ia}^{T} s (k))$ is the activation function of the hidden layer, s (k) is the input state variable.

Let ${\hat{ω}}_{oa}$ be an estimate of ω_oa; then the actual output can be expressed as Equation (20) $\hat{u} (k) = {\hat{ω}}_{oa}^{T} (k) ψ_{a} (ω_{ia}^{T} s (k))$ (20)

The feedback error signal used for tuning action Neural Network (NN) is defined as: $e_{a} (k) = {\hat{ω}}_{oa}^{T} (k) ψ_{a} (ω_{ia}^{T} s (k)) + \frac{1}{2} R^{- 1} C_{u} \nabla ψ_{a}^{T} (ω_{ia}^{T} s (k)) {\hat{ω}}_{oc}$ (21)

A quadratic function based on the error of the Bellman equation is defined as the minimum objective function, as in Equation (22): $δ_{a} (k) = \frac{1}{2} e_{a}^{T} (k) e_{a} (k) = \frac{1}{2} e_{a}^{2} (k)$ (22)

The weight update law for the action NN is designed based on the gradient descent algorithm, which is given by

$\begin{matrix} {\dot{ω}}_{oc} = - α_{oa} ψ_{a}^{T} ({\hat{ω}}_{oa}^{T} (k) ψ_{a} (ω_{ia}^{T} s (k)) \\ + \frac{1}{2} R^{- 1} C_{u} \nabla ψ_{a}^{T} (ω_{ia}^{T} s (k)) {\hat{ω}}_{oc}) \end{matrix}$ (23) where α_oa is the adaptive gain of the action NN.

4) Critic network frame and parameter update

In order to stabilize the closed-loop system and minimize the cost function, the critic network is used to implement the long-term minimization cost function J(x), thereby constructing the error function and cost function based on the Bellman equation:

Firstly, a neural network is utilized to approximate J(x) as follows: $J (x) = ω_{oc}^{T} (k) ψ_{c} (x) + ɛ_{c}$ (24) where $ω_{oc}^{T}$ is for the unkonwn ideal constant weights and ψ_c (x) is called the critic NN activation function vector, ɛ_c is the critic NN approximation error.

The derivation of the cost function J(x) with respect to x is: $J_{x} = ω_{oc}^{T} \nabla ψ_{c} (x) + \nabla ɛ_{c}$ (25)

We can let ${\hat{ω}}_{oc}$ be an estimate of $ω_{oc}^{T}$ , then we have the estimate of J(x) as follows: ${\hat{J}}_{x} = {\hat{ω}}_{oc}^{T} \nabla ψ_{oc} (x) + \nabla ɛ_{oc}$ (26)

A quadratic function based on the error of the Bellman equation is selected as the minimum objective function. Given any admissible control law, it is desired to select $ω_{oc}^{T}$ to minimize the squared residual error $E_{c} ({\hat{ω}}_{oc}^{T})$ as follows:

$\begin{matrix} E_{c} ({\hat{ω}}_{oc}^{T}) = \frac{1}{2} (ω_{oc}^{T} \nabla ψ_{oc} (A^{'} x_{k} + {B^{'} y}_{k} + C^{'}) \\ {+ x_{k}^{T} Q x_{k} + y_{k}^{T} {Ry}_{k})}^{T} (ω_{oc}^{T} \nabla ψ_{oc} (A^{'} x_{k} + {B^{'} y}_{k} \\ + C^{'}) + x_{k}^{T} Q x_{k} + y_{k}^{T} {Ry}_{k}) \end{matrix}$ (27)

The weight update law for the critic NN is presented based on a gradient descent algorithm, which is given by

$\begin{matrix} {\dot{ω}}_{oc} = - α_{oc} σ_{oc} {\hat{ω}}_{oc} \nabla ψ_{oc} (A^{'} x_{k} + {B^{'} y}_{k} + C^{'}) \\ + x_{k}^{T} Q x_{k} + y_{k}^{T} {Ry}_{k} \end{matrix}$ (28) where α_oc > 0 is the adaptive gain of the critic NN, $σ_{oc} = \nabla ψ_{oc} (A^{'} x_{k} + {B^{'} y}_{k} + C^{'}) / (\nabla ψ_{oc}^{T} (A^{'} x_{k}$ +B′y_k + C′) ∇ ψ_oc (A′x_k + B′y_k + C′) +1)

(2) DDPG-LSTM network

It should be mentioned that most of the above algorithm require the models of the controlled plants to be known or at least partially known. However, in practical applications, most models cannot be obtained accuracy in time. Therefore, it is necessary to reconstruct the non-linear systems with function approximators. In this text, we will present the specific method for modeling the IPMSM control systems with LSTM, which are widely used in the dynamical analysis of non-linear systems. Based on the LSTM model, the DDPG algorithm can be properly introduced to deal with the optimal control problems of unknown non-linear systems in time.

The framework of network design is shown in Fig. 6. The input parameters of the LSTM layer are set by X_a = [x_a1,x_a2,x_a3] and X_c = [x_c1,x_c2,x_c3], where x_a1,x_a2,x_a3 and x_c1,x_c2,x_c3 are the output by three convolutional neural networks (CNN), respectively. Inputs of each convolutional neural network are defined as Equations (29) and (30), respectively. After the LSTM layer, the control strategy u^ddpglstm and the control strategy expectation value J^ddpglstm with the enhanced correlation between the current time and the control target can be regarded as output. The schematic diagram is shown in Fig. 9(a) and (b). ${\begin{matrix} x_{a 1} = CNN (u^{ddpg} (k - 2), Δ e (k - 2)) \\ x_{a 2} = CNN (u^{ddpg} (k - 1), Δ e (k - 1)) \\ x_{a 3} = CNN (u^{ddpg} (k), Δ e (k)) \end{matrix}$ (29) ${\begin{matrix} x_{c 1} = CNN (J^{ddpg} (k - 2), u (k - 2)) \\ x_{c 2} = CNN (J^{ddpg} (k - 1), u (k - 1)) \\ x_{c 3} = CNN (J^{ddpg} (k), u (k)) \end{matrix}$ (30)

Fig. 9

Detailed black diagram of the proposed LSTM network.

In Equations (29) and (30), CNN represents a convolutional neural network; u^ddpg represents the d-q axis control strategies u_d and u_q generated by the DDPG agent, and J^ddpg represents the expected value of the control strategy; Δe(k-2), Δe(k-1), and Δe(k) represent the error of the d-q axis current at k-2, k-1, and k, respectively; u(k-2), u(k-1), and u(k) represent the k-2, k-1, and k time control strategy, respectively.

The main consideration of network design is that the magnitude of the control variable is determined by the error of the target variable. Therefore, the data input pair between the control voltage and the target current is established, and the empirical prediction based on LSTM is performed to increase the reliability of the output control strategy.

The new bench calibration method proposed in this text can automatically realize the optimal d-q axis current matching for arbitrary stator current, obtain the optimal d-q axis voltage and maximum torque output in the full speed-torque range only by traversing stator current and speed, respectively, and ensure uniform calibration accuracy. The basic calibration process is shown in Fig. 10.

Fig. 10

Bench automatic calibration process based on DDPG-LSTM.

4 Simulation analysis of d-q axis current matching based on DDPG-LSTM

In order to theoretically verify the effectiveness of the proposed novel optimal control method, the simulation model is built in the Simulink environment and the structural framework is shown in Fig. 11, and the motor parameters are shown in Table 2. Wherein, the reference signal unit (➀)generates the required I_d and I_q from a given stator current I_s, and the DDPG-LSTM agent (➁) outputs control voltages u_d and u_q.

Fig. 11

Black diagram of simulation model based on DDPG-LSTM.

Table 2

Parameters of IPMSM

Parameter	Value	Parameter	Value
Rate Te	100(N.m)	Max Te	240(N.m)
Rate Speed	4000(r/min)	R_s(Ω)	0.007
Rated P	90(kW)	L_d(mH)	0.8047
Voltage	336(V)	L_q(mH)	1.63
Max Speed	8000(r/min)	Flux (Wb)	0.2668

A. Nominal simulation of DDPG-LSTM

Figure 12 is the simulation results of d-q axis current and torque based on DDPG-LSTM algotihm, simulation results show that there are two dynamic adjusting process during the whole simualtion process about 0.4 s through compare d-q axis current and torque, that are training process and stage process. It can be seen from Fig. 9 that with the d-q axis current dynamic optimal matching value and the torque given the relative response during the training process. When t = 0.15 s, I_d and I_q reach a stable stage and the same stage to torque. At the same time, the value of torque is coming to the peak value among the whole simualtion time. Simulation results verify that the characteristics of the trial-and-error optimization and make the maximum torque output while the stator current is constant.

Fig. 12

Simulation results of d-q axis current dynamic regulating process.

Figures 13 and 14 show that the immediate reward values generated by the d axis agent and the q axis agent gradually increase and get closer to the preset value of 500 during the model control learning process. The increase of the immediate reward value means that the action generated by the agent plays a positive role in generating the maximun torque output, which is in line with the original intention of the reward function design, and realizes the optimization of d-q axis current matching to generate maximum torque output.

Fig. 13

The immediate reward of d axis agent.

Fig. 14

The immediate reward of q axis agent.

Simulation results verify the feasibility of the optimal matching method of d-q axis current. It further verifies that this method has the ability to derive the best d-q axis current under any given working condition through the traversal of multiple working conditions, which lays a good foundation for the automatic calibration of the bench.

B. Comparative simualtion

In this section, compare DDPG-LSTM with Feed-forward compensation decoupling control (FFCDC), DTC. The motor parameters are selected to be identical for all three control algorithms, so that the difference in the performance is only due to the difference in their torque generate capability and current optimal capability. Fig. 15 shows the response of d-q axis current and torque with three different algorithms. It can be seen from the figure that the DDPG-based system response lags the FOC and DTC algorithms. In the initial response phase of the system, DDPG needs more time to adapt to the environment through self-learning. When the system is stable, DDPG has the same adjustment capacity as FOC and DTC, as shown in Fig. 15(a). When t = 2.5 s, the system load changes, and all three have almost faster adjustment response capabilities. This is mainly reflected the fact that DDPG can generate greater torque output by optimizing the d-q axis currents. As shown in Fig. 15(b), the torque output capability based on DDPG is significantly higher than FOC and DTC.

Fig. 15

The response of d-q axis current and torque under three different controller.

In order to further verify the advantages of the proposed method in achieving the optimal matching of d-q axis current. the proposed method is realized in the Simulink model compared with the traditional decoupling method. Under the same stator current command, traverse the motor multiple sets of working conditions, observe the maximum torque output and the corresponding d-q axis current. The statistics of multiple sets of simulation data are shown in Table 3. The relative error is caculated by (T_eDDPG-T_eFFCDC)/ T_eFFCDC, changes in parameters such as inductance and flux linkage are not considered in the simulation process. These parameters will change to varying degrees during the actual motor operation.

Table 3

Simulation results of DDPG-LSTM-NDC and FFCDC and DTC

I _s(A)	DDPG-LSTM-NDC			DTC			FFCDC
	T_e(N.m)	I_d(A)	I_q(A)	T_e(N.m)	I_d(A)	I_q(A)	I_d(A)	I_q(A)	T_e(N.m)
10	16.15	–0.31	10.07	15.65	– 0.42	9.77	– 0.374	9.99	15.54
20	32.34	– 1.24	20.13	25.84	– 1.24	19.63	– 2.43	19.85	26.76
30	48.63	– 2.78	30.12	48.33	– 2.38	30.22	– 5.32	29.52	41.73
40	65.06	– 4.88	40.04	55.16	– 5.68	38.79	– 4.69	39.72	53.03
50	81.66	– 7.51	49.85	72.36	– 8.21	49.75	– 8.52	49.26	68.54
60	98.48	– 10.62	59.56	88.18	– 13.62	58.32	– 17.38	57.42	79.05
70	115.5	– 14.17	69.14	102.3	– 14.97	68.94	– 11.44	69.05	99.41
80	131.86	– 17.86	78.05	123.16	– 17.56	77.95	– 18.41	77.85	107.03
90	149.49	– 22.11	87.40	136.32	– 27.21	86.83	– 28.62	85.32	122.72
100	167.45	– 26.68	96.62	147.86	– 26.98	96.22	– 22.71	95.38	135.19

It can be seen from Table 3 that the maximum torque output capability based on the DDPG-LSTM-NDC method is significantly higher than FFCDC and DTC algorithm. The relative error among three algorithms is change between 23.8% and 3.9%, and the average is about 20% higher than that in the FFCDC and DTC control; the absolute value of the optimal I_d based on DDPG-LSTM-NDC is smaller than the optimal I_d based on FFCDC, and the opposite in terms of optimal I_q based on DDPG-LSTM-NDC is higher than the optimal I_q based on FFCDC and DTC, which further proves that based on DDPG-LSTM-NDC, more stator current components can be guaranteed to generate torque, reduce the component of excitation current, and increase the component of torque current. The component of the moment current reflects its ability to matching the optimal d-q axis current.

5 Experimental verification

In order to further verify the effectiveness and feasibility of the proposed calibration method, the bench test verification is divided into three parts: Firstly, comparative experiment under nominal condition is complete. Secondly, the feasibility of the proposed method in optimizing d-q axis current matching is verified; Thirdly, the calibration process of the whole range is verified. The whole bench experimental system consists of IPMSM, motor controller, the eddy current dynamometer and its supporting equipment, the structure of the motor test bench is shown in Fig. 16. CANape is the main tool to complete the whole calibration process, which is a measurement and calibration tool supporting the optimization, test and validation of software-based control systems.

Fig. 16

The structure of the motor test bench.

A. Experiment with nominal and disturbance condition based on DDPG-LSTM

First, a dynamic response experiment based on DDPG-LSTM is carried out. Set the motor to work in the rated state, add load disturbances, monitor the current and torque response of the motor through CANape, and record the experimental results.

The three-phase current and d-q axis current are shown in Fig. 17 and Fig. 18, respectively. From the figures can find that the three-phase current and d-q axis current stabilizes after a period of adaptation. When the external load changes in 0.25 s(A1) and 0.55 s(A2), the system quickly responses to stable, the system comes to convergence within 0.3 s. Meanwhile, the non-decoupling method based on the DDPG-LSTM algorithm can also quickly be adjusted according to the error and robustness to the motor disturbance.

Fig. 17

Diagram for the d-q axis current waveform.

Fig. 18

Diagram for the three-phase current waveform.

The torque response results are summarized in Fig. 19, which indicates that the torque of the system has smooth response characteristics. When the learning process increase, the torque error gradually decreases until it reaches a stable value.

Fig. 19

Variation of the torque error and the change point error with the iterative process.

B. Compare experiment with FFCDC, DTC and DDPG-LSTM

Firstly, in order to better verify the experimental results, all three control methods use the same external environment and only change the control method to ensure the authenticity of the experimental data. Among them, the initial parameters required in the neural network based on the DDPG algorithm are obtained by simulation, and then self-optimized through experiments. Comparative experiments mainly verify the ability of the three algorithms to generate torque at the same stator current.

It is observed from Fig. 20 that DDPG-LSTM is higher than FFCDC and DTC in generating maximum torque after the speed in applied. In other ways, we can find DTC algorithm has high torque ripples during steady state. Among all the generated torque, the FFCDC have the almost equal ability to DTC, and lower than DDPG-LSTM. Anyway, the DDPG-LSTM have the best capability of generate maximum output.

Fig. 20

Comparison of maximum torque generation based on DDPG-LSTM, FFCDC, DTC.

Secondly, many groups stator currents are applied to three motor controller to observe the maximum torque output. The results of maximum torque and d-q axis current generated by multiple groups stator currents are summarized. Meanwhile, a control experiment using traditional feedforward decoupling control is set up, the maximum torque and d-q axis current are obtained by adjusting the PI regulator. The results of the two sets of test data are shown in Table 4, the relative error is caculated by (T_eDDPG-T_eFFCDC)/ T_eFFCDC. Analysis of the data in Table 4 can get the fact that the torque output produced when the bench test is relatively small to the simulation results, the maximun torque output capacity based on DDPG-LSTM-NDC is higher than the conventional FFCDC control about 10%. Similarly, the DDPG-LSTM-NDC can ensure that more stator current components are used to generate torque for optimizing the d-q axis current matching.

Table 4

The experiment results of DDPG-LSTM-NDC and FFCDC

I_s(A)	DDPG-LSTM-NDC			FFCDC			DTC
	T_e(N.m)	I_d(A)	I_q(A)	I_d(A)	I_q(A)	T_e(N.m)	I_d(A)	I_q(A)	T_e(N.m)
100	22.5	– 25.4	98.8	– 27.5	96.1	20.1	– 28.3	97.1	21.1
180	43.7	– 68.8	165.2	– 70.2	165.7	37.5	– 69.2	166.1	39.3
250	64.8	– 122.6	219.1	– 125	216.5	56.1	– 128	211.8	52.3
320	86.5	– 184.4	265.2	– 185.5	260.7	76.7	– 192.2	262.5	77.1
380	105	– 242.8	301.3	– 245	290.5	96.7	– 245	290.5	95.3
450	127	– 309.9	338.5	– 310.8	325.4	120.9	– 315.8	322.6	118.3
530	146	– 371.9	373.3	– 375.2	374.3	140.9	– 374.1	375.3	142.3
610	179	– 442.8	418.2	– 445.2	417	168.5	– 446.1	419	171.2
700	210	– 510.9	476.9	– 512.5	476.8	190.3	– 514.2	471.3	192.3

The results of maximum torque and d-q axis current generated by multiple groups stator currents are summarized. Meanwhile, two control experiment using FFCDC and DTC control are set up. The analysis of the data is shown in Fig. 21. We get the fact that the maximun torque output capacity based on DDPG-LSTM is higher than the FFCDC and DTC control about 10%, and the DDPG-LSTM can ensure that more stator current components are used to generate torque for optimizing the d-q axis current matching.

Fig. 21

Comparative analysis of experimental results based on DDPG-LSTM and FFCDC and DTC control.

In summary, simulation analysis and bench experiment verify that the proposed method achieves the purpose of optimizing the d-q axis current matching by optimizing the control voltage using the empirical trial-and-error method, and the results show that the DDPG-LSTM has advantages in optimizing the d-q axis current matching and the maximum torque-to-current ratio output, which can make the most advantages of the motor and generate larger torque output.

C. Experimental analysis of the efficiency test results of manual calibration MAP and automatic calibration MAP on bench

In order to verify the improvement effect of MAP in the range of full speed and full torque on the motor efficiency of the platform automatic calibration, the motor efficiency measurement of the platform automatic calibration MAP vector control and manual calibration MAP vector control were carried out, respectively, and the experimental data were compared and analyzed.

It can be seen from Fig. 22 that the vector control based on DDPG-LSTM in the range of full speed and full torque is generally more efficient than that based on manual calibration. In the full speed range, the global MAP shows unique advantages. Especially in the medium and high speed range, the traditional vector control efficiency is 82% ∼89%, while the global MAP vector control efficiency is 93% ∼97%, with a maximum difference of 8%. This is mainly because the global MAP vector control in the medium and high speed range of the current distribution is more reasonable.

Fig. 22

The efficiency test results of manual calibration and automatic calibration on bench.

6 Conclusion

In this paper, the principle of IPMSM was analyzed, and got the conclusion that the high-precision calibration results are of great significance to improve the motor efficiency in the process of motor control. In order to achieve the optimal current matching of d-q axis and improve the work efficiency of IPMSM, this paper proposes an automatic calibration method of automotive electric motors platform based on DDPG-LSTM, which can quickly obtain the optimal control MAP in the full speed-torque range of IPMSM. Firstly, a DDPG network based on AC framework was built, and a reward function for optimizing the output control strategy performance of actor network was constructed. Secondly, in order to optimize the optimal control strategy output, the LSTM layer was added to the Actor network and Critic network outputs to enable the control strategy and expected evaluation generated by DDPG to have the ability of associative memory before and after the output. Simulation results show that DDPG-LSTM based control method has significant advantages in optimizing d-q axis current matching and maximum torque current ratio control. In order to further verify the effectiveness and feasibility of the method, the experimental verification was carried out on the bench with dynamometer and other equipment. Through the analysis of multiple experimental results, it is concluded that DDPG-LSTM can generate larger torque output, and the torque output increases by about 10% compared with that of the same stator current in PI control scheme, which greatly improve the efficiency of the motor. Finally, the proposed calibration method was verified by bench test, and the results show that the proposed method can realize the MAP calibration in full speed-torque range, and the precision of the calibration data was greatly improved compared with manual calibration. The results show that the motor efficiency was improved about 8% during medium and high speed range.

Footnotes

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant Nos. 51775082, 61976039) and the China Fundamental Research Funds for the Central Universities (Grant Nos. DUT19LAB36, DUT20GJ207), and Science and Technology Innovation Fund of Dalian (2018J12GX061).

References

Sun

and Wang

, Extension of virtual signal injection based mtpa control for interior permanent magnet synchronous machine drives into field weakening region, IEEE Transactions on Industrial Electronics 62(11) (2015), 6809–6817.

H.M.

, Huang

S.D.

, Luo

D.R.

and Gao

, et al., Dynamic dc-link voltage adjustment for electric vehicles considering the cross saturation effects, Energies 11(8) (2018).

Sue

S.M.

and Pan

C.T.

, Voltage-constraint-tracking-based field-weakening control of ipm synchronous motor drives, Ieee Transactions on Industrial Electronics 55(1) (2008), 340–347.

Casadei

, Profumo

, Serra

and Tani

, et al., Foc and dtc:Two viable schemes for induction motors torque control, Converter Technology & Electric Traction 17(5) (2004), 779–787.

Consoli

, Scarcella

, Scelba

and Cacciato

, An effective energy-saving scalar control for industrial ipmsm drives, IEEE Transactions on Industrial Electronics 60(9) (2013), 3658–3669.

Preindl

and Bolognani

, Model predictive direct torque control with finite control set for pmsm drive systems, part 1: Maximum torque per ampere operation, Ieee Transactions on Industrial Informatics 9(4) (2013), 1912–1921.

Inoue

, Inoue

, Morimoto

and Sanada

, Mathematical model for mtpa control of permanent-magnet synchronous motor in stator flux linkage synchronous frame, Ieee Transactions on Industry Applications 51(5) (2015), 3620–3628.

Khayamy

and Chaoui

, Current sensorless mtpa operation of interior pmsm drives for vehicular applications, Ieee Transactions on Vehicular Technology 67(8) (2018), 6872–6881.

Miao

, Ge

, Preindl

and Ye

, et al., Mtpa fitting and torque estimation technique based on a new flux-linkage model for interior-permanent-magnet synchronous machines, Ieee Transactions on Industry Applications 53(6) (2017), 5451–5460.

10.

Sun

T.F.

, Koc

and Wang

J.B.

, Mtpa control of ipmsm drives based on virtual signal injection considering machine parameter variations, Ieee Transactions on Industrial Electronics 65(8) (2018), 6089–6098.

11.

and Wang

, Maximum torque per ampere (mtpa) control for ipmsm drives based on a variable-equivalent-parameter mtpa control law, Ieee Transactions on Power Electronics 34(7) (2019), 7092–7102.

12.

Urasaki

, Noguchi

, Howlader

A.M.

and Yonaha

, et al., Wide-speed range operation of interior permanent magnet synchronous motor with parameter identification, Electric Power Components and Systems 37(8) (2009), 847–865.

13.

Liu

and Zhu

Z.-Q.

, Online estimation of the rotor flux linkage and voltage-source inverter nonlinearity in permanent magnet synchronous machine drives, IEEE Transactions on Power Electronics 29(1) (2013), 418–427.

14.

Hoang

K.D.

and Aorith

H.K.A.

, Online control of ipmsm drives for traction applications considering machine parameter and inverter nonlinearities, IEEE Transactions on Transportation Electrification 1(4) (2015), 312–325.

15.

Shen

H.L.

, Luo

, Liang

G.L.

and Shen

A.W.

, A robust dynamic decoupling control scheme for pmsm current loops based on improved sliding mode observer, Journal of Power Electronics 18(6) (2018), 1708–1719.

16.

Preindl

and Bolognani

, Optimal state reference computation with constrained mtpa criterion for pm motor drives, IEEE Transactions on Power Electronics 30(8) (2014), 4524–4535.

17.

Ullmann

, Reuss

H.-C.

and Zell

, Calibration system prototype for increasing thelevel of automation in stationary engine testing and calibration, SAE TechnicalPaper 1(4080) (2005), 559–560.

18.

and Xu

, Characterizing the torque lookup table of an ipm machine for automotive application, (2014).

19.

Rabiei

, Thiringer

, Alatalo

and Grunditz

, Improved maximum torque per ampere algorithm accounting for core saturation, cross coupling effect and temperature for a pmsm intended for vehicular applications, IEEE Transactions on Transportation Electrification 2(2) (2016), 150–159.

20.

Uddin

M.N.

and Rahman

M.A.

, High-speed control of ipmsm drives using improved fuzzy logic algorithms, Ieee Transactions on Industrial Electronics 54(1) (2007), 190–199.

21.

Park

J.W.

, Koo

D.H.

, Kim

J.M.

and Kim

H.G.

, Improvement of control characteristics of interior permanent-magnetsynchronous motor for electric vehicle, 37(6) (2001), 1754–1760.

22.

Vafamand

, Arefi

M.M.

, Khooban

M.H.

and Dragicevic

, et al., Nonlinear model predictive speed control of electric vehicles represented by linear parameter varying models with bias terms, Ieee Journal of Emerging and Selected Topics in Power Electronics 1-1.

23.

Uddin

M.N.

, Abido

M.A.

and Rahman

M.A.

, Development and implementation of a hybrid intelligent controller for interior permanent-magnet synchronous motor drives, Ieee Transactions on Industry Applications 40(1) (2004), 68–76.

24.

Chy

M.M.I.

and Uddin

M.N.

, Development and implementation of a new adaptive intelligent speed controller for ipmsm drive, Ieee Transactions on Industry Applications 45(3) (2009), 1106–1115.

25.

Vafamand

, Khooban

M.H.

, Khayatian

and Blabbjerg

, Design of robust double-fuzzy-summation nonparallel distributed compensation controller for chaotic power systems, Journal of Dynamic Systems Measurement and Control 140(3) (2018).

26.

Khooban

M.H.

, Gheisarnezhad

, Vafamand

and Boudjadar

, Electric vehicle power propulsion system control based on time-varying fractional calculus: Implementation and experimental results, IEEE Transactions on Intelligent Vehicles (2019).

27.

H.Y.

, Wang

S.H.

, Shao

K.R.

and Sun

J.B.

, Design novel fuzzy logic controller of ipmsm for electric vehicles, Elektronika Ir Elektrotechnika 20(6) (2014), 35–41.

28.

Uddin

M.N.

and Khastoo

, Fuzzy logic-based efficiency optimization and high dynamic performance of ipmsm drive system in both transient and steady-state conditions, Ieee Transactions on Industry Applications 50(6) (2014), 4251–4259.

29.

Justo

J.J.

, Mwasilu

, Kim

E.K.

and Kim

, et al., Fuzzy model predictive direct torque control of ipmsms for electric vehicle applications, Ieee-Asme Transactions on Mechatronics 22(4) (2017), 1542–1553.

30.

Uddin

M.N.

and Rebeiro

R.S.

, Online efficiency optimization of a fuzzy logic controller based ipmsm drive, Industry Applications Society Annual Meeting, 2009. IAS 2009. IEEE, (2009).

31.

Navid

, Khooban

M.H.

, Khayatian

and Blaabjerg

, Design of robust double-fuzzy-summation non-pdc controller for chaotic power systems, Journal of Dynamic Systems Measurement & Control

32.

Khooban

M.H.

, Vafamand

and Niknam

, T–s fuzzy model predictive speed control of electrical vehicles, Isa Transactions S0019057816300659.

33.

J.P.

, Shi

, Yu

H.S.

and Chen

, et al., Approximation-based discrete-time adaptive position tracking control for interior permanent magnet synchronous motors, Ieee Transactions on Cybernetics 45(7) (2015), 1363–1371.

34.

Subbarao

, Nuthi

and Atmeh

, Reinforcement learning based computational adaptive optimal control and system identification for linear systems, Annual Reviews in Control 42 (2016), 319–331.

35.

Kaelbling

L.P.

, Littman

M.L.

and Moore

A.W.

, An introduction to reinforcement learning, in: Steels

(Ed.) The Biology and Technology of Intelligent Autonomous Agents, Springer Berlin Heidelberg, Berlin, Heidelberg, (1995), pp. 90–127.

36.

Modares

, Lewis

F.L.

and Jiang

, Optimal output-feedback control of unknown continuous-time linear systems using off-policy reinforcement learning, Ieee Transactions on Cybernetics 46(11) (2016), 2401–2410.

37.

, Yang

and Shi

, Reinforcement learning output feedback nn control using deterministic learning technique, IEEE Transactions on Neural Networks & Learning Systems 25(3) (2014), 635–641.

38.

Prasad

L.B.

, Gupta

H.O.

and Tyagi

, Application of policy iteration technique based adaptive optimal control design for automatic voltage regulator of power system, International Journal of Electrical Power & Energy Systems 63(2) (2014), 940–949.

39.

Wang

, Yu

, Wang

and Qi

, A novel adaptive neuro-control approach for permanent magnet synchronous motor speed control, Energies (2018).

40.

Gebregergis

, Chowdhury

, Islam

and Sebastian

, Modeling of permanent magnet synchronous machine including torque ripple effects, Industry Applications IEEE Transactions on 51(1) (2015), 232–239.

41.

Chen

J.H.

, Li

and Qu

R.H.

, Maximum-torque-per-ampere and magnetization-state control of a variable-flux permanent magnet machine, Ieee Transactions on Industrial Electronics 65(2) (2018), 1158–1169.

42.

Kim

, Yoon

Y.D.

, Sul

S.K.

and Ide

, et al., Parameter independent maximum torque per ampere (mtpa) control of ipm machine based on signal injection, Applied Power Electronics Conference & Exposition, (2010).

43.

Bolognani

, Petrella

, Prearo

and Sgarbossa

, Automatic tracking of mtpa trajectory in ipm motor drives based on ac current injection, Ieee Transactions on Industry Applications 47(1) (2011), 105–114.

44.

Dianov

, Young-Kwan

, Sang-Joon

and Sang-Taek

, Robust self-tuning mtpa algorithm for ipmsm drives, Conference of the IEEE Industrial Electronics Society, (2008).

45.

Kim

, Yoon

Y.D.

, Sul

S.K.

and Ide

, Maximum torque per ampere (mtpa) control of an ipm machine based on signal injection considering inductance saturation, IEEE Transactions on Power Electronics 28(1) (2013), 488–497.

46.

Reigosa

D.D.

, Garcia

, Briz

and Raca

, et al., Modeling and adaptive decoupling of high-frequency resistance and temperature effects in carrier-based sensorless control of pm synchronous machines, Ieee Transactions on Industry Applications 46(1) (2010), 139–149.

47.

Rafaq

M.S.

, Mohammed

S.A.Q.

and Jung

J.-W.

, Online multiparameter estimation for robust adaptive decoupling pi controllers of an ipmsm drive: Variable regularized apas, IEEE/ASME Transactions on Mechatronics 24(3) (2019), 1386–1395.

48.

Rahman

K.M.

and Hiti

, Identification of machine parameters of a synchronous motor, Ieee Transactions on Industry Applications 41(2), 0–565.

49.

Hoang

K.D.

and Aorith

H.K.A.

, Online control of ipmsm drives for traction applications considering machine parameter and inverter nonlinearities, Ieee Transactions on Transportation Electrification 1(4) (2016), 312–325.

50.

Balamurali

, Feng

G.D.

, Lai

C.Y.

and Tjong

, et al., Maximum efficiency control of pmsm drives considering system losses using gradient descent algorithm based on dc power measurement, Ieee Transactions on Energy Conversion 33(4) (2018), 2240–2249.