Investigation on energy management strategy of gas-electric marine hybrid propulsion system with improved dynamic programing algorithm

Abstract

An energy management strategy (EMS) is important in marine hybrid propulsion systems while the global optimal solutions are more significant for the EMS to improve the fuel economy of system. Aiming at finding the more accurate global optimal solutions for EMS, an improved dynamic programing (DP) algorithm was proposed to overcome the basic DP algorithms’ weaknesses and improve the fuel economy. Then the model of gas-electric hybrid propulsion system was also established to verify the strategies. What’s more, the improved DP algorithm was applied to the minimum natural gas consumption and charge-sustaining strategy. And a bi-level optimization scheme that decoupled the interaction between the EMS and components matching was also studied. The results showed that the improved DP algorithm not only avoided the shortcomings in the basic DP algorithm such as the weak effect of penalty functions and curse of dimensionality but also gave more accurate solutions. Compared to the basic DP, the improved DP algorithm can reduce the gas fuel consumption by 3.67% and the computational complexity decreases 100 times.

Keywords

Marine hybrid propulsion system gas-electric hybrid propulsion system energy management strategy dynamic programing method bi-level optimization

Introduction

In recent years, energy saving and emission reduction have become the main goals in transportation industry. The shipbuilding industry which is one of the largest sources of pollutants and greenhouse gas emissions is under pressure. Especially, the International Maritime Organization (IMO) set the strict standards for emissions of nitrogen oxides (NOx) and sulfur oxides (SOx) in Emission Control Area (ECA). At present, the NOx emissions standards are implemented in Tier III and the SOx emissions standards limit the oil to 0.5% sulfur oxides. Furthermore, the Energy Efficiency Design Index (EEDI) was proposed by IMO to reduce greenhouse gas emissions which was already in EEDI III.¹ That means that new technologies are required to satisfy these standards. Hybrid propulsion technology is one of the key technologies to meet the stricter IMO regulations. This conclusion was verified in Hybrid Electric Vehicle (HEV) and Plug-in Hybrid Electric Vehicle (PHEV). Similarly, it also provides a reference for ship power plants.

We also noticed that more and more ships were equipped with hybrid propulsion systems.^2,3 Marine hybrid propulsion systems can improve the energy efficiency by optimizing the power distribution between different power sources. And the competitive advantage are a relatively lower fuel consumption and pollutant emissions. On the one hand, the hybrid propulsion system avoids the engine working under the poor conditions such as start-stop and idle conditions. On the other hand, downsizing the engine also improves fuel economy. It also includes other advantages, for example, lower vibration and noise, and great dynamic response performance, etc.^4–6 Therefore, the hybrid propulsion system is more suitable for tugboats, ferries and offshore platforms that the engine operating conditions change frequently and work at the low load.^7–9

Energy management for hybrid propulsion system

The core technology of hybrid propulsion systems is the energy management. And the energy management strategies (EMSs) are mainly divided into offline and online at present. There is no advantage in real-time and rapidity for off-line energy management strategies compared to the online strategies.¹⁰ However, the off-line EMSs based on intelligent optimization algorithms can solve the nonlinear and multiple-constrained energy management questions. Recently, the energy management algorithms based on heuristic algorithms become the research hotspots including Particle Swarm Optimization (PSO),¹¹ Genetic Algorithm (GA),^12,13 Ant Colony Optimization algorithm (ACO),¹⁴ and etc. Most of strategies take the equivalent fuel consumption (ECMS) as objective functions to obtain the optimal power distribution scheme based on heuristic algorithms.¹⁵ As for these methods, it depends on the selection of equivalent factor which lacks theoretical analysis. For improving the comprehensive performance of the system, the multi-objective optimization algorithm is applied to the energy management problems, which concerns the fuel consumption, emissions, and dynamic performance.¹⁶ It is worth noting that the EMSs, single, or multiple targets, based on heuristic intelligent algorithms are usually instantaneous and local rather than global.

Due to the expensive computing cost and the slow computation speed, the heuristic algorithms are difficult to apply in the energy management system in engineering. Therefore, the online EMSs are the main object of engineering application research, for instance, fuzzy Rule-Based (RB),¹⁷ Model Predictive Control (MPC),¹⁸ and Pontryagin’s Minimum Principle (PMP) energy management strategies.¹⁹ The RB algorithm is the most popular for EMSs which is simple and reliable. And the MPC strategies can optimize in real time rely on accurate model. The MPC methods have been tested and verified on hybrid propulsion system which show the great dynamic response performance and better fuel economy.²⁰ In fact, a multi-scheme energy management strategy is advisable and intelligent which means more robust.²¹ In summary, a lot of research show that the energy management is an optimization question, which aims to provide the optimum power distribution schemes with various objectives under different operating conditions.

The ultimate goal of energy management optimization questions is to find the global optimum solutions. As research subjects, the global optimum solutions of energy management for hybrid propulsion systems are defined as the best power allocation schemes for the whole voyage rather than at a moment. It’s also worth emphasizing that the global optimal solutions for the driving cycle is not the aggregation of the local optimal solutions. This paper will focus on the global optimization of EMSs for hybrid propulsion systems. It is worth to note that the global optimal solutions will provide a reference for other algorithms. That is why many articles compare the results of global optimization solutions to other EMSs, and the closer they are to the global solutions, the better results are obtained by the methods.^22–24

Energy management for dynamic programing algorithm

The dynamic programing (DP) algorithm is one of the important methods to solve global optimization problems which has been applied to EMSs on HEVs. Due to the slow development of marine hybrid propulsion, few studies are focused on the DP algorithms in energy management on ships. In HEVs, a fast DP algorithm was proposed by Xiang et al.²² whereby raising the calculation speed by 700 times. The method gave the optimal solutions by dividing the battery state of charge (SOC) and powertrain mode as states, and reduced the fuel consumption by 6.56% and 3.15% in the two driving cycle conditions on the power split hybrid vehicles system. Chen et al.²⁵ combined DP algorithm with neural network and the results obtained by the DP algorithm were implemented to the Power-Split Plug-in Hybrid Electric Vehicle (PHEV) in real time. Vagg et al.²⁶ researched the stochastic DP method on energy management in HEVs, which the vehicle speed, acceleration, gear, and battery SOC were selected as state variables. The results showed that the algorithm can effectively reduce the motor stress by 12% without fuel consumption losing in test. Bianchi et al.²⁷ designed a RB energy management strategy for a series/parallel HEV and this approach referred to the DP algorithm’s results, which is not only real-time but also near-optimal. Boyali et al.²⁸ developed a real-time neuro-dynamic programing method that approximated the DP solutions of optimum power distribution for HEV model. Compared to the basic control strategy, the fuel consumption can be reduced by 4.12% but the DP method reduced by 12.3%. Liu et al.²⁹ proposed a downshifting strategy using cluster-based stochastic DP. The state variables included wheel speed and battery SOC while the input variables were the motor’s torque and gear. The results showed that the fuel economy was improved by 12.3% compared with the traditional rule-based strategy. Except for the research on energy management based DP method, other studies on DP concentrated on comparing the effect of proposed algorithm to the DP algorithm and the details will not be described here.^30,31

All in all, most of the researches on the DP algorithm for energy management is combined with other advanced algorithms or just used as a reference to prove the proposed methods effectiveness. What we found that few researchers focused on the DP algorithm itself. And it is known that the DP algorithm originates from discrete mathematics and is more applicable to the discrete system. So, as for the marine hybrid propulsion system, new questions will inevitably arise, such as curse of dimensionality, and the crude operation of approximation, and interpolation when the DP algorithm is applied to the hybrid system.³² Especially for the new hybrid propulsion system, it is essential to study the special questions for the system. As a result, the EMS of a marine hybrid propulsion system based on the DP algorithm was studied in this paper.

Objectives and highlights

The objective of this paper is to provide an improved DP algorithm to get the more accurate solutions for energy management problem. Naturally, a bi-level optimization scheme that combined the improved DP algorithm and genetic algorithm was also proposed. However, the research in this regard is limited in published literature. The main outline of this paper is as follow:

The structure of gas-electric hybrid propulsion system was proposed, and the model was established to verify the strategies in section 2.

The improved DP methods was given in section 3. And we compared the differences between the improved DP methods and basic DP.

The EMSs based on the improved DP methods and a bi-level optimization scheme that combined the improved DP algorithm and genetic algorithm was proposed in section 4. And the effectiveness of the proposed improved DP methods in energy management was discussed in charge-depleting and charge-sustaining cycle.

Finally, the research conclusions in this paper and recommendations for future research will be provided in section 5.

System description and modeling

Gas-electric hybrid propulsion system

The schematic diagram of the marine hybrid propulsion system is shown in Figure 1, which consists of a natural gas engine, a permanent magnet synchronous machine (PMSM), a lithium iron phosphate battery, the clutch, the gearbox, the propeller, the power electronics, the shore power charging device, and etc. In this powertrain, the natural gas engine and PMSM are mechanically coupled through the gearbox to drive the propeller together. The DC grid consists of two battery packs which are connected by DC/DC to ensure the voltage stability. And the shore power charging device is also connected to the DC grid and charges the batteries when the ship is in dock. It is worth mentioned that the PMSM is reversible, which means it not only works as an electromotor but also as a generator. Therefore, the active inverter (DC/AC) is applied to control the PMSM. This powertrain is particularly suitable for tugs or ferries.

Figure 1.

Gas-electric hybrid propulsion system.

Because the powertrain includes the natural gas engine and PMSM, it is called the gas-electric hybrid propulsion system in this paper. And the working modes are as follows:

Mechanical propulsion mode: in this mode, only the natural gas engine is working and the PMSM stops. The propeller is driven by the natural gas engine.

Electrical propulsion mode: it is defined that the PMSM works in motor mode to drive the propeller and the natural gas engine stops.

Hybrid propulsion mode: the natural gas engine and electric motor drive the propeller together. It is also called power take in (PTI) mode in large two stroke diesel combined with shaft generator system.

Propulsion with charging mode: in this mode, the natural gas engine not only drives the propeller but also feeds power into the grid by PMSM. It is same as the power take out (PTO) mode in large two stroke diesel engines combined with shaft generator systems.

Shore power mode: when the ship at dock, the batteries will be charged by the charging device, and other electric loads will also be supplied.

There are many advantages of the gas-electric hybrid propulsion system. On the one hand, the operating costs will be reduced with shore power. Meanwhile, the parallel structure can regulate the engine operating in an economic area by energy management and the fuel of natural gas is clean, convenient, and efficient compared to diesel oil.^33,34 On the other hand, the weaknesses of natural gas engines (poor dynamic characteristics) are overcome by torque coordinating with PMSM. Furthermore, the electrical propulsion mode will reduce vibration and noise, which is conducive to quiet navigation when the ship berths.

System modeling and simulation

In this paper, the research will focus on how to coordinate the power distribution of the natural gas engine and the PMSM to minimize fuel consumption. And the DP algorithm was selected as the main energy management strategies for the gas-electric hybrid propulsion system. The model of system was established to test and verify the energy management strategies based on the DP algorithm.

It is pointed out that the quasi-static model can meet the requirements of energy management system modeling for marine hybrid power systems.²² On the one hand, the transient dynamic behaviors have little effect on the fuel consumption of the steady-state conditions.³⁵ On the other hand, the universal characteristics map of specific fuel consumption based on the manufacturer’s experimental data is enough accurate for describing fuel consumption and efficiency. Therefore, the fuel consumption of the natural gas engine can be described with output torque and engine rotational speed by fuel consumption map³⁶ and the following relations hold as equations (1)–(4):

g_{e} = f_{fuel} (N_{e}, T_{e})

(1)

η_{e} = \frac{3600 \times 10^{3}}{g_{e} H_{u}}

(2)

P_{e} = \frac{60}{2 π} \frac{N_{e} \times T_{e}}{1000})

(3)

m_{f} = \frac{P_{e} g_{e}}{3600} \times Δ t

(4)

Where $g_{e}$ is the specific fuel consumption of natural gas engine, $\frac{g}{kWh}$ . $N_{e}$ is the engine rotational speed, $rpm$ . And $T_{e}$ is the output torque, $Nm$ . $H_{u}$ is the lower calorific value, $\frac{kJ}{kg}$ . $η_{e}$ is engine efficiency, and $P_{e}$ is the engine output power, $kW$ . $m_{f}$ is the fuel consumption with respect to output power of the engine, $g$ . $Δ t$ is the engine running time, $s$ . It is worth noting that the $f_{fuel}$ function can be approximated with a quadratic function or be an interpolation function.

The PMSM is more efficient and reliable than induction motors.³⁷ What should be concerned is the PMSM’s efficiency for energy management. So, just like the method of engine modeling, the PMSM’s efficiency is modeled by the efficiency map from experiment. In this work, it is defined that the value of torque is negative when the motor works as a generator and positive as an electric motor. The model of PMSM are modeled as equations (5) and(6):

η_{m} = f_{m, eff} (N_{m}, T_{m})

(5)

P_{m} = \frac{60}{2 π} \frac{N_{m} \times T_{m}}{1000}

(6)

The battery efficiency and SOC are needed for the EMS. A rint electrical circuit model is implemented and the open cell voltage is given by equation (7). The battery SOC estimation is mainly modeled by ampere-hour method as equation (8). And the battery efficiency is related to SOC, see equation (9).

V_{oc} = V_{b} - I_{b} R_{i}

(7)

SOC (t) = SOC (t_{0}) - \frac{1}{C \times 3600} \int_{t_{0}}^{t} I_{b} η_{b} d t

(8)

η_{b} = f_{b, eff} (SOC)

(9)

The transmission device mainly includes a gearbox and a clutch, in which the clutch is integrated in the gearbox. Ignoring dynamic characteristic, the transfer efficiency needs to be considered that assumed it to be constant. The input and output relationship of the clutch for rotational speed and torque is expressed as equations (10) and (11).

{\begin{matrix} N_{c, out} = 0 \\ T_{c, out} = 0 \end{matrix} if clt = 0

(10)

{\begin{matrix} N_{c, out} = N_{c, in} \\ T_{c, out} = η_{c} T_{c, in} \end{matrix} if clt = 1

(11)

Where, the $N_{c, in}$ and $N_{c, out}$ is the input and output rotational speed of the clutch. $T_{c, in}$ and $T_{c, out}$ is the input and output torque of the clutch. $η_{c}$ is the clutch efficiency.

The dual-input and single-output gearbox is designed with two ratios. The relationships between input and output of torque and speed are given in equations (12)–(15).

(T_{e} N_{e} + T_{m} N_{m}) η_{g} = T_{s} N_{s}

(12)

T_{s} = (T_{e} i_{e} + T_{m} i_{m}) η_{g}

(13)

i_{e} = \frac{N_{e}}{N_{s}}

(14)

i_{m} = \frac{N_{m}}{N_{s}}

(15)

Where, $η_{g}$ is the gearbox efficiency. And the $i_{e}$ and $i_{m}$ are the rotational speed ratios relative to the output shaft of gear. $N_{s}$ and $T_{s}$ are the rotational speed and torque of the output shaft.

The aim of the propeller modeling is to calculate the thrust and torque of the propeller which are characterized as functions (16) and (17). And the propeller is also the key component that establishes connection between the main engine speed, propeller rotational speed, and ship speed. Therefore, the relationship among the propeller rotational speed and navigational speed is expressed as equation (18) by the advance coefficient. The open water characteristics of the propeller are shown in equations (18)–(21).

T_{p} = K_{T} ρ {N_{p}}^{2} D^{4}

(16)

Q_{p} = K_{Q} ρ {N_{p}}^{2} D_{p}^{5}

(17)

J_{p} = \frac{v_{A}}{n_{p} D_{p}}

(18)

V_{A} = V_{S} (1 - w)

(19)

K_{T} = f_{T} (J_{p})

(20)

K_{Q} = f_{Q} (J_{p})

(21)

Where, the $D_{p}$ is the diameter of the propeller, $ρ$ is the density of sea water. $N_{p}$ is the rotational speed. The $T_{p}$ and $Q_{p}$ are the thrust and torque of the propeller. $K_{T}$ and $K_{Q}$ are the thrust and torque coefficients of the propeller that can be derived from open water characteristics as shown in Figure 2. $J_{p}$ is the ratio of the propeller advance coefficient and $w$ is the wake fraction. $V_{A}$ and $V_{s}$ are the advanced speed and ship speed.

Figure 2.

The open water characteristics of propeller.

Other power electronics, such as DC/DC and DC/AC, only consider the energy efficiency and assume that the efficiency is constant with ignoring tiny changes in working. It will be simple and fast for the calculation and the steady efficiency is sufficient to meet the requirements for the energy management. The parameters of all the components for the gas-electric hybrid propulsion system are shown in Table 1.

Table 1.

The parameters for gas-electric hybrid propulsion system.

Parameter of natural gas engine	Value
Rated power/kW	300
Rated rotational speed/r/min	1800
Parameter of PMSM
Rated power/kW	68
Rated rotational speed/r/min	2200
Parameter of battery
Nominal capacity/kWh	100
Nominal voltage/V	503.7
Parameter of gearbox and clutch
Efficiency of gearbox/%	95
Efficiency of clutch/%	98
Reduction ratio I	6.38
Reduction ratio II	7.58
Parameter of propeller
Propeller area ratio	0.5
Pitch ratio of propeller	0.8
Diameter of propeller	2.1
Wake fraction	0.23
Relative rotational efficiency/%	99

Energy management problem formulation

The EMSs of the gas-electric hybrid propulsion system are shown in Figure 3. The goal of the EMSs is to satisfy the power demand for propulsion and guarantee the minimum fuel consumption. First, the power demand can be explained by the ship dynamic model with respect to the velocity of ship and steering angle. Then, the reference rotational speed PMSM and natural gas engine are given from the driver model. And the torque will be allocated with objective of the fuel consumption and the constrains for battery SOC and other limitations considered. In this work, the objective function that the fuel consumption of the whole voyage is dealt with by DP methods. The monitor and alarm strategies are often included in the EMS system, although they are not discussed in this article.

Figure 3.

The diagram of energy management strategy for the gas-electric hybrid propulsion system.

The DP approach can give the optimal control trajectory with Bellman’s principle. It is usually applied in the multi-stage questions. So, the DP approach is quite suitable for the ship energy management because the ship navigation profile can be divided into the multi-stage. Especially, the historical data is necessary for DP algorithm calculation. It is easier than automobiles to get the historical data for the ship with the fixed route and schedule. For example, the ship will sail a fixed route from city to city or country to country in some days or months. This is just like the New European Driving Cycle (NEDC) for cars.

The basic DP method

Before the discussion of the DP method applied to the energy management for the gas-electric hybrid propulsion system, the basic DP method’s principle is introduced. The DP approach utilizes the backward calculation and looking forward model to give the optimal control trajectory. As for the continuous systems and hybrid systems, the input and state variables should be discretized. The basic model can be expressed as equations (22)–(26).

x_{k + 1} = f_{k} (x_{k}, u_{k})

(22)

x_{k} \in X_{k}

(23)

x_{0} = x_{int}

(24)

x_{N} \in T or x_{N} = x_{end}

(25)

u_{k} \in U_{k}, k = 0, 1, . . ., N

(26)

Where, the $x_{k}$ is the state variable and the $k$ is the time index. And the $f_{k}$ is the state space equation. The $X_{k}$ and $U_{k}$ are the aggregates of the state variables and input variables. And the final state node can be limited to the aggregate $T$ or the state $x_{end}$ , in this paper $T = [0.3 0.8]$ .

In the discrete state space, the initialization of the cost to go function at each node can be calculated as equation (27).

J_{N} (x_{k}^{i}) = {\begin{matrix} c_{N} (x_{k}^{i}), for x_{k}^{i} \in T \\ \infty, else \end{matrix}

(27)

Where, the superscript $i$ is the state index at the node with the time index $k$ . A large number will be given for penalizing if the state beyond constraints.

From the $N - 1$ step to 0, the objective of the backward iteration is shown as equation (28).

J_{k} (x_{k}^{i}) = min_{u_{k} \in U_{k}} {c_{k} (x_{k}^{i}, u_{k}) + J_{k + 1} (f_{k} (x_{k}^{i}, u_{k}))}

(28)

It means that the optimal control signal at every state in time index $k$ is given by the minimizing the whole cost function of the time $k + 1$ to $N$ . That assures the optimal solutions are global.

Therefore, by the backward iteration, the optimal control policy is expressed as equation. (29).

π = {μ_{0}, μ_{1}, . . ., μ_{N - 1}}

(29)

It is important to note that the $μ_{k}$ is the aggregate of the optimal control input for each state node at time $k$ . With the definition of the initial state, the optimal control trajectory will be derived by equations (30)–(32).

x_{0} = x_{int}

(30)

u_{k}^{opt} = \underset{u_{k} \in U_{k}}{\arg min} {c_{k} (x_{k}^{i}, u_{k}) + J_{k + 1} (f_{k} (x_{k}^{i}, u_{k}))}

(31)

x_{k + 1} = f_{k} (x_{k}, u_{k}^{opt})

(32)

The basic DP method will cause the numerical problems. First, the cost to go function needs to be calculated at each state at time $k$ by equation (28) while the cost-to-go $J_{k + 1} (f_{k} (x_{k}^{i}, u_{k}))$ is known for the all state before the time $k - 1$ . So, the cost to go function need to be evaluated at time $k$ and the state $x_{k}^{i}$ should be derived by the equation (22). One question is that the state variable we get may not generally coincide with the grid when the system is continuous. Thus, the interpolation is used to find the least state node on grid to replace the real state. And $J_{k + 1} (f_{k} (x_{k}^{i}, u_{k}))$ should be also interpolated which causes the inaccuracy of the optimal solutions. And what we need doing to overcome the shortcoming is giving a finer grid and increasing the number of grid nodes. These two solutions will also result in prohibitively large amounts of computational time. A large penalty value will be used to handle the constraints in the DP method. Therefore, another question is the effect of leaking of penalty values due to interpolation.³⁸ Although, a level set DP algorithm was proposed to solve this problem by this paper.³⁹ The interpolation is still inevitable.

All in all, there are still some numerical issues for the DP method that result in an inaccurate global optimal solution. In other words, especially for the hybrid propulsion system, it is difficult to achieve minimum fuel consumption. So, this paper will focus on the minimum fuel consumption of energy management based on DP algorithm and improve the performance in energy management to overcome problems of interpolation, curse of dimensionality and etc.

The improved DP method

As we know, because the state variables we get from equation (22) are not always coincide with the grid especially the system is continuous. And the interpolation and approximation operations will be used to deal with the problem. But it will produce the tolerances. What we need improving is avoiding the interpolation and ensure the state variables are coincide with the grid. Therefore, the improvements of basic DP method are described as equation (33).

u_{k} = \bar{g} (x_{k}^{i}, x_{k + 1}^{j})

(33)

Different from the equation (22), the inputs are obtained by the state i to state j which avoids the interpolation. And the additional operation that the constraints should be added on the obtained input variables, just like equation (34).

{\bar{u}}_{k}^{j} = \arg \max {max (u_{-}^{\max} u_{k}^{j}, u_{k}^{j} - u_{k}^{\min}}

(34)

To understand, other differences between the basic DP and the improved DP algorithms are shown as Table 2.

Table 2.

The differences between the basic DP and the improved DP algorithms.

	The basic DP algorithm	The improved DP algorithm
State and input variables constrain	$x_{k} \in X_{k}, u_{k} \in U_{k}$	$x_{k} \in X_{k}, u_{k} \in U_{k}$
State update equations	$x_{k + 1} = f_{k} (x_{k}, u_{k})$	$u_{k} = \bar{g} (x_{k}^{i}, x_{k + 1}^{j})$
Cost to go function	$J_{N} (x_{k}^{i}) = {\begin{matrix} c_{N} (x_{k}^{i}), for x_{k}^{i} \in T \\ \infty, else \end{matrix}$	$J_{N} (x_{k}^{i}) = {\begin{matrix} c_{N} (x_{k}^{i}), for x_{k}^{i} \in T \\ \infty, else \end{matrix}$
The objective of the backward iteration	$J_{k} (x_{k}^{i}) = min_{u_{k} \in U_{k}} {\begin{matrix} c_{k} (x_{k}^{i}, u_{k}) + \\ J_{k + 1} (f_{k} (x_{k}^{i}, u_{k})) \end{matrix}}$	$J_{k} (x_{k}^{i}) = min_{u_{k} \in U_{k}} {\begin{matrix} c_{k} (x_{k}^{i}, x_{k + 1}^{j}) + \\ J_{k + 1} (f_{k} (x_{k}^{i}, \bar{g} (x_{k}^{i}, x_{k + 1}^{j}))) \end{matrix}}$
The optimal control trajectory	$u_{k}^{opt} = \underset{u_{k} \in U_{k}}{\arg min} {\begin{matrix} c_{k} (x_{k}^{i}, u_{k}) \\ + J_{k + 1} (f_{k} (x_{k}^{i}, u_{k})) \end{matrix}}$	$u_{k}^{opt} = \underset{u_{k} \in U_{k}}{\arg min} {\begin{matrix} c_{k} (x_{k}^{i}, x_{k + 1}^{j}) + \\ J_{k + 1} (f_{k} (x_{k}^{i}, \bar{g} (x_{k}^{i}, x_{k + 1}^{j}))) \end{matrix}}$

Because of the state update equation changing, the input variables are related with the state variables, which caused the cost to go function, the objective of the backward iteration, and the optimal control trajectory differ from the basic DP method. But the constraints of the state and input variables are identical. All in all, the main contribution of the improved DP algorithm is the state update equations, which can improve the calculating precision by avoiding interpolation.

Energy management based the improved DP method

In this paper, the battery SOC was taken as the state variable while the torque of the motor was the control variable. Both of them were discretized in the grid, which can be expressed as the following equations (35) and (36). It should be noted that the state of the clutch is coupled with the motor torque. That is, the state of the clutch is off when the motor torque is zero and vice versa. Therefore, the switching state of the clutch will not be defined repeatedly.

x = {SO C_{0}, SO C_{1}, . . ., SO C_{M 1}}

(35)

u = {{T_{m,}}_{0}, {T_{m}}_{, 1}, . . ., {T_{m}}_{, M 2}}

(36)

Because of the shore power, the battery can be charged when the ship comes back in dock. Therefore, the battery will be used first during navigation, and it is also called the charge deplete-charge sustain strategy. And the gas fuel consumption of the natural gas engine in the whole navigation cycle of the ship is taken as the optimum objective function, following equation (37).

min_{u_{k} \in U_{k}} {{m_{f,}}_{N} (SO C_{N}) + \sum_{0}^{N - 1} m_{f, k} (SO C_{k})}

(37)

As we know the torque of the motor, the torque of the engine can be obtained by equation (38).

T_{e} = (T_{p} - T_{m} i_{m} η_{g}) / i_{e} η_{g}

(38)

The gas fuel consumption can be described directly by the torque of the natural gas engine. When the input variable of motor torque is determined, the battery SOC can also be derived. And the objective of the k step backward iteration is as follow equation (39).

\begin{matrix} J_{k} ({SOC}_{k}^{i}) = min_{u_{k} \in U_{k}} {{m_{f,}}_{k} ({SOC}_{k}^{i}, T_{{_{m,}}_{k}}^{j}) + J_{k + 1} ({SOC}_{k + 1}^{j})}, \\ i = 1, 2, . . ., M 1, j = 1, 2, . . ., M 2 \end{matrix}

(39)

For the energy management of gas-electric hybrid propulsion system, the constraints should be concerned with equations (40)–(45). The constraints include the battery SOC limitation, the envelope limitation of the engine and the motor, limitation of charge, and discharge capabilities, and the rotational speed limitation of the engine, and the motor.

x_{k} \in X_{k}, X_{k} = [S O C_{m i n}, S O C_{m a x}

(40)

u_{k} \in U_{k}, U_{k} = [T_{m, \min} (k), T_{m, \max} (k)

(41)

T_{e, \min} (k) \leq T_{e} \leq T_{e, \max} (k)

(42)

P_{b, \min} (k) \leq P_{b} \leq P_{b, \max} (k)

(43)

n_{e, \min} (k) \leq n_{e} \leq n_{e, \max} (k)

(44)

n_{m, \min} (k) \leq n_{m} \leq n_{m, \max} (k)

(45)

After the redefinition of the DP method for energy management, the optimal solutions can be calculated with the basic DP method. But the numerical issues such as interpolation, constraint penalty, dimension disaster, and so on can’t be avoided. Above the analysis of the definition of the DP method for energy management, the state and input variables and cost to go function are not independent. Because the hybrid propulsion system is a typical hybrid system and both continuous and discrete variables are related which is different from the discrete system. It is necessary to improve the DP method to adapt to energy management of the gas-electric hybrid propulsion system and overcome the shortcomings of DP methods.

Firstly, the battery SOC can be calculated by equations (46)–(48) which is mainly affected by the battery open circuit voltage and battery internal resistance. And the internal resistance and open circuit voltage are related to temperature, etc.

P_{b} = \frac{2 π}{60} \frac{T_{m} N_{m}}{1000} \times (η_{ac} η_{dc})^{- sign (P_{m})}

(46)

I_{b} = \frac{V_{oc} - \sqrt{V_{oc}^{2} - 4 R_{i} P_{b}}}{2 R_{i}}

(47)

SOC (k + 1) = SOC (k) - \frac{I_{b} Δ t}{Q_{batt}}

(48)

The research showed that the change of the internal resistance was very small, and that the influence of the internal resistance can be expressed by the battery efficiency in some range of the battery SOC.⁴⁰ At the same time, the battery open circuit voltage in pack can be considered as a constant in a certain range of SOC.^41,42 So, we can suppose that the internal resistance and the open circuit voltage of the battery are constant in the SOC range of 0.3–0.8. The battery SOC can be simplified by equations (49)–(50).

S = 0.5 \times (1 + sign (P_{m}))

(49)

\begin{matrix} SOC (k + 1) = \\ SOC (k) - \frac{\frac{2 π}{60} \frac{T_{m} N_{m}}{1000} \times {(η_{ac} η_{dc})}^{- sign (P_{m})} ({η_{dis}}^{- 1} S + η_{ch} (1 - S))}{Q_{batt} V_{oc}} \times Δ T \end{matrix}

(50)

Where, the $S$ is the factor to distinguish the generator and motor mode. When the $P_{m}$ is positive, the $S$ is 1 and the PMSM works in motor mode. When the $P_{m}$ is negative, the $S$ is 0, the PMSM works in generator mode.

From the above formula (50), we establish the direct relationship between the state variable of the battery SOC and the control variable of PMSM torque. And we can directly calculate the battery SOC through the motor torque. Similarly, the motor torque can be deduced through the battery SOC.

Because of the formula (50), an improved DP algorithm was proposed for the gas-electric hybrid propulsion system. The input variable was driven by the state variable described as equation (51). Therefore, the number of state and input variables is equal in this method.

u (k) = \bar{g} ({SOC}_{k}^{i}, {SOC}_{k + 1}^{j}) i = 1, . . ., M, j = 1, . . ., M

(51)

In other words, it is avoided that the state calculated by the state space equation is not in the state grid nodes. That means it does not need interpolation. But it is also noted that the input variables must satisfy constraints. So, a Level-Set methods was adopted to deal with the constraints of the input variable, which is described as equation (52).³⁹

{\bar{u}}_{k}^{j} = argmax {max (T_{-}^{\max} T_{m, k}^{j}, T_{m, k}^{j} - T_{m, k}^{\min}}

(52)

Due to the input variable in k step being obtained by the state variable between k and k + 1 step, it does not need to calculate the Backward-Reachable Space. In the improved DP method for energy management, the cost to go function is followed equations (53) and (54):

f_{k} ({SOC}_{k}^{i}, \bar{g} ({SOC}_{k}^{i}, {SOC}_{k + 1}^{j})) = m_{f, k} ({SOC}_{k}^{i}, \bar{g} ({SOC}_{k}^{i}, {SOC}_{k + 1}^{j}))

(53)

\begin{matrix} J_{k} ({SOC}_{k}^{i}) = min_{u_{k} \in U_{k}} {f_{k} ({SOC}_{k}^{i}, \bar{g} ({SOC}_{k}^{i}, {SOC}_{k + 1}^{j})) + J_{k + 1} ({SOC}_{k + 1}^{j})}, \\ j = 1, 2, M, i = = 1, 2, M \end{matrix}

(54)

Discussion for energy management by the improved DP method

Minimum natural gas consumption strategy

In order to assess the effectiveness of the improved DP algorithm in energy management, the typical tugboat voyage conditions were adopted in this paper, as illustrated in Figure 4. And the main working modes of tugs were classified as standby, transit, barge moving, and shore power. For the energy management in the gas-electric hybrid propulsion system, the input variable is the reference velocity of ship and the output variables are the torque and speed of the PMSM and engine. And the targets are minimizing the fuel consumption and sustaining the battery SOC. The research scheme in this section was taken as follows. First, compare and analyze the performance differences between the basic DP algorithm and the improved DP algorithm in energy management by utilizing the identical working conditions and same optimization targets. Second, for the questions of interpolation and curse of dimensionality, the influence of penalty functions and dimension of discretization on the improved DP algorithm was discussed.

Figure 4.

Voyage conditions of a tugboat.

Due to the shore power, the battery can be charged at the end of the voyage. So, in order to save the fuel cost, the battery is used as much as possible, and there are no restrictions on the terminal SOC. The minimum natural gas consumption strategy was taken for the energy management based on two DP methods. When the driving cycle begins, the SOC of battery is set as 0.8 and the level of discretization for state and input variables is 1000 × 1000. The changes of natural gas consumption are shown in Figure 5. The results demonstrate that the natural gas consumption of the improved DP method is reduced by 3.7% compared to the basic DP method in the whole voyage cycle. This is because the improved DP algorithm reduces the interpolation and makes the global optimal solutions more accurate.

Figure 5.

Comparison between the basic DP algorithm and improved DP algorithm for gas consumption.

And the torque distribution between the natural gas engine and PMSM is showed as Figures 6 and 7. it is obvious that the PMSM spends more time in generator mode in basic DP algorithm. Similarly, the natural gas engine torque of basic DP is higher than improve DP algorithm. The extra energy will charge the battery, which results in an increase for fuel consumption. Noting that after 6000 s, the mode of PMSM changes frequently. The reason is that the energy management based on the basic DP algorithm give a suboptimal PMSM torque, which will also cause the battery SOC too high or too low and the PMSM mode changing frequently. But the improved DP algorithm avoids the phenomenon.

Figure 6.

Torque of PMSM for the improved DP algorithm and basic DP algorithm.

Figure 7.

Torque of natural gas engine for the improved DP algorithm and basic DP algorithm.

And the natural gas engine and PMSM working conditions are shown as Figure 8(a) and (b). The magenta dash line and the purple dash line represent the average specific fuel consumption /PMSM efficiency for basic DP and improved DP algorithm. Although the fuel consumption of the whole cycle is reduced in the improved DP algorithm, the average efficiency of the natural gas engine is lower while the average PMSM efficiency are higher. That is because the energy management objective is the minimum fuel consumption. The higher PMSM efficiency, the lower fuel consumption. That means the improved DP algorithm is a more effective. That also means the part of the natural gas engine power charged into the battery can improve the natural gas engine efficiency and also the generation efficiency of PMSM.

Figure 8.

(a) The working conditions of the natural gas engine for minimum natural gas consumption strategy. (b)The working conditions of the PMSM for minimum natural gas consumption strategy.

When it comes to the energy management, it is crucial to note that the strategy designed should trade off the fuel consumption, efficiency, and other factors to make the system more competitive. However, in this paper, we will concentrate on the global optimal solutions and improving the DP method’s performance. And in the next section, the optimization objective for energy management will be changed to improve the efficiency of the natural gas engine and work in the charge-sustaining mode. What’s more, it can also compare the performance of the two algorithms on different targets.

As mentioned earlier, the level of discretization determines the accuracy of the solutions for DP algorithms, but the finer discrete grid will lead to the numerical problem of dimension disaster and increased calculation time. The computational complexity of the DP algorithm can be calculated by the following formula (55).

N_{feval} = N \times Π_{i = 1}^{n} N_{x_{i}} \times Π_{j = 1}^{m} N_{u_{j}}

(55)

Where, the $N$ denotes the number of working conditions. The $N_{x_{i}}$ is the number of state variables and the $N_{u_{j}}$ is the number of input variables.

Because of the particularity of the improved DP method, the level of discretization of the input variables is same as the state variables. In order to ensure the fairness for comparison of the two algorithms, the same levels of discretization were also adopted for the basic DP algorithm. The computational complexity and the optimal solution in the improved DP algorithm at different levels of discretization are shown in Table 3.

Table 3.

The performance of the improved DP algorithm at different level of discretization.

Levels of discretization	Gas cost	Function evaluations
100 × 100	21.69	4120,000
250 × 250	21.67	25,750,000
500 × 500	21.62	103,000,000
1000 × 1000	21.25	412,000,000

As the levels of discretization increasing, the gas fuel consumption decreases, which implies the more accurate optimal solution of the improved DP algorithm. But the improvement in accuracy is little while the computational complexity increases significantly. In other words, the improved DP algorithm can give the global optimal solution with less computing if the tolerance is allowed. Thus, its computational complexity can be reduced 100 times. And the curse of dimensionality defines that when the dimension of state variables increases, the computational burden will increase exponentially. Especially for basic DP algorithm, to find the more accurate global solutions, the larger battery capacity is, the greater number of grid nodes will be needed. That will be cause the curse of dimensionality. As shown Table 3, although the levels of discretization increases, the results almost have no changes. That means there is no need to take the high level of discretization and the curse of dimensionality is avoided.

Comparing the basic DP algorithm with 1000 × 1000 discretization to the improved DP algorithm with different levels of discretization. As demonstrated in Figure 9, all the results of gas consumption based on the improved DP algorithm are less than the basic DP. And it can save 3.74% natural gas consumption based on the improved DP algorithm at the same level of discretization. It should be noted that when levels of discretization are below 250 × 250, the calculation breaks down. The reason is that the rough grid makes larger errors with interpolation and the real state is farther from the grid nodes, which caused the state variables to violate constraints. It also indicates that the improved DP algorithm is more adaptable and robust.

Figure 9.

Comparison for gas consumption between improved DP algorithm and basic one with different level of discretization.

Different initial SOC will lead to different optimal trajectories when using the DP algorithm for forward calculation. As shown in Figure 10, the change of SOC was researched based on the improved DP algorithm in the whole voyage. When the initial SOC is low such as 0.3–0.4, the battery is charged for a long time to meet the future power requirement. And while the initial SOC at 0.5–0.8, the battery SOC decreases gradually until the battery reaches the minimum SOC. Although with the different initial SOCs, they have the approximately equal terminal SOCs. There are two reasons for this phenomenon. One is due to Bellman’s principle of optimality, and the other one is because the optimization designed objectives prefer consuming battery power.

Figure 10.

SOC changes for improved DP algorithm with different initial SOC.

Charge-sustaining strategy with SOC constrains

The charge-sustaining strategy is one of the important EMSs in the hybrid propulsion system. In this section, the charge-sustaining strategy with the penalty function was studied. And as previously mentioned in this paper, the basis DP algorithm will cause the weak effect of penalty functions. In order to study the effect of the penalty functions on the improved DP algorithm, the penalty function was added to the objective function to restrict the SOC at 0.5 in equation (56). On the one hand, the battery efficiency is comparatively high when the SOC is in this range. On the other hand, the EMS strategy and the objective function were designed to sustain battery SOC and also minimize the natural gas engine fuel cost. The cost to go function for iteration is described in equation (57).

φ_{k} = m_{f, k} + λ_{0} (x_{k}^{i} - x_{0})^{2}, x = SOC

(56)

\begin{matrix} J_{k} (x^{i}) = min_{u_{k} \in U_{k}} {φ_{k} (x_{k}^{i}, \tilde{g_{k}} (x_{k}^{i}, x_{k + 1}^{j})) + J_{k + 1} (φ_{k} (x_{k}^{i}, x_{k + 1}^{j}))}, \\ j = 1, 2, . . ., M, i = = 1, 2, . . ., M \end{matrix}

(57)

Where, the $λ_{0}$ is the penalty factor.

The change of SOC is shown in Figure 11. When the objective function has a penalty term, the penalty effect of the improved DP algorithm is obviously better than the basic DP. Meanwhile, the SOC fluctuation is also smaller. However, at 3000–4000 s, both fluctuate in a large range. This is because the high ship speed causes the high required propulsion torque while the tug is in the transit mode. And in this mode, the natural gas engine and PMSM should work together. What’s more, the effect of penalty function in the two algorithms was estimated by boxplot as Figure 12. Besides the difference between the maximum and the minimum value of SOC during the whole voyage, which was caused by high ship speed. The other quartile values of SOC are closer to the constraint SOC in the improved DP algorithm.

Figure 11.

Comparison for SOC changes between improved DP algorithm and basic one.

Figure 12.

The boxplot of SOC.

Table 4 shows the gas consumption and the terminal SOC based on Charge-Sustaining strategy for two algorithms with the same level of discretization. The terminal SOC of the improved DP algorithm is relatively lower while the fuel consumption is less. This means that the improved DP algorithm still has the advantage for energy saving. Combined with the changes of SOC in navigation, it can be seen that the improved DP algorithm can ensure more accurate global optimal solutions under the effect of penalty functions.

Table 4.

Comparison between improved DP algorithm and basic one at same level of discretization.

	Basic DP algorithm	Improved DP algorithm
Level of discretization	500 × 500	500 × 500
Gas cost	31.34	31.24
SOC terminal	0.472	0.466
Improvement	–	0.316%

The Figure 13(a) and (b) show the operating points of the natural gas engine and PMSM when the charge-sustaining strategy was adopted for energy management. Contrary to the improved DP algorithm, the working points of the natural gas engine based on the basic DP algorithm are almost in the efficient operation region. However, it also leads to the PMSM working in the inefficient working conditions. In the improved DP algorithm, the reverse applies. The usage of the penalty function makes the penalty effect on SOC stronger in the EMSs based on the improved DP algorithm. In other words, the strategy tends to adopt the mechanical propulsion mode and reduce the PMSM running time. The results can be as a valuable reference for online energy management.

Figure 13.

(a) The working conditions of the natural gas engine for Charge-Sustaining strategy. (b) The working conditions of the PMSM for Charge-Sustaining strategy.

In summary, the improved DP algorithm proposed in this paper has an advantage in solving the global optimization problem. It is very suitable for the EMS in hybrid propulsion systems. And it not only avoids the interpolation in the backward calculation in the basic DP algorithm but also the weakening effect of the penalty functions. What’s more, it does not rely on the large number of nodes in the discrete grid and also does not cause curse of dimensionality.

Bi-level optimization of gas-electric hybrid propulsion system

The battery capacity will influence the performance of the basic DP algorithm and the larger battery capacity means the finer mesh for the algorithm as the SOC changes slowly.³² What’s more, the battery capacity will be crucial both in the EMS and system components matching, which will have an effect on the capital expenditures, and the operating expense. But the solutions of the improved DP algorithm proposed in this article are insensitive to the number of grid nodes. That is, the size of battery is only related to the cost (Capital Expenditures) not the accuracy of the optimal solutions for energy management. Therefore, the interaction in the bi-level optimization between the EMS and the components matching, particularly the size of the battery, is decoupled.

Before optimization, the estimation of the range of battery capacity is necessary. What we can see from Figure 14 is that the battery capacity increases while the gas fuel consumption does not always reduce. The redundant battery capacity causes the high terminal SOC and the expensive capital expenditures. And the best battery capacity is about 250 kWh that is the point of intersection for three objectives.

Figure 14.

Results of capital expenditures and operating expense.

In order to find the accurate optimal solution for battery capacity, the bi-level optimization between the energy management and the battery capacity is studied. And the scheme of bi-level optimization is shown in Figure 15. the lower level includes the operating expense and the EMS is the minimum natural gas consumption. Meanwhile, the GA algorithm is used in upper level whose objective function is described as equation (58). And the constraints for the upper level refer to the estimation range of battery capacity, just like equation (59). Noted that the normalization was taken to ensure the weight for the different objectives equal which can see the equation (60).

\begin{matrix} J_{GA} (Q_{b}) = min {m_{f, norm} (Q_{b}, T_{m}, SO C_{0}) + c_{f, norm} (Q_{b}, T_{m}, SO C_{0}) + SO C_{final, norm} (Q_{b}, T_{m}, SO C_{0})} \end{matrix}

(58)

100 \leq Q_{b} \leq 600

(59)

\begin{array}{l} m_{f, n o r m} = \frac{m_{f, m a x} - m_{f}}{m_{f} - m_{f, m i n}}, c_{f, n o r m} = \frac{c_{f, m a x} - c_{f}}{c_{f} - c_{f, m i n}}, \\ S O C_{f, n o r m} = \frac{S O C l_{f i n a l, m a x} - S O C_{f i n a l}}{S O C_{f i n a l} - S O C_{f i n a l, m i n}} \end{array}

(60)

Figure 15.

The diagram of bi-level optimization.

Noted that the objective function doesn’t include the charge sustaining strategy and just involves the gas consumption in the whole navigation, fixed cost, and the terminal SOC. The Influencing variables not only involve the battery capacity in the upper level but also the PMSM torque and battery SOC in the lower level.

Without the couple in the upper and lower levels, the bi-level optimization algorithm converges more rapidly in 11 iterations, as Figure 15. On the one hand, the lower level based on the improved DP algorithm can give the adequately accurate solutions that are not affected by the number of grid nodes. On the other hand, the upper level with GA has the advantage of the fast convergence speed and escaping the local optimal solutions trap. The results obtained by the optimization are consistent with Figure 16.

Figure 16.

The objective and decision values with 20 iterations.

Although the battery capacity is larger than the initial, the optimal solutions make full use of the battery SOC and minimize the gas fuel consumption. It also finds a balance between the capital expenditures and operating expense. The conclusions are displayed in Table 5.

Table 5.

The results of bi-level optimization.

Term	Value	Unit
Initial SOC	0.8	–
Terminal SOC	0.3019	–
Objective value	0.5278	–
Optimal battery capacity	262.67	kWh
Gas fuel consumption	8.009	kg
Capital expenditures	472810	RMB

Conclusions

In an effort to find the global optimal solutions of EMSs for the gas-electric hybrid propulsion system, an improved DP algorithm was proposed in this paper which was taken in the EMSs. What’s more, because the more accurate global solutions of energy management can be given by the improved DP algorithm, the bi-level optimization schedule was put forward with the GA algorithm in upper level and the improved DP algorithm in lower level. The detailed conclusions drawn in this study are listed below.

The improved DP algorithm can give the more accurate solutions than the basic DP in the question of energy management for hybrid propulsion system which reduces the 3.74% gas fuel consumption. In addition, by comparing the calculation results of different levels of discretization, it was shown that the improved DP algorithm was not sensitive to the number of grid nodes and then the calculation complexity was reduced by 100 times.

For the performance of the improved DP algorithm, it was found that the weaknesses, such as the weak effect of penalty functions and curse of dimensionality, were avoided. Compared to the basic DP algorithm, it can save 0.316% gas fuel consumption by the charge-sustaining strategy while the consumes battery energy more. And the effect of punishment was researched by boxplot. The results showed that the first, medium and third quantile are closer to the limitation of SOC, which means the strong effect of the penalty function.

With the bi-level optimization of the improved DP algorithm in lower level and GA in upper level, the interaction between the energy management strategies and components matching was decoupled. The best battery capacity was 262.67 kWh for this gas- electric hybrid propulsion system while the gas consumption was about 8.009 kg. Although, the fixed cost was high, the capital expenditures and operating expense were traded off.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the High tech Ship Scientific Research Project of China, development of hybrid power system for ocean going ships (CB03N20) and the National Major Science and Technology Projects of China (2017-V-0005-0055).

ORCID iD

Chao Xu

References

Papanikolaou

Zaraphonitis

Bitner-Gregersen

, et al. Energy efficient safe ship operation (SHOPERA). In: SNAME 5th World maritime technology conference, 2015.

Geertsma

Negenborn

Visser

, et al. Design and control of hybrid power and propulsion systems for smart ships: a review of developments. Appl Energy 2017; 194: 30–54.

Jelić

Radica

Račić

, et al. Developments in marine hybrid propulsion. In: 2021 6th International conference on smart and sustainable technologies (SpliTech)2021, pp. 1–6. New York: IEEE.

Sui

de Vos

Stapersma

, et al. Fuel consumption and emissions of ocean-going cargo ship with hybrid propulsion and different fuels over voyage. J Mar Sci Eng 2020; 8: 588.

Castles

Reed

Bendre

, et al. Economic benefits of hybrid drive propulsion for naval ships. In: 2009 IEEE electric ship technologies symposium, 2009, pp. 515–520. New York: IEEE.

Barelli

Bidini

Gallorini

, et al. Dynamic modeling of a hybrid propulsion system for tourist boat. Energies 2018; 11: 2592.

Kunicka

Litwin

. Energy efficient small inland passenger shuttle ferry with hybrid propulsion-concept design, calculations and model tests. Pol Marit Res 2019; 26: 85–92.

Völker

. Hybrid propulsion concepts on ships. In: Papers of 33rd international scientific conference “Science in Practice,”2015, pp. 11–16. Hochschule für Angewandte Wissenschaften Würzburg-Schweinfurt University.

Zhu

Chen

Wang

, et al. Optimal design of a hybrid electric propulsive system for an anchor handling tug supply vessel. Appl Energy 2018; 226: 423–436.

10.

Yuan

Wang

Yan

Shen

Long

. A review of multi-energy hybrid power system for ships. Renew Sustain Energ Rev 2020; 132: 110081.

11.

Huang

Lan

Hong

Y-Y

Wen

Fang

. Joint voyage scheduling and economic dispatch for all-electric ships with virtual energy storage systems. Energy 2020; 190: 116268.

12.

Chen

Duan

Cai

Liu

. Smart energy management system for optimal microgrid economic operation. IET Renew Power Gener 2011; 5: 258–267.

13.

Derollepot

Vinot

. Sizing of a combined series-parallel hybrid architecture for river ship application using genetic algorithm and optimal energy management. Math Comput Simul 2019; 158: 248–263.

14.

Xiang

Yang

. An ECMS for multi-objective energy management strategy of parallel diesel electric hybrid ship based on ant colony optimization algorithm. Energies 2021; 14: 810.

15.

Paganelli

Guerra

Delprat

, et al. Simulation and assessment of power control strategies for a parallel hybrid car. Proc IMechE, Part D: J Automobile Engineering 2000; 214: 705–717.

16.

Shang

Srinivasan

Reindl

. Economic and environmental generation and voyage scheduling of all-electric ships. IEEE Trans Power Syst 2015; 31: 4087–4096.

17.

Khan

MMS

Faruque

Newaz

. Fuzzy logic based energy storage management system for MVDC power system of all electric ship. IEEE Trans Energy Convers 2017; 32: 798–809.

18.

Hou

Sun

Hofmann

. Adaptive model predictive control with propulsion load estimation and prediction for all-electric ship energy management. Energy 2018; 150: 877–889.

19.

Nguyen

German

Trovão

JPF

, et al. Real-time energy management of battery/supercapacitor electric vehicles based on an adaptation of Pontryagin’s minimum principle. IEEE Trans Vehicular Technol 2018; 68: 203–212.

20.

Papalambrou

Samokhin

Topaloglou

, et al. Model predictive control for hybrid diesel-electric marine propulsion. IFAC-PapersOnLine 2017; 50: 11064–11069.

21.

Bassam

Phillips

Turnock

Wilson

. Development of a multi-scheme energy management strategy for a hybrid fuel cell driven passenger ship. Int J Hydrogen Energy 2017; 42: 623–635.

22.

Kalikatzarakis

Geertsma

Boonen

, et al. Ship energy management for hybrid propulsion and power supply with shore charging. Control Eng Pract 2018; 76: 133–154.

23.

Xiang

Yang

. A DA-based ECMS for energy optimisation of parallel diesel electric hybrid ship. Ships Offshore Struct 2022; 17: 889–904.

24.

Yuan

Teng

Fengchun

Peng

. Comparative study of dynamic programming and Pontryagin’s minimum principle on energy management for a parallel hybrid electric vehicle. Energies 2013; 6: 2305–2318.

25.

Chen

, et al. Energy management for a power-split plug-in hybrid electric vehicle based on dynamic programming and neural networks. IEEE Trans Vehicular Technol 2013; 63: 1567–1580.

26.

Vagg

Akehurst

Brace

, et al. Stochastic dynamic programming in the real-world control of hybrid electric vehicles. IEEE Trans Control Syst Technol 2015; 24: 853–866.

27.

Bianchi

Rolando

Serrao

, et al. A rule-based strategy for a series/parallel hybrid electric vehicle: an approach based on dynamic programming. In: Dynamic systems and control conference, 2010, pp. 507–514. ASME.

28.

Boyali

Güvenç

. Real-time controller design for a parallel hybrid electric vehicle using neuro-dynamic programming method. In: 2010 IEEE international conference on systems, man and cybernetics, 2010, pp. 4318–4324. New York: IEEE.

29.

Liu

Wang

, et al. Hybrid electric vehicle downshifting strategy based on stochastic dynamic programming during regenerative braking process. IEEE Trans Vehicular Technol 2018; 67: 4716–4727.

30.

Xiao

Liu

Shen

, et al. Comparisons of energy management methods for a parallel plug-in hybrid electric vehicle between the convex optimization and dynamic programming. Appl Sci 2018; 8: 218.

31.

Lee

Song

Kim

, et al. Comparative analysis of energy management strategies for HEV: dynamic programming and reinforcement learning. IEEE Access 2020; 8: 67112–67123.

32.

Patil

Filipi

Fathy

. Comparison of supervisory control strategies for series plug-in hybrid electric vehicle powertrains through dynamic programming. IEEE Trans Control Syst Technol 2013; 22: 502–509.

33.

Zhao

. Advanced direct injection combustion engine technologies and development: diesel engines. Cambridge: Elsevier, 2009.

34.

Alanen

Saukko

Lehtoranta

, et al. The formation and physical properties of the particle emissions from a natural gas engine. Fuel 2015; 162: 155–161.

35.

Sciarretta

Serrao

Dewangan

, et al. A control benchmark on the energy management of a plug-in hybrid electric vehicle. Control Eng Pract 2014; 29: 287–298.

36.

Jianyun

Lijuan

Bin

. Bi-objective optimal design of plug-in hybrid electric propulsion system for ships. Energy 2019; 177: 247–261.

37.

Liu

Chen

Zhao

, et al. Research on the performances and parameters of interior PMSM used for electric vehicles. IRE Trans Ind Electron 2016; 63: 3533–3545.

38.

Sundström

Ambühl

Guzzella

. On implementation of dynamic programming for optimal control problems with final state constraints. Oil Gas Sci Technol Rev Med Inst Fr du Pét 2010; 65: 91–102.

39.

Elbert

Ebbesen

Guzzella

. Implementation of dynamic programming for $ n $-dimensional optimal control problems with final state constraints. IEEE Trans Control Syst Technol 2012; 21: 924–931.

40.

Scipioni

Jørgensen

Graves

, et al. A physically-based equivalent circuit model for the impedance of a LiFePO4/graphite 26650 cylindrical cell. J Electrochem Soc 2017; 164: A2017.

41.

Gao

Yang

. LiFePO4 battery state of charge estimation based on the improved Thevenin equivalent circuit model and Kalman filtering. J Renew Sustain Energy 2016; 8: 024103.

42.

Yang

Wang

Yang

, et al. A comprehensive investigation of lithium-ion battery degradation performance at different discharge rates. J Power Sources 2019; 443: 227108.