1 Introduction

Trajectory optimization problems in terms of space vehicles [2, 4, 6, 9] have attracted significant attentions. One of the current objectives is the development of Space Manoeuvre Vehicles (SMV) for a dynamic mission profile. The Mach number and the flight altitude of the space vehicle vary largely during the whole flight phase, the aerodynamic feature of the vehicle has large uncertainties and nonlinearities. Due to these reasons, it is difficult to calculate analytical solutions of this type of problems. Therefore, numerical methods are commonly used to approximate the optimal solution. Numerical methods for solving optimal control problems are divided into two major classes: indirect methods and direct methods [1, 7, 10]. However, it is difficult to solve the trajectory design problem using indirect methods based on maximum principle. Hence, direct optimization method has been widely used for trajectory optimization.

All the direct methods aim to transcribe the continuous-time optimal control problems to a Nonlinear Programming problem (NLP). The resulting NLP can be solved numerically by well-developed algorithms such as gradient-based methods or derivative free algorithms. Sequential Quadratic Programming (SQP) and Interior point (IP) methods are used successfully for the solution of large scale nonlinear programming problems. The search direction in the SQP method is determined by solving the Quadratic Program (QP) problem whereas IP transcribes the inequality constraints to equality constraints by introducing slack variables. In recent years, derivative free methods has become more popular in the application of optimal control problems. However, the actual advantage of using a global method such as Genetic Algorithm (GA) and Differential Evolution (DE) is difficult to appreciate, in particular when stochastic procedures are applied. In this paper, a number of global search methods are tested for solving the SMV trajectory optimization problem.

Hereafter, the paper is organised as follows. Section 2 introduces the equations of motion, constraints of the SMV and the method used to discretize the optimal control problem. In Sect. 3, the general procedures of typical gradient-based methods and derivative free methods are detailed. Following that, Sect. 4 presents comparative results between the solution calculated using different strategies.

2 Problem Formulation

The mission scenario investigated in this paper focuses on the atmospheric skip entry, targeting the entry into the atmosphere down to a predetermined position and the required controls involved in returning back to low earth orbit. The overall mission can be found in Fig. 1. General skip reentry can be divided into five phases: initial roll, down control, up control, Kepler and final entry. Considering the mission of the SMV is to overfly the ground target with specific altitude, the most challenging phase 2 and 3 will be considered in this paper.

Fig. 1
figure 1

Mission profile

Full size image

2.1 Equations of Motion

To formulate a Space Manoeuvre Vehicle’s skip entry flight, the equations of three-dimensional motion for a point mass about a static Earth are integrated. The equations of motion can be summarised as:

$$\begin{aligned} \begin{array}{c} \dot{r}=V\sin {\gamma }\\ \dot{\theta }=\frac{V\cos {\gamma }\sin {\psi }}{r\cos {\phi }}\\ \dot{\phi }=\frac{V\cos {\gamma }\cos {\psi }}{r}\\ \dot{V}=\frac{T\cos \alpha -D}{m}-g\sin {\gamma }\\ \dot{\gamma }=\frac{L\cos {\sigma }+T\sin \alpha }{mV}+(\frac{V^{2}-gr}{rV})\cos {\gamma }\\ \dot{\psi }=\frac{L\sin {\sigma }}{mV\cos {\gamma }}+\frac{V}{r}\cos {\gamma }\sin {\psi }\tan {\phi }\\ \dot{m} = -\frac{T}{I_{sp}g} \end{array} \end{aligned}$$
(1)

where r is the radial distance from the Earth center to the vehicle, \(\theta \) and \(\phi \) are the longitude and latitude, respectively. V is the Earth-relative velocity. The relative flight-path angle can be denoted as \(\gamma \). \(\psi \) is the relative velocity heading angle measured clockwise from the north. m is the mass of vehicle and t is time. Angle of attack and bank angle are \(\alpha \) and \(\sigma \), separately. The thrust is defined as T. The states and controls are described as \(X = [r,\theta ,\phi ,V,\gamma ,\psi ,m]^{T}\) and \(U = [\alpha ,\sigma ,T]^{T}\). In the model given by Eq. (1), three autopilot equations are introduced using the technique of first order lag to describe the rate constraint of the controls.

$$\begin{aligned} \begin{array}{c} \dot{\alpha } = K_{\alpha }(\alpha _{c}-\alpha ) \\ \dot{\sigma } = K_{\sigma }(\sigma _{c}-\sigma ) \\ \dot{T} = K_{T}(T_{c}-T) \end{array} \end{aligned}$$
(2)

in which \(\alpha _{c}\), \(\sigma _{c}\) and \(T_{c}\) are the demand angle of attack, bank angle and thrust, respectively. The atmosphere model, lift L and drag D can be defined as:

$$\begin{aligned} \begin{array}{ll} g=\frac{\mu }{r^{2}} &{} \rho =\rho _{0}\exp {\frac{r-r_{0}}{h_{s}}} \\ L=\frac{1}{2}\rho V^{2}C_{L}S &{} D=\frac{1}{2}\rho V^{2}C_{D}S \\ C_{D}=C_{D0}+C_{D1}\alpha +C_{D2}\alpha ^{2} &{} C_{L}=C_{L0}+C_{L1}\alpha \end{array} \end{aligned}$$
(3)

where \(S=2690\) ft\(^{2}\) is the reference area, \(\rho \) is the density of the atmosphere and \(\rho _{0}=0.002378\) slug/ft\(^{3}\) is the density of the atmosphere at sea-level. \(\mu =1.4076539\times 10^{16}\mathrm{ft}^{3}/\mathrm{s}^{2}\) is gravitational parameter of the earth. \(r_{0}=20902900\) ft is earth radius, L and D are the lift and drag whereas \(C_{L}\) and \(C_{D}\) are lift and drag coefficient determined by angle of attack \(\alpha \) and Ma, respectively.

2.2 Boundary and Path Constraints

In this paper, two types of constraint are considered in the skip entry process. To complete the mission, the boundary conditions for the states and controls are:

$$\begin{aligned} \begin{array}{l} [r, \phi , \theta , V, \gamma , \psi , m, \alpha , \sigma , T] \\ =[r_{0},\phi _{0},\theta _{0},V_{0},\gamma _{0},\psi _{0},m_{0},\alpha _{0},\sigma _{0},T_{0}]\\ .[r(t_{b}), r(t_{f})]=[r_{b},r_{f}] \end{array} \end{aligned}$$
(4)

where \(t_{b},t_{f}\) are time points for the SMV reaching the bottom point and going back to the final boundary conditions. More precisely, as shown in Fig. 1, \(t_{b}\) and \(t_{f}\) are the time points of the end of phase 2 and phase 3, respectively. Correspondingly, \(r_{b}\) and \(r_{f}\) are the altitude values at \(t_{b}\) and \(t_{f}\), respectively. Based on the boundary conditions illustrated by Eq. (4), the whole process can be divided into two phases, the descent phase and exit phase.

To protect the structure integrity for the SMV, three path constraints including heating rate, dynamic pressure and load factor must be satisfied during the entire flight phase:

$$\begin{aligned} \begin{array}{c} \dot{Q_{d}}=K_{Q}\rho ^{0.5}V^{3.07}(c_{0}+c_{1}\alpha +c_{2}\alpha ^{2}+c_{3}\alpha ^{3})<\dot{Q_{dmax}} \\ P_{d}=\frac{1}{2}\rho V^{2}<P_{dmax} \\ n_{L}=\frac{\sqrt{L^{2}+D^{2}}}{mg}<n_{Lmax} \end{array} \end{aligned}$$
(5)

where \(c_{0}=1.067 ,c_{1}=-1.101,c_{2}=0.6988,c_{3}=-0.1903\) and \(K_{Q}=9.289\times 10^{-9}BTU\cdot s^{2.07}/ft^{3.57}/slug^{0.5}\), respectively. Mission-dependent parameters are \(Q_{dmax}=200BTU/ft^{2}/s\), \(P_{dmax}=13406.4583Pa\) and \(n_{Lmax}=2.5\) representing the allowable maximum heating rate, dynamic pressure and acceleration, respectively.

2.3 Cost Function

According to the mission requirement given by the industrial sponsor of this project, to complete the mission in the shortest time, minimizing the mission duration (e.g. \(t_{f}\)) is chosen as the objective function. Let J represent objective function:

$$\begin{aligned} J=\min t_{f} \end{aligned}$$
(6)

By setting the cost function given by Eq. (6), the SMV trajectory problem can be considered as an optimal control problem which has minimum cost function value and satisfies the initial and terminal variables constraints, three path constraints and dynamic equations.

2.4 Discrete Method

The SMV optimal control problem is discretized using a direct multiple shooting method. The basic idea of the direct multiple shooting method is to transform the original optimal control problem into NLP by parameterizing only the control variables. The controls can be approximated by interpolation at the discretized time nodes \([\tau _{1},\tau _{2},\ldots ,\tau _{N}]\). Then the equation of motion are integrated with a fourth order Runge–Kutta method. The approximation of controls are:

$$\begin{aligned} u(\tau )\approx U(\tau ) = \sum _{i=0}^{N}U_{i}L_{i}(\tau ) \end{aligned}$$
(7)

where \(L_{i}(\tau )\) is a basis of Lagrange polynomials. After using direct multiple shooting method, the resulting NLP problem is solved by applying different optimization strategies illustrated in Sect. 3.

3 Optimization Strategies

A key ingredient to solve optimal control problems is the ability of solving NLP problems. Numerical methods for solving NLP fall into categories: gradient-based methods and derivative free algorithms.

3.1 Gradient Based Methods

3.1.1 Sequential Quadratic Programming

The most commonly used gradient-based methods are SQP and IP or barrier methods. The aim for SQP algorithm is to transform the original problem to a series of QP subproblems by approximating the augmented Lagrangian quadratically and linearizing the constraints using Taylor expansion. The resulting augmented Lagrangian is:

$$\begin{aligned} L(x,\lambda ,u) = f(x)+\lambda ^{T}h(x)+u^{T}g(x) \end{aligned}$$
(8)

Using quadratic model to approximate Eq. (8), the QP subproblem is:

$$\begin{aligned} \begin{array}{c} \min {\frac{1}{2}d^{T}H(x_{k},\lambda _{k},u_{k})d+ \nabla f(x_{k})^{T}d} \\ h(x_{k})+\nabla h(x_{k})d = 0 \\ g(x_{k})+\nabla g(x_{k})d \le 0 \\ d \in \mathfrak {R}^{n} \end{array} \end{aligned}$$
(9)

where a \((x_{k},\lambda _{k},u_{k})\) presents the current iterate point whereas \(H(x_{k},\lambda _{k},u_{k})\) is the Hessian matrix. Commonly, the Hessian is calculated using \(H(x_{k},\lambda _{k},u_{k})= \nabla _{xx}L(x_{k},\lambda _{k},u_{k})\) or a suitable approximation defined by the user. \(\nabla h(x_{k})\) and \(\nabla g(x_{k})\) are the Jacobian matrix of the vector of equality constraints and inequality constraints, respectively. The index k stands for the number of iteration for the optimization algorithm and \(k=0,1,2,\ldots \).

If the active set is defined as \(\Lambda \), a sequence of linear equations are constructed as Karush-Kuhm-Tucker (KKT) system. Then by using Newton method, the KKT condition of Eq. (9) can then be calculated.

3.1.2 Interior Point Method

Another well-known and effective algorithm is the IP method. Numerous updates and modifications have been done on this approach during the last several decades. The ability for IP converging to a stationary point can be guaranteed theoretically. Before applying the IP to the general form of problems, Eq. (8) should be transformed by introducing \(s\in \mathfrak {R}^{m}, \tau >0\) and:

$$\begin{aligned} f_{\tau }(x)= f(x)+\tau \varSigma _{i=1}^{l} h_{i}(x) \end{aligned}$$
(10)

The IP strategy consists of reducing the inequality constraints in Eq. (8) using slack variables \(s=(s_{1},s_{2},\ldots ,s_{m})\), where all the elements in the vector should be positive. Then, the modified problem can be summarised as:

$$\begin{aligned} \begin{array}{c} \min {f_{\tau }(x)-\mu \varSigma _{j=1}^{m} log(s_{j})} \\ g(x)+s = 0 \\ x\in \mathfrak {R}^{n}, s\in \mathfrak {R}^{m} \end{array} \end{aligned}$$
(11)

And the augmented Lagrangian for Eq. (11) is:

$$\begin{aligned} L(x,s,\lambda ,u) = f_{\lambda }(x)-\mu \varSigma _{j=1}^{m} log(s_{j})+u^{T}(g(x)+s) \end{aligned}$$
(12)

In Eq. (11), the term \(\mu \) stands for a barrier variable and the smaller it is, the closer are the solutions. Both SQP and IP are using Newton iteration to get KKT system and the converge solution.

3.2 Derivative Free Algorithms

In this paper, four derivative free global search algorithms are investigated: Genetic Algorithm (GA) and Differential Evolution(DE) that belong to the generic class of Evolutionary Algorithms (EA), Particle Swarm Optimization (PSO) that belongs to the class of agent-based algorithms, and Artificial Bee Colony (ABC) that is classified to the colony-based algorithms.

3.2.1 Genetic Algorithm

GA [3] is one kind of evolution algorithms, which generates solutions to optimization problems taking inspiration from the natural selection and survival of the fittest in the biological world. Each iteration of a GA involves a competitive selection that eliminates poor solutions. It is regarded as one of the most robust and reliable optimization algorithms which has no requirement for gradient information and initial guess. Recombination and mutation are applied to generate new solutions so that the population can have more diversity. As for the control parameters of GA, the population size is given as 500 individuals. Single values are used for the crossover and mutation probability, \(CR=0.7\) and \(p_{i}=0.5\), respectively.

3.2.2 Differential Evolution

The main attempt of DE [9] is to calculate the variation vector \(v_{i,G+1}\) of a solution vector \(x_{i,G+1}\) by taking the weighted difference between two additional solutions. This can be described as:

$$\begin{aligned} v_{i,G+1} = e[(x_{i3,G}-x_{i,G})+F(x_{i2,G}-x_{i1,G})] \end{aligned}$$
(13)

where \(F=0.7\), i is integer number randomly chosen within the interval [1, NP]. The equation of e can be written as:

$$\begin{aligned} e(x) = \left\{ \begin{array}{ll} 1, &{} rand\le CR; \\ 0. &{} rand>CR. \end{array} \right. \end{aligned}$$
(14)

where rand is a random number within [0, 1]. The selection process is largely depended on the fitness function defined by the user. If the new candidate can have a better fitness value then it can be selected to the next generation.

3.2.3 Particle Swarm Optimization

PSO [8] is a population-based derivative free optimization algorithm developed in 1995. PSO was inspired by the social behaviour of bird flocking or fish schooling. The general concept of PSO consists of changing the velocity of every candidate at each iteration. The new individual in the next generation can be calculated by:

$$\begin{aligned} v_{i,G+1} = \omega v_{i,G}+u_{i,G} \end{aligned}$$
(15)

where \(\omega \) is a weighting function which is proportional to the number of iterations G. The process has two stochastic components given by the two random number \(r_{1}\) and \(r_{2}\). The corresponding terms are elastic component controlled by \(c_{1}=2\) and convergence term controlled by \(c_{2}=2\). The first term tends to recall the individual back to the old position whereas the second term drives the entire population toward convergence. The search is applied until a stopping condition is satisfied.

3.2.4 Artificial Bee Colony

The artificial bee colony (ABC) algorithm was originally presented by Dervis Karaboga in 2007 [5] and was inspired by the collective behavior of honey bees. The ABC algorithm has been tested to exhibit a good performance in the function optimization problems. For each iteration of the ABC algorithm, both global search and local search are conducted so that the probability of achieving the optimum is increased dramatically. In each iteration of the algorithm, the searching principle is defined as:

$$\begin{aligned} v_{ij} = x_{ij}+\varphi _{ij}(x_{ij}-x_{kj}) \end{aligned}$$
(16)

where v denotes the new position. Such a searching strategy adaptively reduces the searching step when the candidate approaches the optimal solution. The control parameters for ABC algorithm are \(NP=200\), \(Iter=100\) and \(Limit=10\), respectively.

4 Simulation Results

4.1 Parameters Setting

The initial, terminal boundary conditions, box constraints and aerodynamic coefficients of the skip process can be found in [2]. Comparative simulations using classical gradient optimization techniques and derivative free optimization methods are presented. It should be noticed that only the skip entry phase shown in Fig. 1 is taken into account in the paper. The initial altitude is around 80 km, where is the assumed edge of the atmosphere.

4.2 Time History of the State and Control for Different Methods

The results of optimal trajectories are shown in Figs. 2, 3, 4, 5, 6, 7, 8 and 9.

Fig. 2
figure 2

Altitude generated by using gradient methods and derivative free methods

Full size image
Fig. 3
figure 3

Speed generated by using gradient methods and derivative free methods

Full size image
Fig. 4
figure 4

Angle of attack generated by using gradient methods and derivative free methods

Full size image
Fig. 5
figure 5

Bank angle generated by using gradient methods and derivative free methods

Full size image
Fig. 6
figure 6

Heating generated by using gradient methods and derivative free methods

Full size image
Fig. 7
figure 7

Dynamic pressure generated by using gradient methods and derivative free methods

Full size image
Fig. 8
figure 8

Load factor generated by using gradient methods and derivative free methods

Full size image
Fig. 9
figure 9

Flight path angle generated by using gradient methods and derivative free methods

Full size image

4.3 Analysis of the Solutions

From Figs. 2, 3, 4, 5, 6, 7, 8 and 9, the general trend of trajectories is split into two phases: descending and climbing.

(1) Descending phase: In order to achieve the target position (around 164000 ft altitude) and minimize the mission time (since the overall objective is to minimize the time duration, it is equivalent to minimize the time duration for each phase), Fig. 2 shows that the SMV goes down directly at the beginning of the mission. To avoid path constraints becoming active, angle of attack should increase to slow down the vehicle so that the heating and dynamic pressure do not increase significantly.

(2) Climbing phase: After reaching the target point, the vehicle fires its engine so that the SMV can have enough kinetic energy to go back to the final point. With the decreasing of air density and mass, the aerodynamic heating, dynamic pressure and load factor will decrease during the climbing phase. The trend of angle of attack can be found in Fig. 4 where the angle of attack is increased during the whole climbing phase. This is because in the climbing phase, without violating path constraints, it can have positive influences in terms of acceleration.

With regard to the performance of different methods, all the global approaches manage to generate skip entry trajectories between the predetermined initial position and terminal position without violating the path constraints. This can be seen from Figs. 6, 7 and 8. When the nonlinearity of the cost functions or path constraints become higher, which means it is difficult to calculate the gradient information for gradient techniques, the global methods become the only way to solve the SMV trajectory optimization problem. However, there are some limits for global strategies. As can be seen from Figs. 2, 3, 4, 5, 6, 7, 8 and 9, the results are significantly different between PSO-based method and others. This can be explained that in this case, the initial guess generated by the PSO-based method is not close enough to the optimal solution. Moreover, to combine the optimization processes with discrete methods, global techniques cannot be as flexible as gradient methods. Also, it is hard to verify the optimality for the solutions from global techniques whereas the SQP and other gradient methods have KKT conditions.

Consequently, all the figures provided above confirm the feasibility of the gradient and derivative free algorithms. By using different optimization strategies, the SMV can reach the target position without violating three path constraints and boundary conditions.

5 Conclusions

In this paper, the gradient-based and derivative free algorithms are applied to solve SMV trajectory design problem. In order to transform the continuous optimal control problem to static NLP problem, direct multiple shooting method is implemented to discrete the equations of motion and path constraints. Simulation results indicated that the proposed two kind of strategies can generate feasible solution for the trajectory design problem. By applying the gradient-based method, the number of iterations, function evaluations, and computational time can be decreased compared with derivative free methods. Therefore, although the solutions generated from derivative free methods can be accepted, there are still a lot of room for improvement in terms of using these techniques in trajectory optimization problems.