Keywords

1 Introduction

Seakeeping performance refers to the ability of a ship to successfully operate at various sea states [1]. The seakeeping performance of a vessel is a critical topic of consideration when evaluating ship operational behaviors. From a design perspective, it is essential to not only analyze seakeeping responses, but to also investigate the relationships between seakeeping responses and other operational aspects. Therefore, establishing a framework that analyzes such interdependencies within the ship system significantly benefits the designers during preliminary design stages [2].

A critical component of seakeeping analysis is ship motion prediction. Historically, a considerable number of seakeeping studies have been focused on this subject. Sayli et al. [3] took advantage of multiple regression models in order to analyze the relationships between seakeeping responses and hull parameters of Mediterranean fishing vessels. This is an example of analyzing statistical methods for a single vessel type. In terms of the theoretical improvement of seakeeping predictions, research has been performed in both the time and frequency domains. A JAVA program has been developed by Augustine et al. [4] to estimate coupled heave and pitch motions in the time domain. Seakeeping Prediction Program (SPP) [5] implements a version of SCORES Program [6] for the approximation of heave, pitch and roll motions in the frequency domain. The advantage of these programs is the reliable response estimations for different circumstances that they provide for designers. However, one disadvantage of these estimations is the poor representation of the interconnections between vessel motions and temporal operational behaviors explicitly, especially when taking into consideration fickle wave conditions worldwide. As a result, an operational evaluation framework which can address such disadvantages is necessary in the continued analysis.

Ship-Centric Markov Decision Process (SC-MDP) framework provides the opportunity to analyze operational evaluation in the preliminary design stage in a novel way. The information generated by the SC-MDP framework serves as a unique reference to identify lifecycle decision path dependencies and relate them back to early stage of ship design, which provides the design data needed to avoid design lock-in [7]. SC-MDP was originally developed by Niese and Singer [8] to study the lifecycle decision process for the ballast water treatment of a ship when taking into consideration potential environmental policy changes. Niese and Singer [9] advanced the process by defining a set of metrics to evaluate changeability of a design based on their previous SC-MDP results. Kana et al. [10] applied eigenvalue spectral analysis to the original SC-MDP for lifecycle compliance of ballast water treatment. The work in these papers demonstrates the applicability and flexibility of SC-MDP in evaluating ship operational events. Current research developments of the SC-MDP framework concentrate on the various impacts of environmental policies, economic benefits, and safety factors [11] during the ship design process. However, the influence of seakeeping performance on vessel design has not been well-developed yet. Ship motions will impact and possibly reduce the reliability of operational behaviors and cause uncertain results, therefore it is critical to incorporate this impact into the SC-MDP framework.

The purpose of this paper is to present the formulation of a modified SC-MDP framework which takes into consideration the impact of seakeeping performance on vessel transit. A commercial ship design for passage over the North Pacific Ocean is discussed as a case study to illustrate this formulation. The main efforts of this paper have been divided into two components. First, an appropriate seakeeping calculation method is required to conduct ship motion predictions. These calculation results will determine the values of transition probability and reward in the Markov Decision Process (MDP) structure according to some transformation rules defined by designers. Second, the MDP framework is defined as a surrogate model to emulate vessel sailing. The inherent relationship between physical responses and operating decisions is embedded within the MDP structure. The following sections outline the seakeeping and MDP methodologies, the results of the commercial ship design case study, and future work to be performed.

2 Method

2.1 Background

Due to the constant increase in international trade, a more advanced design method of commercial ships is required to maintain relevancy in such a competitive industry. One of the most basic missions of a commercial ship is to successfully reach its destination. Without taking into consideration the physical responses of the sea state, it is a desirable decision to take the shortest trade route to complete the vessel’s mission. Unfortunately, these responses are inevitable as long as the ship operates in water. As a consequence, alternate routes that deviate from the shortest trade route must be considered due to the probable existence of adverse ship motions.

The ultimate value of this SC-MDP framework is to differentiate multiple ship designs through the evaluation of the trade-offs between seakeeping responses and operational performance with limited information provided in the preliminary design stage. Currently, the work in this paper accomplishes the formulation of offering rational results for a single ship design. In the future, the differentiation of various ship designs will be realized based on the utilization of those results. The case study of the single ship design is generated according to such background and the following methods are selected for the appropriate evaluation of seakeeping responses and operational performance.

2.2 Seakeeping Prediction Program (SPP)

SPP, a University of Michigan seakeeping software, applies strip theory and long-crested wave assumptions in the frequency domain [5]. This program offers relatively reliable estimations of seakeeping responses given only the main hull parameters, which makes it an effective method during the preliminary stage. It utilizes hull approximation, ship speed, ship heading relative to the waves, and sea spectrum data as inputs and generates the response spectrum of heave, pitch, and roll.

The main parameters required to define a hull include waterline length (LWL), beam (B), and draft (T), in addition to some ship-form coefficients such as prismatic coefficient (CP), maximum section coefficient (CX), and waterplane coefficient (Cwp). These coefficients are necessary for approximating the sectional area curve and waterline curve of the vessel using a mathematical model in the program. Furthermore, SPP applies Lewis Form to model hull sections at each station. These procedures produce a rough form based on the provided hull parameters.

Ship speed, in this case study, is a constant value, meaning that variations in speed throughout the vessel’s operation are not taken into consideration. The variable factor of the utilized vessel is its direction during the transit, resulting in various values of ship headings relative to the waves during temporal steps.

The sea spectrum is used to represent the real irregular waves in which the vessel is operating. For the purpose of calculating responses during trans-ocean operations, the ISSC spectrum has been utilized to characterize the fully developed wave conditions. The ISSC spectrum is a two-parameter spectrum which depends on significant wave height in meters (Hs) and the mean wave period in seconds (T), whose formula is shown in Eq. (1) [12]:

$$ S_{w} \left( \omega \right) = \frac{{173.6H_{s}^{2} T^{ - 4} }}{{\omega^{5} }}\exp \left( { - 694.4T^{ - 4} \omega^{ - 4} } \right) $$
(1)

where ω is the wave frequency in radians per second.

In combination with the selected hull parameters, the sea spectrum \( S_{w} \left( \omega \right) \) can be converted into the encounter frequency spectrum \( S_{w} \left( {\omega_{E} } \right) \) which the ship will actually experience. Furthermore, the energy spectrum \( S_{motion} \left( {\omega_{E} } \right) \) for any given motion can be obtained through Eq. (2):

$$ S_{motion} \left( {\omega_{E} } \right) = \left[ {RAO_{motion} \left( {\omega_{E} } \right)} \right]^{2} S_{w} \left( {\omega_{E} } \right) $$
(2)

where \( \omega_{E} \) is the encounter frequency in radians per second and \( RAO_{motion} \left( {\omega_{E} } \right) \) is the response amplitude operator for the corresponding motion.

From these equations, numerous values can be obtained. These values are the statistical parameters that can be found from the heave, pitch, and roll motion energy spectrum. The values include the root mean square, the average of the 1/3 highest motion response and the average of the 1/10 highest motion response which can be calculated for heave, pitch, and roll spectrum. The average of 1/10 highest values indicate the extreme motion amplitude, which are extracted from SPP for use in MDP framework in order to learn the maximum trade-offs between physical responses and deviations away from the shortest route during the operational mission.

2.3 Markov Decision Process (MDP)

MDP focuses on sequential decision-making problems in regard to stochastic conditions. The outcomes of this framework are influenced by both human control and uncertainties, while including current and future opportunities. A stationary and fully observable MDP is a 4-tuple <S, A, P, R>, where:

  • S, is the set of states that represent the system;

  • A, is the set of actions that can be executed in each state;

  • \( {\text{P}}({\text{s}}^{{\prime }} |{\text{s}},{\text{a}}), \) is the transition probability of achieving state s’ from state s through the execution of action a;

  • R(s, a), is the reward obtained due to taking action a in state s, a discount factor γ may be introduced to count for the preference of current reward over future reward.

The MDP outputs an optimal policy which yields the largest cumulative reward defined as the Bellman equation [13]. In this case study, the optimal policy is solved using a value iteration algorithm [14].

State.

Different locations over the ocean are modelled as states in the MDP framework. The wave system in the North Pacific Ocean is selected as an example of the operating environment. To be more specific, the latitude range is designated from 0 to 60N and the longitude range is designated from 120E to 104W (256E). The overall range is discretized to distributed points with respect to certain latitude and longitude intervals. These points at varying latitudes and longitudes are the states for which the ship is in transit. Figure 1 displays a map including all states when intervals are 5° and 8° along latitude and longitude respectively. For example, locations M and N are two samples of state representations.

Fig. 1.
figure 1

A map representation of states in the case study

Full size image

Each state contains relevant wave information including significant wave height, mean wave period, and mean wave direction. A public dataset from ECMWF [15] has been utilized to obtain these values. In order to analyze the worst-case scenario, a month containing large significant wave heights and long mean wave periods is preferred for use in this case study. Figure 2 summarizes the rough variation of significant wave height.

Fig. 2.
figure 2

A demonstration of significant wave height distribution from ECMWF dataset

Full size image

Figure 3 and Fig. 4 represent the extracted values of mean wave period and mean wave direction which occurred in the selected month.

Fig. 3.
figure 3

A demonstration of mean wave period distribution from ECMWF dataset

Full size image
Fig. 4.
figure 4

A demonstration of mean wave direction distribution from ECMWF dataset

Full size image

Action.

At each state in Fig. 1, a ship is assumed to travel to the surrounding states through 8 different actions. In other words, the ship is allowed to continue the operation in 8 azimuths approximately. Therefore, there are 8 elements in the set of action A, each of which is recorded by an action index in Table 1.

Table 1. Summary of actions
Full size table

Performing different actions in a certain state requires operation through each wave environment at varying relative headings, which will ultimately cause diverse ship motions. Such motions depreciate the original expectation of an action, making it more challenging to execute the corresponding action in addition to making it less desirable to select that action than one in a circumstance without seakeeping.

Transition Probability.

The transition probabilities are assumed to be affected by the roll responses of the vessel. Intuitively, even though the roll responses in all 8 directions may not exceed normative seakeeping criteria, the different degrees of difficulties that they will impose on a vessel in transit must be taken into consideration. The target of a certain action is to successfully operate to the next location in its direction. Though, the roll responses may decrease the probability of success of maintaining the required angle and reaching the target. These probabilities are written as \( {\text{P}}\left( {{\text{target|current}},{\text{a}}} \right) \) where P is the probability of arriving at the expected target in the corresponding direction through the execution of action a at the current state. Otherwise, failure of an action a means that the ship stays in place at the current state, with a probability written as \( {\text{P}}({\text{current}}|{\text{current}},{\text{a}}) \). Therefore, the following transformation rules demonstrated in Table 2 are assumed in this case to distinguish the impact of different ranges of roll motions on determining transition probabilities.

Table 2. Transition probability rules
Full size table

Utilizing states M and N in Fig. 1 as an example, if the roll response at state M to go north equals 3.5°, then \( {\text{P}}\left( {{\text{N}}|{\text{M}},{\text{north}}} \right) \) will be 0.7 and \( {\text{P}}\left( {{\text{M}}|{\text{M}},{\text{north}}} \right) \) will be 0.3 in the transition matrix. In order to achieve the goal of obtaining optimal policy without roll impact using the same MDP structure, the successful probability \( {\text{P}}\left( {{\text{target|current}},{\text{a}}} \right) \) is always assumed to be 0.99 as a dummy value in calculations.

Reward.

For a voyage across the North Pacific Ocean, the strategy of this operation is assumed to follow the great-circle azimuth as close as possible at each step. The great circle of two points on the earth directs the shortest route, which is used as a reference of operational efficiency in the case study when the ship speed remains constant.

Reward is formulated in Eq. (3), which reflects the deviation from that great circle:

$$ {\text{R}}\left( {{\text{s}},{\text{a}}} \right) = - \left| {{\upvarphi }\left( {{\text{s}},{\text{destination}}} \right) - {\upvarphi }\left( {\text{a}} \right)} \right| $$
(3)

where \( \varphi \left( {s,destination} \right) \) is the great-circle azimuth between the current state s and the destination; \( \varphi \left( a \right) \) is the azimuth associated with the action.

Logically, an action that makes the ship closer to great circle will generate smaller absolute deviation and a larger reward in return due to the negative sign. What’s more, traveling at certain deviations means experiencing the related pitch motions in the seaway. It is possible that a large pitch motion will worsen the previous reward calculated by Eq. (3). If the impact of pitch motions on depreciating reward R(s, a) is added, Eq. (3) will be transformed to Eq. (4) with a depreciation amplifier factor f,

$$ {\text{R}}\left( {{\text{s}},{\text{a}}} \right) = - \left| {{\upvarphi }\left( {{\text{s}},{\text{destination}}} \right) - {\upvarphi }\left( {\text{a}} \right)} \right| \times {\text{f}}\left(\uptheta \right) $$
(4)

where \( {\text{f}}\left(\uptheta \right) \) is a function of pitch motion θ, using \( {\text{f}}\left(\uptheta \right) = e^{\theta } \) in this case.

Decision Criteria.

The objective of the SC-MDP framework is to find an optimal policy with the least accumulative deviation from the great circle at all steps. The optimal policy can be solved via the Bellman equation, shown in Eq. (5):

$$ {\text{U}}\left( {\text{s}} \right) = \mathop {\hbox{max} }\limits_{a \in A\left( s \right)} [R\left( {s,a} \right) + \sum\nolimits_{s'} {\gamma P\left( {s^{\prime} |s,a} \right)U\left( {s^{\prime}} \right)} ] $$
(5)

where U(s) is the expected utility of state s and γ is the discount factor where γ = 1.

Equation (6) gives the expression of the optimal policy. Some insights and implications can be gained through further analysis of these results.

$$ \uppi\left( {\text{s}} \right) = arg\mathop {\hbox{max} }\limits_{a \in A\left( s \right)} [R\left( {s,a} \right) + \sum\nolimits_{s'} {\gamma P\left( {s^{\prime} |s,a} \right)U\left( {s^{\prime}} \right)} ] $$
(6)

3 Result

The ship design sample parameters which are imported into the SC-MDP framework are shown in Table 3. It is possible for this ship to take any action at each state where the probable seakeeping responses are summarized in Fig. 5. The horizontal axis in Fig. 5 is the action index, each of which represents one of the actions mentioned in Table 1. And the vertical axis stands for the responses in degrees. Overall, the range of roll motions that can be experienced equate up to 12° approximately, and the maximum value of pitch motions is roughly 3.2°. As mentioned previously, these responses are only static values and their influence can be seen in the SC-MDP.

Table 3. Hull parameters of the design sample
Full size table
Fig. 5.
figure 5

Scatter plot of roll and pitch responses over the North Pacific Ocean

Full size image

There are two scenarios that exemplify the application of the methods, which provide a new way to identify vessel operational performance in the preliminary stage. Two parameters are applied to describe the main characteristics of vessel operational behaviors with respect to physical responses.

On one hand, the introduction of seakeeping responses will change some optimal policies that are desired in a circumstance without seakeeping. The percentage of such changes reflects the major range of seakeeping impact over the ocean. In the case study, this percentage parameter refers to the ratio of the number of states that show different optimal policies between seakeeping and non-seakeeping situations to the total number of states located in the sea area, written as P(policy change). A vessel with a smaller P(policy change) is more capable of handling the predefined wave conditions when emulating transits in this framework, which indicates a potentially preferable ship design for the operation.

On the other hand, the overall deterioration of operational performance is demonstrated in not only the change of optimal policies, but also the increase in accumulative deviation cost during transits. In this framework, the utility value from Eq. (5) represents such accumulative deviations at each state, so it is helpful to compare the two sets of utility values between seakeeping circumstances and non-seakeeping circumstances to learn the differences caused by physical responses. The second parameter, namely E(utility difference), which is defined as the average of all the differences between those two sets of utility values, is calculated as another quantification of the deterioration of operational efficiency. In the early stage evaluation, ship designs with small values of E(utility difference) are anticipated to perform efficiently with seakeeping impact.

The two parameters above, P(policy change) and E(utility difference), serve as unique references to reflect the degree of impact from seakeeping responses. If multiple vessel designs are analyzed in the preliminary stage, vessels that yield smaller results of these parameters will be distinguished from others for better operational performance. As a start point, this paper only tests the simulation of a single ship design to explain the rationality of the results. Moreover, the future utilization and modification of these results to realize differentiation of vessel designs are discussed at the end of this section.

3.1 Scenario 1 (Destination Location: 45N 128W)

In scenario 1, the destination is set as 45N 128W (232E), which can be regarded as a representation of a generic United States western ports. In order to traverse the North Pacific Ocean, the vessel mainly operates in the direction from west to east. Compared with the optimal policy without seakeeping responses (black arrows), the impact of roll and pitch individually is highlighted in bold red arrows in Fig. 6 and Fig. 7.

Fig. 6.
figure 6

The impact of roll response on optimal policy to destination 1

Full size image
Fig. 7.
figure 7

The impact of pitch response on optimal policy to destination 1

Full size image

In order to evaluate the impact of roll responses, values of P(policy change) and E(utility difference) are summarized in Table 4.

Table 4. MDP results with roll impact (destination 1)
Full size table

The black arrows in the Fig. 6 denote the optimal policy without seakeeping consideration. Except for several states that are constrained by land area, the arrows in the open sea area follow the tendency of great-circle strategy. A bold red arrow denotes an optimal policy change different from the non-seakeeping result, which indicates the impact of the related motion. Two main changes are displayed in Fig. 6, which reflect a logical relationship between seakeeping responses and operational transit.

First, some optimal policies between 35N and 45N are converted to east instead of northeast. According to the wave environment shown in Fig. 2 and Fig. 4, action northeast meets beam seas in large significant wave heights resulting in extreme roll angles from 8° to 11°. Such large roll responses are associated with high probabilities of maintaining station-keeping, while action east means experiencing roll motions at around 1 or 2° and accumulating the second smallest deviation from the shortest route. Therefore, this can be regarded as a typical change to avoid large roll motions in bad wave conditions.

Second, starting near latitude 20N and longitude 208E when operating to the destination location, the optimal policy changes from northeast to north. The environment conditions are quite mild along this diagonal line, and most roll responses caused by action northeast and north both approximate to 5°. In other words, these changes are not a result of avoiding large roll responses. In fact, the introduction of uncertainty depreciates the utilities of states along the northeast direction more than those along the north direction. As a result, it is worthy to take an action to get away from the northeast route. In general, these changes derive from roll motions implicitly, which are difficult to identify without the help of such framework.

Similarly, in order to evaluate the impact of pitch responses, values of P(policy change) and E(utility difference) are summarized in the following Table 5.

Table 5. MDP results with pitch impact (destination 1)
Full size table

The changes in Fig. 7 are mainly distributed in the area from 5N to 20N. These new policies manifest relevant operational behaviors of a ship to adapt to pitch motions. In this area, a ship is more likely to encounter head seas with the action northeast, which may cause the consecutive negative effects. Firstly, an action that ends up in experiencing head seas or following seas leads to larger pitch angles than that of the other actions. Secondly, those large angles are exponentially exaggerated based on the assumptive amplifier function f(θ) in reward calculation. Thirdly, another action is chosen to avoid either the unsatisfactory reward at the immediate step or the summation of such rewards at future steps.

Moreover, the impact of both roll and pitch motions can be incorporated at the same time, which is depicted in Table 6 and Fig. 8. It demonstrates similar changes to Fig. 6 and Fig. 7, but it is not a simple addition of red arrows in those figures. It is a new round of interaction based on this framework with 36.02% optimal policy change and 359.78° change in utilities on average.

Table 6. MDP results with roll and pitch impact (destination 1)
Full size table
Fig. 8.
figure 8

The impact of both roll and pitch on optimal policy to destination 1

Full size image

3.2 Scenario 2 (Destination Location: 30N 128E)

The second destination is assigned at 30N 128E, a representative of ports in China, which will motivate the transit across the North Pacific Ocean in the opposite direction. Figure 9 through Fig. 11 represent the corresponding optimal policies of scenario 2 (Fig. 10).

Fig. 9.
figure 9

The impact of roll response on optimal policy to destination 2

Full size image
Fig. 10.
figure 10

The impact of pitch response on optimal policy to destination 2

Full size image
Fig. 11.
figure 11

The impact of both roll and pitch on optimal policy to destination 2

Full size image

It is apparent that there are few changes under the impact of roll responses but significant changes due to the pitch responses in scenario 2. The degrees of change are primarily determined using the initial definitions in transition probabilities and rewards. The transition probability \( {\text{P}}\left( {{\text{target|current}},{\text{a}}} \right) \) decreases gradually with a broad range of roll angles in this case study, making the optimal policy more sensitive to extreme roll responses and less sensitive to small ones. The reward R(s, a) is exponentially deteriorated by pitch responses in this case study, therefore the optimal policies have been modified starting at small pitch angles.

The results in scenario 1 and scenario 2 are accurate reflection of model settings in this specific case study, which are appropriately used as a proof of the formulation. The operational changes under different circumstances are summarized in total in Table 7 and Table 8 for reference.

Table 7. Summary of P(policy change) due to roll, pitch, roll and pitch together
Full size table
Table 8. Summary of E(utility difference) due to roll, pitch, roll and pitch together (in degrees)
Full size table

These results describe the operational performance of a ship design in the early stage. They can be used as elementary information to evaluate vessels that operate through seaway. However, they are not convincing enough to conduct meaningful comparisons of multiple ship designs. It is essential to expand on these results in the following two aspects to support further ship differentiations.

  • Two scenarios have been used as examples in the case study. If this framework is activated by random destinations over the predefined area, designers will obtain more general statistical values of the relevant parameters for the evaluation and differentiation of ship designs.

  • This framework takes advantage of designers’ subjective opinions in defining transformation rules of physical responses. Other than the given definitions in this case, designers should adjust the definitions based on their own requirements to compare different ship designs.

4 Conclusion and Future Work

The authors propose a formulation of an extension to the SC-MDP framework to incorporate the impact of physical responses on transits across the ocean. A case study has been discussed in this paper to prove the feasibility of generating rational logic and results from this formulation. The influence of seakeeping responses has been explored over the entirety of the North Pacific Ocean utilizing the combination of SPP and MDP, which demonstrate a possible method to process the information constraints in the preliminary stage. However, this framework is simply an elementary formulation in order to prove the mechanism of introducing appropriate seakeeping tools to the SC-MDP framework. There are simplifications and assumptions that restrict its application for more problems. Consequently, significant future work is required to improve this framework.

Further improvements should be made on the SC-MDP framework in order to expand its access to other seakeeping parameters, which will broaden the framework’s overall inclusion of physical information. Additionally, the transformation rules of adding seakeeping components to MDP are anticipated to enable sensitivity analysis in the future. New functions will likely be included within this framework to support operational evaluation for ships in the preliminary stage. In reality, seakeeping analysis is related to far more than roll and pitch motions, it is a significant component impacting the overall feasibility of a vessel. Therefore, vessels must be designed to successfully operate in diverse circumstances.