Preparing Reports on Risk Evaluation and Assessment in Rail Transport Based on the Polish Experience

Abstract

The European Union Regulation on risk evaluation and assessment has resulted in the need for risk management in railway undertakings when they decide, based on analysis, that a change is significant and affects rail transport safety. Then it must be proven that the risk management process caused by this change is correct. The analysis of past experience in this field in Poland shows the need to improve this process, both in terms of cooperation between different entities involved in this process and in order to understand the objectives of the regulation and its key provisions. The fundamental problems and mistakes in reporting on risk evaluation and assessment include mistakes in defining the system undergoing change, mistakes in defining system boundaries, mistakes in identifying risks, improprieties in selecting risk acceptance principles, the improper use of codes of practice, ignoring safety requirements etc. These problems have triggered the analytical research on the assessment of the adequacy of risk management in rail transport as a result of a technical, operational and organizational change. The results of the analysis have been presented in this paper.

1 Introduction

The issue of safety in rail transport is an issue that requires constant search for new, effective and efficient solutions, which aim to reduce risks in rail transport and improve the reliability of used devices. The role of a risk management process in rail transport is increasing [1]. The need for dealing with this issue results from the applicable European Union Directives [2–4]. According to A.M. Zarembski, J.W. Palese, in recent years, railways have turned to the discipline of risk management to improve safety and reduce the potential risk of accidents or derailments. Since accidents and derailments are very low probability of occurrence events, it is necessary to focus on the derailment causes themselves and develop risk management tools that quantify and analyse the “risk” for each key derailment or accident area [5]. It should be highlighted that railway undertakings are obliged to implement and maintain safety management systems when risk management is mandatory. It is not easy to apply the principles of risk management in rail transport due to the complexity of its transport and maintenance processes. It is a complex process that requires a holistic perspective based on very good knowledge of technical processes [6]. Railway undertakings are obliged to manage risk as part of their operating activities. This process should be part of organization operations and analysed in terms of all aspects of railway traffic safety. The risk should be evaluated when a new product is implemented, and when the railway system is designed, operated and maintained. These cases cause changes in the railway system and they should be analysed in terms of their impact on railway traffic safety. The awareness of railway undertakings managers of the need for efficient and effective risk management in the event of changes in the railway system is increasing. In Polish conditions, there are a number of shortcomings in applying the regulations related to this process. The Polish and European experience, due to the relatively new view of the risk management process, shows a significant need for further legislative work in this field, as well as changes in accepting responsibility for introduced changes. The process of risk assessment in the event of a significant change in the railway system entails a lot of necessary and required actions, which is often inconvenient for managers in terms of logistics. It results also from implementing new technologies and adapting them to new conditions [7]. The difficulties are mainly due to poor planning and lack of imagination of those who introduce changes as regards the impact of these changes on railway traffic safety. This leads to problems causing, for example, delays in performing investment processes and the need to find bad solutions resulting, for example, in unreliable risk analysis, and in particular, decisions that a significant change is non-significant in terms of safety. It is a widespread and reprehensible phenomenon. In addition to such situations, many other problems, mistakes and irregularities in applying legislation on risk management in rail transport can be observed. Key problems associated with implementing the requirements of Regulation 402/2013 [8], whose provisions govern the core assumptions of the risk management process, have been presented in the paper, based on the review of the relevant literature, including regulations and the author’s own experience. The study is qualitative research based on the analysis of case studies.

2 Change Management in the Railway System

A change to the railway system is a factor creating new conditions for the functioning of this system. A change generates new circumstances, often affecting the level of safety. A change to the railway system, therefore, should be the subject of analysis in terms of safety. A railway undertaking that monitors changes in the designed, operated or maintained railway system increases the level of the culture of safety, at the same time eliminating any potential risks which, if there was no reaction, would increase the level of risks in rail traffic. A change, therefore, has been considered a very important element of the functioning of the railway system, the consequences of which are crucial for maintaining an acceptable level of safety. The identification of the types of changes that affect safety has been described in Regulation 402/2013. Such changes may be of a technical, operational or organisational nature. Such a division of changes makes it possible to monitor all the activities of railway undertakings in terms of an impact on safety. A technical change occurs when a railway system is introduced, modernised or renewed. An operational change is any changes affecting the rules of operating the railway system in terms of using its functionality. As regards organisational changes, only those changes which could impact on the operational or maintenance processes shall be subjected to consideration under the rules 402/2013. Change as a natural result of business activity in the railway sector has been subjected to monitoring and has become one of the key issues significantly affecting railway traffic safety. The examples of changes to the railway system in the areas of infrastructure, rolling stock and organization include:

Infrastructure:

• 1. The construction of a new railway line or extension of the existing one by another track.

• 2. The construction of the platform/station in a new location within the existing railway line.

• 3. The construction of a new or reconstruction of an existing railway subgrade resulting from changes in the geometry of the track system with a possible change in the embankment slopes/ditches tilt resulting in increasing or adjusting the maximum axle load to the designed category of line and/or increasing the design speed on the line.

Rolling stock:

• 1. The installation of ETCS on-board devices level 1.

• 2. The installation of ETCS on-board devices level 2 and higher.

• 3. The redevelopment of the gear system.

• 4. The redevelopment of the brake system.

Organization:

• 1. The introduction of new/modified rolling stock into operation.

• 2. An increase in the amount of rolling stock used.

• 3. The introduction of a new product on the rail market [9].

• 4. The presented changes show how wide the range of the discussed issue is. There is an infinite number of changes that may affect the rail industry and railway traffic safety, therefore this subject is now a key issue in the area of maintaining and increasing railway safety.

3 Risk Evaluation and Assessment in the Context of Significant Changes

The process of risk evaluation and assessment requires the selection of appropriate methods and techniques. When explicit risk is estimated, appropriate quantitative and qualitative methods are used. While the risk management process does not require that any specific tools should be applied, many of the more well-known techniques will be relevant, including:

• structured group discussions;

• checklists;

• task analysis;

• hazard and operability studies (HAZOPs);

• hazard identification studies (HAZIDs);

• failure mode and effects analysis (FMEA);

• fault trees; and

• event trees [10].

In addition to the above methods, it is reasonable to use comprehensive solutions for risk evaluation and assessment. Figure 1 shows the assumptions of such a comprehensive approach, that is the Risk Score Matrix model based on the assumptions of DIN V VDE V 0831-101 norm. The Risk Score Matrix (RSM) consists of the application of a risk matrix and score tables for assessment of the barriers, similar to RPN schemes. The final result consists of hazard rates (HR) related to the functional failures of the technical system and the assumptions on which the analysis rests, which may turn into safety-related application rules (SAR) [11].

Fig. 1.

Overview of the Risk Score Matrix model [11].

An important and widely used method for estimating risk is the ALARP principle.¹

The ALARP principle ensures that the risks of any system with serious consequences in terms of human loss and injuries, is kept to a level which is As Low As is Reasonably Practicable. ALARP defines three risk levels:

Intolerable Risk, which cannot be justified or accepted, except in extraordinary circumstances Tolerable Risk, which can be accepted only if risk reduction is impractical or if the cost or risk reduction greatly exceeds the benefit gained.

Negligible Risk, which is broadly acceptable and does not require risk mitigating measures. If risk is determined to be at the intolerable level, measures must be taken to reduce it immediately to tolerable level. If risk is found to be at tolerable level, risk mitigating measures should still be applied, provided that a cost benefit analysis is in favor of it [12]. The presented status of risk is important from the point of view of risk classification and identification of appropriate risk reduction measures to fulfil safety requirements. In Polish conditions, the FMEA method has been widely accepted and it is mostly used to determine explicit risk, that is a situation when the codes of practice or other reference system do not apply to an identified hazard. Despite many shortcomings, which include the high level of subjectivity of this method, it is still very popular. It seems reasonable to use solutions that combine several methods at the same time; then it is possible to reduce the number of mistakes that the FMEA method may entail.

When a report on risk evaluation and assessment is drawn up, it is necessary to extend the range of issues it covers. Based on the author’s experience, the structure of the report on risk evaluation and assessment can be developed on the basis of the following subjects:

• 1. The impact of a change on safety (Yes, it matters - further action/No, it does not matter.)

• 2. The general description of the system before change (system objective e.g. intended purpose)

• 3. The description of a change - a type of change: technical, operational, organizational, the detailed description of the system:

  • 3.1. The functions and elements of the system after change (including e.g. human, technical and operational elements);

  • 3.2. The system boundary, including other interacting systems;

  • 3.3. Physical (i.e. interacting systems) and functional (i.e. functional inputs and outputs) interfaces;

  • 3.4. The system environment (e.g. energy and thermal flow, shocks, vibrations, electromagnetic interference, operational use);

  • 3.5. Existing security measures and the definition of safety requirements identified by the risk assessment process (at subsequent stages).

  • 3.6. Assumptions determining the limits for the risk assessment.

• 4. The criteria for assessing the significance of change:

  • 4.1. Failure consequences: a credible worst-case scenario in the event of failure of the system under assessment, taking into account the existence of safety barriers outside the system.

  • 4.2. Novelty used in implementing the change; this criterion includes innovation in both the entire railway sector, and what is new just for the organization implementing the change.

  • 4.3. Complexity of the change.

  • 4.4. Monitoring: the inability to monitor the implemented change throughout the system life-cycle and take appropriate interventions

  • 4.5. Reversibility: the inability to revert to the system before change; and

  • 4.6. The assessment of the significance of change taking into account all recent modifications to the system under assessment and were not judged as significant.

• 5. The identification and classification of hazards (what might happen):

• 6. The choice of the risk acceptance principle (Using the codes of practice during risk evaluation, reference systems, assessment and evaluation of explicit risk).

• 7. The rules and scope of applying these risk acceptance principles in relation to defined hazards.

• 8. The identification of risk acceptance principles.

• 9. Confirmation that all defined hazards and risks fulfil the acceptance criteria.

• 10. Final conclusions and presenting the limitations of the document.

The presented issues do not exhaust the subject because in certain situations the scope should be expanded and clarified to match the context of the system undergoing change. Therefore, this description is illustrative, though quite detailed.

4 An Impact on Safety and the Evaluation of the Significance of a Change as a Milestone in the Risk Management Process

The greatest difficulty in the risk management process in the context of introduced changes is to assess their impact on safety and consequently, assess the significance of the change. In the first step it is necessary to assess the impact on safety in order to further assess the significance of the change according to specified criteria. If the proposed change has an impact on safety, the proposer shall decide, by expert judgement, on the significance of the change based on the following criteria:

• a. failure consequence: credible worst-case scenario in the event of failure of the system under assessment, taking into account the existence of safety barriers outside the system under assessment;

• b. novelty used in implementing the change: this concerns both what is innovative in the railway sector, and what is new for the organisation implementing the change;

• c. the complexity of the change;

• d. monitoring: the inability to monitor the implemented change throughout the system life-cycle and intervene appropriately;

• e. reversibility: the inability to revert to the system before the change;

• f. additionality: assessment of the significance of the change taking into account all recent safety-related changes to the system under assessment and which were not judged to be significant [13].

Figure 1 shows the logic of conducting the analysis leading to decisions about the impact of change on railway traffic safety. The presented diagram proves that first it is necessary to assess whether the change has any impact on safety. If it does, it must be determined whether it is significant. If it is significant for railway traffic safety based on the relevant assessment criteria, then a report on risk evaluation and assessment is drawn up, which is subsequently verified by an accredited assessment body. It should be noted that it is a railway undertaking or manufacturer that is responsible for deciding whether the change is significant or not. In any case, the decision should be documented and clearly identify specific individuals responsible for making a decision on the significance of the change (Fig. 2).

Fig. 2.

The algorithm describing the decision-making process with regard to significant changes [own study]

The issue under discussion is particularly important as regards railway transport safety. Transferring responsibility for deciding that a change is significant and non-significant to railway undertakings, based on the subjectivity of the evaluation, has resulted in the situation that many changes are classified as non-significant, which reduces the role of accredited assessment bodies in participating in the process of change management and reduces the significance of changes to the railway system. Therefore, in order to decrease the number of situations when a group of railway undertakings has no responsibility, the control and supervisory role of the NSA (National Safety Authority) is important in supervising the implementation of changes in the railway system.

5 An Example of Application of the Method Risk Score Matrix in the Valuation of the Risks Associated with Rupture Axis of the Freight Wagon

Risk Score Matrix Method is a method which can be considered now as optimal from the point of view of technical risk assessment and the requirements of the Common Safety Method for Risk evaluation and assessment.

This method is based on the assumptions of the method of the event tree ETA (Event Tree Analysis) and allows the determination of the probability of occurrence with regard to the applied by the company railway protections.

It allows to confront the determined value of the risk to the safety requirements laid down in Regulation 402 and defined as: highly improbable “Means an occurrence of failure at a frequency less than or equal to 10 9 per operating hour and” improbable “Means an occurrence of failure at a frequency less than or equal to 10 7 per operating hour”.

Existing methods such as. Method FMEA (failure mode and effects analysis) does not take into account the calculated probability of the occurrence of hazards from the point of view of a railway undertaking protections.

It is also no possible to designate a target of the risk in the context of the requirements posed by that regulation 402.

As part of the FMEA method calculated the number of priority risk is designated a value of not relating to the safety limits laid down in Regulation 402/2013.

Therefore, commonly used methods FMEA already adopted in the case of the requirements of Regulation 402 is not justified because of the limitations described above.

Figure 3 shows an example of the calculation of the risk of using the event tree for the risk of rupture of the wheelset axle in a freight wagon.

Fig. 3.

An example of using the event tree ETA (Event Tree Analysis) for example cracks wheelset axle freight wagon [own study]

Were defined four safety barriers namely: Effectively was used documentation of maintenance, is used system EVIC (European Visual Inspection Catalogue), are carried out non-destructive testing axis of maintaining level P4 (major repair) and are implemented non-destructive testing axis of maintaining level P3 (review periodic chief) and are also implemented non-destructive testing axis of level P2 (review current). These four defined protections barriers led to the development of seventeen scenarios have been assigned a probability of risk in the context used a combination of protections.

The calculated probability values can be analysed with using the safety requirements laid down in Regulation 402.

Analysing the results of considerations it can be stated that the designated nodes in random probability estimates contribute to the final calculation of the probability. Important are used tools for safety protection.

The next step in this method is to apply the calculated value of the matrix Risk Score Matrix portfolio, taking into account the level of protections versus class of accidents.

Summing up the results of the calculation indicated in the case described, it is possible to use such a pattern of conduct with the risks for which the probability of danger will be included in the expected protections what is requirement of rules 402/2013.

6 The Preparation of Reports on Risk Evaluation and Assessment - Key Experiences and Problems

In scientific research six selected reports on risk evaluation and assessment were used, drawn up by infrastructure managers, railway construction investment companies and manufacturers, assessed by an accredited assessment body in terms of compliance with the ISO IEC 17020 norm [14]. Qualitative research based on the phenomenological paradigm was applied [15]. When analyzing selected cases of the examined reports, repeated irregularities were shown in the process of drawing up reports on risk evaluation and assessment and decisions about the significance of changes. Grouping the cases allowed for the assessment which subject areas are the most difficult for those who prepare documentation on documenting the impact of changes on rail traffic safety and they are often recommendations by the accredited assessment body. It should be noted that the experience in applying Regulation 402/2013 is not wide in Poland. This is due to the fact that the regulation has been in force for a short time. The paper is one of the first in Poland describing the study of selected aspects of conducting risk evaluation in the context of technical, operational and organizational changes in rail transport. As regards the scope of this study, six reports were assessed, out of which four related to operational changes introduced by the infrastructure manager, resulting from the introduction of new traffic control devices, including a control structural sub-system - track-side devices and two reports on the modernization of railway lines carried by railway construction investment companies, the scope of which included an energy structural sub-system. The reports were prepared in various forms. Sometimes the report on assessing the significance of change and the report on risk evaluation and assessment were a unified whole, and in other cases they were separate documents. Most reports (four) were prepared by a proposer, the other two by consulting companies assisting the proposer. All of the examined reports were ultimately positively assessed by an accredited assessment body, but all of them required prior alterations, which were collected and research findings were formulated based on them and the accompanying documentation. The following are key recurring problems that the authors of the reports on risk evaluation and assessment came across. They were identified by the accredited assessment body and described in the reports on safety assessment as part of the independent process of evaluating the adequacy of the risk management process in the context of the requirements of Regulation 402/2013.

• 1. The inaccurate description of the system being changed and the system before change. The authors do not accurately show what the subject of the change was, and do not describe how this change has influenced specific new hazards in rail traffic. They do not describe key interfaces between devices, either. To a large extent, these new interfaces resulting from the new functionalities of the system and changed configuration pose new risks, which should be covered by appropriate risk control measures. Some mistakes can also be observed in defining the boundaries of the railway system undergoing change. This is particularly important when a change affects a number of structural sub-systems according to TSI (Technical Specification of Interoperability), e.g. when a change relates to control sub-systems, energy and infrastructure.

• 2. It is imprecisely described which specific criteria influenced a decision about the significance of the change, as well as the reasons why a particular criterion occurred. There are also obvious factual mistakes in the interpretation of the significance of changes. Moreover, individual evaluation criteria are misunderstood.

• 3. The incorrect identification of the codes of practice within the scope of CSM RA (Common Safety Methods Risk Assessment) in the context of the defined hazards. Proposers often overlook this aspect and assess explicit risk, for example, by the FMEA quality method. Another problem is misunderstanding the definition of codes of practice and assigning a specific code of practice to specific hazards, as well as using codes of practice when a risk is only partially covered by the assigned code of practice.

• 4. The incorrect identification of the scope of using qualitative methods, e.g. FMEA compared to other risk acceptance principles. It should be also noted that the range of methods assessing explicit risk in Poland is reduced only to the uncritical application of the FMEA method, with no interest in other quantitative and qualitative methods.

• 5. The key problem is also failure to identify all possible risks, which in a given situation are important for the changed operation of the railway system. Industry experts from assessment bodies think this aspect demands improvements in particular.

• 6. Reports on risk evaluation and assessment are often drawn up by specialists who do not have sufficient experience in the railway sector, which is shown in the low quality of the description of technical issues as well as their detail. Another problem is insufficient knowledge of safety requirements with respect to the specific character of the railway system undergoing change.

The presented research findings indicate that the quality of reporting on the risk evaluation and assessment by accredited assessment bodies needs improving. The authors of these reports do not fully understand the seriousness of the impact of technical, operational and organizational changes on railway traffic safety.

7 Conclusion

The process of risk evaluation and assessment in terms of changes in the railway system is a complex process that requires the participation of many stakeholders. This issue is increasingly understood in Poland, however many mistakes have not been avoided, which the author has tried to highlight in the paper. Conducted qualitative research of selected reports on risk evaluation and assessment have helped to formulate the following conclusions:

• 1. The authors of the reports on risk evaluation and assessment make a series of mistakes of a methodological nature, which is related to the lack of knowledge of the correct interpretation of various issues such as: a change to the railway system, the codes of practice or safety requirements.

• 2. It is observed that decisions about the significance of a change are avoided. Railway undertakings are afraid of a long process of risk evaluation and assessment and the costs of participation of accredited assessment bodies in the risk management process.

• 3. The reports are often too general and do not present technical aspects in detail, as well as functionalities resulting from the changes, especially in the context of interfaces with other rail devices or systems.

The presented findings should provide a foundation for improving risk management processes by railway undertakings. The paper highlights the need to initiate a broad discussion on the quality of reports on risk evaluation and assessment and the problem of classifying changes as non-significant in order to avoid the complicated procedure for documenting changes to the railway system and the reliable assessment of their impact on its safety.