Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The problem of No Fault Found (NFF) is a widely known and recognised especially among those who deal with complex systems.

1.1 No Fault Found—Background

Following study of the literature, [1, 2], the most common set of maintenance activities resulting in the NFF can be described as follows:

When a unit is diagnosed as faulty at the “on-platform” test level, it is removed from the system and sent to its subsequent “off-platform” test level for further investigation. If the recognition or localisation of the fault is unsuccessful then the unit in question is tagged as NFF.

Thus the often used generic definition of a fault which is classified as NFF is “a reported fault for which the root cause cannot be found” [3]. Such faults and the way they affect systems are also described in literature as: erroneous removal (ER), no problem found (NPF), can-not duplicate (CND), and re-test OK (RTOK). Attempts to provide a systematic approach to classify and describe the NFF phenomena can be found in literature [4, 5]. However, despite great efforts from the researchers and industrial practitioners, the problem of NFF still remains an open and challenging area.

A simplified example of a maintenance process is shown in Fig. 15.1. It can be seen that the NFF issue can occur at a various steps within the maintenance process. The sequence of maintenance, from fault reporting to fault resolution, can be described as follows: A system operator records an error (e.g. fault code), maintenance personnel are notified to find the cause and provide a solution to the reported problem. If no root causes can be determined, the reported problem will be tagged as ‘NFF’.

Fig. 15.1
figure 1

Repair process during a maintenance action [4]

Full size image

There may be one or many reasons that contribute to such an outcome, and it is described in more detail in this chapter when NFF root causes are discussed. In literature such situations are often associated with faults concerning electric and electronic systems, both in aerospace [6] as well as in the automotive [7] application.

1.2 Common Causes of No Fault Found

The following categories of NFF causes are indicated in available literature about NFF: the utilisation of a system and the support or maintenance of a system. Within these categories, there are many potential direct and indirect causes of NFF and to list them all would be impractical. Attempts to classify and provide a taxonomy of NFF, however, can be seen in various sources. Khan and Phillips [8] divided the potential causes and grouped them into the following four categories:

  1. 1.

    Fault diagnosis

  2. 2.

    System design

  3. 3.

    Human factors

  4. 4.

    Data management.

Fault diagnostics will include a range of technical equipment together with diagnostic processes and procedures some of which will use specialist test and diagnostic equipment. Diagnostic equipment also includes sensors and Built-in Test (BIT) routines or processes and the hardware that operates these tests, the BIT equipment (BITE). Processes and procedures will also include manuals, sometimes called Fault Isolation Manuals (FIMs). These manuals will often specify calibration standards for the equipment or for the process itself and include environmental considerations if appropriate. When diagnostic processes and procedures are inadequate, inappropriate, or the thresholds for pass or fail have been incorrectly set, then the diagnostic process will result in a NFF. If the sensors or the BITE are faulty or operate intermittently, then again the diagnostic process may result in NFF. If the BIT process or procedure has been poorly designed with inappropriate thresholds then NFF will inevitably result. This situation leads us onto the second cause, system design.

System design includes not only the design of the test and fault diagnostic procedure but also the design and clarity of the manuals and publications that are provided. These should always be improved and developed using feedback from operational circumstances that bring realistic diagnostic experience into continually improving the diagnostic processes. If this feedback, doesn’t happen then design of the processes will still rely on theoretical information which may be wide of the mark. Design of diagnostic equipment and processes will also initially be based on a theoretical foundation of knowledge so will need to be improved using actual operational experience to ensure it does not generate NFF.

The third category to cause NFF is Human factors. There is often a reluctance to highlight the inability to find a fault where the engineer does not want to appear unsuccessful. Other human factors such as pressure to deliver a result and being seen to do something may result in the wrong component being changed. In this situation NFF may not be declared, but was in fact the real result. Lack of time, taking short cuts, lack of training and experience can all be contributory factors in causing a NFF. Pressure due to warranty claims will also feature in some industries and can be loosely grouped under the Human Factor causes of NFF. Both lack of communication and misinterpretation of symptoms are also examples where NFF can easily be caused or be the result.

The final category is data management which is a big area and particularly focuses on the ability to have enough data and data of the right quality available for fault resolution. It stems right from the time when the fault is reported; if this reporting is inadequate, the data provided can lead the diagnostic technician down the wrong diagnostic path and a NFF will occur. If knowledge of conditions when the fault occurred is not available the maintenance technician will not have a comprehensive set of data available with which to start the diagnosis. Good clear data is essential in first describing the fault symptoms and the operational conditions when the fault first occurred. Ensuring that all the relevant data is available is therefore critical as is past fault history so that data or fault trending can be investigated. Data mining in a comprehensive and accurate database is a key aspect of reducing the occurrence of NFF.

2 Cost of No Fault Found

The real cost of NFF remains unclear, as many studies have shown. However, the cost, dependability and safety, are listed as a critical stakeholder requirements which may be affected by NFF events [1].

A method to estimate a true cost of NFF in military systems was given in a presentation to the Machine failure Prevention Techniques Conference in 2014 and 2015 [9, 10] and the subsequent report [11]. In the work, a distinction is made between the costs of platform unavailability due to NFF issues and other maintenance and support actions. Another example of quantifying the cost associated with NFF events has been proposed in [12] where the described research outcomes provides stakeholders with a method to estimate the cost of NFF within their organisation and also across the whole supply chain. The proposed methodology in [12] is to estimate the cost of NFF events has been based on the Soft Systems Methodology proposed initially by Checkland in 1989 [13]. The key is to capture the main cost drivers of NFF across the supply chain and to build a framework to estimate the cost associated with NFF events. The Checkland’s Soft Systems Methodology has been adopted and a three-phase method for NFF cost estimation has been offered in [12]. In the Phase 1 the scope of the problem is defined, the Phase 2 deals with defining the data analysis process, and the Phase 3 focuses on the main cost drivers analysis as defined in the previous stages. The proposed method is in essence a dynamic time based modelling approach that is applied to represent the cost of NFF across the supply chain. The NFF events cost estimation model has been developed using an agent based modelling approach. Whiting the model two main groups of agents are included: the corrective maintenance agents and preventive maintenance agents. Each group consists of three agents to represent the supply chain, namely: Customer, original equipment manufacturer (OEM) and supply chain. Input to this model requires the cost data, i.e. preventive and corrective maintenance cost, as well as weighted coefficients related to each cost driver determined in Phase 2. Based on the above, the proposed methodology enables to estimate the total cost of NFF events. Predominantly, it is a design towards identification of abnormal cost drivers and its behaviour and evaluation of associated performance metrics with cost implications to allow analytical and heuristic information sources to be used alongside process history, costs and associated risks. The key drivers in both approaches is similar—the knowledge of how an NFF affects the overall system in terms of costs helps to determine where to concentrate and where to minimise effort in tackling the NFF root causes.

Different organisations and stakeholders will assess or count the cost in different ways. Operators and users will see the cost in loss of availability, providers of support services will see NFF as a cost in maintenance man hours for faults that cannot be found or that are repeated without resolution. Different businesses will in fact have many different and perhaps specific sources of costs that need to be identified. Once they have been there will be an assessment of the impact of these costs within whatever contract arrangements are in force.

For Operations Departments, NFF will generally reduce availability while maintenance staff struggle to find a reported fault and perhaps end up changing several items to ensure that the fault is removed, so ensuring one of the items will be tested NFF further down the supply chain. There will be system or machine unavailability and this may affect reputation and future business. Operators will also see a cost in warranty cover with equipment being returned for faults than cannot be replicated. Depending on contractual arrangements and the industry concerned the cost will fall on the operator, the distributor or the OEM. Maintenance providers will see the cost as lost man-hours and lengthy diagnostic investigations where there is no clear resolution of the reported fault; this may involve re-testing which could be expensive in the use of facilities and resources. Stakeholders in general may also have a safety cost and this can difficult to assess in merely financial terms. OEMs will have costs of re-design and almost certainly warranty costs where an endemic fault is difficult to resolve but must be solved to protect reputation and future business. There are also costs all the way down the supply chain where suppliers need to protect their own reputations and will have costs for re-testing or supply of additional parts.

There will also be indirect costs throughout the supply chain where disruption and certainty occur because of a NFF situation. Uncertainty as to whether the fault has been resolved can also feed back to the safety considerations and the need for additional checks and supervision in order to clear the system for further operation when no fault can be reproduced. The uncertainty will require further checks and balances and supervisory effort, particularly where there is no redundancy or for safety critical systems.

Finally loss of profit will occur in some industries and organisations and often needs to be balanced perhaps with considerations of safety or reputation. In some cases it is a perfectly valid option to accept the costs of NFF further down the supply chain if loss of reputation would occur. Customer satisfaction may well be more important and any detrimental effect from faults taking time to be resolved would affect business profitability in the future. Each industry and organisation must first identify the costs of dealing with NFF in order to be able to balance the decisions of where these costs should best lie.

3 Impact of No Fault Found

Ongoing efforts to estimate the real impact of NFF events can be seen across different industries and in the published academic research work. It has been shown already that NFF is often being misreported and is thus under-estimated, hence there is a great difficulty in assessing the true impact and cost associated with it. The following four areas of an equipment life-cycle and operation life are mostly impacted: operation and maintenance, stakeholders, original equipment manufacturers (OEM) and the overarching supply chain [12]. When operations and maintenance are considered, the NFF impacts can be seen in the lost man hours, direct maintenance cost, warranty cover, production cost, machine unavailability as well as further intangible costs, such as the loss of future business. This often coincides with the impact on the stakeholders which can be observed in the increasing costs and losses to the business as a result of getting a bad reputation, warranty cover cost, cost of in-tolerance failures and system operation training and safety. The NFF impact can be observed throughout the entire value chain. OEMs are mostly impacted by the direct and indirect capital expenditure cost such as those costs due to increased inventory maintenance, warranty liabilities, obsolescence and repair cost. Further to that the impact on the overall supply chain, with a strong emphasis on reverse logistics, can be observed by the increasing intangible cost for example due to loss in productivity, packaging and handling costs, a downtime and transportation cost.

An important fact is that beyond the direct impact on the maintenance, such as the costs of components and manpower, which are relatively easy to quantify once the NFF is recognised, there are other major impacts, often hidden, that are not easily quantifiably and its mechanisms are not fully understood yet. An example can be the loss of customer confidence and company reputation [3].

Regardless whether it is a military or a civil system, the occurrence of NFF events causes a disruptive effect on the successful delivery of through-life customer support. Whilst commercially NFF may cause a huge loss of revenue, in the military environment, where costs are less obvious and may be hidden, losses may be even higher. A lack of visibility of the true cost is an issue that must be tackled in the most informed manner. It is essential to stress here that numerous maintenance terms are used rather than declare a ‘no defect’ situation; terms such as CND and RTOK are all NFF and are important contributors to the overall hidden costs that nugatory maintenance causes which in turn deliver potentially high levels of operational disruption.

A significant impact due to NFF events can be also registered in the high cost of warranty returns. If a product is warranted and returns are much larger than initially forecasted it may generate high cost to the manufacturer, both in the fiscal terms as well as the reputation for unreliability and high rate product replacements.

3.1 NFF in Aerospace Industry

The NFF issues are certainly not restricted to any particular industry, however it can be observed that the aircraft industry has been the strongest advocate of tackling the NFFs as equipment is more expensive and the NFF incurred downtime causes significant loss in revenue. A nugatory and wasteful maintenance activity generates high losses and greatly contributes to the disruption to the overall operation. In-service support, that includes maintenance and repair, constitute the majority of life-cycle of aero platforms. It has been argued in the literature that for equipment having an in service life of around 20 years, such as aero-platforms, the operating and service/maintenance activities may account for as high as about a 60–80% of the whole life cycle cost of the equipment [11, 12].

In the case of aircraft systems, NFF events manifest themselves during a high stress, such as high g-forces, extreme thermal cycles, high vibration levels, or a combination of stresses [11]. High percentage of aircraft NFF problems, mostly accredited to the intermittent faults, are related to aircraft ageing, especially to problems with wiring and connections as a result of environmental and operational conditions such as exposure to vibration resulting in cracking, oxidation, heat cycling and spark-erosion, etc. [14]. It is a growing problem, mostly in legacy aircraft systems, where the effects on electric and electronic equipment, such as broken wires, cracked solder joints or corroded wire wrap are more prevalent [1].

Maintenance, repair and overhaul activities are also recognised to significantly contribute to the NFF rate [4, 10]. An often encountered inability to recreate a fault condition during bench testing results in the component being declared NFF or CND and returned as serviceable. The issue here is that bench testing is either not representative of the flight conditions or has higher thresholds for failure. The test bench itself may not be sufficiently sensitive to find faults and to determine the root cause of the original fault or symptoms seen on the aircraft. Quite often faults that have no relevance to the original fault will be identified on the test bench and consequently repaired yet these had no relevance to the original fault which will now remain dormant. Whilst not recorded as NFF on this occasion, the fact that the real cause of the fault has not been found will ensure that the root cause still becomes a driver of future fault symptoms and removals which will be designated NFF.

3.2 NFF in Automotive Industry

The automotive industry has been and still is evolving at a very high pace. This industry is seen as a leading exponent of setting new standards and trends within various engineering domains. This perhaps makes it more vulnerable to NFF problems, when compared with aerospace industry. The cost of single NFF incident in an automotive system is considerably lower than that in an aerospace system, especially when considering the potential severity and the direct and indirect effect it has on its ecosystem. To demonstrate this, an example can be considered from an everyday operational scenario; an immobilised road vehicle can case a minor disruption on the road and in most cases a low cost of recovery, especially when compared with the ripple effect that an immobilised air vehicle can cause on the runway or within a busy airport and combined with the high cost of recovery and subsequent maintenance costs.

Production demands in the automotive industry are ever increasing. According to the Society of Motor Manufacturers and Traders, in 2014 the production of passenger cars in the UK was around 1.53 million units with Nissan, Land Rover, and MINI being the leading brands contributing to these production figures. Future demand in this sector is also expected to increase as car manufacturing volumes are on course to break all-time records by 2018. With the increase in demand, customers of the automotive industry are also showing strong interest for more in-vehicle technologies and digital services. Just as phones got smart, so will cars. The rise in the digital economy and an increase in demand for customised products have caused modern automotive vehicles to become a complex system [15]. Cars are now well equipped with ever more complex systems that are connected with other remote systems and data centres, e.g. warning devices, navigation and traffic information services, infotainment systems and safety features, and thus are commonly called networked cars. The number of networked cars will rise by 30% a year for the next few years. It is estimated that by 2020, one in five cars will be connected to the Internet [15]. Thus the usage of embedded software and electronic systems will play a dominant role in the coming years for the automotive industry. The OEMs are therefore looking for ever more innovative solutions; products with shorter time to market will also be required in order to attract customers and maintain market share. Figure 15.2 shows the current usage of electronic systems on a normal passenger car [16].

Fig. 15.2
figure 2

Automotive electronics [16]

Full size image

When a unit or component is diagnosed as faulty at any test level, the unit is removed from the system and sent for test at the next maintenance level. If the diagnosis or localisation of the fault is unsuccessful then their unit is simply tagged as NFF [1]. A common definition of NFF, also to be found in [3], is “a reported fault for which root cause cannot be found”. NFF and intermittent failures have been reported in the automotive, aerospace, telecommunications, computer and consumer industries where they represent a significant percentage of reported warranty returns and field returns, resulting in significant costs [17]. The phenomenon of NFF is very well known in the aerospace industry, and particularly in avionics. A study performed by Boeing, Texas Instrument and General Dynamics and reported by Pecht [18] showed various military and commercial avionics systems and radar systems which had been analysed, showed that, depending on the type of system, between 21 and 70% of cases have been attributed to NFF. Also in more recent studies [19] similar rates of NFF can be observed in automotive, telecommunications, computers and consumer electronics industries. However, in the automotive industry it is less understood as a problem for availability, rather it is a problem for the distributors and the OEMs of warranty and maintaining reputation. Consequently the reliability of the complex electronic systems has become an important issue for the automotive industry.

The architecture of an electronics system in an automobile includes a combination of different sensors, electronic control units (ECUs), actuators and user interfaces to perform complex electrical functions [20]. Due to the addition of flourishing on-vehicle services, user and software functions are interrelated at the component level and at the whole vehicle network level. Communication between the components has become difficult to manage and performance is hard to predict in terms of the run-time behaviour of these networks [21]. The complexity brought about by embedded software and electronics has created unavoidable challenges in maintenance and repair, threatening customer satisfaction and causing negative effects on the costs of repair and replacements. From the software engineering perspective, around one third of problems are associated with software and electronics [15]. This high level of complexity and associated failures in vehicle electronics has given rise to a significant increase in NFF events and warrant issues for many manufacturers.

The NFF phenomenon also can have a negative impact on the system safety, dependability and it inevitably increases lifecycle costs. The manufacturer, component and system suppliers, the service centres and of course the operators, or users, are all considered as stakeholders of NFF. NFF events can be basically classified into two categories: those that affect safety and those that do not. One sub-category of NFF events that affect safety is where a test process at the user or operator level fails to recognize and localize the reported fault. Another safety-related sub-category is NFF events where the subsequent test process at the next service and repair level is unable to localize the actual fault. However, an NFF event where at the preceding test level, a fault is recognized that does not exist, i.e. a false alarm, this does not directly affect safety. Unfortunately, it is not always easy to decide which category of NFF event has actually occurred [22].

The major impact of NFF in the automotive sector is on the warranty or support budget which is wasted trying to find the root cause of the reported faults. Such events tend to increase the spare parts inventory and the cost of work and manpower [7]. Very often reported faults are due to operator or user error and their unfamiliarity in using and operating the system. Within the aerospace industry NFF can cause up to 90% of the total maintenance costs related to aircraft electronics. The automotive industry, like many other manufacturing industries, struggles with the high cost of improving fault diagnostics. Some authorities claim that the cost of poor fault diagnosis can be as high as 20–30% of finished products [1]. Consequently it is a big issue that no manufacturing company can afford to ignore.

Without understanding the root cause, companies and particularly the automotive dealerships, often find themselves implementing a workaround or swap-out solution of the suspect part with no real idea of whether it will work. Considering the major impacts of NFF issues, any step-change improvements are scarce and hence the lack of data on the cost impact means that dealing with the problem is unlikely to be escalated. Since “what gets measured gets done”, there is consequently a need for a generic tool that can be used (within any industry) to evaluate an organization’s ability to deal with the problem.

4 Solution

Once an organisation has realised that there is a NFF problem, a useful solution is to use a NFF Impact Avoidance Benchmarking Tool. Such a tool was developed by the EPSRC Innovative Manufacturing Centre for Through-Life Engineering Services at Cranfield University (The TES Centre) and has been applied in both aerospace and the automotive industry. The tool enables a thorough analysis of the NFF problem and the processes used in the organisations together with identifying the attitudes and culture surrounding its resolution. The tool was developed after extensive research and industry consultation allowing key aspects of the NFF process of where NFF occurs to be established. The breakdown of the processes was important for each different organisation in order to identify and establish where root causes were identified and where faults were diagnosed successfully. However, there are also difficult organizational concerns and human factors aspects to take into account as they also cause NFF in addition to technical causes; such organizational, human factor and process issues are usually easier solve and to provide mitigations for once they can be identified. The tool thus concentrates the mind on the process and the underlying attitudes using a team approach where a number of engaged and knowledgeable experts assess the ability to reduce NFF.

Benchmarking using this tool approach is a structured way of assessing an organisation and where changes are required and may they may be accepted and adopted. Benchmarking also identifies areas where easy wins can be achieved and thus contributes to a continuous improvement process (CIP) for the organization. The main purpose of the benchmarking tool is defined as [10]:

“To achieve a minimum level of non-value added activity in the timely diagnosis and resolution of complex system fault indications with the minimum amount of time and other resources expended to confidently restore the customers operation, robustly isolate and remove the cause of the fault indication, and to learn lessons and prevent future impacts.”

The process model created was initially based on the aerospace industry and includes the on-platform and off-platform processes which may or may not result in NFF. Customization of the process to adapt to the automotive industry was simply done using inputs from dealership, OEM and distributors. A high-level view of the generic maintenance process in which NFF occurs is shown in Fig. 15.3.

Fig. 15.3
figure 3

Generic model of maintenance action

Full size image

The first step is thus to adapt the model with any changes to the process. Once agreed the assembled team of subject matter and departmental experts, uses it as a self-assessment tool which requires some subjective assessment in specifying the target performance level for each process step. The tool then scores the selections and the team scores can be compared or averaged. Differences in scoring are not important and indeed demonstrate different perceptions across the organisation. Any low score allow the team to highlight where the organization is weak and enables it to focus on solutions and options for mitigation.

Whilst it was initially developed for the aerospace industry, it has been easily adapted and tailored to suit the automotive industry, thus proving its versatility. The key motivations across all industries are universal—to uncover where NFF is impacting maintenance and warranty costs and to provide mitigations, recommendations and solutions. The key steps of the benchmarking tool which should be taken into account when adoption into different class of problems are shown in Fig. 15.4.

Fig. 15.4
figure 4

Main steps of the NFF impact avoidance benchmarking process

Full size image

The possible mitigation strategies were developed from expert knowledge, best practice examples and from the accumulated research knowledge within the Cranfield TES Centre. Potential actions for end-user organizations include for example, new tools, where support is needed, required infrastructure improvements, team actions and training; mitigation strategies will always need to be customized but are provided to allow an organization to move its current assessed capability and performance, to the desired higher level in avoiding and reducing NFF issues. The tool allows mitigation options to be pre-set and to be generated as a direct result of feedback and post-analysis. The organization can decide then to make any necessary transition slowly (evolutionary), or take a more rapid approach (revolutionary). The organization’s own experts will be the best judge of what is possible and can best be delivered by revolution or evolution.

In order to address the NFF issues specific to the automotive industry an NFF impact avoidance benchmarking tool has been developed that will analyse the ability of automotive suppliers and manufacturers to deal with the NFF problems. The tool was based on the previous research work that developed a ‘NFF benchmarking tool for the aerospace industry’ The approach has been evaluated and tested in partnership with an automotive manufacturer. The combined knowledge—gathered from the literature survey on NFF, the background research of the automotive industry as well as the internal procedures of the partner organisation helped to define the problem and to deliver a suitable solution. The key objective was to control the warranty cost attributed to NFF, and to achieve a minimum level of non-value added activity in the timely diagnosis and resolution of complex system fault indications with the minimum amount of time and other resources expended.

5 Conclusions

NFF is a serious problem across many industries yet many organisations do address the problem and this can be for a variety of reasons. Most common is the inability to assess the true costs and whilst the impact may be noticed, there is an inability to identify mitigation actions that would reduce the problem. Cost and impact are difficult to estimate without good information. Some examples of cost and impact modelling are available but useful literature and descriptions are few and far between. Addressing the issue by using a benchmarking tool such as described in this chapter will be an essential first step in identifying the process around NFF occurrence and will systematically expose where to concentrate effort for the most rapid gain and progress.

The impact of NFF in aerospace industry mostly relates to high value components and its effects, such as disruption to service and safety breaches, whereas in the automotive industry, the key impacts are felt with much higher volume of albeit, less-costly effects. In both cases, however, the stakes are high.

In the automotive industry the occurrence of NFF events is considered to be one of the major threats to the overall reliability of the system and an important factor in warranty costs. Following the example of aerospace industry, it has become essential for the automotive manufacturers to avoid NFF events by means of effective fault diagnostics and correct identification of root causes of fault at every step of the maintenance process. This will certainly help minimise the impact of NFF, and subsequently help to lower the cost of warranty claims. However, communication between the dealerships and the OEMs is poor and yet to be properly incentivised so that both share and solve the problem.

An excellent proposed solution, as a first step, is to identify the non-value added activities in the fault diagnostic process and provide mitigation strategies to address the NFF events as proposed in the NFF Benchmarking method described in this Chapter.