TiV-Model—An Attempt at Breaching the Industry Adoption Barrier for New Complex System Design Methodologies

Abstract

System design and engineering are fundamental to the creation of the devices and technologies that have become a large part of our lives. Technology companies, motor vehicle manufacturers and inventors go through this process to develop all kinds of luxuries and necessities for everyday life in the twenty-first century.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

4.1 Introduction

System design and engineering is fundamental to the creation of the devices and technologies that have become a large part of our lives. Technology companies, motor vehicle manufacturers and inventors go through this process to develop all kinds of luxuries and necessities for everyday life in the twenty-first century.

Often from the perspective of the user, the means as to how their products were created is not of concern; it works so it does not matter. To the designer, the methods and methodologies are very important tools in their belt, but some tools are better suited to the job than others. Current methodologies, such as the Mechatronic V-Model, provide decision-making knowledge and support to designers and enable simple platforms as the basis for development.

This information source is important, and it tells the designer what it is that needs to be known, a crucial component of the process for engineers, especially when designing complex systems. Research finds novice designers to be only aware of 35 % of their knowledge needs in the aerospace industry [1], showing that there is a very high competency barrier associated with complex systems.

This high competency standard is but one of the many difficulties that arise from complex system design relative to conventional system design, but there are many more, and researchers and companies will always be interested in looking for new ways to do things. The interest in developing more efficient and effective methodologies for the design of complex systems can thus be argued for on economic terms alone.

Take for instance the example of BAE Systems, one of the world’s biggest and most successful developers of complex systems in the form of naval, aerospace and ground platforms for various functions. With £1.3 billion in revenue in 2014 [2], a small investment in research into the design process improvement even for tiny reoccurring percentile gains would be a simple choice. Academia is one environment in which to study the application of new methods and methodologies, but as Birkhofer et al. [3] show in their work, methodologies born of academic research are rarely or reluctantly adopted into practice. The reasons for existence of these adoption barriers range from the lack of perceived usefulness, bad communication of concepts and absence of “proof of usefulness”.

This chapter will introduce the TiV-Model, a design methodology for complex system projects that aims to put to rest concerns facing the adoption of the methodology into practice. The next section will contain a description of the TiV-Model, how it was developed and will show how it plans on solving design related issues. This will be followed by the validation planning of the methodology and future plans for development concerning predicted future challenges within the industry.

4.1.1 Complex System Design

To understand the difficulties in Complex Systems Engineering (CSE), it is necessary first to distinguish between regular systems engineering and CSE. Traditionally, engineering design is considered to be an iterative design process “concerned with the creation of systems, devices and processes useful to, and sought by, society.” [4]. In short and in a way, CSE is concerned with the investigation of the means to the creation of complex mechatronic systems such as robotic systems. The most common understanding of this is the engineering design process, a general term used to express the series of steps involved in the design of systems. Figure 4.1 shows the design engineering process in simple form.

Fig. 4.1

Engineering design process (after [5])

Design methodologies and the engineering design processes are, for the most part, interchangeable. Most methodologies have some focal point such as an optimised critical path, DfX¹ methods or some other element that provides for more favourable results in certain areas relative to other methodologies. A core difference between complex and conventional methodologies, and also a key identifier of the former, is that conventional projects try to balance out manufacturability and repeatability with the product quality. Complex projects in contrast tend to spare no effort in achieving their goal, even with the use of expensive or difficult manufacturing processes, particularly where the design is a one-off.

4.1.2 Complex Versus Conventional System Design

The engineering of complex systems comes with additional challenges that need to be accounted for in the engineering design process. A majority of these differences stem from the increased scale and complexity of the project. Issues such as an increased number of parts and manufacturing operations due to the design’s physical size can be easily accommodated. Processes related differences due to budgetary and time constraints, such as the reduced accessibility of physical prototyping, will have to be explicitly addressed and made aware to the designer. Table 4.1 sets out some of the qualitative properties of conventional products, mechatronic products and complex projects.

Table 4.1 Conventional, mechatronic and complex projects [6]

Table 4.1 serves to highlight some of the core issues surrounding the completion of complex projects, namely.

Complex Design Management—The data and physical output of large scale CSE projects can be overwhelming when compared to traditional systems engineering. Large capacity servers are required to manage the amount of data, but the data itself is more varied. For example, a CAD model of a satellite design may contain separate models of the electronics, chassis, fixtures, mechanisms and heating elements, possibly further divided by subsystem or payload. This added layer of complexity must be accounted for in the methodology and the management system. The sheer volume of files must be tracked and accounted for as well as appropriately labelled for use in a group environment.

Complex Knowledge Base —Complex systems are multidisciplinary in nature and require a firmer grasp of the required knowledge bases. Tolerances are smaller, requirements more demanding and designs more convoluted than for regular mechatronic engineering. Documenting and tracking this knowledge is more important and computer aided tools are essentially mandatory to ensure each team is up to date with the huge amount of information, such as operating principles and design specifications. This wider range and expertise of knowledge means that specialist teams will be more common; allocating these to areas of the project that need them is an additional planning complication.

Increased Uncertainty and Risk —As with any high budget project, the more money invested into it, the more money is wasted on failure. The increased complexity, in the form of increased points of failure, tighter tolerances and non-standard design practices also brings additional uncertainty in both process and design. Hiring new graduates and novice engineers may be perceived by management as detrimental to the project as experienced engineers are expected to take the lead and perform a disproportionate amount of the work. Design teams require more information, skill and agency to complete the tasks relative to that of conventional systems engineering.

Design Evaluation and Non-Destructive Testing —High budget projects generally have the freedom, and are encouraged to develop working prototypes to test and validate the “real-world” behaviour of their design. In large scale CSE projects, the nature of the design solution is, however, often one that cannot be wholly prototyped as cost, time and resource constraints prevent this. In a best case scenario, subsystems or components can be prototyped, but not full systems. If full systems are to be tested, it would be in the post-fabrication stages, thus non-destructive testing is the only way to preserve the system integrity. Reliance on simulation and on-paper calculations can be considered mandatory otherwise.

4.1.3 Methodology Adoption Resistance

Badke-Schaub et al. [7] summarise the perceived issues with new design methods and methodologies. Figure 4.2 then shows the common industry reasoning for the lack of integration of new design models and methods.

Fig. 4.2

Industry perspective barriers to methodology adoption

Performance issues relate to the absence or uncertainty of proof that the methodology will work as intended or produce results. This stems from a lack of validation on the part of the creator or of follow-up case studies. The presentation of the methodology refers to the effective communication of information and its clarity. Process relative issues often involve the intra-task efficiency of the model, for instance the trade-off of time/cost/flexibility.

If the issues from the CSE perspective are combined with the adoption barriers list, it is possible to effectively create an issue matrix specific to design methodologies within complex system engineering industries by adding an extra column to Fig. 4.2 as in Fig. 4.3.

Fig. 4.3

Comprehensive issue matrix for CSE methodology adoption

The matrix of Fig. 4.3 then provides a list of problems that can be solved at the methodology level and it is to address the TiV-Model that has been created, a CSE design model that aims for industry adoption by focusing on the issues that commonly prevent industry adoption as well as the issues faced by CSE designers.

4.2 TiV-Model

The TiV-Model of Fig. 4.4 is a CSE design methodology that possesses multiple traits that make it highly beneficial for use in the complex systems industries with a focus on spacecraft and satellite development. The development focused on taking an existing model platform and adjusting certain characteristics.

Fig. 4.4

The TiV-model

Categorised Sequential Task Process —By categorising tasks into stages as a sequential process, much like traditional design processes, the process can be simplified into a step-by-step programme of tasks. At a glance one would think that this hinders the application of concurrent engineering. However, by using knowledge databases, the general stage tasks can be partitioned into discipline specific tasks and goals. This will allow either a traditional approach to the design process, or the more modern concurrent approach depending on the preference of the organisation or team. Allowing for both of these approaches ensures the general flexibility of the model.

Goal Oriented Process—Traditional design methodologies will sometimes incorporate specific methods as part of the design process; having methods that are well proven to work for that specific application can be beneficial, but ultimately means that the overall flexibility of the methodology is compromised. Additionally, by focusing on the task rather than the short term goal, new designers may not understand the purpose of doing such a task, leading to a possible chance of failure. The TiV-Model instead states which deliverables are required at that stage, while providing possible, but not definite, methods to accomplish the task.

This allows organisations to adopt the model without changing their pre-implemented methods or tools. This also eliminates the risk of the design team performing a task simply for the sake of performing a task, which can occur if leaders do not specify the “Why?” behind it. By focusing on what is needed of them as an end result, designers can understand the process, focus on the output and are still free to use whichever method preferable to obtain that output. Again, this flexibility maintains the value of the methodology across industries and applications.

Idealised Requirements for Accurate, Non-Destructive Validation of Design —Typically, the prototyping stage in system design would be used to validate specific functions or systems, the cost/time/resource constraints of CSE make this form of validation much more inaccessible. The reliance on computer aided means of validation can, however, be accommodated in the methodology by ensuring that multiple system and manufacturing/assembly models are created.

Discipline specific models to evaluate parameters such as thermal properties, yield, kinematics and geometric interferences will be integrated into multifaceted models, designed to simulate the actual environmental and loading conditions of the model. For example, launch resonance conditions and the effect of rocket shroud heating can be modelled independently, but running both of these aspects together will ultimately give a more reliable result. Combine this with iterative retroactive “reality checks” for the simulation data, and the need for prototyping and destructive testing can be effectively reduced.

Simplified Resource Allocation Recommendations —For certain stages of the design process, teams will be formed in some capacity, either functional or disciplinary. The project planners then have to allocate these teams to tasks pertaining to their expertise. Specialists may also be required for temporary contracting depending on the variety of the in-house design team. To plan for situations like these, the specialist knowledge types have been categorised into databases and linked to the stages where this expertise would be required. In doing so, the project planned can look ahead at the kind of disciplines required for the project and hire ahead of time, reducing cost and time.

Communicable and Understandable Language and Processes —One of the most crucial aspects of the methodology is its ability to be easily understood, time spent educating team members does not directly add value to the project, so as little as possible is the ideal. A framework that is easy and quick to learn will be welcomed by novice and experienced designers alike, as it enables the newer designers to pick up the slack earlier in the project without being carried by experienced designers.

The “Tiv” component in the model name enables a simple memory trigger to remember the stage names and general contents at will (QualitaTive, LegislaTive, etc.). A simple memory game like this can help boost first time retention of model concepts. By segmenting the tasks, deliverables and databases into a neat column-row dichotomy it is hoped that the model can retain a visual appearance that aids recognition of elements and understanding of task/goal flow.

4.2.1 Model Description

The TiV-Model has the essential steps required for any design process, each of these are labelled with a memorable name pertaining to nature of the stage. Tables 4.2 and 4.3 then show the type of information being presented.

Table 4.2 TiV-Model stage descriptions

Table 4.3 TiV-model column descriptions

Performance —The core problems associated with adoption from this perspective is the lack of study into validation of the methodology and “proven” usefulness. The TiV-Model will be built on the provable performance and is currently undergoing the verification and validation process. By performing necessary validation of the methodology through verifiable means and by performing post experiment case studies on the implementation of the methodology, empirical data can be given to prove the validity and performance of the TiV-Model.

Presentation —Successful communication of a model’s core principles involves considering the designer’s point of view during the model’s development. TiV-Model was initially designed with ease-of-use from a designer’s perspective in mind. Many of the changes from the base version of this model, the 3-column model, have involved redefining task and timeline taxonomy to “clean up” the presentable information on the core methodology view [8].

The organisation of presentable information involved;

• Refining the concept of system models as deliverables representative of the system overall.

• Refining the concept of Manufacturing/Assembly models as deliverables that represent the details required for buy-in, manufacturing and assembly of components.

• Showing the critical task path and the key deliverables for that stage of the process.

• Splitting the middle stages of the methodology (Qualitative-Evaluative) by design discipline and showing rough critical path for each.

• Identifying knowledge databases by discipline.

• Displaying when particular knowledge is needed at which stage.

• Displaying key deliverables with suggested methods, maintaining the option for alternative methods.

The information displayed on the TiV-Model allows the designer to make a quick and accurate extrapolation of the meaning behind the visuals and the wording. Methods are “advertised” and encouraged, but ultimately subject to change depending on the approach of the designer or organisation. This flexibility is communicated by showing that the task is outcome oriented, with methods paths only suggested and not enforced. Designers with the most basic systems engineering knowledge can develop an understanding of the process and a natural experiment that shows this will be discussed.

Knowledge base taxonomy is divided into general disciplines that are shown in the model as well as where they are best applied. This ensures planners recognise where knowledge is to be applied within the project.

Process —The process issues were addressed through changes made by logical reasoning, the effectiveness will be demonstrated in the experiment referred to above and discussed later. As already mentioned, flexibility is ensured by goal orienting the tasks, leaving the method open to the organisation’s preference, yet offering options and suggestions for placeholder methods. This aids new designers in making decisions that would otherwise require more information or expertise. Support from management is an extension of how well integrated the methodology is from bottom-to-top in the organisation. However, the success of integration is subject to acceptance at both management and user level. Direct benefits to management of the project would come from the interactive program planned for the final development stage of the TiV-Model.

Complex Systems Engineering —With the increased uncertainty associated with CSE, measures taken in the methodology can offset this. As mentioned before, by presenting suggestions for methods and clarifying where specific knowledge should be used, the uncertainty can be minimised and thus the project risk associated with that uncertainty reduced. Solving the problem of a high part and file count for a CSE project would be the responsibility of the management system in place and this is addressed as an interactive component of the methodology.

Additionally, the entry skill barrier to new engineers can be reduced by recognising the knowledge gap between them and more experienced engineers, what knowledge they need and when. TiV-Model, while being goal oriented, makes suggestions for possible methods to use to accomplish the task. These methods are optional, and organisations with prior operating principles can implement their own methods, but in the absence of that knowledge the designer has the capacity to retain their agency.

4.2.2 Potential Benefits

Designer

• Easy to use and understand current tasks.

• Information needed is provided at the time it is needed.

• Transparency in planning allows greater agency and communication.

• Novice engineers enabled to contribute more.

• Experienced engineers not relied upon too heavily.

• Choice of method, tools and style dependant on designer or organisation.

Project

• Computer-aided validation focus has higher chance of ensuring correctness first time.

• Concurrent design options may help improve systems integration quality.

• Clear deliverables helps improve error checking and identifying points of failure.

• Documentation of each stage is part of deliverables required, meaning retroactive checking and changes can be made during the project.

• More means for design validation.

Planning and Management

• Stage and task breakdown is categorised to ease timescale planning and rough resource allocation.

• Sequential tasks broke down by discipline, allowing for either a traditional or concurrent engineering approach.

• Knowledge requirements for each stage outlined, allowing plans for specialist help.

• Planning is transparent and thus easily communicable.

Organisational

• Flexible goal-oriented design means tools and methods need not change.

• Keeping tools and methods means very quick and easy implementation into organisation.

• Reduce costs by;

  • Supporting inexperienced engineers.

  • Using computer-aided design validation as opposed to prototypes.

  • Retaining in-house tools and methods.

Industry

• Methodology validation breaks down industry barriers for academic model acceptance.

• Stepping stone example for new, improved design methodologies.

• Hiring of inexperienced engineers will be justifiable, as risk is reduced.

• Non-destructive and computer-aided means of design validation could reduce project costs across all projects.

• Flexible and modular methodology sections means TiV-Model can be adopted into any CSE industry, a potential for a standard.

4.3 Methodology Validation

In order for a new methodology to be accepted as a working, feasible alternative, it must first be scientifically verified and validated. Verification of design methodologies involves confirming that the internal logic of the methodology is consistent, validation involves proving that the methodology will provide the desired output effectively and efficiently.

4.3.1 Validation Methods

In the realm of engineering design methodologies, research into the validation of models is somewhat rare. Three suitable models were considered for use in the validation of TiV-Model.

Technology Acceptance Model (TAM)—The TAM was introduced as a means of validating tools, models and methods from a usability perspective. The model was designed specifically for the validation of computer systems, but can be expanded for general use. The TAM focuses on the acceptance of a model by measurement of the users intentions; perception of use quantifies validity in this sense [9].

Method Evaluation Model (MEM)—MEM is a method that focuses on the validation of design models and methods for information systems [10]. Validation is comparable in many ways to TAM, however, the focus was on validation by user perception in order to obtain projected performance estimates. However, due to the limited case study evidence supporting this model’s success, and the need for a more solid evaluation structure, this model was not selected.

Validation Square—The method that was ultimately chosen to validate the TiV-Model’s experimental data was the Validation Square [11]. This is a model used specifically to demonstrate the validity of design methods by scrutinising the method in four key areas, as shown in Fig. 4.5.

Fig. 4.5

Validation square

The Validation Square was picked due to its suitability to the field; it was created for the purpose of evaluating design methods, the conditions for validation are far more stringent than the other listed models and the Validation square also goes as far as to validate itself. The model uses both the theory behind the method and the empirical data achieved from experiments to verify the method’s structure and validate its performance. The validation is achieved by challenging the methodology with six logical statements that must be proven true.

• 1. The individual constructs of the method(ology) are valid.

• 2. The method(ology) construct is internally consistent.

• 3. The example problems are relevant to the method(ology).

• 4. The method(ology) is useful to the example problems.

• 5. The usefulness is a result of applying the method(ology).

• 6. The method(ology) is useful beyond the example problems.

These six statements have been proposed to be proven true across three experiments. Statements 1–3 can be demonstrated in an uncontrolled natural experiment, showing that the methodology is designed to be used in the systems engineering context. Statements 4 and 5 can be shown in a controlled group experiment designed to prove the usefulness of the methodology compared with other successful design methodologies. The sixth and final statement is justified in the Validation Square as a “leap of faith” once the other five statements have been proven true; the methodology is as good as valid. However, the project aims to go one step further and use the TiV-Model in an actual CSE scenario, upon which a case study will be built to prove the methodology’s effectiveness.

4.3.2 Experiment 1—Natural Experiment

The natural experiment is a means of showing that the TiV-Model is capable of the core function of producing a complex design solution as part of a mechatronic design project. This will be the first soft implementation of the methodology in a realistic use environment with the purpose of obtaining usage data from the respondent. The experiment is rather simple; a fourth year design engineering student was tasked with the design of a robotic solution for the automated application of icing on cakes, this involved a focus on the design of the mechanism but also included the control and electronics at a conceptual level. Post-project feedback is obtained from the respondent in the form of a qualitative feedback survey, followed up by an informal feedback session, where efforts will be made to obtain suggestions to improve suitability for CSE and usability of the model.

4.3.3 Experiment 2—Controlled Comparative Study

The second experiment aims to prove the 4th and 5th statements of the Validation square. The methodology can be proved to be useful to the example problems by comparing “usefulness” of the TiV-Model to that of existing successful models. Some questions for this approach are:

• What variables constitute usefulness in a design context?

• What successful methodologies are valid for comparison?

• How can an experiment be designed to extract these variables?

While the specifics of which methodologies to use are being planned, it is likely the experiment which will take the form of previously published comparative studies. In a previous study focused on comparing the V-Model with other life cycle development tools, comparison extends no further than the literature and logic [12]. In the experiment it is aimed to demonstrate the hypothesis of these comparisons via controlled environment, where teams of designers will each be using one of three design methodologies in a performance incentivised CSE project. Effectiveness will involve the comparison of output design qualities and efficiency will, much like the natural experiment, focus on qualitative feedback from the designers as users.

To understand effectiveness there needs to be measurable variables generated by the project that can be compared. The TiV-Model’s ideal competency is that it is thoroughly validated, so efforts were made to understand the testing parameters of arguably the most rigorously tested field of all; medicine.

Validation Lessons from Medicine—When it comes to experiment design and testing standards, few organisations are more stringent than those involved in medicine. This is perhaps due to the nature and risk associated with the development of pharmaceuticals. There may be no testing standards for experiments with design methodologies, but methodologies used in medicine can be a useful equivalent benchmark. Frey and Dym [13] discuss in great depth how medicine can be taken as a useful analogy towards the validation of design methods. Analytical methods for laboratories have to follow set standards such as those of the US Food and Drug Administration (USFDA), Current Good Manufacturing Practice (cGMP) and ISO/IEC in order for their methods to be eligible for validation. The standards include parameters that can be tested that reflect the success of the end product. These variables are;

• Accuracy	• Precision
• Specificity	• Limit of detection
• Limit of quantisation	• Linearity and range
• Ruggedness	• Robustness

By setting an acceptable threshold for these quantifiable values, medical researchers can determine effective “success” of a treatment or drug and compare it to other solutions. Design research can learn from this as many of these factors have equivalents in a design context.

The specifics of such comparisons are still up for debate, but on a “closest match” standard. The relevant factors can be determined for the evaluation of design solutions as opposed to medical ones. Table 4.4 provides the context for this evaluation in the medical domain.

Table 4.4 Relevant success measures in medicine compared to design

With these new parameters that determine success, based off equivalents in medicine, it is possible to continue with designing an experiment that will extract these parameters and enable the evaluation of the methodology to take place.

4.3.4 Experiment 3—Case Study

To prove the sixth statement, and determine that the methodology is indeed fit for practical use, a CSE project will be undertaken using the TiV-Model as the methodology of choice. This project will involve the design of a multifunctional mechatronic gripper for fixture on board spacecraft and structures. This is sufficiently within the intended design area of the methodology as a complex mechatronic project, demonstrating its original focus. This project will be documented and examined as a case study, evaluating the success or failure of the project based on similar measurable variables as the second experiment. If the project is successful, the validation will be complete and presentable as proven fact, more than most academic models can claim.

4.4 Next Steps and Conclusions

4.4.1 Interactive Software Integration

Relating to the goals of increased user-friendliness, the capacity to manage large scale projects and integrate with an organisation from top-to-bottom, the TiV-Model will be further developed into a comprehensive methodology and life cycle management system. By doing so, it is possible to effectively tie methods together with their respective tools, for example; evaluation of concepts by weighted convergence matrix is meta-linked to a dynamic group shared file that contains a House of Quality style matrix. This goal is very much inspired by the PLM systems developed by companies such as AutoDesk, and in heavy use by the likes of BAE Systems.

4.4.2 Closing Remarks

TiV-Model is a proposed solution to the question often asked in design research; “why are new models slow to come to practical adoption?” It can be shown that there are various concerns expressed by industry about the suitability of new models as well as their performance and usability. The key issue, however, is the lack of proven effectiveness or validation of said methodologies. By ensuring TiV-Model is thoroughly valid, it can act as an example of breaking down the barrier of acceptance to industry. It also aims to hold its own as a user-friendly and flexible alternative for the CSE industry. Validation of the TiV-Model will use the Validation Square, a suitably stringent means for design method evaluation, to prove that it can perform well. This validation process will encompass three experimental steps that mirror the nature of practical use more with each step. By showing TiV-Model can succeed and even thrive in similar projects, it is possible to remove many of the doubts industry may have about this new academically rooted model. It also works to satisfy future needs; the need for an overarching set of tools, methods and methodologies that encompasses CSE is predicted [14]. The TiV-Model will work towards the goal of a universally compatible architecture to accommodate new design methods and tools. Alternatively, by providing a verified and validated foundation, future method development can springboard from TiV-Model, perhaps even merging as a powerful supplement to the methodology.