Keywords

1 Manufacturability Assessment Knowledge-Based Evaluation (MAKE) Background

According to McCall and Fuller (2018), the Manufacturability Assessment Knowledge-based Evaluation (MAKE) tool draws upon a taxonomy of manufacturability concerns (i.e., life cycle cost drivers), based on functional areas of a manufacturing system (quality, EHS, supply chain, etc.). It exists to identify concerns within areas of each manufacturing system that are impacted by characteristics of the design. These concerns drive the determination of a manufacturability metric, which can be used to compare alternatives of a design at levels from the individual components up to the final assembly.

The Department of Defense (DoD) science and technology communities support the analysis of model-based engineering early into the design process to support decision-making for analysis of alternatives (AoA). Analysis of alternatives is a DoD requirement of military acquisition policy to ensure that multiple design alternatives have been analyzed prior to making costly investment decisions (US Office of Management and Budget 2008). The objective of this research involves the development of a methodology, more specifically a metric, intended to reflect the manufacturability of a product design. The metric may reflect the manufacturability of a total product design and subcomponents or subassemblies of that design. Ultimately, the metric is intended to provide some guidance during AoA or trade-off studies in order to understand the cost drivers or risk inherent to a particular design. Through the evaluation of different design options, users can arrive at design solutions that best meet the mission goals.

McCall et al. detailed the approach to the research which began with development of a methodology at life cycle Milestone C, where fidelity of the design is at a stage where relevant design and manufacturing parameters exist on which to base the development of the architecture for the manufacturability assessment. As the research progressed, more effort has been spent to understand how far to the “left” in the product life cycle a particular design can be assessed. That is, “what is the earliest point in the life cycle timeline at which a useful assessment can be performed?” In addition, there is also the driving question of what is required of the methodology to allow for the assessment of such designs in the early phases where design fidelity is minimal and multiple alternatives are being considered. Figure 1 depicts the strategy of the manufacturability development.

Fig. 1
figure 1

Strategy of MAKE

Full size image

Previous case study assessments described in prior work by McCall et al. (2016, 2017) focused on analysis of systems near Milestone C and utilized the methodology referenced in Fig. 1 as MAKE C. Significant research effort has supported understanding this scope of work and the extent to which the current methodology can be applied at Milestones A and B. Understanding the limitations of the current methodology established a research framework for determining the architecture of the manufacturability methodology necessary for design fidelities inherent to Milestone A.

2 MAKE Current Capabilities to Support Tradespace Analysis

2.1 Existing Methodology and Tool Features

The existing structure of the MAKE tool allows for a user to perform an analysis of alternatives in support of the decision-making process during the product design. As part of the assessment process, the user creates a parts list and subsequently a hierarchical bill of materials (BOM). By asking the question, “What is the impact of a particular aspect of design on a particular aspect of manufacturing?”, the user is able to review each part of the BOM, documenting design concerns and recommendations for each of 21 different interactions. The “Scores” area of Fig. 2 shows the manufacturability interaction matrix (MIM) that guides the assessment.

Fig. 2
figure 2

MAKE tool showing BOM and part assessment

Full size image

After documenting the concerns for a particular part of the BOM, the assessor then assigns a score to each of the interactions. The tool allows the assessor to quickly and easily add, remove, and substitute parts and subassemblies in the BOM. The design’s manufacturability metric is automatically “rolled up” from the BOM giving instant feedback to the assessor as modifications are made. Once the parts have been evaluated, the assessor can evaluate several variations of the design by modifying the BOM as needed and documenting the resulting metric for that variation.

In addition to assigning a score and documenting the concerns for each part, the assessor can upload additional information such as part drawings, comments, and photos to further document and justify the score for a specific part. Maintaining the score and auxiliary information at the part level allows the assessor to:

  • Assemble information needed to define the variation.

  • List the concerns and recommendation for that variation.

  • Assemble data that highlight issues within a variation.

  • Collect, develop, and store other information pertinent to the variation.

2.2 Output Includes List of Prescriptive Measures for Decision-Making

In addition to documenting the concerns identified during the assessment, documenting recommendations for prescriptive measures to mitigate the concern is an important element of the manufacturability assessment. The tool allows recommendations for a concern to be documented while the assessor is evaluating the part and captures the immediate response to remedy the concern (e.g., “using screws in design rather than glue”). This, however, does not prevent assessors or other experts from later revisiting the concerns and modifying or adding recommendations.

The tool allows the assessor to rate the level of concern and effort to implement a recommendation and provides a listing of concerns for a specific part along with the associated recommendations for mitigation. This list can be produced for an individual part or “rolled up” for a part and its subassemblies. The tool also provides a concern versus effort graph, which is a graph of the score of a concern versus effort to implement recommendations to mitigate that concern.

2.3 Reliance upon Subject Matter Experts

The MAKE methodology relies heavily on subject matter experts (SMEs) to evaluate the design areas in need of assessment. Furthermore, the entire manufacturability product assessment is based on the existence or creation of a reliable BOM and a thorough review of each part and assembly within that BOM. The assessor’s knowledge of the best practices within the design area that he/she is evaluating is paramount to the accuracy of identifying the concerns and providing viable prescriptive measures for mitigation of manufacturing risk. The knowledge basis of the SME is essential to the accurate portrayal of a product’s manufacturability risk. While the reliance on SMEs may be acceptable for a MAKE C evaluation, it is a significant concern as one looks toward an assessment for early life cycle designs.

2.4 Challenges of Applying MAKE to Early Life Cycle Assessments

The challenge of developing a methodology (MAKE A, Fig. 1) to assess the manufacturability of conceptual designs at Milestone A, where minimal system design information is available, involves both the use of SMEs and the fidelity of the design at this stage. A primary goal for early life cycle manufacturability assessment is to support understanding the impact of these early design decisions on manufacturing cost, time, and quality. From a practical standpoint, using SME judgment on a design’s manufacturability at this stage lacks feasibility. The most prominent issue relates to the inability to evaluate the large number of unique alternatives in the tradespace environment.

The practice of DFM analysis typically involves having access to design geometry and specifications with some consideration of the manufacturing processes. However, in the conceptual stage, the challenge involves assessing manufacturability based on a significant number of unknown parameters. Pre-Milestone A design analysis primarily focuses on performance, cost, and other metrics for which the methodology to evaluate at this stage has been developed and optimized through years of experience and research.

Another challenge exists in the foundation for the tradespace evaluations. Previous analysis of point-based design processes (using an existing design as a foundation) has demonstrated that later iterations to refine that design solution can be time-consuming and costly and lead to a suboptimal design (Iansiti 1995; Kalyanaram and Krishnan 1997). The ability to examine many designs is made possible by relying on past products and extrapolating information from those past designs. This extrapolation process relies heavily on years of experience and research. The attempt to similarly provide the manufacturability metric for each of these designs is extremely problematic.

Tradespace exploration research by investigated various methods to integrate cost models with tradespace analysis while requiring minimal user interaction. This research used predictive modeling using artificial neural networks to develop a surrogate model. Furthermore, various expert system methodologies could be investigated for implementation with the MAKE A methodology (Viral and Bhushan 2014). Using similar methods could provide the ability to reduce the reliance on SME input and further automate the process for efficient analysis.

3 MAKE 2.0

3.1 The Connection with Tradespace Exploration

By using tradespace exploration in the AoA process, program analysts and decision makers are provided with early system design and development analysis to support understanding of potential system capabilities, gaps, and potential compromises and implications. It informs decision makers regarding opposing system options and the significance of decisions across various missions and objectives.

Tradespaces are essentially a matrix of information which contains design parameters of a variation of a product’s design and the associated results of various analyses for that design. During the product design, in addition to defining various design variations, researchers investigate system attributes in order to derive other parameters for that variation, such as suitability, performance, cost, maintainability, etc. The design parameters and analysis results are collected into the tradespace. The tradespace is analyzed by the teams using common data analytic techniques and system engineering tools (e.g., multi-objective decision analysis) to determine an optimal design.

Generating the manufacturability metric for each design variation of a large tradespace could fully integrate the MAKE methodology into tradespace exploration. However, attempting this with the current tool methodology is impractical due to the requirements for intensive user input for each design. Attempting to use the tool as previously described could require a cost prohibitive amount of effort. The possibility of an objective-based assessment provides the means in which to make the connection between manufacturability and life cycle cost.

Previous research by Buchanan et al. (2018) investigated analyzing set-based design and incorporation of cost estimation using a notional cost model for ground vehicles. The cost methodology in Fig. 3 from Cherwonik (2017) illustrates the flow from physical design parameters (red) combined with programmatic cost drivers (blue) and the cost factors generated from historic data (green) prior to integration. A potential connection point for manufacturability assessment to support early Milestone A tradespace analysis could be made by interfacing with the cost methodology through the “step-up” factors. For example, the manufacturability assessment rating could provide a multiplication factor between the prototype manufacturing (1.04) and production development (2.02) costs. Linking the manufacturability assessment rating to the cost model estimation process could provide decision makers with additional cost and manufacturing insights while making system decisions for a program.

Fig. 3
figure 3

Cost methodology

Full size image

3.2 Transitioning from Subjective to Objective Analysis

As stated previously, the current MAKE methodology is highly subjective due to the extensive use of SMEs’ knowledge of the manufacturing process. The desire is to remove as much subjectivity as possible and to create an objective system. The advantages of such a system are apparent, such as:

  • Remove biases that are the result of isolated events in the SME’s career.

  • Available tradespaces with manufacturability metric, if the designs have the fidelity required to analyze.

  • Ability to perform the assessment without years and years of experience of an SME.

  • Ability to mix manufacturing processes without multiple SMEs participating in the assessment.

Converting to an objective-based manufacturability assessment will require creating computer models that assess the design against a set of manufacturability criteria. The fact that the assessment will be used in the tradespace analysis will also drive the decision as to the type of models needed to satisfy both requirements. Several possible techniques that could be applied include:

  • Physics-based models – These models would calculate the manufacturability score of alternative design options based on certain physical characteristics of the design (e.g., geometry), thus providing the basis for an objective manufacturability assessment model. However, due to the lower fidelity of data available in early life cycle, they would not be realistic for use in the tradespace analysis. In addition, even the basis of a physics-based model would require some level of heuristic knowledge in order to truly identify the manufacturing impact.

  • Expert system (rule-based) models – These models would utilize expert systems, where the SMEs’ knowledge and experience for an interaction are coded into rules and then passed through an inference engine to obtain the final metric. Unfortunately, expert systems are not appropriate for use where much of the input is unknown or vague due to the rigid structure of the system.

  • Artificial intelligence/machine learning (AI/ML) models – These models use modern AI techniques to build models that can handle the “fuzziness” of the input. Bayesian networks, a type of probabilistic graphical model, appear to be very promising. The networks can better deal with missing or unknown information. The Bayesian network also assigns a probability to the computed value providing a measure of confidence to value. Other techniques such as neural networks not only provide the model but also are able to “learn” when unknown conditions arise.

  • Combination/ensemble models – These models use combinations of various models, attempting to use the best part of each technique used. One such possibility is the expert system combined with neural networks, under the assumption that the neural network would learn new rules to handle the fuzziness of the input.

While all the above techniques could possibly meet the requirements, the pure AI/ML techniques may be the most promising in order to meet the requirements of an objective, early life cycle manufacturability analysis resulting in a manufacturability metric that could support for tradespace analysis.

4 Conclusions

While the current version of the MAKE methodology is suitable for a Milestone C assessment, there is a desire to perform early life cycle assessments at Milestone A or pre-Milestone A timing. A secondary goal is to transition the methodology from a very subjective process to an objective assessment, which will integrate the methodology into tradespace environments. This will serve to optimize the assessment process by removing biases from SMEs and reduce the time and expertise needed to perform the assessment.

It is possible to achieve these goals, by using various computer models such as artificial intelligence, machine learning, expert systems, or physics-based models. Each of these options has positives and negatives associated with them. Another option is to use a combination of the aforementioned models. That transition would, theoretically, allow the framework to provide the manufacturability metric for any number of design variations in a tradespace.

Since most tradespaces are created to support the acquisition of new products, solving the early life cycle challenge of low design fidelity is paramount. More research will need to be done in the area of early life cycle assessments.