This work explores the design and application of argumentative-driven assessments (ADA) within the context of challenge-based learning as a mechanism of feedback and evaluation of knowledge and competencies. It discusses a combined model that integrates the essential elements of challenge-based learning with the Toulmin argumentative structure. This work attends the need for a step-by-step guide and overall recommendations for the ADA design (including an evaluation rubric) that includes a mid-term ADA in the Mechanical Vibrations challenge based on the proposed guidelines. The ADA's implementation showed that the students developed competencies related to critical thinking, problem-solving, engineering design, and data analysis. The students proposed different solutions to the open-engineering problem while backing each assumption with relevant information and calculation. ADA offers the students the opportunity to demonstrate their ability to predict and discuss different scenarios in the challenge. The ADA-challenge design and implementation's results suggest that contextual argumentation with immersion is essential to promote a sense of purpose in the students. ADAs are essential tools that facilitate identifying and evaluating the development levels of various competencies while providing a forum for students to discuss, express, and apply acquired knowledge. ADAs can also improve the students' learning experience by providing continuous feedback in the context of conceptual and structural scaffolding. The ultimate goal is to give the students the relevant feedback that correctly identifies their achievements and monitors their progress and opportunity areas in their learning process.
In any teaching–learning technique, the evaluation of competencies and knowledge is challenging for teachers. The design of an evaluation system is a key element to verify that the student is learning. A course's overall evaluation system comprises self-evaluations, co-evaluations, diagnostic evaluations, formative evaluations, and summative evaluations. While diagnostic evaluations are designed to estimate the future academic performance and potential of the student given their knowledge background in a subject, the formative evaluations seek to systematically assess the learning process of the student to improve it and to verify the development of the competencies needed to solve a given problem [1, 2]. Formative evaluations help the students acquire self-knowledge, learn responsibility, and develop self-esteem. Along with their professor and peers, the students develop action plans to reinforce and apply concepts learned [3].
Formative evaluation is a fundamental mechanism to continuously validate knowledge acquisition and develop competencies while providing relevant and timely feedback. This type of evaluation could be seen as a teachers' instrument to evaluate the transfer of knowledge and adapt the learning activities to meet the students' expectations, objectives, interests, and learning needs [4]. Therefore, formative assessments are synonymous with feedback and adaptation.
A proper formative evaluation system should contain continuous evaluation instruments, such as small activities, material review, and verification evaluations that give the class a great sense of dynamism while encouraging the review of additional material [5]. A typical form of quick and constant evaluation for a recently learned key concept is the use of brief quizzes (duration time: five minutes or less) weighted as extra points for future aid. These extra credit quizzes have two main purposes: 1) to evaluate and document the continuous learning process of the students and 2) engage, motivate, and encourage the students in their learning process. Moreover, this type of evaluation allows gamification elements and e-quizzes to create a virtual community and competition [6].
Formative evaluation is a key component in the development of competencies in various student-centered pedagogical models. In models based on the Challenge-Based Learning (CBL) framework (such as the TEC21Footnote 1 educational model), engineering students assume the role of experts who need to solve real-world industrial problems (challenges). In a CBL framework, the students apply theoretical knowledge and empiricism to develop and implement solutions for various high-impact industrial problems of the twenty-first century [7, 8]. Under this model, the students acquire, strengthen, and apply the necessary competencies, skills, and engineering knowledge to reach a solution. A key component for a successful implementation of the challenge is the context or background of the problem. This context must be meaningful and engaging, so the students feel that the knowledge learned has a meaningful purpose [9, 10]. Argumentation and contextualization in a challenge become crucial in students' motivation and immersion. Formative assessments and evaluations must be oriented to develop, apply, and evaluate competencies aligned with the challenge's context. The competencies and assessment criteria should be defined as clearly as possible, considering that competencies are achieved gradually. Thus, an instrument of a formative evaluation system is needed to track the level of development of different competencies in a CBL framework.
Argumentative-driven assessments or examinations (ADA) consist of a series of research-based queries that the student answers, explaining scientific/engineering phenomena, backing the arguments with calculations, providing evidence from trusted sources (research), and applying concepts, competencies, and skills acquired during the course [11, 12]. In an ADA, the students are encouraged to demonstrate conceptualization and substantiate claims as they deliver coherent and convincing reasoning to validate their problem resolution [13]. Thus, engineering exams should become academic/industrial challenges that lead the students to reflect, find missing information and data, deduce, propose solutions (innovate), justify the answers, and promote scenarios [8, 14]. Argumentative examinations evaluate competencies, such as content knowledge, reasoning, and critical thinking [15]. Students validate their knowledge through a series of logical and mathematical arguments, having a forum to interpret their acquired knowledge. A proper scientific argumentation gives the professor insight into the students' mastery level in a given subject. Moreover, ADAs allow personalized feedback that helps students identify their achievements and those elements and concepts that need improvement. It is noteworthy that the CBL framework uses mainly conceptual, metacognitive, and strategic scaffolding [16] to reinforce the students' argumentation during the learning process.
Although argumentation has been used as a form of competency evaluation in engineering courses, most of them are well-defined and structured problems with data or information given to the students. In this work, an argumentative-driven assessments (ADA) as an evaluation tool under a challenge-based learning (CBL) framework is proposed to address the need for the proper evaluation of competencies in engineering courses while providing an opportunity for the students to search and interpret missing information while backing each step taken (through arguments). Moreover, this study presents both challenge-based learning elements and the general structure of argumentative-driven assessments with a general step-by-step design guide and an evaluation rubric. A case study of a mid-term ADA design and implementation of a problem solution to a challenge posed in a Mechanical Vibrations course is presented. It includes discussing and identifying the key elements of the argumentation structure underlying the students' proposed solution.
2 Challenge-based learning and argumentation
Challenge-Based Learning (CBL) is a student-centered learning technique developed by Apple Inc. [7, 17] in which students are asked to solve real-world problems in a collaborative and hands-on framework. Under this learning technique, the students develop and strengthen competencies and knowledge acquired during lectures and asynchronous activities to solve twenty-first-century problems [7, 8, 18]. CBL allows the students to apply fundamental engineering knowledge in an active learning environment to address socio-technical problems or open engineering problems. CBL not only promotes critical thinking as a competency but also fosters innovation, creativity, teamwork, self-learning (learning to learn), and problem-solving skills, thus, moving from a theoretical to a more pragmatic approach. These competencies are required to solve various open-engineering challenges.
CBL involves the contribution of three major stakeholders for its design and implementation:
(1)
Academic mentor (Professor): Responsible for the course design and the required activities, the synchronous and asynchronous content, guiding students to develop competencies, and co-designing the challenges with the industrial research mentor.
(2)
Industrial research mentor: Responsible for co-designing the challenge and aligning it with the course syllabus and the company's primary objective. The industry mentor is a leader and expert in solving real-world problems related to a company, research field, or industry and provides continuous advice and feedback based on their experience while acting as the "client" [19].
(3)
Students: The main actors during the challenge. The students must engage in their learning by applying the knowledge and concepts learned from lectures to propose a feasible solution that meets the industrial mentor's requirements. Students assume different contextual roles and responsibilities as they undertake the challenge.
During CBL, the students receive mentoring sessions from the academic and industrial mentors and receive continuous feedback from them on their progress reports. The elements that comprise the challenge are presented in Fig. 1.
The first element, Big Idea—Challenge, involves the problem context, the principal issues to solve (challenge), the main objective proposed by the industrial mentor, and the purpose and significance. The challenge should be engaging and immersive and instill a sense of purpose [9, 10, 20, 21]. Moreover, this element encourages the students to discuss what information and concepts are needed to start solving the challenge.
The second element, research, and road map can be conceived as a hybrid element of the challenge. The academic and industrial mentors could provide different conceptual queries or arguments that demonstrate the minimum requirements and steps that the students need to follow. Basic information, elements, or problem schematics are given to the students to begin. The students are encouraged to research various missing information, concepts, or materials needed to propose a solution. Each decision and all information provided by the students need cited sources and arguments that validate their methodology.
During the third and fourth elements of the challenge, solution proposal—refinement, the students actively work on the challenge by following the proposed methodology and answering each essential question. The students propose different solution scenarios based on assumptions and required calculations. Like the second challenge element, every assumption to simplify the problem and all calculations must be backed either by a reference in the literature or estimation and discussion of a parameter within a valid range (provided by the industrial mentor). Once the students obtain the first approximation of a solution, they consult the mentors to validate the simplifications made and then make the adjustments necessary to approximate a good solution to the challenge that meets the company's requirements. This can be done, for example, by including more variables, higher degrees of freedom, and nonlinearities, among other things.
The last element of the challenge is the solution presentation, which involves composing a technical report detailing each element, the decisions made, calculations, discussions, and analysis for each proposed solution. Moreover, the students are encouraged to present the results decisively and convincingly to a panel of experts (mainly to the industry experts and mentors).
The elements mentioned above follow a logical outline. All challenge components are subject to logical argument and critical thinking, demonstrating the students' mastery of these key competencies. The argumentation follows the Toulmin argumentative model [22] shown in Fig. 2.
Fig. 2
Structure of Toulmin's model of argumentation [22]
Facts/data are required to validate a certain statement/hypothesis; warrants are the reasons needed to justify the link between facts and conclusions (logical argumentation). These justifications must be supported by some basic knowledge (backing) under a certain set of conditions in which the claims are or are not valid (qualifier). Even though the students reach a conclusion or claim, their arguments are subject to a rebuttal in which counter-arguments/data are provided to indicate the boundaries and limitations of their conclusions.
Furthermore, facts/data are explicitly mentioned throughout the challenge's argumentation, while warrants could be expressed explicitly or implicitly. Both elements must be present in each argument by the student. In contrast, backing information/calculations and modality may not be used in each argument. In other words, a backing is only used when the warrants cannot be directly accepted and need further discussion [22, 23]. Rebuttal arguments can be seen as the exemption of the rules or different perspectives that contribute to the claim's acceptance or rejection (because it may require further study or validation). Consequently, the validity of a claim or concluding argument could be strengthened or weakened (in case of rejection) by a modal qualifier.
Argumentative-driven reports and assessments (ADAs) could be used to easily identify and evaluate the development of competencies in a CBL framework. The combined model of challenge and Toulmin argumentative elements developed is showed in Fig. 3. It can be seen that different elements of argumentation characterize each element of the challenge. Rebuttal arguments are typically observations from the mentors' feedback that could take the style of what-if questions. Initially, the facts/data and warrants are linked to the first solution proposal or the first approximation of the challenge. Next, come the mentors' rebuttals, needed to improve the solution proposal, strengthen the discussion (modality), and reach valid conclusions for the challenge.
Fig. 3
Argumentative-driven structure in a challenge-based learning framework. The arrows represent the main path of CBL approach, while auxiliary lines represent the steps or activities needed to reach the next milestone. These activities are crucial points for argumentation (italic font) and for feedback
The assessment of ADAs and technical reports gives the students the necessary feedback to reflect on their knowledge and limitations and promote the search for missing information and data to improve their methodology. The students can then auto-assess their proposed solution/model, improving it considering different scenarios and perspectives (see dotted lines in Fig. 3). Thus, we highly suggest conceptual and strategic scaffolding during the challenge to help students improve their argumentative skills [11, 16, 24].
Lastly, it is worth emphasizing that ADAs should be designed in a CBL framework that demonstrates the strong relationship between the knowledge learned through lectures and the various applications, perspectives, critical thinking, and solution approaches that simplify the proper evaluation of competencies and knowledge acquisition. Moreover, critical thinking and the ability to argue are key competencies evaluated in an argumentative assessment. These competencies are developed through continuous feedback and examples from the professor.
3 ADA design guidelines
This section presents a series of steps (guidelines) and recommendations for the design of an argumentative-driven assessment (ADA). Figure 4 provides a quick guide and reference of the elements required by each step and an easy tool to design an ADA.
Step 0 Planning—Planning is the key to a successful ADA design. The professor and the industrial mentor co-design a challenge segmented into thoughtful, conceptual questions that emerge from a complex system. ADA design planning starts by defining the object of study: Which system of the company presents a problem that requires analysis and solution by the students? Once the object of study is defined, it is necessary to determine which competencies and concepts are required to resolve the challenge. The contextual competencies and concepts to be evaluated must align with the challenge and the course material provided to the students synchronously and asynchronously. The challenge must promote critical thinking skills while strengthening the students' understanding of concepts and knowledge.
ADAs can be designed as a collaborative or individual activity, depending on the challenge's complexity and scope. For collaborative assessments, the challenge must be sufficiently significant in scope that the team members adopt different roles in contributing to the challenge solution. Individual assessments typically involve splitting up the challenge into smaller problems, so the component of each student's challenge is shorter.
Lastly, the professor must define the evaluation criteria and the number of points that each question will weigh in the final assessment score. Evaluation sheets that rate the level of mastery of each competency are highly suggested. Table 1 shows a template for the evaluation sheet developed by the authors. Both the professor and the industrial mentor complete these evaluation sheets to arrive at a global perspective of the students' performance. Moreover, we suggest that the professor shares the evaluation criteria and rubric with the students, so they know what is expected from them in their solution proposals.
Table 1 Competency evaluation rubric used in the ADA
It is worth emphasizing that ADAs in the CBL framework are open engineering problems with possible multiple solution proposals or conclusions, depending on the students' analyses and assumptions. Part of the evaluation assesses how the students reach the solution, including their arguments and decisions that back their solution proposal to the open problem.
Step 1 Context and Motivation—The first element of the assessment describes the students' challenge. In this introductory phase, elements of "role play" could be added to encourage student engagement. Next follows a statement of the company's overall problem/big idea and the challenge that the students are expected to solve. It is noteworthy that context and background play an important role in the students' engagement and motivation. Therefore, the challenge must allow argumentation to easily identify the relevance and implications of the challenge and the necessity to solve it, giving the students a sense of purpose in their work. Likewise, it is recommended that the company or the professor provide some empirical insights on the challenge, perhaps discussing their experience with the problem, providing information about faulty components, or imparting approaches to a numerical model.
Step 2 Additional Information—Following the context and problem statement, additional information must be provided to the students. Flow or process diagrams, for example, might illustrate the problem's basic elements as a visual aid for the students to understand the problem and the system under analysis better. Similarly, equations, models, or additional explanations of the diagram's components should be given to the students as a starting point. However, this does not imply that all the information should be given to the students. ADA requires that the students make assumptions based on their research and data discoverable in the literature. Thus, the additional information might include at least one reference to the literature or reference material as a starting example for their research. One common mistake in ADA design is to assume that the students know how to obtain or extract information from research papers, books, and proceedings. A means to minimize this problem is to prepare workshops and exercises that teach research and critical thinking skills. (See the template for analyzing research papers developed by Paul and Elder.) (2003) [25].
It is worth noting that the industrial and professor's context and additional information follow the Toulmin model, i.e., in these sections, facts, data, warrants, and basic backing information are provided to the students.
Step 3 Conceptual Questions—The evaluation of specific concepts, competencies, and skills learned throughout the course must be carried out after designing thoughtful assessment questions aligned with the challenge. These questions must evaluate the basic competencies needed to solve complex problems and prepare the students for arguing rebuttals. For instance, it is required to evaluate whether the student can propose a numerical analysis in MATLAB for a different phenomenon. Hence, the assessment question designed must evaluate 1) the overall understanding of the phenomenon, 2) the capability of the student to represent i the system to solve in mathematical expression, and 3) their numerical and software skills in arriving at a solution. Moreover, these questions could consider simple elements or sub-systems of the challenge under a controlled environment with specific data. The main objective is to evaluate the students' understanding of the concepts and their competencies.
In a conceptual question, the students have the limited information given in the context presented to encourage them to search for missing data or find similar models in the literature to make the assumptions needed to solve the question. It is important to remember that the questions and the students' solutions must function in the context of the challenge and meet the overall challenge conditions.
Step 4 Rebuttals (What if questions)—After the conceptual questions, a series of rebuttal conditions or questions (the Rebuttal element in the Toulmin model) are given to the students to evaluate their capability to identify and use the information to propose methodologies and approaches to solve complex systems. The rebuttal questions do not necessarily contemplate the students reaching a specific numerical solution. Instead, the students are asked to identify new elements, make different assumptions, propose a new methodology, and evaluate the previous solution in terms of feasibility and risk minimization. The students are encouraged to propose a solution or design algorithm, using flowcharts, diagrams, schematics, or graphics to back their argumentation. One key idea to consider when designing rebuttal questions is to create a forum in which the students express, discuss, and apply the "learned" knowledge from the conceptual questions in the context of the challenge. The rebuttals must be open questions in which the students argue, discuss, and back with information the viability of the new scenario proposed by the professor or the industry mentor in their feedback. It is expected that the students provide warrants and backing information to reach a conclusion.
4 ADA implementation: case study—mechanical vibrations challenge
Argumentative-driven assessments (ADA) were carried out in a case study in the Mechanical Vibrations 2020 course (Tecnologico de Monterrey, 2020) as part of the mid-term and final evaluations of the course. This course uses challenge-based learning (CBL) and Flipped Classroom (FC) as the main teaching–learning techniques. Initially, the students solve specific or model problems to apply the concepts learned and develop the competencies necessary to solve a real-world problem. However, these problems were structured as challenges with increasing complexity as the course approached presenting the main industrial problem. The students had to present a specific numerical result and the methodology and decisions to reach their solutions.
The following section presents an argumentative-driven assessment applied in the mid-term exam of the Mechanical Vibrations course. For this assessment, the students had one week to reach a solution and justify each decision and assumption. Moreover, the competencies evaluated involved problem-solving, interpreting physical phenomena, applying engineering criteria, and critical thinking. The specific skill required to solve the ADA required numerical modeling and analysis using MATLAB or similar software. The study object was the absorber and suspension design of vehicles with 1, 2, and 4 degrees of freedom (DoF).
Note that ADA implementation is not restricted to a specific course, study program, or degree. However, its application should be carried out for courses that follow a challenge-based learning model since ADA's main contribution is to evaluate competencies to solve real-world problems. ADA complexity depends on the student's level and degree.
4.1 Challenge context and motivation
The context of the mid-term Mechanical Vibrations Challenge Assessment 2020 (MVC) is presented below. The role-playing elements are noteworthy; also, the challenge is an open-engineering problem directed to a plausible design proposal from the students.
Let us assume that you are the Vibrations Engineer in a prestigious heavy-machinery company responsible for designing a shock absorber system for the company's newest product. The company has received many complaints from current customers. They argue that the suspension system is "too hard" and that they expected better "comfort" when riding on uneven surfaces (roads, which can resemble sine wave signals). Your job is to conduct a study to redesign the shock absorber system and select the appropriate system components (c and k). The redesign should please upset customers and improve the suspension system to feel "less hard".
4.2 Additional information
The additional information with facts, data, warrants, backing information, and diagrams is presented below for the MVC course:
The company provides you with a series of technical materials and elements (shown in the figure below). This figure illustrates the different ways to model the suspension system of the heavy machine. The models vary from the simplest one with only 1 degree of freedom to a more complex model. A) A first approach to the damping system's design problem is to consider the simplest model, corresponding to one degree of freedom (Fig. 5A). This widely studied model corresponds to a mechanical vibratory system given by the differential equation Eq. 1 (Warrant), where variable m (mass) corresponds to the total weight of the vehicle; k corresponds to the stiffness of the four springs (two for each wheel), and c corresponds to the four total shock absorbers, as shown in Fig. 5A (Facts). The force F0 represents the amplitude of the forces caused by a certain profile of disturbances that excite the damping system; these are caused by the irregularities in the surface (Facts, Data). Initially, it can be considered that the excitations have an oscillatory harmonic shape (Eq. 2). Therefore, the vibratory system would take the form of Eq. 3 (Warrant, Backing).
Fig. 5
Additional Information: different models for the suspension system used in the ADA
It is worth noting that, during the six-week, real-world, industrial challenge, the additional information given to the students is limited (not as complete as the above mid-term challenge). Therefore, the students must propose the assumptions, guiding activities, and methodology, and properly solve the problem with their justifications. This includes approaching a complex problem by dividing it into simpler components or systems and deciding the equations they need to use. In contrast, for the mid-term ADA, the academic and industrial mentors design specific questions that relate the concepts learned during the course with the challenge. These questions must support evaluating competencies needed to solve complex problems, as described in Sect. 4. The following questions were used in the mid-term ADA. In the questions below, additional information is given to the students; however, questions are designed to encourage them to use each argumentative element of the Toulmin model (Figs. 2, 3).
(I) Please calculate the values of c and k for the vehicle whose weight is 370 kg (Data). Assume that the range of F0 forces varies from 500 to 1000 N, and the range of frequencies varies from 20 to 200 Hz according to the surface of the uneven road (Need Backing Information). (II) Perform the analytical study. Use MATLAB simulations to demonstrate that your proposed redesign causes the damping coefficient to be over-damped by a slightly larger damping factor than 1, for example, 1.05 (Need Backing Information and Modality). (III) What is your opinion about designing the suspension system if we consider a damping factor value < 1, such as 0.95? What would the behavior of the forces specified above be? (F0 of 500–1000 N and the frequency range of 20–200 Hz) (Need Warrants and Conclusion). (IV) Study the Bode diagram (frequency response) as an analytical tool and present the simulation's various responses over time. (V) Finally, justify your results extensively and establish your conclusions and technical analysis (Need Backing and Rebuttal Information).
These questions are framed so that the students discuss their decisions and search for the information needed to improve the ride "comfort." Even though these questions seem like straightforward calculations, the students must answer them to meet the challenge conditions.
4.4 Rebuttals
A second and a third set of questions were used in the mid-term ADA to evaluate the students' ability to identify, evaluate, and discuss higher-level-complex systems. For these questions, the students will not reach a specific numerical solution; however, they will be asked to identify the new elements needed to reach a solution, such as the variations between systems, newer assumptions, and the relationships and similarities among components, as described in Sect. 3. Additionally, these questions aim to evaluate the students' ability to predict how the system will react or change with a given perturbation or variation in the parameters without using calculations. The following second set of questions was used in the mid-term ADA.
(B) A second approach to the problem of designing the damping system is to consider the moderately more complicated model, i.e., to use the two-degrees-of-freedom model (Fig. 5B). (I) Obtain the differential equations of this two-degree freedom model. (II) Explain what the two masses represented in the model correspond to, and the constants, k, and c. (III) Assume the same F0 force conditions when using the model of one degree of freedom (Eq. 2). (IV) What would be your process for designing the shock absorber system for this two-degree of freedom model? Just describe how you would solve it conceptually and qualitatively. (Do not solve it, just describe your approach.) You can use a flowchart, process chart, or a graphic scheme to support your explanation.
Finally, the third set of questions used were:
(C) A third approach to design the shock absorber system is to consider the more complex but more complete model, which would mean using the four-degree-of-freedom model (Fig. 5C). (I) In your opinion, what would be the contribution of having a complete model? (II) How would the analysis be modified? (III) To which physical parameter corresponds each mass, spring, and shock absorber shown in Fig. 5C? Please justify each question technically.
These questions are framed so that the students discuss their decisions and search for the information needed in terms of the ride "comfort" that needs to be improved (Warrants and Conclusions). Even though these questions seem like a straightforward calculation, the students must answer them to meet the challenge conditions.
4.5 Students' solution
The next step is for the students to solve the challenge while documenting their methodology, decisions, calculations, and conclusions, following the basic argumentation steps presented in Sect. 2. Initially, the students need to assume certain values and establish the facts and warrants while backing every assumption and calculation made with the basic knowledge learned in the course. Moreover, the students establish the basic methodology or steps they will follow to reach each inquiry's solution. This corresponds to the big idea, challenge, essential questions, and guiding activities sections of the industrial challenge (Fig. 1). The first solution to the conceptual question solution is shown below.
To determine the total mass, one must add the mass of the rider and the vehicle chassis. Assume a weight of 100 kg (Data) for the rider and 370 kg for the vehicle chassis, so the total weight is 470 kg. This information was obtained from the company clients' average weights (Backing). So then, m = 470 kg. The natural frequency of vibration is extremely important, and it is a fundamental parameter in calculating the vehicle's suspension parameters (Warrant, Backing). Once we determine the natural frequency, we can directly calculate the spring constant, k, using the equation wn2 = k/m. Now, the problem is summarized to find an adequate natural frequency of oscillation of the vibratory system. We consider that the human body's sensitivity is between 3 and 20 Hz, so we want to avoid the oscillations occurring in this frequency range. It is where the human being feels more sensitive to vibrations.
The above argumentation clearly states the basic data needed to start the computations. Even though the students begin by arguing the importance of the human body's sensitivity, more information is required, specifically the sensitivity of three main body parts: chest plus stomach, head, and internal organs. Therefore, the following argumentation from the students (in the form of a schematic diagram, Fig. 6) provides the information needed regarding the vibration modes from the human body while discussing and qualifying each element in terms of the solution (Modality, Rebuttal, and Warrants to reach a Claim). Research-based inquiries such as the above encourage critical thinking while the students develop competency in information searches of peer-reviewed journals, technical reports, proceedings, and others.
Fig. 6
Students' analysis: schematic of body vibrations modes. Each element provides the needed data
After all the required information is stated and backed, the students begin making decisions concerning the main challenge, namely, the passenger's comfort while riding in the vehicle. The students present a series of calculations and computations to validate their assumptions and reach a solution. This corresponds to the first approach to the industrial challenge's solution and claim (Figs. 1, 3).
We have two options: designing a suspension with a natural frequency greater than 20 Hz or selecting a natural frequency of less than 3 Hz to avoid the human sensitivity frequency range (3–20 Hz). For example, if we select a vehicle suspension with a natural frequency between 1 and 3 Hz under this criterion, then we would have the spring constants of 18.55 kN/m (natural frequency of 1 Hz), or 74.4 kN/m (natural frequency of 2 Hz), or 166.9kN/m (natural frequency of 3 Hz), derived from the equation: \({\text{k}} = {\text{m}}\upomega _{{\text{n}}}^{2}\). (Remember that total mass m = 470 kg).
Once this selection has been made, we can calculate the appropriate damping factor that allows an adequate and smooth suspension, \({\text{c}} = 2\rm{\varsigma \omega }_{{\text{n}}} {\text{m}}\). That is, it meets the comfort criteria of the rider. Remember that the rider has a certain tolerance to vertical vibrations; the most sensitive range is between 3 and 20 Hz (Modality). To illustrate this analysis, consider Fig. 7.
Fig. 7
Students' analysis: results of damping factor and natural frequency evaluation
It is noteworthy that the students' results shown above illustrate every methodological step, rebuttal condition, and calculation in an argumentative structure that focuses on the design criteria of comfort. Every decision made by the students is backed by computations and discussions of their implications for the clients' comfort. The arguments illustrated in Fig. 8 respond to the inquiry about the damping coefficient; the students analyze its performance considering the obtained results.
Fig. 8
Students' analysis: evaluation of vehicle suspension and performance
The following graph shows that the proposed redesign causes the damping coefficient to be over-damped by a slightly larger damping factor, above 1, as requested (Backing, Claim). Moreover, the proposed vehicle suspension and its performance are shown in Fig. 8 (Backing, Modality, and Claim).
Finally, for the first set of conceptual questions, the students reached a solution that satisfied the company and their customers' demand by showing the suspension performance at the different conditions (damping factors). The students gave the final suspension design recommendations for the one degree-of-freedom system.
As it can be seen (Fig. 9), if we consider a damping factor below 1, the overdamped suspension systems protect the driver from hard vibration; however, the response is slow, and, consequently, the rider's comfort does not improve (Rebuttal, Warrants, and Claim). Therefore, it is recommended that the vehicle suspension be designed using the values mentioned above to increase the rider's perception of comfort (Conclusion).
Fig. 9
Students' analysis: the proposed vehicle suspension
Note that the students make a recommendation or solution proposal. In this challenge, the students are exposed to multiple generations of possible solutions involving a design that considers a vibratory mechanical system's relevant parameters. Students are encouraged to present their design proposals (responses to the challenge) based on the validation of their decisions using solid engineering arguments associated with mechanical vibrations (damping, vibration isolation, response, and response speed) and on important physical quantities in a vibratory mechanical system (displacements, speeds, and accelerations in each mass, or inertia). In this cognitive-learning exercise, students are exposed to real engineering design situations where they must make decisions and propose different solutions or scenarios. There is no one particular answer but rather a set of valid answers within a domain of solutions presented in a computational model.
For the conceptual questions (the second and third sets of ADA questions), the students identify the new components and their implications. They evaluate the information needed to propose a methodology to solve a complex problem: the two and four DoF systems. During this part of the evaluation, the professor uses conceptual and strategic scaffolding to link concepts and knowledge previously acquired to new models and provide insight into the complex models. Students are encouraged to critically argue the methodology developed in the previous solution and its validity for complex models. Critical thinking is crucial for the successful performance of the students during the ADA. The students' answers, shown below, clearly indicate the new components added while providing insight into how the system can be modeled and how different solutions change the dynamics. The elements of this argumentation correspond to the rebuttal, warrants (and backing if required), and conclusions. Thus, the ADA method can be seen as very useful in the solution, improvement, and solution implementation of the industrial challenge (Figs. 1, 3). Students' argumentation:
The sprung mass (mvehicle-mrider) represents the mass of the vehicle + the rider's mass. The 4kstrut represents the vehicle's resorts (stiffness), and the 4cstrut represents the vehicle's dampers. The unsprung mass (2mwheel) represents the mass of the wheels. Lastly, the 2ktire represents the equivalent stiffness of the tires. The displacement of the sprung mass (mvehicle + mrider) will oscillate with the same frequency of the input (Force) but with a different magnitude and phase. Even though the new systems consist of a two DoF model, with the above elements in consideration, the shock absorber design follows the same methodology as the one DoF case (assuming the same rider weight, same conditions of overdamping coefficients, and the human vibrations modes).
Finally, the third set of questions involves the rebuttal arguments of the implications of a complete model. The students had been prepared with two DoF systems and, with the given information, could analyze the difference between systems, parameters, and equations (involved). Moreover, the students can argue how the system could consider different dynamics not taken into account in simpler models. The final students' argumentation is showed below.
Having a complete model means the addition of two more degrees of freedom. This may result in analyzing other motions and factors, such as the acceleration and braking motions, the dynamics of the chassis, the tires' dynamics, and the effect of vibrations on human riders. Moreover, the methodology proposed for one and two DoF will be modified since the system has two extra degrees of freedom, and it requires at least four differential equations plus a pitch motion. These equations must include the following parameters: the mass of the rider (mrider), the stiffness of the driver (krider), the damper of the driver (crider), the mass of the vehicle chassis (mvehicle), the stiffness of the suspension (2kstrut) and dampers (2cstrut), and finally, the masses of the wheels (mwheel).
Overall, the application of a mid-term ADA resulted in the proper evaluation of the students' competencies. The students generally showed their capability to solve real-world open engineering problems by segmenting them into simpler models while validating their knowledge that supported their facility in proposing and developing methodologies for models of varying complexity and precision. The students became aware of the importance of applying physics and mechanical laws when they must generate simplifications and hypotheses to validate their proposed models.
Further analysis of the ADA showed that the students developed critical thinking, teamwork, and problem-solving competencies. The students identified the problem and applied physical and mathematical principles in different engineering design criteria in problem-solving. Additionally, the students' argumentation made it evident that they developed competencies related to acquiring and applying new engineering knowledge to support complete and validated solutions. (This is related to the conceptual and strategic scaffolding discussions during lectures with the professor.)
5 Conclusions and final recommendations
This work described the elements and criteria of the design and application of argumentative-driven assessments (ADA) in a challenge-based learning framework. During the project, we continuously assessed the students' development of competencies while documenting their learning process. Different components of the argumentative assessment based on the Toulmin model were adapted into the challenge structure and context. The key components of a successful implementation of ADA in a CBL structure are planning and contextualizing the problem background. ADA planning involves the object of study, the competencies to be developed by the students and evaluated, the duration of the assessment, and the evaluation criteria. Contextual argumentation and immersion play a fundamental role in engaging the students and giving them a sense of purpose. Students are motivated to solve complex problems because all the knowledge and skills they acquire are meaningful and purposeful.
A mid-term ADA was carried out in the Mechanical Vibrations course as a case study for STEM students. The students were asked to design a suspension for a vehicle that would improve rider comfort. The students successfully proposed different solutions, backing each assumption, additional information and data, and calculation. Moreover, the ADA results showed the students' ability to predict and discuss how the system will be affected by changes in parameter values, additional forces, or systems with extra variables (acquisition of new knowledge).
For a successful ADA design and application, the professor should consider the following recommendations:
1.
Challenges must be aligned with the competencies and concepts to be evaluated.
2.
The evaluation rubric must assess the level of mastery of each competency. The professor and the mentor could perform this evaluation together.
3.
Each level of mastery must be defined and provided to the students for them to know what is expected and understand the levels.
4.
A complex challenge that requires more than six weeks to resolve could be evaluated by dividing it into smaller ADA assessments, such as a mid-term evaluation. The duration of the ADA must be congruent with the complexity and scope of the challenge.
5.
The context and background of the real-world challenge should be novel and engaging, preferably enriched by the industrial mentor's insight; it should include role-playing elements.
6.
The additional information and the conceptual questions give the students clues about how to begin tackling the challenge.
7.
Emphasize the open nature of the challenge. The students must be encouraged to conduct information searches, form assumptions, make decisions, and propose a solution under open conditions.
8.
Before engaging in the ADA, design activities that foster literature research and critical thinking.
9.
Conceptual questions should point to a straightforward evaluation of a concept or skill.
10.
Rebuttal arguments and questions should promote discussions among team members.
Finally, ADAs are useful tools for the evaluation of competencies, skills, and concepts. ADAs could be implemented in any course or discipline that uses a challenge-based learning framework as the main teaching–learning technique to develop competencies.
Schildkamp, K., van der Kleij, F.M., Heitink, M.C., Kippers, W.B., Veldkamp, B.P.: Formative assessment: a systematic review of critical teacher prerequisites for classroom practice. Int. J. Educ. Res. 103, 101602 (2020). https://doi.org/10.1016/j.ijer.2020.101602
Rivers, M.L., Dunlosky, J., Joynes, R.: The contribution of classroom exams to the formative evaluation of concept-level knowledge. Contemp. Educ. Psychol. 59, 101806 (2019). https://doi.org/10.1016/j.cedpsych.2019.101806
Zainuddin, Z., Shujahat, M., Haruna, H., Chu, S.K.W.: The role of gamified e-quizzes on student learning and engagement: an interactive gamification solution for a formative assessment system. Comput. Educ. 145, 103–729 (2020). https://doi.org/10.1016/j.compedu.2019.103729
Savery, J.R.: Overview of problem-based learning: definitions and distinctions. Interdiscip. J. Probl. Learn. (2006). https://doi.org/10.7771/1541-5015.1002
Amelia, R., Budiasih, E., Yahmin, G.: Promoting the scientific argumentation skills of students using ADI-S and ADI models in chemical kinetics teaching. In: AIP Conference Proceedings, p. 020001 (2020)
Walker, J.P., Sampson, V., Grooms, J., Anderson, B., Zimmerman, C.O.: Argument-driven inquiry in undergraduate chemistry labs: the impact on students’ conceptual understanding, argument skills, and attitudes toward science. J. Coll. Sci. Teach. 41, 74–81 (2012)
Academic Vicerectory Tecnologico de Monterrey: Internal Reports on Argumentative Assessments. Monterrey, Mexico (2020)
Waychal, P., Mohanty, R.P., Verma, A.: Determinants of innovation as a competence: an empirical study. Int. J. Bus. Innov. Res. 5, 192 (2011). https://doi.org/10.1504/IJBIR.2011.038781
Harrell, M., Wetzel, D.: Using argument diagramming to teach critical thinking in a first-year writing course. In: The Palgrave Handbook of Critical Thinking in Higher Education, pp. 213–232. Palgrave Macmillan, New York (2015)
Hannafin, M.J., Land, S.M.: Technology and student-centered learning in higher education: issues and practices. J. Comput. High. Educ. 12, 3–30 (2000). https://doi.org/10.1007/BF03032712
Apple: Challenge Based Learning: A Classroom Guide. Apple, Inc. (2010)
Membrillo-Hernández, J., Ramírez-Cadena, J., Martínez-Acosta, M., Cruz-Gómez, E., Muñoz-Díaz, E., Elizalde, H.: Challenge based learning: the importance of world-leading companies as training partners. Int. J. Interact. Des. Manuf. 13, 1103–1113 (2019). https://doi.org/10.1007/s12008-019-00569-4
Savery, J.R., Duffy, T.M.: Problem based learning: an instructional model and its constructivist framework. Educ. Technol. 35, 31–38 (1995). https://doi.org/10.2307/44428296
Chan, T.-W., Looi, C.-K., Chen, W., Wong, L.-H., Chang, B., Liao, C.C.Y., Cheng, H., Chen, Z.-H., Liu, C.-C., Kong, S.-C., Jeong, H., Mason, J., So, H.-J., Murthy, S., Yu, F.-Y., Wong, S.L., King, R.B., Gu, X., Wang, M., Wu, L., Huang, R., Lam, R., Ogata, H.: Interest-driven creator theory: towards a theory of learning design for Asia in the twenty-first century. J. Comput. Educ. 5, 435–461 (2018). https://doi.org/10.1007/s40692-018-0122-0
Kadayifci, H., Yalcin-Celik, A.: Implementation of argument-driven inquiry as an instructional model in a general chemistry laboratory course. Sci. Educ. Int. 27, 369–390 (2016)
The authors would like to acknowledge the Scientometrics team led by Nathalie María Galeano Sánchez and the Faculty Development and Innovative Education Center for the pedagogic support during this work. The authors would like to acknowledge the technical support of Writing Lab, Institute for the Future of Education, Tecnologico de Monterrey, Mexico, in the production of this work. The current project was funded by Tecnologico de Monterrey and Fundación FEMSA Foundation (Grant No. 0020206BB3, CAMPUSCITY project). The authors would like to acknowledge the financial and the technical support of Vicerrectoria Academica y de Innovacion Educativa of the Tecnologico de Monterrey in the production of this work.
Author information
Authors and Affiliations
Tecnologico de Monterrey, Escuela de Ingeniería y Ciencias, Ave. Eugenio Garza Sada 2501, 64849, Monterrey, N.L., Mexico
Enrique A. López-Guajardo, Ricardo A. Ramirez-Mendoza, Adriana Vargas-Martinez, Wang Jianhong, Armando Roman-Flores & Genaro Zavala
Jiangxi University of Science and Technology, No. 86, Hongqi Avenue, Zhanggong District, Ganzhou City, Jianxi Province, People’s Republic of China
Ricardo A. Ramirez-Mendoza & Wang Jianhong
Facultad de Ingeniería, Universidad Andres Bello, Antonio Varas 880, 7500973, Santiago, Chile
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
López-Guajardo, E.A., Ramirez-Mendoza, R.A., Vargas-Martinez, A. et al. Argumentative-driven assessments in engineering: a challenge-based learning approach to the evaluation of competencies.
Int J Interact Des Manuf17, 79–91 (2023). https://doi.org/10.1007/s12008-022-01188-2
