Evaluation of information and communication technology in education programs for middle and high schools: GENIE program as a case study

Abstract

The Generalization of Information and Communication Technologies in Education program (GENIE) has made way for computers, video projectors, interactive whiteboards and multimedia rooms into many public schools in Morocco, and has worked to add the communicative dimension to the process of technology-assisted teaching. The program also worked on qualifying human resources to be more responsive within the new Information and Communication Technology (ICT) enriched environment by means of occasional workshops and Massive Open Online Courses (MOOCS). Still, in the absence of return on investment (ROI) studies or impact evaluations, the program stirred long controversy over its merits. This paper aims to study the impact of GENIE on teachers and students in middle and high school using an evaluation model conceived by Daniel Kirkpatrick and Thomas Guskey. The process of data mining and analysis took advantage of both quantitative and qualitative approaches, with more emphasis on the second. The study concludes that the flagship ICT integration program in the country is a promising one; however, it falls short of delivering its promise of engaging the Moroccan school into the information society. The program’s 4 axes of operation (infrastructure, training, digital resources and development of use) endure serious impediments that disturb the attainment of the program’s objectives throughout all GENIE’s three phases of execution and will certainly hinder the realization of the ministry’s 2030 vision.

1 Introduction

Mastery of educational technologies available at school is such an indispensable requirement that raises a teacher’s profile and is a substantial criterion to hire one. Accordingly, technology, as a pedagogical tool, is a motivation catalyst for both students and teachers to digitalize their teaching/learning process without necessarily being academically trained or aware why, when and how it could be used (Davies 2011). Teaching with technology academically, rather, is not about making the process of learning easier, but more meaningful, challenging, interesting and foremost communicative (Haydn 2014).

With the introduction of computers, internet, mobile phones, tablets and other communication devices to schools, the interest of scholars has shifted towards Information and Communication Technologies (ICTs) as potential substitutes for (instructional technology) IT tools driven by the communicative, competency and project-based approaches (Brown and Green 2009). In order to approximate this global ever-evolving field of research, the Moroccan ministry of education launched GENIE program (Generalization of Information and Communication Technologies in Education) in 2006 to establish a nationwide strategy that systematizes the abrupt occasional initiatives by teachers and voluntary associations whose effectiveness remained, for a while, questionable and based on mere intuition.

The ministry of national education in Morocco celebrates GENIE as the most elaborate collaborative ICT program in which the government, and its pedagogical and technical partners, invested a colossal budget between 2006 and 2013 over 2 phases 2006/2008, 2009/2013 (Ministry of Education 2015) (Jerrad 2015), not to mention the deployment of a considerable amount of human resources. To date, the sweeping majority of available studies conducted by the ministry of education, interested bodies and scholars focus on the technical side of the program with some sort of quantitative logic. These studies, such as the ones conducted by Messaoudi (2012); Hamse (2015); GENIE (2012, 2016) are close to balance sheets and tend to focus on coverage and training rates rather than answering the most basic primitive question: What difference did the program make on our pedagogical system?

The ministry never conducted a thorough impact assessment of the program that was initially supposed to last only three years to end up taking over a decade. Even the decision of program protraction was based on perfunctory studies lacking the academic background and often neglect the direct stakeholders concerned with the process of teaching and learning; namely the teacher and the student. These “studies” examined, foremost, whether the equipment and the training are sufficient without bothering about measuring their effectiveness and appropriateness.

Like any other educational program, it is essential to place GENIE under a thorough evaluation to determine its strengths and failures. “When the evaluation is done, we can hope that the results are positive and gratifying, both for those responsible for the program and for upper-level managers who will make decisions based on their evaluation of the program” (Kirkpatrick and Kirkpatrick 2006, p. 3). To date, the program has been operational for twelve years, which is sufficient to make it subject to examination and criticism from a pedagogical perspective.

The present study investigates factors of success and failure from a bottom-up perspective relying heavily on feedback from teachers and students. It is divided into 5 parts. The first part is dedicated to the introduction of GENIE Program and how it matured over 15 years. The second part is dedicated to the literature relative to the evaluation models used to perform this appraisal. State of the art is presented in part 3 followed by the methodology in part 4. Finally, part 5 incorporate findings and implications.

2 GENIE program

2.1 GENIE I (2006–2009)

In its initial version, GENIE was granted a period of operation lasting for three years with three principal axes; infrastructure, training and digital resources:

• Infrastructure: setting up multimedia environments with internet connection in partnership with international hardware and software companies.

• Training: It was based on a waterfall approach starting with “Master Trainers” at the central level and ending up with regional coaches from each of the 16 regional academies of education and training (RAETs).

• Digital resources: also called content development aims at providing digital resources and establishing a national laboratory of digital resources and a national web portal for Information and Communication Technology in Education (ICTE).

In order to set the program in motion, an agreement between the ministry of education represented by GENIE directorate and INTEL was signed in 2006 and 2007 based on project pedagogy model. This would allow teachers to acquire and sharpen ICT related competencies and enhance their day-to-day classroom practices via “Intel Teach to the Future” (Kabbaj et al. 2009). Due to a number of inhibiting factors, the program barely achieved 25% of its target goals and a renewal of the mandate was imperative.

2.2 GENIE II (2009–2013)

The firm willingness expressed by the ministry of education to disseminate and integrate ICTs in educational institutes is widely confronted by enormous obstacles, constraints and inhibiting challenges (Alj and Benjelloun 2013). Therefore, and 2 years past the program’s inauguration, GENIE program directorate, under the supervision of the NART, conceived a moratorium in 2008 reflecting on the importance of ICT integration into education. Studies (about the program) revealed that raising awareness, communication, empowerment, reviewing and updating the training process in terms of logistics, coaching, supervision and content are necessary and essential for the improvement of GENIE (Kabbaj et al. 2009).

GENIE II was particularly characterized by the introduction of a fourth axis to be added to infrastructure, training and digital resources; that is of usage development. The new mission sets a number of priority objectives such as the acquisition of digital resources, launching an ICTE web portal, organizing sensitization campaigns and sharing workshops. It also investigates and tracks what the end users do with ICT. Although the pace of realization has tangibly improved, but the program fell short again of achieving 100% of the target goals.

2.3 GENIE III (post 2015)

Within the Strategic Vision of Reform 2015/2030 launched by the Supreme Council for Education, Training and Scientific Research (SCETSR), particularly in the sixth lever, the council calls for the equipment of educational institutions with the necessary infrastructure, equipment, didactic material, and digital libraries. It also calls for the equipment of classrooms with audio-visual aids and ICTs. The vision has lifted the ban on GENIE and freed it from any fixed-term plans.

Starting from 2016, the program has for the first time opened up on Open Source programs thanks to the National Laboratory of Digital Resources (NLDR) and the Morocco-Korean Centre of ICTE Training (MKCT) by means of several projects.

3 Program evaluation

3.1 Donald Kirkpatrick’s levels of evaluation

In 1959, Donald Kirkpatrick proposed 4 basic levels of evaluation published in the Training and Development Journal to make up a reference mark for most, if not all, subsequent models of evaluation. When launched for the first time, it made part of a project on evaluating a supervisory training program, yet the model’s simplicity, effectiveness and comprehensiveness required in any evaluation process makes it a good fit for a wide range of study fields including medicine, higher education, vocational education in enterprises, blended learning, ICT, etc. (Moldovan 2016) (Tamkin et al. 2002). Because of the ever-evolving research on evaluation, Kirkpatrick had to consistently adapt or update the levels’ guidelines, while the four levels (reaction/learning/behavior/evaluation) remained unchangeable. The levels are also referred to as steps or even taxonomy as each one leads to a more elaborate level that is “more difficult and time-consuming, but … also provides more valuable information” (Kirkpatrick and Kirkpatrick 2006, p. 25).

• a. Reaction: Kirkpatrick also calls it a “measure of customer satisfaction” (Kirkpatrick 1996). A customer according to him is anyone who takes part of the training course whether they paid for it or not, whether it was voluntary or forced by an organization. Although the model was conceived about 60 years ago, Kirkpatrick adopts a bottom-up approach to the evaluation process as he believes that the positive reactions of trainees are important for trainers and for those who make public programs.

• b. Learning: This step measures the effectiveness of learning process and the impact it made on the learners at one of these levels: knowledge, skills or attitudes. Certain programs target enhancing one of these competencies such as languages or engineering, while others can incorporate integrative approaches to enhance two or even three such as motivation and communication courses. The evaluator, therefore, must determine clearly their objectives to remain on a safe side.

• c. Behavior: This step is referred to as transfer of training. It examines whether the training has impacted the learner’s behavior at work or school as intended by the institution after attending a particular training. Kirkpatrick, as stated earlier in this chapter, draws attention to the fact that institutions that carry out evaluation are likely to skip behavior and results evaluation; nevertheless, some institutes bypass the first two levels to address particularly behavior evaluation from the very beginning. He disapprovingly does not recommend the procedure and even calls it a “serious mistake” because a program’s failure to deliver at the level of behavior does not impulsively mean that it failed to deliver at the level of reaction and learning.

• d. Results: This step examines the final results and the effects of the training on learners and institution as well. Optimal results should, for instance, reveal an increase of profit, better quality products, better graduation rates, cost reduction, reinforcement of desirable practices and values, lower drop-out rates, etc. “It is important to recognize that results like these are the reason for having some training programs. Therefore, the final objectives of the training program need to be stated in these terms” (Kirkpatrick 2009, p. 33)

3.2 Guskey’s critical five levels (2000)

Dr. Thomas Guskey, the expert in evaluation design, analysis, and educational reform, finds that Kirkpatrick’s 4 levels could be adopted and adapted to the arena of education. Guskey’s model (2000) gained currency and academic eminence since it is tailored to meet the needs of educational purposes. Still, in his attempt to advance his own amendment of the model, he suggests 5 critical levels. The success of each depends on the success of the preceding one (Guskey 2013).

• a. Level 1: Participant reaction:

Just like Kirkpatrick’s model, the reaction evaluation investigates and measures the participants’ satisfaction and impressions about the training, material, timing, the environment, the leader, etc. It is the easiest of all levels and is usually conducted using a questionnaire at the end of the session. The feedback is used to enhance the program’s design and delivery.

• b. Level 2: Participant learning

This evaluation is meant to make sure that the participants have successfully acquired the intended knowledge and skills. Different instruments are deployed at this level such as portfolios, demonstrations, personal reflections, simulations, paper-and-pencil tests, etc. This evaluation is imperative to the improvement of content, format, and organization of work.

• c. Level 3: Organizational support and learning

This level is a defining feature of Guskey’s model that makes it distinct from Kirkpatrick’s. At this level, focus shifts towards the analysis of how the organization, or the governing body, supports and reinforces the newly acquired skills and knowledge. Negative results at this level are by no means an indicator of training failure but are rather the result of obstructive policies that undermine implementation efforts and consequently blur the success achieved in the first two levels (Guskey 2013). This evaluation is conducted on a large scale by program stakeholders via district meetings, questionnaires addressed to school administrators, structured interviews or unobtrusive observations.

• d. Level 4: Participant use of new knowledge and skills

This level is similar to Kirkpatrick’s behavior evaluation that investigates whether the participants make use of their newly acquired skills and knowledge and implement them in real life or business contexts. As coping with new practices and behaviors does not take place overnight, the assessment of participant’s use of new knowledge and skills should be gradual and belated. Like the preceding level, the evaluation is conducted by means of questionnaires, interviews, observations of superiors, etc.

• e. Level 5: Student learning outcomes

Unlike other evaluation models, such as Stufflebeam’s, Phillip’s and even Kirkpatrick’s at a later stage that particularly examined the merits of a program by means of its ROI, Tomas Guskey had the student and only the student in mind as the most salient area of evaluation. “Using five critical levels of evaluation, you can improve your school’s professional development program. But be sure to start with the desired result—improved student outcomes” (Guskey 2002, p.45). Still, the information gathered by stakeholders or commissioned experts could be used to assess the ROI, although it is not the target point of this level by itself. This evaluation serves to correlate the student learning objectives with the learning outcomes. These outcomes may take different shapes and forms ranging from cognitive (knowledge) and affective (attitude) up to psychomotor (skills and behavior).

4 Related work

Similar evaluation studies have been carried out in other parts of the world on similar ICT programs such as SchoolNet in Canada (KPMG Consulting LP 2000). The umbrella program of many programs is Canada’s SchoolNet that was launched in 1993 as a partnership that involves provincial and territorial governments, the education community and the private sector with the aim of improving ICT in learning. In December 2000, KPMG Consulting LP, was mandated by Industry Canada to draft an evaluation report on the SchoolNet 1 initiative. The report raised eight issues that were meticulously investigated separately in the form of conclusions, background, findings and results from interviews and surveys. Although the report highlights successes, especially at the level of apparatus equipment, yet it saves no reserve to point out to several lacunae in the implementation process such as ICT use.

As far as GENIE is particularly concerned, two internal evaluations have been carried out by GENIE directorate. In 2012, a GENIE department in charge of evaluation, was mandated to conduct an internal assessment of the program. Not long after the release of the 2012/2013 report, the ministry launched a new assessment in 2014 following allegations of public money squandering. In addition to inspectors, school headmasters and teachers, the evaluation surveys, for the first time, students in accordance with the orientations and key indicators considered by the UNESCO (UNESCO, 2009). The credit of these evaluations might be challenged for different reasons. Being conducted by the National Observatory of ICTE Uses (NOICTEU), which operates under the umbrella of GENIE directorate may result in an infringement of impartiality. Conducting the evaluation by an “insider” evaluator may lead to bias and subjective tendency to favor the program over other alternatives (Smith et al., 1997 as cited in Hurley et al. 2002). The reports also include some issues related to balance and representation: most if not all participants represent scientific disciplines; inspectors provided feedback on behalf of teachers; some categories were not given a chance to voice out their recommendations. It is also noted that the figures are notably and inexplicably inconsistent in the two reports.

One of the renowned studies conducted on e-learning engineering in Morocco is that of Faouzia Messoudi (2012) within the framework of her PhD research. Messoudi’s study may serve as an exhaustive outline of ICT use in Morocco that covers public and private institutes, strategies, programs, challenges, etc. In her project, she examined GENIE that comes to concretize the national strategy to generalize ICT in education (Messaoudi 2012). Messaoudi, in an attempt to bring about a diagnosis of every aspect in relevance to ICT use in Morocco, her study summed up programs, strategies and theories in a methodology that misses focus. Also, there is no second reading of the statistics provided by GENIE, which are blurry and not clear enough.

Studies investigating the issues that hinder the optimal use of ICT at Moroccan schools are numerous. Nachit et al. (2013), investigated the issue of ICT training as the majority of Moroccan Math teachers, for instance, are poorly qualified to use ICT for educational purposes because they do not receive an adequate official training by the ministry. Another study concluded that although some teachers benefited from a GENIE training course, they do not use ICT in the classroom by reason of lacking motivation (Alj and Benjelloun 2013).

Bouziane (2019) conducted an exhaustive synthetic survey of master and doctoral theses in which he investigated success and failure experiences relative to ICTE in Morocco. By outlining official ICT initiatives with their triumphs and challenges, the study attempts to generate a balanced, yet constructive, perspective of ICT uses in Morocco. The study affirms that online learning, in particular, is not given the attention it is worth by decision makers, which proved to be indeed a strategic pitfall during the Corona pandemic. Bouziane lists a number of recommendations for stakeholders who are urgently invited to reconsider the adopted model of governance by focusing on the frontline actors’ needs and aspirations, improving the quality of material and encouraging academic research in the field.

5 Methodology

In an effort to abide by the highest reliability and validity possible standards of evaluation, this appraisal study incorporates data generated using quantitative as well as qualitative approaches. Still, the nature of this evaluation, which calls into question the outdated routine assessment approach used by the ministry and mandated departments, dictated the qualitative approach more priority whenever possible. Therefore, the process of data collection and analysis were founded on the following:

5.1 Population of the study

The population of the surveys involves students and teachers who represent the bottom line of GENIE stakeholders whose success or failure closely reflects the worthiness of GENIE and related investments capable of realizing an effective ICTE integration in Moroccan schools. The teachers’ sample population (N = 249) could be divided into two major groups; teachers of scientific subjects (including Math, Physics, Science, Technology and ICT), and teachers of foreign languages (French and particularly English). The teachers were surveyed via web-based polling platforms, particularly Google Forms and Surveymonkey to ensure maximum outreach. As for the students’ questionnaires, the sample population (N = 304) involves students from 4 public institutes of different critical variables: two high schools that speak for the urban and rural milieu located in Fez and the outskirts of Moulay Yakoub, in addition to two middle schools that represent both an under-privileged area (placed in a precarious environment), and an upscale area (downtown) whose students have better accessibility to ICTE. 4 classes have been chosen from the sample middle schools, and 6 others have been chosen from high schools. The levels range from grade 9 up to grade 12 (the second-year baccalaureate) and the host institutes are all equipped with GENIE multimedia room. The students were interrogated by means of hard copy surveys that investigate their usage of ICT for educational purposes. The key representatives/informants, namely GENIE training coaches, GENIE, representative of the National Centre for Educational Innovation and Experimentation. were surveyed via semi-structured interviews in-person and via telephone.

5.2 Data processing

• The analytical process of feedback did not take advantage of all the retrieved data; only items that serve better the evaluation model were exploited.

• The statistical analysis was performed using Microsoft Excel (bars and pies) and IBM SPSS (Statistical Package for the Social Sciences) V21.

• As the nature of entries is qualitative on the first place, the functions used are correlation, standard deviation and index reference.

5.3 Adaptation of Kirkpartick’s and Guskey’s models in the study

The questionnaires, interviews and examined official circulars should provide answers to the following 5 integrated evaluation levels adopted from the three models stated above.

• 1. Reaction: This evaluation would gauge the teachers’ impression about the ICTE training they received under the auspices of GENIE program without referring to the training content. The evaluation allows room for a better and closer understanding of the conditions under which the training took place.

• 2. Learning: This evaluation inspects the content of cursus, what the teachers particularly have learned and what they think of its utility at school. We will look into the program’s official training courses, statistics and related sources of data

• 3. Behavior: This component examines the potentials of new behaviors that may have taken place thanks to GENIE trainings.

• 4. Organizational Support: Because any behavior change is conditional upon reinforcement, it is indispensable to examine the organizational support provided by the administration and how efficient it is. Direct testimonies of teachers, superintendents and students would be of much relevance.

• 5. Outcomes: Finally, we would see if the program have had any impact on teachers and students being the ultimate target population or “customers”. The impact may have different shapes or forms and we would not prefer to focus on a single parameter.

6 Study findings

6.1 Reaction

It is surprising, and a bit disappointing, that GENIE’s official narrative on the number of training beneficiaries sharply contradicts with our research findings. According to GENIE directorate (2016), 100% of the ministry staff took advantage of training workshops sometime between 2006 and 2016; while according to our survey sample, whose vast majority of participants have an experience of over 10 years, only 33% of teachers did. This brings into question GENIE’s criteria to count trainees, given that our survey was conducted between 2016 and 2017; ten years after the commencement of the program. Some interviewees mentioned that giveaway CDs distributed in schools may have been counted as actual trainings; the allegation has not been confirmed by this study.

In order to measure their first impressions or reaction, as stated in Kirkpatrick’s model, vis-à-vis the conditions through which the training took place, the study sheds light on four major elements that are material, time, coach, and overall satisfaction (Table 1).

Table 1 Teachers’ first impression about GENIE training

The analysis of the feedback shows, however, a sort of contradiction; the accumulative percentage of reactions related to material, time, and coach is not consistent with the population’s overall assessment of the training conditions. The training material scored 53% of partial or full satisfaction; time scored 74% of partial or full satisfaction; interaction with the coach scored 63% of partial or full satisfaction. The gross satisfaction rate of the three variables is 63,33%. Conversely, only 50% expressed their partial or full satisfaction about the training conditions in general. On an index reference scale, the variable would get 3/5 or ((5*5) + (5*1) / N) = 30/10 = 3, which is a positive non-neutral index mark. In this study, both methods were used, and they proved to be harmonious on the proportional as well as the index reference scales. The inconsistency of values and poor scoring of the general first impression might be explained by the teachers’ predominant judgment of the whole experience where the assessment of outcomes overlaps with the assessment of procedures.

To conclude, the teachers’ attitude towards GENIE’s training conditions remains positive to a large extent. A large proportion of the study population (almost 70%) expressed their absolute readiness to take the training course again if allowed to, and they highly recommend the training to their fellow teachers (89%). Still, interviewee coaches recommend taking measures to ensure that the apparatus is operational and sufficient in quantity.

6.2 Learning

Having concluded that the training environment is relatively favorable according to the study participants, despite the frequent issues related to maintenance of desktops, projectors and mobile briefcases as ICT teachers testified, it is no guarantee that the learning process was also effective. The content of the training modules reflects the directorate’s wish to systematize and standardize the process of training, yet it undergoes several anomalies we would like to highlight. To begin with, the official syllabi was conceived 5 years past the inauguration of the program (Court of Audit, 2014), which marks a sign of arbitrariness before that point when coaches relied on mere headlines and their own creative talents.

The content of the training creates endless controversies among trainees who think that it is either drowned in theoretical, glittering yet ineffective big words, or presenting unsuitable material of already transcended word processing drills. It is also confusing that the manual does not bluntly designate Microsoft word and data processing workshops as the principal component of trainings. As a result, a small proportion, as little as 20%, feel that the training helped them sharpen their ICT skills (Figs. 1 and 2).

Fig. 1

Q. Did GENIE training course have a strong impact on you?

Fig. 2

What is the nature of material provided in the training course?

More than the double of that proportion (42%) are convinced that the trainings did not serve them by any means. The rest remained doubtful about the utility of the training. These results are a natural repercussion of poor planning, lack of creativity and quasi-ignorance of real-life needs of teachers and students in the classroom (Fig. 3).

Fig. 3

Does the training provide innovative content?

To value the worthiness of content for classroom needs, teachers, including GENIE alumni, were asked if the training helped them in the classroom or made their tasks any different. 57% of the participants think that the process had very little or no impact at all on their classroom practice. 31% think that the utility of training was limited or not efficient enough. 12% think that the training impacted positively their savoir-fair in the classroom as the Table 2 below shows.

Table 2 Relevance of training to teachers in the classroom

It is seriously alarming to witness that only 12% of participants find that their competencies have been positively impacted thanks to the program. When the content of a training course provided by a lavishly funded program like GENIE scores as little as 2,16/5 on an index reference scale, it is imperative to take prompt measures and reconsider the worthiness of the program.

6.3 Behavior

The first question pertinent to behavior was about the frequency of technology use in the teachers’ habitual classrooms. All the 249 participant teachers, including those who did not participate in GENIE trainings, have been asked the question to draw a contrastive analysis of their tendencies and see if GENIE trainings had any impact on the alumni’s attitudes (Fig. 4).

Fig. 4

Q: Do you use technology in your classroom? (e.g. your PC)

On both a proportional and standard deviation scales, the answers demonstrate that GENIE alumni have higher disposition to use technology relying on their own material (Table 3). Although the answers provided by ICT and Technology teachers were disregarded for fear of tipping the balance, the result was not any different as ICTE can serve the teacher and the student as well regardless of the study field.

Table 3 Q: Do you use technology in your classroom?

The next question of the teachers’ attitude towards technology reveals that the teachers’ positive stand is a pre-existing assumption more than it is a result of GENIE trainings, at least for almost 71% of participants (Fig. 5).

Fig. 5

Q: Did GENIE change your attitude towards ICT in the classroom?

On the other hand, seeking alternative solutions by using mobile gadgets in the classroom does not seem like a reliable solution for many participants when they run short of hardware material. Around 40% of participants never or rarely use their mobile gadgets in the classroom, including GENIE alumni. The rejection rate is higher when it comes to allowing the students to use their mobile gadgets, which suggests that the program does not target enhancing alternative IT solutions for fear of violating regulations in certain academies that strictly forbid the use of smartphones in the classroom for both the teacher and the students (Table 4).

Table 4 Q: Are your students allowed to use their mobile gadgets?

One more element that impacts the adoption of new work/learning behavior is related to the axis of digital resources, which is not helping any better. While online resources provide an invaluable opportunity to establish a formal learning model that meets the conditions of conformity, reliability and credibility, lots of challenges are undermining the axis full potentials. The availability of resources is notably disproportionate in favor of scientific fields such as Physics, Chemistry, Science and Math. Applications of pertinence to languages and humanities are a bit ignored. Lately, under the unexpected outbreak of Corona pandemic and emergent need for D-learning material, the ministry had to abruptly cram the platform with videos from YouTube, many of which are uploaded by amateur youtubers and vloggers. The study asserts that 89% of students never visit websites sponsored or launched by the ministry. In an interview with a sample group of students, they justified their answers by the fact that they know nothing about Taalimtice.ma, and they prefer to look up elsewhere on “YouTube” and “Wikipedia”.

6.4 Organizational support

Internalizing newly acquired skills and knowledge requires a synergy of efforts and a synchronous action plan where every component serves as an essential cog, without which the whole process is doomed to failure. The results in relevance to the Organization Support reveal probably the most serious sicknesses of GENIE. The semi-structured interviews with headmasters, teachers and coaches revealed several anomalies that can be summarized in the following. Many headmasters have issues with the administrative management of hardware and multimedia room (MMR), poor ICT qualification and strictly limited autonomy to resolve the MMR problems. Non-ICT teachers are constantly challenged by the denial of access to the MMR, the impossibility to network with their peers and the lack of follow-up accompaniment. ICT teachers, on the other hand, complain about the frequent apparatus failure and the exploitation of the MMR for administrative purposes. Last but not least, GENIE coaches endure hardship with the uneven ICT skills of trainees, lack of candidates targeting, apparatus failure and limited practicality of the training syllabus.

One of the program’s biggest pitfalls, especially in the training axis, is being conceived by foreign IT companies, namely Intel and Microsoft whose IT engineers and technicians are missing the academic background. The datum can explain why GENIE directorate, focused more on the axis of infrastructure in GENIE I and ignored the other axes. For instance, trainings before 2010 were ushered by general outlines; the coaches had to improvise the workshop material, methodology, timing, etc. A standard training methodology was conceived in 2010 in which the Change Management approach is explicitly stated for the first time as an academic framework for the program. The study, however, concludes that the mechanisms by which GENIE runs the 4 axes of operation endure either a superficial understanding of the CM approach or a complete negligence of its imperatives. The mediocracy of GENIE I outcomes and the testimonies of our directly concerned interviewees confirm the hypothesis that the planning phase was chaotic. The criticism addressed to GENIE II and III by scholars, ministry officials, organizations and the Court of Audit disclose the control weakness throughout the execution phase after targeting mainly the purchase and installation of apparatus. As GENIE alumni deny the existence of administrative support or follow-up visits upon completion of their trainings, one may argue that the stage of maintaining change has been a bit disregarded. The NOICTEU, being the department in charge of usage development was qualified to undergo this mission, but, instead, it was assigned to conduct “soft” internal evaluations and pay exclusive “inspection” visits to ICT teachers.

7 Outcomes

7.1 Teachers’ outcomes

Both groups of teachers who received a training under GENIE program and those who did not, highly agree that ICT helps their students achieve better outcomes including grades and behavior. Comparing the answers of the two parties, by means of proportions and standard deviation, leads to the conclusion that GENIE has limited or no credit at all to the elevated teachers’ awareness of ICTE relevance in the information age classroom (Fig. 6 and Tables 4 and 5).

Fig. 6

Q: Apart from raising their motivation, do you think technology helps students achieve better results at school?

Table 5 Q: Do you think technology helps students achieve better results at school?

It is more likely an innate conviction the teachers have developed throughout their in-service trainings and professional experience. Nevertheless, when the teachers were asked if they had empirically tested and compared the efficacy of ICTE themselves with their students, the vast majority’s answers were negative leading us to assume that their conviction is rather a hypothetical supposition. The finding does not, by any means, deny the positive impact of ICT on learners’ achievement at school, but it highlights the fact that, apart from raising the students’ motivation, the benefits of ICTE are barely verified by practitioners.

Due to lots of ethical and procedural constraints, it was impossible to test or measure the teachers’ mastery of ICTE material upon the end of GENIE’s training. To compensate for this shortcoming, the author decided to measure the familiarity of study participants with 5 umbrella terms that are recurrent and widely popular in ICT trainings worldwide (MOOCs, E-learning, M-learning, LMS and Moodle). Positive or negative results might be interpreted as a strong or weak acquaintance with satellite areas of study related to ICTE. It was expected that apart from ICT teachers, GENIE alumni would be able to identify at least 3 out of 5 key terms, which is quite satisfactory. The result concluded that except for e-learning, the 4 other terms are largely ignored by our participants. It is also noted that the minority of participants who provided positive answers consists mostly of ICT teachers (Fig. 7).

Fig. 7

Familiarity with key components of ICT trainings

On an index reference, teachers who have been assessed on the basis of their familiarity with frequent ICTE jargon scored as little as 1,57 out of 5. The finding implies that the training outcomes are poor in terms of standards as interviewees mentioned earlier. When compared to international ICT trainings, the rewards of word and data processing workshops (Microsoft Word/ Excel) are mediocre and not consistent with invested provisions, which urges the need to bring about a discussion on the relevance of these trainings and their alignment with international standards.

7.2 Students’ outcomes

The students’ outcome section, which is relatively more significant than the previous one, investigates the indicators of establishing an ICT empowered school at the information age and verifies their alignment with the ministry’s vision in this regard. Although it is an established piece of information that not all of the students enjoy access to the MMR at school, the 304 sample participants from 10 classes belonging to 4 public schools have attended classes of some GENIE alumni teachers. The results whether positive or negative are not the direct aftereffect of GENIE program; we only try to study the school ICT environment and examine the abundance of factors that favor and encourage directly or indirectly the use of ICT for educational purposes, which is the role of GENIE program.

The first question reveals a widespread innate awareness of the ICT relevance at school, and a readiness to be involved in ICT-based projects. The answers suggest that there is no affective filter towards technology that might, eventually, cause the process to slow down or even stall. As self-efficacy is amongst the most important assets GENIE capitalizes on, 45% of students have confirmed their complete dependency on ICT to perform school projects (Fig. 8).

Fig. 8

Q: Can you conduct a school project without using ICT?

The number, which came lower than expected, might be interpreted by the students’ willpower not to abandon traditional learning methods and core skills that view technology as an instrument not an end itself.

The study, based on the students’ feedback, confirms the existence of a significant positive correlation between the tendency to use of conventional technology and m-learning strategies in the classroom.

It is concluded that the more teachers develop a tendency to use technology in their classrooms, the more they become open to using alternative learning models such as m-learning. The process, however, might be subject to vulnerability because the students are not encouraged by their teachers to use their mobile phones, despite their abundance in all the classrooms surveyed.¹

The improvement of productivity and performance at school is a major impact indicator that solicits the relevance of ICTE, thus, we asked students whether they think technology helps them achieve better results and higher marks (Table 6).

Table 6 Correlation of ICT use and m-learning

Almost 85% of students bear a positive attitude towards the impact of technology on their performance at school, yet in the absence of a recorded history of the students’ outcomes pre and post ICT integration we had to take refuge to the statistical analysis of students’ performance represented by grades (being an empirically measurable variable) versus the frequency of MMR use (Tables 6, 7 and 8).

Table 7 Q: Do you think ICT enhances your productivity at school?

Table 8 Correlation of MMR frequency of use and grades

The comparative statistical analysis of the two variables concludes that there is no significant correlation between the frequency of MMR access and achievement; consequently, it would be safe to presume that the MMR and ICT use in general may not contribute to enhancing performance (grades in our study), but it might contribute to enhancing the learning environment, teaching model and students’ attitude. They may still affect other variables that are not under the scope of this study, which calls for further research in the future.

8 Conclusion

The study concludes that GENIE, being the embodiment of Morocco’s bid on ICT in education, is a promising program that requires prompt and candid redress. By comparing the ICT hardware assets of Moroccan schools back in 2005 with those of 2018, one would find no reason to doubt the paradigm shift that has taken place thanks to GENIE. However, a bottom-up analysis approach of ICT practices at schools that involve first-line stakeholders, namely the teachers and students, reveals numerous anomalies that should, eventually, lead to eminent failures. For the first level of evaluation (reaction), the results are satisfactory, but as the evaluation progresses to the four other levels (learning, behavior, organizational support and outcomes) the results come less reassuring and range between below average and mediocre for especially three reasons. First, GENIE alumni teachers endure an acute weakness at the level of key notions that constitute the core of e-learning workshops around the world. Second, the mechanisms by which GENIE runs the 4 axes of operation endure either a superficial understanding of the Change Management approach or a complete negligence of its imperatives. Third, the figures communicated by GENIE incorporate several anomalies and inconsistent data, which lead us to assume that the program benefited from sufficient resources but did not enjoy an equivalent amount of strategic planning. The author, though, still believes that in the midst of a fractured educational system in Morocco, it would sound a bit peculiar to await miraculous solutions to the spreading educational challenges from technology alone.