Keywords

1 Introduction

Information literacy (IL) indicates the ability to work with information effectively. Competencies specified in IL standards are necessary for adequate participation in the information society. The Global Media and Information Literacy Assessment Framework [1] (MIL) and Framework for Information Literacy for Higher Education [2] belong among IL standards the most often used and supported by organizations, such as ALA, ACRL or UNESCO. The first one focuses on life-long learning and active participation in the information society; the second emphasizes the competencies necessary for higher education and the professions that require it.

Universities have included IL courses in their curricula for the adoption of requested skills. However, many first-year university students lack the skills required to successfully manage their studies [3], primarily academic writing. IL courses should include core competenscies in all three areas defined in MIL – to find, evaluate and use relevant information [1]. The form of courses can vary greatly. For instance, universities provide face-to-face, blended, and purely online courses, massive courses, and small courses embedded for a specific field, and individual consultations. Each form could be effective if proven procedures are followed (see Sect. 2 Literature review).

Tertiary education in the Czech Republic is open to high school graduates to increase their qualifications in various specializations. Czech law divides university education into three basic levels: bachelor’s, master’s and doctoral (PhD.). Education at state universities is provided free of charge, in contrast to private ones that collect tuition fees. Students of the latter expect a higher quality of instruction and a greater focus on practice [4,5,6]. Education at both types of schools respects the same law and rules, but private universities are generally more connected with negative bias about the quality of education (e.g., “purchased university degree”) [6].

This paper aimed to determine how effective was the blended IL course focused on first-year private university students. At the same time, we wanted to compare the IL level of students at public and private universities.

2 Literature Review

IL courses have been an essential part of university curricula for over thirty years [7]. The content should reflect current recognized standards, including critical thinking and the production and sharing of new information [1, 2]. Some authors pointed out the spread of these standards throughout the syllabi of university IL courses [8]. Others criticized thematic deviations and a decline in standards observance [9]. The content of IL courses is evolving, with new topics emerging, for example, critical thinking [10]. Some researchers discussed content differences and the absence of country- and culture-specific contexts [9]. Other authors demonstrated the positive impact of IL training on the management of academic information resources – pointing to more efficient search and improved use of the library and other reliable sources [3, 11].

As a result of ICT advances, academia has to adapt to students’ changing needs. Although the current digital-born generation is assumed to be computer literate [9], several studies pointed to insufficient IL skills and competencies [12, 13]. Teaching IL via e-courses seems to be an alternative solution [14]. Because e-learning allows an individual pace of study and unlimited access to materials, it can be a viable method to acquire theoretical knowledge and practical skills [15]. The tutor’s role is crucial [16], but the credit-awarding scheme also supports students’ motivation [17]. Blended learning courses have been growing in importance [18, 19] because massive online courses allow only limited individual communication [20]. Some authors even suggested a retreat from mass education [21]. A review [22] of the effect of IL teaching methods at universities showed no overall preference of delivery method of IL teaching: face-to-face, online, or blended learning. In addition, there is no overall statistically significant difference between formats in student skill outcomes [22].

A nationwide survey in the form of IL self-evaluation at state-run universities in the Czech Republic showed that IL level factors were the study program, personal motivation, course completion, and the participants’ gender [23]. Many studies showed the positive impact of IL courses. Especially navigating through information resources and determining their relevance saw a significant improvement [11].

The original study [17], followed by this contribution, found problems in digital and IL and also all MIL components. The worst self-evaluation was in the Evaluation component; the worst objective evaluation was in the Creation component. Students often overestimated their ability to acquire information and IT competencies. The study proved suitability to educate in traditional topics (e.g., library services) and the positive impact of massive general online IL courses, which should be followed by smaller practical, individualized lessons in the library.

3 Methodology

The present paper aimed to describe the IL of students of private Ambis University and compare it with the original study based on the MU [17]. Both studies focused on the students’ level of IL and detecting improvements after completing a semester IL course. Questioning the presumption of generally low IL levels at Czech private universities, the outcomes will help develop IL courses. Because of limitations at the two universities, we do not want to generalize our results.

There were several similarities and differences in compared environments. IL courses at both schools were designed primarily for first-year students. They focused on the competencies defined in the MIL [1] and the ACRL standards [2]. The courses were comparable in their content and form, exploiting the potential of e-learning, both lasting for 12 weeks (one semester).

We perceive the fundamental difference in the status of the schools – a large state-maintained Masaryk University (MU) and private university Ambis. The IL course run at MU took place purely online, and it was optional for all students. At Ambis University, the course was taught to all undergraduates as a compulsory subject by all departments. The Ambis course had the form of blended learning. Having attended a four-hour lecture and a two-hour seminar, students continued online using six e-learning modules and six compulsory tests, finishing the course with a face-to-face written test. To assess the effectiveness of both courses, we conducted self- and objective evaluation before and after course surveys.

We formulated the following four hypotheses:

  • There is a relationship between self-evaluation and objective evaluation before IL course completion at Ambis University.

  • Ambis students raised their IL level significantly after completing the IL course.

  • The entry IL level was higher at MU than at Ambis University.

  • Ambis students raised their IL level more than MU students after their course.

We replicated the original MU’s research design [17] at the Ambis University in the 2019 winter semester. Both studies used a self-evaluation questionnaire and pretest and posttest for objective evaluation. Both evaluation types covered twelve topics covering all items of the MIL framework [1]: defining the topic, the Internet as a source of information, databases, information resources and information retrieval (Access in MIL); organization of information, evaluating resources and information, effective reading (Evaluation); text creation, formal text processing, visualization, presentation and publishing, sharing and collaboration (Creation). We examined each topic using one five-point Likert scale question for self-evaluation and two multiple-choice test questions with one correct answer and three distractors in each data collection phase. In the Ambis study, we modified several questions to reflect slightly different course content, while the competence set remained the same. The Ambis study left out the possibility of “I do not know.” in the pretest. Another methodology difference was using the test as pretest and posttest. It was the same as the original pretest; in the original study, questions were applied to the different text in each test version.

We administered online questionnaires to all eligible students enrolled in the IL course at the time of data collection. The questionnaires (covered both self- and objective evaluation questions) were completed by 1163, respectively, 580 Ambis students (compared to 1287/602 MU students in the 2016 and 2017 autumn semesters).

4 Results

42% of male and 58% of female students composed the set of respondents (including all test-takers). Unlike the MU research, only undergraduates participated in the survey, but they represented all three Ambis University departments – security and law (52.4%), economics and management (39.9%) and regional development (7.7%).

As in the original research, students self-assessed their competencies quite similarly in all three components (see Fig. 1). We found both the best (searching on the Internet) and the worst (searching in databases) self-evaluated topics in the access component of MIL. The second worst self-evaluated issue was creating text. All the other problems had almost the same self-evaluation (between “rather satisfied” and “something between”, closer to the lower point of satisfaction).

Fig. 1.
figure 1

Self-evaluation before the course

Full size image

We found the correlation (Spearman’s rho) between self-assessment and objective assessment before the course only in two thematic areas and in both weak and negative – visualization (ρ = −.085, sig. at the 0.01 level) and presentation (ρ = −.067, sig. at the 0.05 level). The result refuted our first hypothesis. Testing the correlation between self- and objective evaluation after completing the course rested on the assumption of increased awareness of the topic area and students’ knowledge. However, no correlation appeared. Despite this, there was a slight improvement in self-evaluation after the course completion (still mostly between “rather satisfied” and “something in-between” but closer to a higher level of satisfaction for almost all items). Again, students scored the lowest on database searches and text creation, but these topics gained the highest improvement in self-assessment, followed by information evaluation (all three topics improved more than half a point).

Because of the limited relationship between self-assessment and objective assessment, we focused more on identified problems in competencies by test questions. In contrast to MU, we detected small differences between pretest and posttest results at Ambis University (see Table 1). The t-test, nevertheless, indicated that they were large enough to be statistically significant (t = 6.556, df = 481, p = 0.000). There was some improvement, though. In the pretest, 60.3% of students got 12 or fewer points (half of the answers correct), compared to 38.3% in the posttest.

However, a closer look at the examined competence revealed a less optimistic picture (see Table 2). The average scores for pretest items showed IL gaps of Ambis students. We found the lowest values for topics: narrowing the topic, self-presentation, graphs and argumentation. But fourteen items scored less than half a point. Backing-up, visualization and suitability of resources lay at the opposite end of the scale. Posttest showed some (but not all) similar topics. The lowest values were for self-presentation, search engines, argumentation, databases and graphs. After finishing the IL course, the presumed improvement proved problematic as the results are even worse in seven areas.

Table 1. Pretest and posttest descriptive statistics
Full size table

Table 2 included a comparison with MU. Students at Ambis got significantly better results in fifteen questions of the pretest compared to five questions where students of MU were better. But if we looked at the improvement after the course (points in posttest minus in pretest), MU students shifted their knowledge level more in all questions (only for one question is this result not statistically significant). We found the highest differences (>0.5 points) for self-presentation, search engines and databases. Thus, we refuted both the third and the fourth hypotheses.

We tested differences in data from Ambis according to gender (t-test) and study department (ANOVA test). Men had better pretest results (p = .002), but women had higher improvement (p = .004), and there was no statistical difference in the posttest. When we focused on topics, women were better only in sharing in the pretest and improved more in referencing. Men got better results in six issues in the pretest (access to full text, tagging of information, evaluation of resources, referencing, terms and services, and social networks). Still, they did not improve statistically in any topic. We found no difference in the pretest and only one difference in improvement according to the department (a form of text where department of security and law got the best results and the department of economy and management got the worst results).

5 Discussion

A survey carried out at Ambis University yielded different outcomes from those produced at MU. The current state of IL at privately-run higher education institutions in the Czech Republic is unknown, either in its research or school practice (for instance, implementation of IL courses, integration into strategic plans). Except for Ambis University, represented by the co-author of this article, private schools are not members of the professional commission Information education and IL, Association of University Libraries of the Czech Republic. Not being mapped by research, the situation of private universities is reflected only in unsubstantiated assumptions. Their comparison with actual data on students’ IL is also the goal of the present study.

Table 2. Students’ average points in different topics (* sig. <0.05, ** sig. <0.01).
Full size table

Previous MU research identified a correlation between self- and objective evaluation [17]. However, Ambis University’s survey did not confirm the same, either before or after the IL course. The pretest revealed considerable closeness of self-evaluation in individual areas, except for searching information on the Internet and databases, that is, the skill that students can assess without in-depth knowledge. While students searched on the Internet daily, professional databases are often a novelty, especially to first-year students who have little idea of how they work and why they are worth utilizing. Other competence categories, in their complexity, seem to be more difficult for students to evaluate. Reacting primarily to the name of the given topic area, the respondents assume that they have demonstrated the respective skill before. But since the university expects more of them, they have to take a compulsory course in IL. Hence, they place their responses in the middle of the self-evaluation scale, not differentiating between various themes too much. Students’ self-evaluation in the respective schools varies according to how it is affected by prior knowledge and skills, objective evaluation showing differences favouring Ambis students. These considerations set the direction for any follow-up qualitative research on their IL needs.

Compared to MU students, Ambis undergraduates achieved better outcomes in many pretest items. This difference may be because approximately half of Ambis University students take up their studies after several years of work experience, acquiring some practical IL competencies. Both courses and tests focused on practical skills rather than mere theoretical knowledge. Some research suggests that students drawing on their prior experience have a competitive advantage even before classroom instruction begins [5].

We taught both courses as massive online courses. That projected into them the advantages and disadvantages associated with the use of this educational environment, like the rare application of collaborative and interactive activities. Despite that, the results showed that massive online courses are a promising approach for developing IL skills, according to other research [9].

Having completed the course, Ambis students raised their level of IL only slightly compared to MU. It might be a consequence of the format of the educational process. A substantial part of the course took place in the massive online course platform designed for hundreds of primarily first-year students who had not yet developed the study habits necessary for efficient e-learning [9]. Compared to that, the MU course included fewer students and students of all degrees. Another limiting factor was the lack of tutors, a single teacher in charge of all the communication. It weakened the provision of relevant feedback during the course, which negatively affected the understanding of the subject matter [16]. Even IL courses implemented via blended learning mode are reaching their limits, mainly due to the massification of higher education. That collides with the opposite direction of its personalization, the path that ICT-mediated education should take, not just relying on thousands-of-students’ responsible approach [20]. Blended learning proved to be more effective than traditional e-learning [18]; however, mass tertiary education requires tools to improve feedback and communication.

The course format affected the perception of its content. At MU, the IL course was optional, with fewer students compared to Ambis University. The compulsory/optional difference was one of the research limits. Students choose an optional course when they are generally interested in the issue (here IL). Moreover, Ambis tutors provided the learners with weekly individual feedback on their assignments. Due to their large number, tutors did not assign Ambis course participants tasks requiring the teacher’s correction. All six obligatory tests performed throughout the semester were assessed automatically by the information system, without students knowing where there were gaps in their knowledge that they should close. Students who possess a certain pre-understanding of the subject had to attend the compulsory Ambis IL course. The lecturer faced a tricky task to convince them how beneficial the training was to them personally. The above drawbacks affected the students’ motivation to develop their IL skills further, the respective role of motivation confirmed by research [23].

Compared to MU, the knowledge improvement of Ambis University students was insignificant. The results could also be influenced by the fact that MU students, unlike Ambis ones, had an “I don’t know” option in the pretest. When the latter students did not know the answer, they guessed, that is, chose the option that seemed most likely to be correct. That could affect the pretest outcomes and thus the overall rate of improvement. Another potentially influential factor was the attitude of the participants. The study efforts of some private university students are not driven by a long-term vision of the future practical application of acquired skills and knowledge but rather by a short-term pragmatic goal of passing exams and finishing their studies [5]. Since the posttest was not part of the exam, they used only the least possible effort to pass it.

6 Conclusion

The present paper examined the IL level at a public and a private university in the Czech Republic. It compared the respective participants’ self-evaluation, input knowledge, and IL skills improvement. In terms of content and form, the courses were similar. Some format differences, however, might significantly affect students’ competencies.

The research focused on whether the results of private Ambis University students would indicate the same significant post-course improvement as that of MU students. Despite having had a higher entry-level of IL, Ambis students’ competence progress after the IL course was tiny compared to MU course graduates. The underlying reason is the excessive massification of higher education and the subsequent reduced student-teacher interactions and insufficient feedback from the tutor to each student. Because of the massive increase in student enrolment and weak feedback, even a well-designed IL training course seems to be just one of many compulsory curriculum subjects to complete, the substantial reduction of skills and knowledge deficits being of secondary importance.

The results of the research led Ambis University to improve the content and form of the course. More tutors were involved, and they set up more frequent communication with students. We are going to conduct further comparison in the coming year.