Preservice teachers’ perceptions of an ICT-rich learning environment: Development of an instrument

Abstract

This article reports the development of an instrument to measure preservice teachers’ perceptions of the Study Landscape (SLS), which is an ICT-infused learning environment that encourages preservice teachers to direct their own learning to build a two-way relationship between theory and teaching practice. This study comprised a qualitative and quantitative part. Six factors derived from interviews with users formed the basis of the instrument: (1) Support of Learners’ Initiatives; (2) Support of Information Searches; (3) Support of Interaction; (4) Relationship with Fellow Students; (5) Relationship with Teacher Educators; and (6) Relationship with SLS Staff. Once developed, the questionnaire was responded to by 186 students. Analysis of the data supported five out of six factors.

The study landscape: A constructivist learning environment in teacher education

Studies that explore user perceptions of learning environments usually focus on a classroom situation in which students and teachers interact. The learning environment central to this study, however, has little resemblance to a classroom. It is an area of interrelated spaces in which each space represents a particular function. Future teachers are invited and facilitated to direct their learning in this environment, referred to as the Study Landscape (SLS).

This SLS, constructed in 1999 at Edith Stein Teacher Education College in The Netherlands, has a surface of 2400 m². The purpose of the SLS is to facilitate an enactment of the college’s constructivist view on teacher and learning. This view implies that learning is an active and cooperative process situated in real-life contexts (cf. Richardson 1997). As a consequence, the SLS should help to bring about team-based, interdisciplinary work on meaningful tasks utilising a variety of technological means (cf. Salomon 1998).

To stress the interrelations of the SLS functions, its design was guided by a metaphor of a city. In this ‘city’, students address their learning needs by visiting a market place, specialty shops, a terrace and a bar. Table 1 clarifies the SLS metaphor.

Table 1 Explanation of the SLS metaphor

The ‘market place’ consists of information that serves the daily needs of the students. The function of the ‘specialty shops’ is to stress the distinctive features of the Edith Stein Teacher Education College. There is a special program that prepares preservice teachers to teach in Montessori schools; hence one of the shops is devoted to the work of Maria Montessori. After students have ‘shopped’ for information, they process it on the ‘terrace’, which contains a large computer room of 50 computers along with various other rooms for group and individual work. When they want to relax, preservice teachers can go to the ‘bar’, a study café where they can have soft drinks and talk to each other or check their e-mail.

As mentioned earlier, the SLS is meant to be a learning environment that facilitates active, cooperative and authentic learning by prospective teachers. However, realising the potential of a learning environment depends on its users. In this respect, Vermetten et al. (2002) introduce the concept of a ‘subjective’ learning environment that is about how students perceive and use it. This article focuses on developing an instrument to assess how students perceive the SLS.

A new instrument is necessary because of the unique characteristics of the SLS as a learning environment. It is not a single classroom, computer laboratory, or e-learning environment but, at the same time, it is not so wide as to encompass the whole of the school either. Existing instruments that measure student perceptions tend to either examine an enclosed learning environment or examine the whole school level (cf. Fraser 1998a). The development of a new instrument could deepen our insight into student perceptions of a hybrid and open environment and provide a tool for measuring these perceptions.

Research procedure

Instruments that measure perceptions of learning environments often refer to the pioneering work of Moos (1979) in describing human environments (e.g. Fraser 1998b; Fraser et al. 1993; McRobbie et al. 1998).

Moos distinguishes three dimensions: (1) the relationship dimension; (2) the dimension of goal orientation and personal growth; and (3) the dimension of system maintenance and change. The relationship dimension comprises the nature and intensity of personal interactions within the setting. The second dimension measures the basic goals of the learning environment, that is, the areas in which personal development and self-enhancement tend to occur. The dimension of system maintenance and change measures the extent to which the environment is orderly and clear in its expectations, maintains control, and is responsive to change.

In our effort to develop an instrument that measures Preservice Teacher’s Perceptions (PTP) of the Study Landscape (SLS), the three dimensions distinguished by Moos were used as a starting point. However, the environment of the SLS does not resemble the learning environments central in Moos’s work. Therefore, we used the dimensions as a conceptual framework to arrange our findings. In the qualitative phase, which preceded the development of the instrument, we explored what is important in the students’ experience in the SLS. On the basis of these insights, we developed our own scales and related them to Moos’s dimensions for the particular environment of the SLS.

The next four sections elaborate the development of the instrument to measure PTP of the SLS and explore student perceptions of the SLS: a study validating the intentions behind the SLS according to its designers; an overview of an explorative study with students to particularise Moos’s dimensions for the SLS context; the development and validation of the PTP-SLS questionnaire; and student perceptions of the SLS as assessed by the PTP-SLS questionnaire.

Validating the intentions behind the SLS

An articulated and validated description of the ideal SLS was needed to construct the instrument because it gives direction to the PTP-SLS questionnaire. The question that guided this validation study is as follows: “How do the SLS designers express their ideal functioning of the SLS and compare it to the actual functioning?” In order to get a clear and specific picture of the designers’ intentions, it was decided to confront them with a case description. In the case description, preservice teachers were asked to design a history project about World War II for the upper elementary level. All components of the SLS and its corresponding functions were incorporated in the case description. In this description, the researchers specified their view on how the ideal SLS would work. This case was discussed in an interview with two of the SLS designers. To detect discrepancies between the ideal and the actual functioning, attention was given to both in the case and interview. The results of the interview are summarised in Table 2.

Table 2 Interview results: Case description, SLS function and designer intentions

We concluded that the ideal SLS as represented in the virtual case is in line with the view of the SLS designers. The SLS offers many facilities (computers, books, etc.) to help students to obtain the information that they need. Affordance in face-to-face and virtual communication among students is well established in the SLS. Because human support is also considered important, SLS employees should be available to help students and teacher educators. The SLS facilitates interaction between the theoretical part of the curriculum and the practical work in elementary schools.

However, it became clear that the ideal SLS, as pictured in the case, is still developing. In particular, the role of the teacher educator in the ideal SLS is more profound than it is in the realised SLS. Also, in the case description, the explicit way in which the SLS is integrated and interwoven into students’ curriculum doesn’t correspond to the actual situation.

The insights gained from this part of the study are used in the next section in which the target group, teacher education students, are the focus of attention.

Towards factors in student perceptions

In the second and exploratory phase, the connection was made between the factors derived from the literature and the conclusions of the first part of the study. The following research question guided this part: “How do students perceive the SLS?” This part of the study aimed to identify factors of students’ experience in the SLS. Initially, instruments previously used to measure experience were skimmed to gain preliminary insight into possible factors determining student experience of the SLS. These instruments were also used to construct a set of plausible factors of experience. Next, individual semi-structured interviews were carried out with three students and three teacher educators. These persons were selected by means of purposive sampling. We looked for students and teacher educators who are often present in the SLS and are articulate and eager to talk about their experiences. These interviews were used to acquire better insight into the environment, to get a first impression of the student experiences in the learning environment, and to give an indication of the feasibility of the listed factors for the intended instrument. A checklist with SLS components and activities facilitated by the SLS guided the interview. The data from this study were transcribed and ordered in Contact Summary Sheets (Miles and Huberman 1984). The individual interviews resulted in a large list of items that might play a role in student perceptions of the SLS. These items were then ordered according to Moos’s three dimensions. This list provided input for the group interviews, which were the main tool of investigation in this part of the study.

In the group interviews, 16 preservice teachers participated. The most talkative and critical students were selected using the intensity sampling strategy because we wanted to obtain as much relevant information as possible (cf. Patton 1990). The factors and underlying variables resulting from the individual interviews guided the four group interviews.

During the interviews, students were invited to give their opinions of the SLS by naming three strong and three weak points of it. They were then asked to think of a concrete assignment that they executed in the SLS. Considering this assignment, they were asked about the relationship with different people in the environment, their interaction with and use of different materials. Finally students were asked about their experiences with specific units in relation to that assignment. Figure 1 illustrates the structure of the interviews and following analysis.

Fig. 1

Structure of group interviews and analysis

Transcriptions of the interviews served as input for analysis. They were analysed according to the structure described above and are presented below in coincided form.¹

Strong points of SLS

The students mentioned the availability of computers and other materials as the greatest advantages of the SLS. The diversity of the equipment and work area make for a stimulating environment where they can undertake different activities within one large area. In this kind of environment, students can work and socialise at the same time.

Weak points of SLS

The technical facilities are a main source of student frustration because they are often occupied or malfunctioning. The library’s search system is also unclear for the students. The ‘teaching materials centre’ (where students can borrow materials for use in schools and get advice about suitable activities) is not open often or long enough. Also, according to the students, some staff members are not always able to provide adequate information or assistance.

Relationships with people

Students

Students interact intensively with each other both inside and outside the SLS when conversing about the assignments and work strategies. However, most of their assignments are executed individually. In the SLS, students coordinate and discuss collaborative assignments, but most students divide them into individual parts and work separately on them. They then join the different parts at the end with little or no additional editing. When students work together, they do so in the SLS as well as at home, depending on the availability of resources and their schedule.

Staff

Students especially appreciate the staff who have content-related roles in the SLS—those who work in the ‘teaching materials centre’ and the specialised units. The interaction with them leads to concrete solutions for classroom activities. The relationship with other staff members with general assistance tasks is less satisfying.

Teacher educators

Teachers and students rarely interact within the SLS because teacher educators are not often in the SLS. However, students indicate that teachers are available for assistance when needed but that it takes place outside the SLS.

Relationships with the system

Computers

Computers in the SLS are used intensively. Much of their use is devoted to short tasks such as checking e-mail and finding information on the local intranet. The use of computers for writing reports or completing school assignments varies among students. Many students prefer to work at home where they are sure of having a computer available with no disruption. Others try to work specifically in the SLS because then they have all the materials within reach and can consult other students when necessary.

Media collection

Students make selective use of the available literature. All of them use the available school books for elementary education to search for interesting ideas for classroom activities. Other professional literature is used infrequently. Not many assignments demand the use of additional literature above the compulsory study materials. Additional literature is difficult to find using the electronic search system.

Study rooms

The small individual study rooms are viewed as pleasant working spaces. They make it possible for students to isolate themselves and work individually or in pairs. All students mentioned that theses rooms are always fully booked. Therefore, if they want to make use of them, they have to reserve them far in advance.

Group facilities

These rooms are used less frequently for study purposes. Most of them have no computers, but students state that computers are necessary for working in the SLS. The rooms with computers are always fully booked and are therefore not available when students want to use them.

Study café

Use of the study café varies between students. The flexible rules of the study café make it a pleasant place to work in. Students can talk freely and eat while they work. Students use it to confer about assignments rather than for writing them.

Use of specific units

The use of the specific units varies significantly. Units that have a permanent staff available are more popular with the students and are used frequently. Units that are not staffed are seldom used. When they are used, it is mostly for activities that are not aligned with the specific purpose of the unit like meetings.

Following the first analysis, these experiences were interpreted with use of the three dimensions identified by Moos (1979). The general factors and dimensions were related to the specific situation of the SLS. The results and conclusions are summarised below according to the three dimensions of Moos (1979):

• Goal dimension: Participants perceive the SLS as a place where students interact, organise their group activities and find information. At the same time, the intensity of learning-related collaboration is not very high. The best example of active learning within the SLS is the preparation for the teaching practice; the SLS is a suitable environment for finding the right textbooks and other information.

• System dimension: The students use some parts of the physical environment intensively. They use the group rooms and computers when working on their assignments, but also for reading their e-mails and checking their marks. The SLS helps them to prepare for teaching practice by providing materials and by offering facilities such as computers and copiers. However, students do not regard the unmanned units as functional parts of the SLS that facilitate their learning. The units do not encourage student activities in the way in which the group rooms, the library or the computer room do.

• Relationship dimension: The SLS seems to serve as a central meeting point for students. Relationships with their fellow students are perceived as the most intensive and substantive of all relationships that they have within the SLS. The role of teacher educators in the SLS is negligible. Other staff members in areas such as administration, computer help desk and room reservations support the processes in the SLS, although it is difficult for students to have a clear understanding and expectation of these staff members’ roles.

For students, the SLS appears to serve as a central meeting point, where their relationships with fellow students are perceived as the most intensive and substantive within the SLS. The SLS helps them to prepare for teaching practice by providing materials and by offering facilities such as computers and copiers.

Next, the factors that play a role in student perceptions of the SLS were formulated. The starting points were the original ideal functioning of the SLS and what the teacher education college is pursuing in relation to the student experiences in the SLS. The results of the student interviews were used to give the ideal SLS a form and content which students could recognise and to which they could relate, as well as enabling students to see which experiences in the ideal SLS are relevant to them. In a deliberative and iterative process, the results of the qualitative inquiry have been reorganised and regrouped in relation to the experiences in the ideal form. This has led to the identification of six factors that were deemed prominent in relation to the students’ experiences in the SLS and comparable learning environments. This has also led to the contextualising of the factors in the students’ current experience, thus making them recognisable to the students. The identified factors are as follows:

• 1. Support of Learners’ Initiatives: extent to which there are possibilities for learners to develop their own initiatives

• 2. Support of Information Searches: extent to which the SLS facilitates the learner’s need to search for information

• 3. Support of Interaction: extent to which interaction that facilitates learning is supported by the SLS

• 4. Relationship with Fellow Students: extent to which a positive relationship with fellow students is facilitated by the SLS

• 5. Relationship with Teacher Educators: extent to which a positive relationship with teacher educators is facilitated by the SLS

• 6. Relationship with SLS Staff: extent to which a positive relationship with SLS staff is facilitated by the SLS.

Three of these factors belong to the system and goal dimension (Support of Learners’ Initiatives, Support of Information Searches, Support of Interaction) and the others belong to the relationship dimension (Relationship with Fellow Students, Relationship with Teacher Educators and Relationship with SLS Staff). In this instrument, Moos’s third dimension of goal orientation and personal growth was not defined as a separate dimension of experience. This is due to a slightly different conceptualisation of the system dimension in this environment. In other instruments, the system dimension is restricted to the extent to which the environment is orderly and clear or responsive to change. Due to the integrated nature of this learning environment, the system dimension in this instrument is seen as the extent to which the system in its physical characteristics is supportive to the goals of the environment. In this way, the goal dimension is practically integrated into the system dimension and therefore does not appear as a separate dimension. We therefore use the title of ‘system and goal dimension’ for this dimension.

Development of the instrument

After defining the prominent factors in the SLS experience of the students, the next step was to construct an instrument to measure these experiences in a systematic way that can be repeated at regular intervals.

Construction of the instrument and pilot study

Based on the outcomes of the qualitative inquiry, items were formulated for every factor mentioned in the previous section. The learning environment’s different components were examined for their relevance for each factor. The ‘study café’, for example, is relevant for the support of interaction but not for the support of student initiatives or information searches. The ‘teaching-aids centre’ supports students’ initiatives for lesson planning but has no function in supporting interaction or information searches. In relation to this, items were constructed for every assumed relationship between an SLS component and a factor in the experience. Attention was also given to possible differences in goals of the students’ work in the SLS. This was done to avoid confusion in interpreting the items because an activity in the SLS can have divergent goals: a specific assignment for a course, a lesson plan for practical training, or a question arising from their practical training that they want to explore further.

Some factors have more items than others because they are more prominent in the SLS and are therefore manifested in more learning environment components. Table 3 shows the number of items per factor as well as examples of items.

Table 3 Number of items and sample item for each scale

Items have a five-point Likert response scale consisting of Totally Agree, Mostly Agree, Neutral, Mostly Disagree, and Totally Disagree. Moreover, the response category No Opinion was added to distinguish between answers of respondents who are neutral about an element and those who have had no experience with it. Based on the qualitative exploration, we believe that some parts of the intended learning environment are not yet fully realised and that therefore some students have not yet experienced these elements. Distinguishing between these options can deliver important information about the realisation of the different SLS elements. This can help in avoiding possible confusion around the Neutral answer and perhaps heighten the reliability of the responses to it.

The pilot study consisted of an expert appraisal and tryout with six preservice teachers. The results of the pilot study were important in ‘debugging’ weak areas of the questionnaire to make the items more understandable for the preservice teachers.

Sample and procedure

A stratified sample of 186 preservice teachers out of a population of 982 was asked to participate. The sample was stratified according to the study years. Of each level (ranging from first to third year students), three whole classes were randomly selected to participate. Students from these nine classes were asked to fill out the questionnaire during class time. This procedure was done in order to ensure a high response rate. This was deemed more important at this stage of instrument construction than a totally random sample. Responses were obtained from 183 students, which represents a 98% response rate. After a short introduction, the participants were asked to complete the questionnaire, which took about 20 minutes. The data were analysed by means of computing Cronbach’s α coefficient for the pre-established scales and the discriminate validity, using the mean correlation between the scales as a simple index. (The sample was not large enough to conduct factor analysis.)

Reliability of the questionnaire

Table 4 shows the values of the α coefficients computed for the factors of the instrument. The values vary from 0.66 to 0.85. Except for one factor, Relationship with Teacher Educators, all factors can be considered reliable (following Spector’s [1992] minimum reliability value of 0.7). A possible explanation for the fact that Relationship with Teacher Educators could not be measured reliably is the lack of the teacher educator’s role in the SLS as was seen in the interview results. However, the questions related to this factor have not yet been removed from the questionnaire as this factor is considered important to the initiators of the SLS and is expected to be developed further in the future. Two items were eliminated from the questionnaire to improve the reliability of two scales without lowering their content validity. In hindsight, the content of the items was not considered to have added value or provide additional information.

Table 4 Reliability and discriminant validity for PTP-SLS scales

The discriminant validity varies from 0.32 to 0.45. These values indicate that the constructed factors measure different aspects of the environment. At the same time, a degree of overlap is noticed. This might be because of the construction of the SLS. After all, components of the learning environment do not just achieve one goal; they influence multiple factors.

Exploring student experience

The first results from use of the instrument are discussed further in this section. Discussing results from using an instrument is not customary in studies that deal with the initial construction of questionnaires and of scales that point to factors in the concept measured. The focus of these studies is to establish the reliability of the scales. They are primarily based on factor analysis to check or establish a factor structure. In this case, the structure of the scales was not altered as a result of the examination of the scales’ reliability. Therefore, initial results are examined. Moreover, considering the growing popularity of varied and ICT-infused learning environments, a short depiction of the students’ experiences of the SLS can contribute to an understanding and the design of such a learning environment.

The results for the different scales in Table 5 show how the students experienced the different dimensions of the learning environment. The mean of each scale can vary from 1 to 5. The factor Support of Interaction has the highest mean (3.81). Students seem to experience the SLS particularly as a place where they can interact with their peers at different levels. This conclusion is strengthened because both factors of the relationship dimension (Relationship with SLS Staff and Relationship with Fellow Students) have the next highest means (3.66 and 3.51, respectively). These results are also corroborated by the interviews held in the first stage of the research which showed that students see the SLS predominantly as a social interaction environment and to a lesser extent as an environment that supports initiatives and information searches.

Table 5 Mean standard deviation and sample size for each scale

Support of Learners’ Initiatives has the lowest mean of all scales (2.88), which leads to the conclusion that students experience this aspect in a limited way compared to others. Students feel that, as of yet, they are not supported and stimulated through the learning environment to take initiatives and explore their own wishes in the SLS. Factors of the system and goal dimension (sample sizes of 75, 82 and 95) have a notably lower response rate than those of the relationship dimension (N = 161 and 122). This might be attributed to the students’ clearer perception of these interpersonal aspects rather than the system’s support of different aspects of their work.

A representation of the results in the form of box plots (Fig. 2) illustrates the different scoring pattern of the different scales. Factors of the relationship dimension, though tending to have higher means than the other factors (in terms of median and mean), show more spread in the results. This is also the case with the standard deviation (both have a SD of 0.59). This also highlights that Support of Interaction has the lowest spread and standard deviation, but the highest mean and median. The spread of results in the box plot for Support of Interaction is even smaller when the score is under the median. Students seem to experience this factor in a similar way and agree on this aspect’s level of success in the learning environment. This might support the initial finding that Support of Interaction is an aspect of the learning environment that has been accomplished in the best way so far.

Fig. 2

Box plots of the results on the different scales. Note: Circles represent the outliers in the different scales. These outliers lie more than one and a half inter-quartile length under the middle 50% of the scores

Summary and conclusions

The Study Landscape (SLS) at Edith Stein Teacher Education College is designed as an ICT-rich learning environment that supports active, self-directed and cooperative learning among preservice teachers. ICT facilities are integrated into the environment by means of up-to-date computer hardware, Internet access, multimedia cases of exemplary teaching practices, an e-learning environment and facilities for video recording and editing.

This study involved the development of an instrument for measuring the way in which preservice teachers experience the SLS. In the first part of the study, we gained insight into the factors that influence the way in which preservice teachers experience the SLS. We started our investigations by validating the designers’ intentions of the SLS. Based on existing documents, we constructed a case-description, representing the work of students in the SLS, in which all of the functions and components of the SLS were incorporated. Based on this case, we interviewed the designers. The conclusion of this first part of the study was that the functions included in the case covered the designers’ intentions for the SLS. Moreover, the case helped to give these functions a more concrete focus. In hindsight, we think that doing a well-focused interview on the basis of a case description is a good way to initiate a conversation about basic rationale and intentions underlying a learning environment.

In the second part of the study, we conducted individual and group interviews with preservice teachers who frequently visit the SLS. From this, we learned that the preservice teachers do not perceive the SLS as an integrated learning environment that consists of interrelated parts. They seem to perceive the different components as independent facilities that can be used according to their specific needs. Preservice teachers are especially attracted to the SLS by the opportunity to interact with peers and obtain information directly related to student teaching.

Based on the results of this study, six factors of experience were defined: (1) Support of Learners’ Initiatives; (2) Support of Information Searches; (3) Support of Interaction; (4) Relationship with Fellow Students; (5) Relationship with Teacher Educators; and (6) Relationship with SLS Staff.

The focused group interviews provided insight into the learning activities within the SLS. Interviewing a group of preservice teachers turned out to be a rich and useful source in gaining insight into how students perceive and interact in and with the SLS.

Based on the results of the qualitative studies, a 76-item questionnaire was developed with six scales. Five out of six scales were found to measure the students’ experiences in a reliable way. The value of Cronbach’s α coefficient for different scales ranges from 0.66 to 0.85. One scale (Relationship with Teacher Educators) could not be classified as reliable because the Cronbach’s α was lower than 0.70. A possible explanation for this result is that the teacher educator’s role in the SLS, though part of the ideal operation of the learning environment, has not yet been sufficiently realised. An initial exploration of the results of administering the instrument showed that the SLS, at this stage, is predominantly experienced by students as a place in which to interact with their peers and the SLS staff. However, its function in supporting student initiatives and information searches is experienced to a lesser extent. These results, validated by responses gathered in interviews at the qualitative stage of this research, highlight the importance of the human factor in the learning environment. The relationships and interactions in the learning environment play a central role in the students’ experience and seem to strongly determine the use of the environment and its different parts. They also give the different components of the SLS their functionality. Functions and components of the SLS that do not include interaction seem to be experienced as less functional. This can also be true for other learning environments; overall, it stresses the importance of considering the nature and intensity of human relationships in the design of learning environments.

The absence of teacher educators is unfortunate because how the teacher interacts with the student in various circumstances seems to have a very positive influence on group learning both inside and outside the classroom (Yan and Kember 2003). Therefore, it seems highly advantageous for the teacher’s role in relation to student work in the SLS to be strengthened in the future.

This study focused on the reliability of the PTP-SLS questionnaire and, to a lesser extent, its validity. When the discriminant validity between the scales was computed, some scale overlap was found. This is not surprising because the components of the SLS can serve various, interrelated functions. For example, the computer facilities might support the search for information as well as communication between students.

We also strived for validity by integrating previous research into our line of thinking and by conducting small-scale qualitative investigations at the beginning of this study. These small-scale studies proved to be very useful in the design of the PTP-SLS questionnaire. They also helped us link previous research on learning environments to the unique characteristics of the SLS, which is an open, hybrid ‘click and brick’ learning environment. Moreover, the qualitative studies helped us to distinguish between the ideal and the existing SLS. Last but not least, the qualitative studies helped us to learn the language of our respondents, which was beneficial in formulating the items of the PTP-SLS questionnaire.

Further development of this instrument and others like it is required both in the context of the Study Landscape of Edith Stein Teacher Education College, as well as in other constructivist ‘hybrid’ learning environments.