1 Introduction

The Israeli Jewish educational system consists of preschool, elementary, junior high and high school levels and is divided into three major sectors, namely state secular, state modern-orthodox and ultra-orthodox administered by independent departments under the umbrella of the Israeli Ministry of Education. These educational sectors exist side by side and enjoy sectorial autonomy with inspectors, who belong to the different sectors, responsible for supervising the educational process in each sector. The Ministry of Education coordinates the educational processes that characterize the different sectors and is responsible for the teacher certification of all teachers in all three sectors as well as for the curriculum in the state secular and state modern-orthodox sectors. For religious and ideological reasons, within the framework of the National Education Act, the ultra-orthodox sector was granted by the KnessetFootnote 1 (1953) a significant measure of autonomy regarding the curriculum implemented in its schools. The present study focuses on Talmud Torah (TT) schools in the ultra-orthodox sector.

The aim of the researchers in the present study was to analyse the study skills of Talmud Torah students in a newly introduced GemaraFootnote 2 study program in TT schools. This study is one of the first studies conducted in this topic for two reasons: Firstly, no one except for Dembo et al. (1997) has ever studied Talmud Torah schools (Spiegel 2011), and there are no studies on teaching Gemara, which is a very special and central subject in these schools. Although there have been various sociological studies (e.g., Caplan 2007; Freund and Band-Winterstein 2013, 2016; Friedman 1986a; Taub and Werner 2016) and psychological (e.g., Feinson and Hornik-Lurie 2016; Grazi and Wolowelsky 2015; Hess 2014; Weiss et al. 2013) performed on the ultra-Orthodox community, no research of ultra-Orthodox schools has been undertaken due to the community’s strict refusal to allow external scrutiny of its school system. Therefore, the Israeli Ministry of Education has only minimal educational and organizational jurisdiction over learning and education in Talmud Torah schools, and it is completely unprecedented for researchers to be given access to teaching and pedagogical processes in these schools.

Secondly, the study and teaching of Gemara are very special (Brandes 2016; Hayman 2009; Levinsohn and Fendrick 2013) and radically different from other disciplines. This stems from the Gemara’s unique religious importance, its textual and linguistic features and its centrality in the school curriculum.

2 Background

2.1 Talmud Torah system

Talmud Torah schools are semi-private elementary schools for boys in the Jewish ultra-orthodox sector. In Israel, the ultra-orthodox boys’ school system is the ideological and social heart of ultra-orthodox society, which views the religious injunction to study Torah and live strictly by its precepts as a sacred value (Caplan 2007; Spiegel 2011). The Talmud Torah system moulds its students and influences their lives and their families. These schools serve a conservative society, in which ultra-orthodox tradition is paramount and Jewish religious precepts are strictly obeyed. Ultra-orthodox communities generally live in separate ultra-orthodox neighbourhoods and settlements where they follow special codes of conduct. Ultra-orthodox society is a conglomeration of ultra-religious communities, where each has its own mind-set and behaviours. The Talmud Torah schools are the basis for the growth and existence of the diverse ultra-orthodox communities they serve.

The ultra-orthodox society in Israel has grown in size and strength in recent decades (Siebzehner and Lehmann 2014). The ultra-orthodox society is so large, that in the early 2000s it numbered several hundred thousand, compared to the early decades after the establishment of Israel, when it was a marginal social group of a few tens of thousands (remnants of the ultra-orthodox Jewish world destroyed in the Holocaust). It is estimated that in present-day Israel, 150,000–180,000 children attend ultra-orthodox schools (Spiegel 2011; Truaan 2007; Vurgan 2007). In 2017, about 30% of Jewish elementary school students (boys and girls) belonged to the ultra-orthodox sector of the Jewish population.

Ultra-orthodox schools differ from the secular and the modern orthodox Israeli state schools in that they are semi-private, and are funded by the state for up to 75% of their budget, with the balance usually coming from philanthropic contributions. The state has very little control over TT pedagogy, administration, or budget, which are autonomous, even though the state jointly funds this system.

TT schools do not teach the Israeli state curriculum. The TT curriculum mainly consists of Jewish religious subjects, with a few weekly hours devoted to secular studies (including mathematics and Hebrew language), depending on the grade level. Secular subjects are only studied at a very basic and elementary level, and TT graduates therefore do not obtain a matriculation certificate and cannot enter higher education. Their schools resist any formal external evaluation, including the standardized Israeli student assessment tests (Meitzav), or international tests such as PISA or TIMSS (National Authority for Measurement and Evaluation in Education [RAMA], 2017). Not only does the state not interfere in the Talmud Torah curriculum, there is also little intervention in teacher training and supervision, contrary to the state school system. Most of the teachers in TT are graduates of Yeshivas (theological colleges) who participated in a short training course for becoming teachers.

Any discussion of ultra-orthodox society needs an understanding of the ethos of Torah study and its key role in ultra-orthodox society (Caplan 2007). This unique ethos stems from two Biblical injunctions: “This book of the law shall not depart from your mouth, but you shall meditate on it day and night” (Joshua 1:8) and “These commandments that I give you today are to be on your hearts. Impress them on your children. Talk about them when you sit at home and when you walk along the road, when you lie down and when you get up” (Deuteronomy 6:7). In the ultra-orthodox world, religious study (mostly Gemara) is therefore a sacred requirement, unlike any other subject of study.

One result of this is that ultra-orthodox society is a learning community which prizes learning above anything else. Ultra-orthodox men study for many more years than the norm in Israeli society, so much so that it is considered a “society of learners” (Friedman 1991). Furthermore, ultra-orthodox society reveres great religious scholars and spiritual leaders who show remarkable intellectual strengths and abilities in intensive Gemara study and have a great capacity for learning. These are the real and primary role models for all ultra-orthodox boys. Another result is that the subjects studied in ultra-orthodox schools are almost all religious. They include Bible, Mishna, Gemara, Halacha (writings on Jewish law), each with its distinctive knowledge structures. In traditional Yeshivas (beginning at high school age), Gemara study exceeds all other subjects in terms of time, effort, and perceived prestige (Spiegel 2011).

2.2 Gemara study

The analysis of the teaching of religious texts is a challenge for researchers (e.g. Carroll and Collins 2015; Ryan 2014) in this research we focus on a unique text—the Gemara. The Gemara is an ancient text. It was finalized in the fifth centuryFootnote 3 and is the basis for halachic law, which governs all aspects of orthodox Jewish life. The Gemara is a compilation of 37 books, summarizing the main halachic (jurisprudence based) and aggadic (folklore based) thinking of leading rabbinic scholars (Kulp and Rogoff 2014; Schlesinger et al. 2011). In many ways, Gemara study and teaching is unique. According to Hayman, the Gemara is:

Unlike any text in western civilization. It is not text at all. A western text is generally the work of a given author, from a given place and time presenting a given story, thesis or experience. Gemara is the work of hundreds of authors, from multiple places and times, presenting laws, stories, theses, and experiences… (Hayman 2009, p. 96).

Thus the Gemara contains references and quotations from a wide variety of sources. These quotations all refer learners to other sources. If we translate this into modern terms, then Gemara text is analogous to a multi-layered hypertext (Rosen 2001). In addition, it is not a continuous text and is unlike any other text. The sugiyotFootnote 4 (plural of sugiya) of the Gemara are based on exchanges between Gemara scholars and describe discourses between people from different eras as if they are conducting a face-to-face dialogue (Weiss-Halivni 2013). Furthermore, the Gemara is based on an inductive process of generalization following case analyses and situations that describe everyday life in the ancient world. For example, who is legally responsible for a bull that escaped from the barn and caused damage (Steinsaltz 2014). Today’s student has difficulty understanding the social-economic-cultural context of the situation.

Students of Gemara must therefore understand the different approaches to the topics they are studying and the approaches of the discussants involved as well as the historical period of the discussions.

Two additional characteristics contribute to the particularly challenging nature of teaching Gemara: language and high-order thinking. Hebrew is not the only language needed to study Gemara, since the text is almost entirely in Aramaic. Furthermore, as indicated by Brandes (2016), the study of Gemara needs teacher mediation that is not only based on a literary understanding of the text but also on an understanding of the cultural background underlying the discourse. Therefore, students must learn and master this language in order to learn Gemara. In the cognitive domain, learning Gemara is an especially challenging intellectual undertaking, which peaks in the Pilpul—a sharp dialectic analysis of the text through intense textual dissection in an attempt to either explain conceptual differences between various halachic rulings or to reconcile any apparent contradictions presented from various readings of different texts (Schloss 2002).

In light of the above, it can be seen that the Gemara is not only unique in its textual characteristics, but also because of its central role and special place in the ultra-orthodox student’s world. As noted above, the Gemara is the main text studied in the vast majority of ultra-orthodox boys’ schools. The students have no secular education from the age of 12 and know little apart from Gemara. In this system, there are no other academic options or alternative channels for learning and development for students who find it hard to study Gemara. We can postulate the distress of youngsters, parents, and teachers who struggle in this sphere.

As mentioned, these are conservative schools in which learning centers on one subject only, and the students are required to exhibit high ability in this field. Furthermore, this is a closed society in which those excelling in the learning of Gemara are admired and even revered. Therefore, it is possible to understand the distress of children who are not enthusiastic about the subject or have major difficulties in learning this subject. These children have few possibilities for development in their school.

This is the fertile soil in which Rabbi Yeshayahu Weber’s method developed. Rabbi Weber is a well-known Gemara scholar and an experienced Gemara teacher. He is self-taught with no formal psycho-pedagogic training. Rabbi Weber first chose individual students who exhibited difficulties in learning and understanding Gemara, since if a student has difficulties learning Gemara, there are no other subjects in the Talmud Torah system which he can enjoy and in which he can succeed. After his method proved to be successful among the children who had difficulties, Rabbi Weber decided to introduce it into the classroom. In his opinion, if the achievements of these children in Gemara will improve, their standing in the school will also improve. The program focuses on the individual’s needs and learning abilities. The current research focused on Rabbi Weber’s new program for teaching Gemara (Weber 2002).

2.3 Rabbi Weber’s method

The goal of Rabbi Weber’s Gemara teaching method is to equip students with the tools necessary for independent study. The most important goal emphasized in this method is to understand the discourse of the Gemara. Coverage of large quantities of material is a secondary goal (Weber interview 12.7.15). The method is designed to provide Talmud Torah students with a strong basis and significant benchmarks which will help them study Gemara for the rest of their lives. According to Rabbi Weber, his approach is the most effective, systematic method for studying Gemara. It trains students to study independently, extensively, and on a high level, which is the requirement for them in their Yeshiva K’tana (religious studies high school) and afterwards. This method includes five elements in learning Gemara: conceptual understanding; pre-textual thinking (i.e., thinking about the text before actually studying it); organizing and processing the information in the sugiya; text comprehension and oral and written expression.

According to Rabbi Weber’s method, the Gemara lesson begins with a discussion between the teacher, known as a melamed, and his students. This preliminary discussion precedes engagement with the text which consists of two components: conceptual understanding and pre-textual thinking.

The pedagogy used in the Weber method intuitively echoes Vygotsky’s (1978) idea that mediation functions according to the principle of “zone of proximal development” as explained by Feuerstein et al. (2010). Thus, when analyzing the pedagogic characteristics of the Weber method, mediation between learner and text should be borne in mind.

2.3.1 Conceptual understanding

The initial stage of the lesson aims to help students understand the concept with all its different strata, beyond the dictionary meaning, and to understand the non-literal meaning of the text. The melamed discusses the idea and shows his students how it refers to everyday situations that are understood by the students who are then able to analyze the problem using their experience and common sense. The goal is for the students to grasp the concept and its boundaries. For example, the idea of “a lost object”. The questions associated with this issue include: How do I know that something I found is lost? If I find a rag in the street, did somebody lose it? Can something with no monetary value be treated as lost? Such discussions help the students understanding the notion of “loss” and to what it can be applied.

2.3.2 Pre-textual thinking

In this stage, the students have not yet looked at the sugiya text. The melamed describes an everyday situation with which the students are familiar, which resembles the situation discussed in the Gemara. For example, if the sugiya deals with a situation in which a person finds some baby chicks behind a fence (Bava Metzia 25b), the melamed can ask the class what would happen if I found something behind the door of a shop. The melamed leads the discussion and helps the students understand the complexity of the situation and that things are not as simple as they seem. The melamed describes different alternatives of changing scenarios and encourages complex hypothetical thinking using concepts and situations learned in the past to enable students to consider differences and similarities between different situations. During the discussion, the melamed uses different syntactical structures, first and foremost conditional phrases (“if…..then”) and cause and effect phrases. The aim of the “pre-textual thinking” stage is to prepare students for the type of dialogue they will find in the Gemara and its discussions.

The next two stages take place with the Gemara page open and occur simultaneously.

2.3.3 Organization and processing the information in the Sugiya

In this stage, which is regarded as cognitively more complex, the student begins to read the sugiya in the Gemara. During the dialog between the melamed and his students, he encourages them to organize the general picture in the Gemara, with all its different elements. This discourse helps students to process everything in their working memory, and to ascribe different weights to all the elements they learned about in the first two stages and use them to understand the situation described in the Gemara.

Sometimes they do not read the entire sugiya at once. Rather, they slowly introduce new ideas from the text. The melamed encourages the students to think hypothetically using the whiteboard to organize the various situations described in the Gemara which are reflected in the class discussion. To do this, the melamed must plan what he will write on the board to ensure it is well-thought out and clear. His aim is to create categories and criteria that will help organize the information in the sugiya and summarize the principles presented in the Gemara.

2.3.4 Reading comprehension

This element goes hand in hand with the Organizing and Processing stage. The main goal of this stage is to enable students to understand the language in which the Gemara is written and its special linguistic codes. The language of the Gemara is extremely condensed, with nuances that require accuracy and acute discrimination. Dealing in the linguistic codes and the steps in the sugiya helps students to understand the kind of thinking needed for Gemara study.

2.3.5 Expression

Throughout all these stages, the students are required to describe the concepts, the Gemara debate, and the argumentations on the issue in their own words. The ability to describe what the Gemara is saying in their own words is an important aid for their understanding. It enables the student to untangle knots and makes the complex Gemara text more understandable.

Rabbi Weber’s approach is unique in that it is student-centred. It focuses on the student and his difficulties and abilities. This contrasts with the traditional approach, where the focus is on the text and the text is learned according to the order in which it appears. In the traditional method, the teacher stops to explain if he notices that his students are having problems comprehending the issue. According to Rabbi Weber’s method, learning the text is secondary to helping the students develop their understanding and ability to study the Gemara (Iluz and Katz 2012; Weber 2002).

The differences between Rabbi Weber’s method and the traditional method can be summarized in that Rabbi Weber believes that although students learn fewer sugiyot with his method than with the traditional method, their learning has more depth and is essential for developing the students’ understanding of the Gemara and its stages, the language codes it employs, and the level of abstract reasoning it requires. Rabbi Weber claims that this is an effective way for cultivating an independent learner who will be able to decipher dialectical arguments which comprise the core of the Gemara text.

3 Research hypothesis and questions

The research hypothesis was that the comprehension test scores of the students who studied using Rabbi Weber’s method will be higher than the scores of students who learned through the traditional method on both the general measure and on the test components (informative knowledge, deduction and application, prior knowledge of Gemara, Aramaic vocabulary).

An additional question to be asked is “what is the relative contribution of each of the tested abilities (as tested prior to the intervention): general ability, verbal ability and knowledge of Aramaic vocabulary for the specific sugiya) to the score in the Gemara comprehension test.

4 Method

The research compared Rabbi Weber’s method with the traditional method by measuring students’ learning achievements.

4.1 Participants

The participants included 159 sixth graders from two Talmud Torah schools in Israel who came from similar ultra-orthodox and socio-economic backgrounds. One school consisted of the students in the experimental group (77 boys) and the other school was comprised of the students in the control group (82 boys). The schools are located in ultra-orthodox areas and serve closed ultra-orthodox communities which are part of the mainstream of ultra-orthodox society. It should be noted that these schools were chosen because they agreed to cooperate with the research process even though the ultra-orthodox sector does not customarily cooperate with researchers.

4.2 Instruments

Two types of research instruments were used: tests examining student abilities and a Gemara comprehension test.

The students’ pre-intervention level in both groups was tested using: 1. A General Ability Questionnaire, 2. Verbal Ability Questionnaire; 3. An Aramaic Vocabulary Test. The post-intervention test was: 4. A Gemara Reading Comprehension Test. Tests three and four were developed especially for this research.

Raven’s Standard Progressive Matrices (Raven 1958; Raven et al. 1998) was used as a general ability test. This test is usually used to examine an individual’s understanding of relations between abstract items. The easier sets require the subject to accurately differentiate between items. The more difficult sets involve analogy, shape changes, and exchanges in patterns and other logical connections (Anastasi and Urbina 1997). The test includes five sub-tests (A–E), with 12 shapes, giving 60 shapes in all. In each sub-test, one element is missing, and respondents must identify the missing element and complete the pattern of 6 or 8 pictures (Janda 1998). Raven’s Test was scored according to Glantz (1989), which was validated for Jewish students in Israel and attained a Cronbach alpha reliability coefficient set at 0.85.

Verbal Ability Test (Glanz 1989) was used to examine the verbal ability of the students. This is a multiple-choice questionnaire with three sections from Glanz’s Abstract Verbal Ability Test. The sections were: synonyms (identifying the word closest to the base word); antonyms (identifying the word closest to the opposite of the base word), and interpreting proverbs in traditional Jewish texts. Each student is graded on each of the three measures, and a mean score is calculated as the verbal ability score (α = 0.76).

Aramaic Vocabulary Test (baseline) test was especially constructed for this study in order to compare the groups’ baseline Aramaic vocabulary skills. The test examined Aramaic vocabulary, since Aramaic is the main language used in Gemara study and is essential for understanding the Gemara text. The vocabulary test consisted of a 20-word multiple-choice test (α = 0.87). The test was compiled by the research team after consulting teachers who are familiar with the material taught to sixth grade TT students. Scores were converted into percentages for standardization.

We first administered the test questionnaires to eight students and then made minor changes in the formulation of the questionnaire. Items with a very low answer distribution were deleted. The questionnaire was then given to four experts in teaching Gemara for validation. These experts analyzed the questions separately and expressed a high degree of agreement regarding the content validity of the test.

Gemara Comprehension Test included 20 open and closed questions and was the main research instrument developed for this study (α = 0.66). Its aim was to test the comprehension level of the student in a sugiya which was learned by both groups of subjects, as detailed below.

We should note that because of the unique nature of the Gemara texts, a model that could reflect the different levels of student understanding was needed. The test was therefore based on Pearson and Johnson’s (1978) model, which represents a reading comprehension taxonomy: “…designed to capture the relationship between information in the text and that which has to come from the reader’s store of prior knowledge” (Chikalanga 1992, p. 700).

This taxonomy includes “three types of question–answer relationships so that comparison of their performance on the three types of questions could be analyzed. The largest portion of textually explicit (TE) questions reflected a stress on such basic reading skills as perceptual match and recall of details explicitly cued in the language of the text. Textually implicit (TI) questions and “scriptually” implicit (SI) questions are equally divided, which measure meaning construction and prior knowledge” (Wang 2006, p. 21).

Four measures were constructed in the Gemara Comprehension Test based on this taxonomy: (A) Informative knowledge reflecting textually explicit information; (B) Deduction and application reflecting textually implicit knowledge. Due to the Gemara text’s unique characteristics, the measure of prior knowledge reflecting “scriptually” implicit knowledge was divided into two measures: (C) Prior knowledge relating to the Gemara, and (D) Understanding the conceptualizations underlying Gemara vocabulary in the context of the studied sugiya.

4.3 Design

The study consisted of three stages:

Stage 1: Both the control group and the experimental group students completed the Raven’s Standard Progressive Matrices Test, Verbal Ability Test, and the Aramaic Vocabulary Test.

Stage 2: Students in both groups studied a 90-min lesson focused on a new sugiya. The experimental group studied using Rabbi Weber’s method and was taught by a guest teacher who was experienced in this method. The control group was taught using the traditional method by a teacher who was the regular class melamed. A research assistant systematically observed the lessons taught by both teachers in order to verify that the lessons were conducted according to the research design. Note should be made that the two teachers received similar training as Gemara teachers, had similar seniority and experience in teaching Gemara, both specialized in teaching at the Talmud Torah school level and both came from similar ultra-orthodox communities that had a similar socio-economic level. In addition, neither group of students had prior experience with Rabbi Weber’s method.

Questionnaires and tests were administered by a research student, supervised by the authors.

Stage 3: A Gemara Comprehension Test was administered after the lesson.

5 Results

5.1 Pre-intervention results

The main aim of this research was to evaluate the outcomes of Rabbi Weber’s innovative program. First a baseline was established for both groups. This ensured that both groups had similar general abilities. The participants were examined in three areas: general ability, verbal ability and Aramaic vocabulary.

The students’ general ability was examined using Raven’s Test and no significant differences were found between the experimental group (M = 98.8, SD = 13.0) and the control group (M = 95.7, SD = 16.9) on the Raven’s nonverbal matrices test, t(149) = 1.2, p > 0.05.

A one-way multivariate analysis of variance (MANOVA) was performed in order to examine verbal abilities. Significant differences were found between the two groups, F(3, 146) = 9.9, p < 0.01, η2 = 0.17.

Table 1 presents the means, standard deviations, F values, and eta-squared values for both groups. The results indicate either better performance of the control group or similar performance for the two groups.

Table 1 Means, standard deviations, F, and eta-squared values for the three inventory factors
Full size table

Based on these results we constructed a general measure using the three language measures. The independent samples t test indicated that the control group’s score (M = 11.6, SD = 1.9) was higher than the experimental group’s score (M = 10.4, SD = 2.3), t(148) = 3.9, p < 0.001. We also tested the students’ Aramaic vocabulary (baseline). This test indicated that the control group achieved higher (M = 83.2, SD = 12.1) on Aramaic vocabulary than the experimental group (M = 77.6, SD  = 14.2), t(146) = 2.5, p < 0.05. Thus, the results of the control group were superior to the experimental group in some areas (synonyms, proverbs, and Aramaic vocabulary), and no differences were found between the groups in other areas (antonyms, Raven Matrices Test).

We would like to note an interesting finding revealed in the results, even though it does not concern the original aim of the research.

The study data allowed us to compare the ultra-orthodox students in the study to the general Israeli student population of the same age as indicated by Glanz (1989) in his study of verbal ability and intelligence based on Raven Matrices Test in the general student population. The data indicated that the ultra-orthodox students attained similar results to those achieved by students in the general population on both verbal ability and the Raven Matrices Test. In the verbal ability test, the standardized scores for each sub-test were weighted and produced a scale with a mean of 10 and a SD of 3. The ultra-orthodox student scores were not significantly different from their non-ultra-orthodox peers: Synonyms (M = 9.99, SD  = 2.5), antonyms (M = 11.07, SD  = 2.9), proverbs (M = 11.1, SD  = 2.5). We also compared scores for the Raven’s non-verbal matrices test for ultra-orthodox students and general population students. There was no apparent difference between the mean for the students in the general population (M = 100, SD  = 15) and the Talmud Torah students (M = 97.3, SD  = 15). This finding is important as thus far there has never been a possibility to compare this cultural minority population with the mainstream Israeli school population.

5.2 Post-intervention results

The research question dealt with the effect of Rabbi Weber’s method, which emphasizes the mediation of the teacher on the study of Gemara. The research hypothesis was that the scores of students who studied using Rabbi Weber’s method will be higher than the scores of students who learned through the traditional method. A t-test for independent samples was performed to test this hypothesis.

The results indicated a significant difference between the two groups: t(157) = 2.1, p < 0.05. The students who were taught using Rabbi Weber’s method achieved higher scores and a better understanding of the Gemara sugiya (M = 64.0, SD = 12.1) than their peers who were taught using the traditional method (M = 59.9, SD = 11.1), in spite of the equality and even advantage of the control group in certain areas in pre-intervention tests including verbal abilities.

A MANCOVA was performed in order to analyse the differences between the groups, where the questionnaire components served as the dependent variables. The analysis indicated significant differences between the groups: F(4, 141) = 10.2, p < 0.001, ƞ2 = 0.22. Co-variables were: knowledge of Aramaic vocabulary F(4, 141) = 3.1, p < 0.05, ƞ2 = 0.08 and verbal ability: F(4, 141) = 7.2, p < 0.01, ƞ2 = 0.17.

Analysis of variance results demonstrated that the experimental group performed better in two areas: inference and application of informative knowledge. No differences were found between the groups regarding knowledge of vocabulary and prior knowledge of Gemara (see Table 2).

Table 2 Means, SDs, F, and eta-squared values for the experimental group and control group for the components of the Gemara Comprehension Test
Full size table

The next research question was intended to clarify the relative contribution of each of the tested abilities (as tested prior to the intervention): general ability, verbal ability and knowledge of Aramaic vocabulary. A hierarchical regression examined the differences between the experimental group and the control group more closely in terms of the predictor variables’ contribution to success in the Gemara test.

General ability (Raven’s test) and verbal ability were introduced in the first step. Aramaic vocabulary was introduced in the next step. The results in Tables 3 and 4 indicate an interesting finding: although the percentage of explained difference is similar in both groups (39% in the experimental group and 35% in the control group), the prediction pattern is different. In the experimental group, verbal ability is the only variable that contributes significantly to predicting success in the Gemara test, while in the control group the most significant predictors of success were general ability and Aramaic vocabulary. This result provides further support for the findings to date, regarding the benefits of Rabbi Weber’s approach, since this approach emphasizes the verbal aspect, whose contribution is apparent only in the intervention group.

Table 3 Regression coefficients prediction of students’ Gemara test achievements: Experimental group
Full size table
Table 4 Regression coefficients prediction of students’ Gemara Test achievements: Control group
Full size table

6 Discussion

Ultra-orthodox Talmud Torah schools in Israel are totally isolated from the Israeli educational system both because they have their own special curriculum and because they are not exposed to any educational research. This study represents one of the first experiences of its kind that provides a glimpse into the closed world of the Israeli Talmud Torah schools. The study aimed to examine Rabbi Weber’s method of teaching Gemara in ultra-orthodox Talmud Torah schools and to compare it with the traditional approach usually used in these schools. In the traditional way of teaching Gemara, the teacher follows the order of the Gemara tractate, while addressing any comprehension problems raised by the students as they go along. In contradistinction, Rabbi Weber’s approach is characterized by providing skills that develop an independent learner, with particular emphasis on the understanding of the Gemara linguistic style. The amount of content studied is secondary, in contrast with the traditional method where quantity of content is of paramount importance.

The significantly superior results achieved by students who studied via Rabbi Weber’s approach compared to students who studied by way of the traditional method, are the most important results of the present study. The experimental group had no a priori advantage over the control group and both groups had a similar baseline regarding general ability (as indicated by the results of the Raven Progressive Matrices Test before the onset of the study). In addition, the experimental group had lower baseline scores than the control group on verbal ability and Aramaic vocabulary measures.

The students who studied using Rabbi Weber’s method generally achieved higher scores. Furthermore, a comprehensive analysis of the two groups’ achievements on the different components of the Gemara test revealed that the experimental group scored higher on the two components of sugiya comprehension: explicit (informative) knowledge and implicit (deduction and application of) knowledge. No differences were found on the two other elements of the test: prior knowledge and Aramaic vocabulary relating to the sugiya studied for the research. These findings also validate the benefit of Rabbi Weber’s method, since the experimental group scored higher on the tests which examined their comprehension of the new sugiya and not in areas reflecting their prior knowledge. On the sub-tests that examined comprehension of the ideas in the sugiya (explicit knowledge) and the ability to draw inferences (implicit knowledge), the students in the experimental group performed significantly better than the control group, thus confirming the efficacy of Rabbi Weber’s method.

We suggest that Rabbi Weber’s method demonstrates Pearson and Johnson’s (1978) view that reading forms a bridge between old knowledge and new knowledge by helping learners to establish better connections between their old and new knowledge (Duke and Pearson 2008). In spite of their existing knowledge, the control group students’ comprehension and their ability to draw inferences was limited and the teacher’s classroom mediation was not adequate.

In addition, results of the regression analysis which examined the relative contribution of the predictor variables to achievement in the Gemara test clearly indicate the major contribution of verbal ability to achievement attained by the experimental group, whereas general ability was the major predictor of achievement attained by the control group.

Rabbi Weber’s approach emphasizes linguistic ability, in particular linguistic accuracy and precision in understanding the sugiya. In addition, Rabbi Weber’s method provided the students in the experimental group with a distinct advantage in confronting the Gemara text despite the fact that these students achieved lower scores on the verbal ability pre-test than their counterparts in the control group. The research findings clearly indicate that the initial verbal ability advantage of the students in the control group was clearly offset by Rabbi Weber’s method which provided the students in the experimental group with the necessary verbal ability to better understand the text of the Gemara.

The importance of Rabbi Weber’s method is that it is the first attempt to tackle Gemara study from the learner’s point of view. Rabbi Weber’s method recognizes that young students only have a limited ability to conceptualize. His approach therefore attempts to overcome this problem by expanding their conceptual boundaries through on-going dialogue between concepts that the students recognize from everyday life and the wider implications of the same concepts presented by the Gemara through its special structure and language. This approach is compatible with the cognitive-developmental stage of the children, in the transition to abstract thinking, and leads them from the concrete examples to understanding abstract principles (Piaget 1985).

In contradistinction, traditional Gemara study tries to cover large amounts of material in great depth and in the order of the text, starting from the beginning of the sugiya, without providing a pedagogical perspective which considers the students’ needs. Rabbi Weber’s approach represents a complete shift in focus, away from the text, and concentrates on the children’s ability to think and conceptualize (Weber 2002).

It is interesting that Rabbi Weber, who grew up without knowing modern psychology which comprises an infrastructure for teaching and learning in modern society, and who grew up in the closed ultra-orthodox world, intuitively designed a didactic method of teaching Gemara that allows the teacher to be an active mediator in the enhancement of students’ understanding of the Gemara. It is very possible that the success achieved by students who studied via Rabbi Weber’s approach may be explained by Vygotsky’s concept (1978) that mediation that functions according to the principle of “zone of proximal development” (Feuerstein et al. 2010) is central to successful learning. Vygotsky (1978) indicates that when the distance between the present stage of the learner and the potential learning stage is diminished by mediation of the teacher, effective learning takes place. The effective functioning of the teacher as a mediator introduces order and logic into the learning world of the student and allows for the understanding of relationships and their underlying constancy and regularity. It appears that Rabbi Weber’s approach enhances students’ ability to understand major Gemara concepts and to come to grips with relationships in the Gemara text that are based on these concepts, a mission that the student cannot successfully achieve without suitable assistance.

6.1 Comparison between ultra-orthodox students and general students in Israel

Since this study enabled first time access to ultra-orthodox boy students and background data collected on their general and verbal abilities, we would like to compare the ultra-orthodox students and the general Jewish Israeli student population.

Despite the fact that ultra-orthodox society is generally described as a low socioeconomic status minority group (Gottlieb 2007; Gurovich and Cohen-Kastro 2004) with a separate educational system that focuses exclusively on religious studies, to the exclusion of almost everything else, and is not exposed to the Israeli curriculum, we found no differences between the ultra-orthodox students and general Jewish Israeli students of the same age on their general and verbal abilities as indicated by Glanz (1989). These findings are interesting and invariably indicated on standard ability tests (e.g., Gardner and Deadrick 2008; Helms 2008). One possible explanation for the above observation could be the special character of the ultra-orthodox population. This is indeed a unique group living as a minority within Israeli society. It is characterized by low socio-economic status due to low income, educational unsuitability for the local market, and a rejection of secular higher education. But it is also a learning group that sees learning as paramount and values study and intellect revolving around religious subjects as a central ethos (Friedman 1986b, 1995; Stadler 2009). The poverty of this group is considered a poverty of choice and group members perceive it as a worthy sacrifice since it allows them to uphold their traditional lifestyle. Thus, this group is different from other minority groups which exhibit economic and cultural backwardness in standard ability tests as described by Ogbu (1990) and Ogbu and Simons (1998) who distinguished between voluntary and involuntary minorities. Accordingly, an involuntary minority (e.g., the Afro-American community in the US) exhibits lower learning achievements than the majority, whereas a voluntary minority, which migrated to the United States freely (e.g., Asian immigrants), attains higher learning achievements than the majority. According to Feniger and Ayalon (2015), when analyzing Israeli society in light of Ogbu’s characterization of voluntary and involuntary minorities, it is clear that the ultra-orthodox community is a voluntary minority.

Israel’s minority ultra-orthodox group differs from the minority groups addressed by Ogbu, which were formed through immigration, because although the ultra-orthodox minority is ethnically part of the majority group, it lives separately from the mainstream society in terms of its value system, lifestyle, employment, and education (Caplan 2007). Israel’s ultra-orthodox population has chosen not to participate in the Israeli education system but it does not opt out of learning. On the contrary, it is a learning society which elevates learning and learners (Friedman 1995). This can explain the students’ performance in the achievement tests in their general and verbal abilities which is similar to that of the general Jewish student population in Israel.

This research contributes to both pedagogical research and practice. In terms of pedagogical research, this was one of the the first studies to examine pedagogical processes in a previously un-researched sector. The researchers were allowed to study schools that had previously refused any contact with the Israeli educational establishment. In terms of pedagogical practice, the study offers the program’s developers and educators who are involved in introducing the program into Talmud Torah schools important data regarding the efficacy of Rabbi Weber’s method.

Finally, the Gemara test developed for this study, based on Pearson and Johnson’s (1978) model, can be a basis for systematic assessment of a sector which knows very little about educational measurement and does not regularly use tests and evaluations apart from tests and assessments by the teacher in the classroom. We hope that the cooperation established between the researchers, the ultra-orthodox sector representatives, the ultra-orthodox school principals, and the decision makers will allow this to happen. For the first time, an infrastructure has been set up which can improve the link between the research community and the ultra-Orthodox community and offer a foundation for future field studies.

7 Limitations and future research

We realize that although the study offers valid and informative data on Talmud Torah schools, the students focused on only one sugiya and for a short time. In addition, neither the design of the study nor the size of the sample easily permit valid comparisons with other student populations. However, it is important to keep in mind the pioneering nature of this research and the fact that it was the first time that any researchers have been allowed into Israeli Talmud Torah schools to examine and analyse learning. A key goal for future research will therefore be a more comprehensive examination of the program, including sampling, study time, and number of sugiyot learned, and to construct appropriate measures using the model described in this research. Another important research direction for future study is classroom teaching and learning processes. Future studies should examine the teachers’ work in the classroom and how they implement Rabbi Weber’s method. We believe that this exploratory project will lead to future studies designed to examine other pedagogical as well as methodological issues.