1 Introduction

Open collaboration emerged as an effective way to produce information and products and to foster innovation by leveraging the effort of volunteer communities (Panciera et al. 2011; Levine and Michael J. Prietula 2014; Forte and Cliff Lampe 2013). Notable examples of these communities include Wikipedia, Open Street Map, Linux, Open Office, and Mozilla Firefox. The success of these communities frequently depends on the influx of new contributors (Forte and Cliff Lampe 2013), since they are a source of innovation and social capital (Kraut et al. 2012). As stated by Forte and Cliff Lampe (2013), open collaboration communities rely on environments with low barriers to entry.

The Open Source Software (OSS) movement works in a symbiotic way. Communities need to motivate, engage, and retain new developers to remain sustainable (Qureshi and Yulin Fang 2011), and projects attract a large, globally distributed community of developers willing to learn, gain visibility, benefit society, and get jobs (Parra et al. 2016; Singh and Lila Holt 2013; Riehle 2015). However, new developers are typically required to find a task that they can implement and figure out how to contribute to the project. Newcomers, therefore, face various barriers when attempting to contribute (Steinmacher et al. 2015b), and, since delivering a contribution to an OSS project is usually a long, multi-step process, they lose motivation and even give up (Steinmacher et al. 2013, 2018).

Mentorship is a frequently-adopted strategy in open collaboration communities for helping newcomers overcome the barriers faced during their first steps (Hsieh et al. 2013; Fagerholm et al. 2014; Musicant et al. 2011). In offline communities, assigning mentors to new members has proven effective at helping them overcome challenges (DuBois et al. 2002). Some OSS communities also offer mentoring initiatives (Steinmacher et al. 2015b; Fagerholm et al. 2014; Silva et al. 2017), including well-known and established programs like Google Summer of Code.Footnote 1 Through mentoring, newcomers are trained to acquire the technical, social, and organizational information they need (Fagerholm et al. 2014; Labuschagne and Reid Holmes 2015; Musicant et al. 2011; Panichella 2015). Thus, understanding how to help mentors might benefit the newcomers joining process as a whole.

While research has looked at the onboarding process in OSS communities and the barriers faced by newcomers (Steinmacher et al. 2015b), the literature has overlooked the challenges faced by OSS mentors. A better understanding of the barriers enables communities and researchers to design and produce tools, and to conceive strategies and processes for better supporting mentoring. It also enables new mentors to be aware of the hurdles that they may face.

Additionally, there is no research on mentors’ perspectives on the barriers that newcomers face during the joining process. Understanding the mentor’s perspective is particularly relevant since they work closely with a variety of newcomers during several onboarding activities and have a broader view of the project’s goals and characteristics.

Therefore, our goal in this paper is to identify the barriers that affect mentors and their newcomer mentees from the perspective of mentors. Moreover, we identify a set of strategies that mentors use to help newcomers on the barriers they encounter, as well as explore OSS onboarding challenges for women, who are underrepresented in this context (Robles et al. 2014). To guide our research, we defined the following research questions:

  • RQ1. What are the barriers that affect OSS mentors during newcomer mentorship?

  • RQ2. What are the barriers that affect OSS newcomers from the mentors’ perspective?

  • RQ3. What are the strategies employed by mentors to help newcomers overcome barriers?

  • RQ4. What are the additional challenges that affect women onboarding to OSS projects?

To answer our research questions, we qualitatively analyzed data collected from interviews with software developers who mentored newcomers in Open Source Software projects. We found 44 barriers: 19 that affect mentors; and 34 that affect newcomers (9 are shared, affecting both newcomers and mentors). From the 34 barriers that affect newcomers, 16 had not been previously identified (Steinmacher et al., 2014, 2015b, 2015a). Our analysis indicates that social factors are a significant challenge for the onboarding of newcomers – mentors and newcomers are subject to 29 social barriers (66% of the identified barriers). In addition to the barriers, we identified 10 strategies that were mentioned by the mentors as effective for supporting newcomers. Interestingly, most of them (7) pertained to overcoming social barriers. However, these strategies cover only 9 out of the 29 social barriers, opening possibilities for future research in the area. Finally, we identified gender-specific challenges, which emerged from our initial analysis and were further investigated in a follow-up study with nine additional structured interviews.

This paper contributes to the literature by (i) identifying a set of barriers faced by mentors while onboarding newcomers to software projects; (ii) adding to the existing literature on barriers faced by newcomers by considering the mentors’ perspective; (iii) unveiling strategies used by mentors; and (iv) exploring the challenges that are specific to women.

2 Related work

In this section, we present previous work on OSS onboarding, mentoring, and gender diversity.

2.1 Newcomers’ onboarding to OSS projects

The onboarding of newcomers has been studied in different online collective production communities, including in Wikipedia (Halfaker et al. 2013; Halfaker et al. 2011; Bryant et al. 2005; Choi et al. 2010) and OSS projects (Jensen et al. 2011; von Krogh and Eric von Hippel 2003; Steinmacher et al. 2015b; Nakakoji et al. 2002; Ducheneaut 2005; Hannebauer et al. 2014; Lakhani and Robert Wolf 2005). Newcomer onboarding also affects commercial software development settings, as described by Dagenais et al. (2010) and Begel and Beth Simon (2008).

Among the studies that focus on newcomers to OSS projects, some report scripts, paths, and cases of developers successfully joining projects. Von Krogh and Hippel (von Krogh and Eric von Hippel 2003), for example, propose a joining script for developers who want to take part in a project. Nakakoji et al. (2002) also studied OSS projects, proposing eight possible participation roles structured in concentric layers—a structure later called “the onion patch.” In addition, some previous work focuses on the motivational forces driving developers to contribute to OSS projects, such as learning opportunities and personal improvement (Bonaccorsi and Cristina Rossi-Lamastra 2004; Roberts et al. 2006; Hars and Shaosong Ou 2002; Krogh 2003; Lakhani and Robert Wolf 2005; Singh 2012). Ye and Kouichi Kishida (2003), for example, built on the Legitimate Peripheral Participation (LPP) theory (Lave and Etienne Wenger 1991) to claim that learning is a strong force motivating newcomers to join OSS. Also relying on LPP, Lakhani and Robert Wolf (2005) report that situated learning and identity construction behaviors were positively linked to long-term participation.

Other researchers focus on understanding and dealing with the barriers that influence newcomers’ onboarding (Steinmacher et al. 2015b, 2015c; Jensen et al. 2011). Jensen et al. (2011) analyzed whether emails sent by newcomers are quickly answered, if gender and nationality influence the kind of answer received, and if the reception of newcomers differs. Similarly, previous work by Steinmacher et al. (2013) analyzed how the answers to newcomers’ first emails influenced their retention. Additionally, Steinmacher and colleagues (Steinmacher et al. 2014, 2015b) conducted a mixed-method study and identified 58 barriers faced by newcomers. They relied on data collected from newcomers, core members, and the literature (Steinmacher et al. 2015a) to build the model. We use this model as a baseline to compare our findings.

2.2 Mentoring

As a well-known strategy, mentoring is explored in management literature as a way to help new employee socialization (Allen et al. 2017; Payne and Ann H Huffman 2005; Street 2004), and in education literature as a way to help new teachers acclimate (Martinez 2004; Redman et al. 2015; Rockoff 2008) and students to overcome learning challenges (Nugent et al. 2004; Crisp and Irene Cruz 2009; Gershenfeld 2014). Part of this literature analyzes the challenges faced during mentorship. For example, Ragins (1989) conducts a literature review analyzing the challenges related to gender in the mentor-mentee relationship. In the education domain, Martinez (2004) explores the problems encountered in mentoring new teachers, while (Kumar et al. 2013) explore the challenges faced by faculty members while mentoring online doctoral students.

Mentoring is often used to offer support for newcomers to online communities (Musicant et al. 2011; Hsieh et al. 2013), and it was an object of study in Software Engineering (Berlin 1992; Sim and Richard C. Holt 1998). In closed source settings, it is a common practice to offer formal mentorship to newcomers to support their first steps (Begel and Beth Simon 2008). Dagenais et al. (2010) reported that teams that proactively mentor newcomers make integration easier.

However, in OSS projects that rely on volunteers, it is not a widely-spread approach to offer formal mentorship programs. Nevertheless, this topic attracted the attention of some researchers interested in supporting the onboarding of newcomers to OSS. Malheiros et al. (2012), Panichella (2015), and Canfora et al. (2012) proposed different approaches to identifying and recommending mentors to OSS newcomers, claiming that mentoring would benefit newcomers’ onboarding. Steinmacher et al. (2012) proposed a recommendation approach to help newcomers find the most appropriate project member to mentor a specific technical task. To assess the impact of mentoring support on developers, Fagerholm et al. (2014) conducted a case study that found mentoring to significantly impact newcomer onboarding, allowing them to become more active. In addition, Schilling et al. (2012) studied the impact of mentoring on training and retention of developers in OSS projects. Based on their findings, they proposed mentoring as a training method for OSS projects, and introduced a measure for assessing mentoring’s capacity to facilitate learning and retention among developers. In contrast, Labuschagne and Reid Holmes (2015), who studied Mozilla, evidenced that onboarding programs may not result in long-term contributors, despite the fact that mentored newcomers considered the mentorship program valuable.

2.3 Gender diversity in OSS communities

Discussions and research related to diversity and gender in software engineering are becoming more common. Vasilescu et al. (2015), found that gender and tenure diversity are significant and positive factors that increase productivity. A recent study (Beckhusen 2016) shows that the proportion of women in information technology-related jobs is still low (25%). Women are even more underrepresented in OSS, comprising a small percentage (about 11%) of contributors in the OSS community (David and Joseph S. Shapiro 2008; Robles et al. 2014). This number is even lower considering the top developers, reaching ≈ 3% when analyzing the top-500 developers of GitHub (Wang et al. 2018).

One recent study reported that when women contributors’ profiles identified their gender, their contribution acceptance rates were 12% lower than women whose genders were not identifiable from their profile (Terrell et al. 2017). In addition, recent research (Burnett et al. 2016) has shown that the individual differences in how people problem-solve and use software features often cluster by gender, and, further, that many software features are inadvertently designed around methods used predominantly by men. For example, research spanning approximately ten years across numerous populations shows that men and women differ in (at least) five ways that can directly impact the ways they use software: (1) their motivations for using the software; (2) their style of processing information; (3) their computer self-efficacy; (4) their attitudes toward technological risks; and (5) their preferred learning styles in learning technology.

Research is also beginning to emerge on social/cultural issues that particularly discourage women from joining OSS communities, and on the benefits to OSS communities for solving these issues. For example, OSS communities function as so-called “meritocracies” (Feller and Brian Fitzgerald 2000), in which women developers report experiencing “imposter syndrome” (Vasilescu et al. 2015). Participant observation of OSS contributors found that “men monopolize code authorship and simultaneously de-legitimize the kinds of social ties necessary to build mechanisms for women’s inclusion” (Nafus 2012). By interviewing women newcomers and experienced women online contributors to Stack Overflow, Ford et al. (2016) identified 14 barriers that affect women. Because of the dearth of women in technical online communities, they also found that women disproportionately experience a lack of “peer parity” (seeing other women contributing to their community) (Ford et al. 2017). In addition, by analyzing a subset of the barriers identified previously (Steinmacher et al. 2015b; Mendez et al. 2018) found that over 73% of the barriers the software professionals found had some form of gender bias. Moreover, most of the instances of gender bias were implicated with multiple facets, implying a pervasive lack of support for problem-solving strategies common among women.

2.4 Section remarks

Although numerous studies focus on newcomer onboarding to OSS projects, none of them consider the mentors’ perspective. In addition, regardless of the potential benefits brought by mentoring in OSS projects (Fagerholm et al. 2014; Schilling et al. 2012), the literature does not consider the potential challenges faced during the mentoring process, as has already been done in other domains. We also contribute to the gender diversity in OSS literature by bringing evidence on the specific challenges faced by women newcomers and mentors.

3 Research method

The main goal of this study is to identify barriers that affect the work of mentors and newcomers in OSS development settings. To achieve this goal, we conducted a qualitative study of responses obtained from interviews. Since the purpose of our study was to evaluate the mentors’ perspective on the barriers faced in OSS software development environments, we selected participants who have at least two years of experience in mentoring newcomers. An overview of our research method is presented in Figure 1.

Figure 1.
figure 1

Research method overview

Full size image

3.1 Participants

We recruited 10 experienced OSS mentors (two women and eight men). Five reported also having experience in industry closed-source projects, and one (P9) had experience working in OSS and academia. We compensated participants with a 25-dollar gift card for participating in the interview.

We used the snowball strategy to recruit participants. At the end of each interview, participants were asked to introduce qualified participants for the study. To recruit the participants, we sent out recruitment emails, in which they were explicitly asked to talk about their experience in onboarding new developers to their projects. We conducted interviews until we came to an agreement that saturation was reached for the barriers identified. According to Strauss and Juliet M. Corbin (2007), sampling can be discontinued once the collected data is considered sufficiently dense and data collection no longer generates new information.

We reached out to 18 people; among them, 13 were interested in taking part in our study, but only ten were considered, since 3 of them had no or little experience in mentoring in OSS settings. Table 1 shows the demographic information for the 10 participants.

Table 1. Demographics.
Full size table

3.2 Data collection

To identify the barriers and strategies, we conducted semi-structured interviews, which consist of a mixture of open-ended and specific questions that are designed to elicit foreseen and unexpected information types (Seaman 1999). In this kind of interview, the questions are planned, and we seek to answer them, but they are not necessarily asked in the same way or order as they are listed (Runeson and Martin Höst 2009). We designed our interview script according to the literature recommendations (Runeson and Martin Höst 2009; Seaman 1999).

Before interviewing the participants, we conducted four pilot interviews with Ph.D. students who had experience working in industry or OSS environments to validate the script and confirm whether the interview would fit in a 40-minute time slot. The pilot participants answered all the interview questions and provided us feedback about the flow of the script. We also analyzed the questions and answers to ensure that they provided data that would answer our research questions. The final interview script is depicted in Table 2.

Table 2. Interview questions.
Full size table

The interviews were conducted remotely and lasted around 40 minutes. The interviews were recorded with the participants’ consent and transcribed directly after their conclusion.

3.3 Data analysis

We qualitatively analyzed the transcripts by applying card-sorting techniques. We started by selectively applying open coding, whereby we identified concepts and their properties. Simultaneously, we grouped these concepts into higher-level categories according to their properties.

The first and third authors of this paper coded the interviews using negotiated agreement. Figure 2 illustrates one of our analysis sessions. Furthermore, we held weekly meetings in which all the authors discussed the resulting codes and classification until we reached an agreement.

Figure 2.
figure 2

Card sorting session

Full size image

3.4 Follow-up: gender-specific challenges

Since interesting findings related to gender-specific challenges emerged from our analysis, we decided to further investigate this specific aspect through follow-up structured interviews with other women mentors. We recruited nine women who had participated as mentors in Google Summer of Code 2017 projects.Footnote 2 We manually inspected the project entries and personally invited mentors who could be identified as women from their GitHub profile. In Table 3, we present the demographics of the participants of our follow-up interviews.

Table 3. Demographics for the follow up interviewees.
Full size table

The follow-up interviews comprised profiling questions and three open-ended questions related to gender differences, as follows:

  • What are the main challenges you face as a women mentor?

  • From your perspective as a mentor, are there differences in the challenges that women newcomers face? If yes, what are they?

  • What are potential strategies or initiatives that you think will reduce gender-related barriers in OSS projects?

Once again, the data was analyzed applying card-sorting techniques.

4 Results

In total, we identified 44 barriers faced by mentors and/or newcomers, which were further classified as:

  • Social barriers: those that involve or directly influence human social interactions. These barriers were further classified as personal barriers – the barriers related to personal characteristics of newcomers or mentors; and Interpersonal barriers – those related to the relationship among community, mentors, and newcomers.

  • Process barriers: those imposed by the organization, or by internal procedures or practices.

  • Technical barriers: those directly related to or caused by technology, including frameworks, programming languages, and/or tooling used in the project.

Figure 3 presents all 44 barriers identified in this study. The barriers are presented hierarchically according to the aforementioned classification and further grouped when appropriate. We associated barriers with graphics: a graduation hat for barriers identified as only impacting mentors; a Venn diagram for barriers shared by both mentors and newcomers; and a star for newcomer barriers that had not been identified in previous work (Steinmacher et al. 2015b). We used Steinmacher and colleagues work as our baseline, since it includes barriers collected from multiple studies and sources, including a set of barriers cataloged by means of a broad systematic literature review (Steinmacher et al. 2015a). In this section, we only discuss newly identified barriers.

Figure 3.
figure 3

Overall view of the barriers

Full size image

From the 34 barriers that affect newcomers (including the shared ones), 16 had not been previously identified by Steinmacher et al. (2015b): 11 of them are social, 1 is technical, and 4 are process barriers. We believe that this high representation of social barriers relates to the focus on the mentors’ perspective, since mentors have a closer and more personal relationship with the newcomers.

In the following sections, we present our results according to our research questions.

4.1 RQ1: what are the barriers that affect OSS mentors during newcomer mentorship?

In our study, we found 19 barriers that mentors reported facing when onboarding newcomers, as presented in Table 4. We could identify only one technical barrier that affects only mentors. This seems reasonable, since mentors usually have been on the project longer, have the programming background and skills, as well as the understanding of tools and technologies used by the community. Accordingly, 16 out of the 19 identified barriers are social barriers – 12 interpersonal and 4 personal.

Table 4. Mentor-only barriers.
Full size table

4.1.1 Personal barriers

We identified four personal barriers that impact mentors. The barriers relate to their ability or lack of ability to manage the responsibilities that come along with mentorship. Handling a large number of mentees can be overwhelming, as stated by P6: “I really wish it was easier to deal with a lot of people.” This barrier is related to scheduling, which also creates difficulty in switching context between helping mentees and doing their own work. P10 explained that “if you are not actively focusing your attention on [your mentee] continuously, context switching can be difficult between doing my work and helping them with theirs.” As a part of mentorship, mentors are expected to complete their own work and be available to help their mentees. Three participants mentioned that difficulty in time-management can be challenging, since mentors must choose how to allocate their time to the project, sometimes weighting different activities, such as working on code, mentoring, and reviewing. Other than these, mentors can also encounter problems in aligning their schedule with newcomers, as mentioned by P4: “being able to contact them [the newcomers] and give feedback was sometimes difficult.” Finally, difficulty in managing different accounts was mentioned as a barrier, since “it’s really annoying to have a lot of accounts to keep track of.”

4.1.2 Interpersonal Barriers

We identified twelve interpersonal barriers that impact mentors: more than what we found for newcomers. This fact suggests that, from the mentors’ point of view, social aspects are more challenging to deal with than process or technical issues, as social interactions play a key role in mentoring.

First, since people who work in an OSS project may come from diverse cultures, cultural differences can be challenging for newcomers and mentors. P8 mentioned “in some cultures, people get more upset when people criticize their code… which can be tough.” Moreover, when newcomers and mentors are geographically distant, they do not have the opportunity for face-to-face interaction, which can, for example, inhibit informal communication and reduce trust. Therefore, communication issues related to time zone and place affect the communication process during onboarding. Also related to communication in global settings, lack of English language skills was mentioned by P9 as hindering the mentorship process: “My English is so-so … when both parties have difficulties communicating, it is challenging to overcome and they don’t have good tools for that.” Although previously identified by Steinmacher et al. (2015b), we highlight this barrier, since English is the dominant language in OSS projects.

We also observed that a mentor’s inability to interact with newcomers (lack of mentor’s interpersonal skills) can greatly impact a newcomer’s decision to continue contributing to the project. Mentors frequently highlight the importance of social aspects, as evidenced by P3: “… the biggest pitfalls of the mentor are: not being responsive and not engaging in other ways than just coding. These projects are about community effort and more than just the code.”

Mentors also face barriers in adapting to how different types of people learn and take in the information presented to them. Two mentors reported that adjusting interaction style to different mentee personalities is a barrier, since mentors are likely to collaborate with diverse people who have unique personalities and working styles, as stated by P9: “[…] you always have to adapt based on each individual newcomer […] one solution doesn’t always work for everyone.” Mentors need to understand their mentees and tailor aspects of the coaching to fit them. For a mentor, determining how to be an effective teacher for a mentee can be difficult. Four mentors mentioned difficulty guiding mentees who are resistant to coaching. Sometimes mentors are required to face the challenge of teaching newcomers who lack a desire to learn. In this sense, P5 mentioned “But I still don’t know how to help people who don’t want to learn.” Also related to coaching, mentors reported that providing constructive feedback based on the mentee’s background is challenging. Mentors must tailor their comments and criticism to the way a newcomer learns, while taking into account their prior experience and level of self-efficacy. P4 reported that “being able to understand the student’s background and the way they see this stuff and give proper feedback is kinda hard.” Some mentees value feedback, while others may not easily perceive it a constructive manner.

Moderation is sometimes required for mentors when dealing with newcomers. For example, newcomers who are eager to contribute something relevant to the project tend to start with a task that may be too large or complex for their skills set. Convincing people to start small rather than big was reported as a difficulty, as explained by P6: “the other challenge is convincing people to start small rather than big because lots of people want to make big changes but I can’t help them with those.”. This relates to the process barrier called “difficulty in identifying appropriate tasks for newcomers.”

Ensuring mentees finish their work was reported as a barrier by P3, who mentioned that “the biggest challenge is making sure they are working and making sure they will finish the project. Otherwise, it is a fail for the mentor if the mentee doesn’t finish.”

As the project community grows, the diversity of contributors grows in parallel. Mentors mentioned the difficulty in creating an inclusive community as a barrier. Mentors try to ensure that newcomers feel comfortable and are not discriminated against. P3 explained, “It’s about the community. There has been a lot of discussion about gender pronouns and this is very important to take into account to make sure the community is inclusive of all, especially for newcomers.” Inclusion is important for attracting newcomers, as well as retaining them and increasing their productivity (Vasilescu et al. 2015). The participants of our study seemed to be aware of this and placed particular emphasis on this barrier. We further discuss this point in Section 4.4.

Finally, a frequently mentioned barrier was harsh project atmosphere (mentioned by 8 out of 10 mentors). This barrier affects mentors, since they face difficulty in supporting newcomers who fear disagreements among committers in the community, as stated by P1: “I may find a patch to be fine and ready to commit but some other committer may look at it and not agree that it is fine.” This is a particularly problematic challenge for mentors, since it is largely out of their control.

4.1.3 Process barriers

We found that mentors are significantly less affected by process barriers than newcomers. However, if any processes are unclear, the mentor must figure out how to get their mentees the information they need. Not having aformal procedure for introducing the community was reported as abarrier by P9, who stated that

“[…] the challenges I have faced are related to how to decide which part of the community to introduce first to the students. It is not totally clear in KDE since we have many processes and don’t have aformal procedure for the introduction.”

In addition, the barrier difficulty in identifying appropriate tasks for newcomers, which was previously identified as an important barrier for newcomers (Steinmacher et al. 2015c), was pointed as a challenge for mentors as well. According to P3, “to keep them [the newcomers] engaged you need […] to pick a task that is appropriate for them… something that is interesting, which can be a challenge for mentors.” When a newcomer’s background and goals are unclear, it can be difficult for the mentor to point them to a specific task.

4.1.4 Technical barriers

We identified only one technical barrier that affects both mentors and mentees: differences in the devices that mentors and mentees use. When mentors and mentees are not using compatible devices or operating systems, it is hard for amentor to help resolve anewcomer issue. P2 stated

“the operating system and distribution my computer is running is very different to what the newcomer is running. If anewcomer has an issue, Itry to reproduce it, and Imay not have this issue which makes it harder to help.”

4.2 RQ2: what are the barriers that affect OSS newcomers from the mentors’ perspective?

Our interviewees reported 34 barriers that newcomers encounter while onboarding to OSS projects. A summary of these barriers is found in Table 5. In this section, we aim to add to the existing literature by identifying the barriers faced by newcomers from the perspective of mentors. We focus our discussion on the barriers that do not appear in Steinmacher et al.’s barriers model (Steinmacher et al. 2015b).

Table 5. Newcomer-only barriers.
Full size table

Among the 34 reported newcomer barriers, 16 are new compared to our benchmark (Steinmacher et al. 2015b). Most of them (11) have a social nature, while 4 are process-related barriers and only 1 comprised a technical barrier. We believe that the perspective of mentors brought this social focus to the identified barriers.

4.2.1 Personal barriers

We identified 11 personal barriers; among them, 8 are not included in Steinmacher et al.’s barriers model (Steinmacher et al. 2015b).

We identified three barriers related to self-efficacy, including Low self-efficacy. Some newcomers believe they will be unable to finish the tasks assigned to them and give up. P1 stated that “[the newcomers] think they aren’t good enough or they don’t know enough.” Fear of judgment and performance anxiety are the two other barriers related to self-efficacy. Regarding the former, P4 included a personal example: “The biggest barrier is being afraid of being judged […] — some people are afraid because the feedback sometimes isn’t very polite or very welcoming […]. It was also something that prevented me from joining open source before. I was really afraid of sending code that would be judged to be bad quality.”

We also observed that a newcomers’ inability to adapt their personality to the team and project environment (newcomers’ personality doesn’t fit with the role) can become a barrier. P8 explained his experience working with a mentee who failed to get onboard, “his code style wouldn’t be right and he just wouldn’t listen and make the same mistakes over and over.” Also regarding personality, the newcomer’s inability to improve upon criticism in a positive manner was considered a barrier. As P9 said, “I know some people may start contributing and then give up after a harsh review. Mainly how to receive criticism, criticize, and improve skills from that criticism is key.”

Two mentors (P9 and P10) reported an additional and interesting personal barrier. According to them, lack of clear professional goals that can hinder newcomers, since “it can be really difficult to figure out which of the issues or features that are listed in a product road map or bug tracker are actually a good fit” [P10].

Additionally, we found that difficulty in managing different accounts and difficulty in time-management were considered to negatively impact newcomers’ first steps.

4.2.2 Interpersonal barriers

We identified 9 interpersonal barriers; among them 3 were not identified by Steinmacher et al. (2015b). Two of them relate to communication.

Communication issues related to time zone and place affect the communication process, impacting newcomers and mentors during the onboarding process. The second communication barrier is the lack of mentor’s interpersonal skills.

Difference in work experience and age was also reported to be a barrier for newcomers. Sometimes, people with high levels of experience forget how it felt to be a newcomer and what kinds of tasks can be difficult for newcomers, as explained by P8: “it’s hard for me to identify sometimes when people don’t get something just because I’ve been doing it for so long...” Although experts possess deep knowledge about how to do their job, they may struggle to surface this knowledge and explain it to others (Shim and Gene L. Roth 2007).

4.2.3 Process barriers

Among the 8 process barriers identified from our interviewees, the following four do not appear in Steinmacher et al. (2015b).

When attempting to contribute to aproject, some newcomers believe that they need to make abig change in their first contribution. However, in many cases they are unable to do so, or the community will not expect this from anew member .This willingness to start with acomplex task may cause newcomers to lose motivation and quite the project if they are unable to complete it. According to P10,

“there is this mismatch in expectation and so you’ll see people be like ‘oh, that issue looks too small’. And they don’t want to do it, because they want to make abigger more significant contribution... I’ve seen this mismatch make newcomers feel disheartened and like they are not actually contributing. It’s tough because in acertain sense they are not actually contributing.”

In addition, issues with project micro-climate were identified as a barrier that impacts onboarding; it was also previously reported by Zhou and Audris Mockus (2015) as a factor that influences the retention of developers. This barrier is mainly related to the schedule, as summarized by P9, who said that “we have things we can and cannot do based on our release schedule. It’s a barrier because it says how our work as developers impacts the work of the others in the community.”

Difficulty in choosing anewcomer-friendly project can be abarrier when newcomers do not know which project matches their interest and expertise. This can demotivate newcomers during their first steps. P2 explained this issue, reporting that:

“[newcomers] come and they really like to join a particular open source community and start contributing to aproject but they don’t know which one. […] this is because not all projects are easy to start either so anewcomer doesn’t know exactly what is going on.”

Lastly, project processes taking too long refers to impediments related to the internal processes of a project that slowdowns or stops newcomers from contributing to software development projects.

4.2.4 Technical barriers

We identified 6 technical barriers that hinder newcomers’ onboarding. Among them, only difference in the devices that mentors and mentees use was not previously identified by Steinmacher et al. (2015b, 2015c). This barrier is detailed in Section 4.1.4, since it also impacts mentors.

4.2.5 Discussion: shared barriers

Among the barriers identified, we found that a subset influences both mentors and newcomers. In fact, 20% of the barriers we identified are shared barriers (9), which are presented in Table 6. As expected, since they affect both sides, interpersonal barriers frequently appear (5 out of 9), such as lack of English language skills.

Table 6. Shared barriers between newcomers and mentors.
Full size table

In addition to the interpersonal barriers, we found that two personal barriers (difficulty in time-management and difficulty in managing different accounts), one process barrier (difficulty in identifying appropriate tasks for newcomers), and one technical barrier (differences in the devices that mentors and mentees use) that hinder both newcomers and mentors during the mentorship process.

These barriers were identified during our analysis as having implications for both newcomers and mentors. However, from our current data it was not possible to understand to what extent or how each of these barriers impacts the stakeholders. This is an interesting future direction of this research.

4.3 RQ3: what are the strategies employed by mentors to help newcomers overcome barriers?

We also asked our participants about the strategies that they use or know of to help newcomers overcome barriers. Table 7 depicts the strategies suggested by the mentors and the list of barriers that they assist in overcoming. In the rest of this subsection, we present these strategies.

Table 7. List of strategies suggested by mentors to overcome the barriers.
Full size table

Newcomers are not aware of the typical steps required for working on atask. Working on abug or issue together with mentees (S1) can show them how to work on their future tasks and how to overcome potential barriers. P10 stated that

“it helps people to be independent and autonomous by teaching them how the project works, how open source works, what their resources are and helping them, working with them as they get the sense of the types of problems they can do on their own.”

Another interviewee, P7, mentioned that he uses this strategy to help newcomers overcome technical barriers. Thus, mentors related this strategy to the following barriers: 1) high code complexity (N-T3); and 2) lack of newcomers’ background knowledge (N-T4).

Holding training sessions for newcomers (S2)

Our participants believe that training sessions for newcomers help them overcome most technical barriers, as described by P9: “For technical barriers, we usually minimize them by initially doing some workshops on our technologies.” We found that this strategy can help in overcoming the following barriers: 1) lack of newcomers’ background knowledge (N-T4); and 1) difficulty in learning related tools or technologies (N-T5).

Flagging newcomers, so others are welcoming to them (S3)

Although many experienced members want to help newcomers, they have other daily duties and responsibilities that prevent them from being available to all the people who need help. This fact can make the project atmosphere harsh and not receptive to newcomers. Therefore, with a newcomer tag, others can recognize them and be more patient, welcoming, and responsive, as stated by P6: “We have some ideas of ways to flag when someone is a newcomer so they can be explained things in a more gentle way.” This strategy was reported as a way to overcome: 1) harsh project atmosphere (N-I9); and 2) low response rate (N-I1).

Communication through different means (S4)

Contributors may be distributed across the world and in different timezones, making it difficult for them to communicate instantly. Thus, offering multiple communication forms benefits newcomers, since they can choose the communication channel in which they feel most comfortable. P4 informed us that, “there is always the language barrier, but in Open Source, the communication is done through email or IRC… It helps me in the communication and also the cultural barrier and I would say timezone.” It was mentioned that providing different means of communication helps in prevailing over some barriers, such as 1) cultural differences (N-I5); 2) communication issues related to time zone and place (N-I6); and 3) lack of English language skills (N-I7).

Giving newcomers small and interesting tasks (S5)

When the first tasks that are assigned to newcomers are too large, complex, or uninteresting to the newcomers, they may lose interest and become afraid that they will be unable to finish the task appropriately. Therefore, mentors need to provide them with a task small enough for them to make progress. This strategy was evidenced by P9, who said that: “If you try to make a newcomer work on highly experienced contributions, that won’t work.” However, it is important to note that choosing an appropriate task can be a barrier for mentors as well, as stated by P3: “To keep them [newcomers] engaged you need the community to pick a task that is appropriate for them. You must give them something that is interesting to them, which can be a challenge for mentors.” This strategy is suggested to help newcomers in overcoming: 1) lack of interest (N-Per1); and 2) performance anxiety (N-Per6).

Giving newcomers rewards to keep them motivated (S6)

Newcomers need to allocate a considerable amount of time and effort to onboarding to the project. Since many of them voluntarily contribute, they can easily become discouraged. This strategy was reported to help in overcoming lack of interest (N-Per1), as stated by P2: “Giving rewards to newcomers as they get through their guide and keep them motivated.”

Having the newcomers share their work to have more exposure (S7)

P9 stated that “we have sessions for newcomers to present their work. We also encourage them to write blog posts, so people know what they are doing.” By presenting their work to others, newcomers have the opportunity to both familiarize others with their work and also face their fear about other’s opinion and judgment about their work and performance. This strategy was reported as a way to help newcomers reduce: 1) fear of judgment (N-Per4); 2) low self-efficacy (N-Per5); and 3) performance anxiety (N-Per6).

Tagging the tasks according to their complexity (S8)

Having issues tagged based on their complexity helps newcomers choose from the list of open issues. P5 stated: “Things had gotten much easier from when I started. There was no documentation or guidelines, and mentors wouldn’t tag bugs suited for newcomers. I am glad things have changed and become easier for newcomers to contribute.” This is a strategy used by some big projects, like Apache, Mozilla, Gnome, and KDE. This fact was mentioned by P10: “There are some large projects out there that have put a lot of time and thought into identifying every newcomer task.”

Having local groups in each country (S9)

Local groups that share adegree of language and culture can help newcomers “feel home,” thereby reducing some initial barriers. P9 stated that,

“Starting things alone is harder than when you have alocal group. In KDE, we have lots of local groups in China, India, US, and Korea, and having those groups is important to welcome newcomers to free software communities. It helps to have people to talk to in your mother language with asimilar culture, so that makes ahuge difference in attracting newcomers.”

Keeping documentation concise and updated (S10)

Presenting structured documentation, with clean and organized information, orients newcomers and increases their self-efficacy (Steinmacher et al. 2016). Our interviewees indicated that it helps newcomers overcome different barriers that they face. Providing documentation happens in a variety of ways, including pointing newcomers to appropriate information, maintaining websites and wikis for each project, and writing about the accepted social conventions in the team. This fact was indicated by P5: “… [newcomer guidelines] make things easier and help people get along. We don’t have to teach the rules; they’re already there for the newcomers.” Moreover, P8 mentioned that “there shouldn’t be too many [process barriers] since we document everything.” In summary, keeping documentation concise and updated was reported as a way to help newcomers overcome technical and process barriers, in addition to cultural differences (N-I5) and harsh project atmosphere (N-I9).

The strategies reported here were mentioned by the interviewed mentors. We have no evidence of the extent to which these strategies actually help newcomers. Still, we could not identify strategies explicitly reported to help newcomers to overcome all social barriers. Therefore, it is an interesting direction to further investigate the reported strategies and to consider complementary strategies that might support newcomers in overcoming the reported social barriers.

4.4 RQ4: what are the additional challenges that affect women onboarding to OSS projects?

To answer RQ4, we explicitly asked our women participants questions about gender differences. In addition, we conducted nine follow-up interviews to gain further insight into the challenges women face in OSS environments.

4.4.1 Gender-specific challenges for newcomers

Our women mentor participants (P6, P10) reflected that, amongst their newcomer-mentees, women seemed to have lower self-efficacy. P6 stated, “They [women mentees] always can but they feel like they can’t.” This phenomenon was also mentioned by three mentors during our follow-up (F3, F4, F5, F7, F8, F9). F4, for example, reported that “[women mentees] often feel like they don’t have competency/fluency in the task and don’t trust their own skills;” this observation was furthered by F5, who said that women newcomers “feel shy, timid, under-confident.” This lack of confidence, according to F8, “is often the main challenge for newcomers (even the most brilliant ones).” Prior work has also found that women statistically have lower computer self-efficacy (confidence) than males within their peer sets, which can affect their behavior with technology (Burnett et al. 2010, 2011; Cazan eta l. 2016; Hartzel 2003; Huffman et al. 2013; Fisher and Jane Margolis 2002; O’Leary-Kelly et al. 2004; PiazzaBlog 2015). Mentors thus have a key role in offering a supportive environment; as F8 described, “it’s important that mentor will constantly remind that it’s okay and that no one expects full expertise from a newcomer.”

In our first interview round, mentors also mentioned that women contributors feel less comfortable with and accepted by their counterparts who are men when compared to their women colleagues. P10 explained having “had conversations with women mentees that they [mentees] probably would not have had with men mentors… about how OSS isn’t super welcoming to women, how do I navigate that… they wouldn’t ask that to a male mentor.” Later, she added, “women are socialized to be more [open] about their emotional state with each other than with men”, which might affect whether they convey their concerns to a male mentor. F5 explicitly said that “some male colleagues may try to undermine them and they might feel weak,” and proposed that “regular feedback from opposite genders related to work and involvement in community” would help reducing gender-related barriers in OSS projects. Along similar lines, recent research found that men, in OSS communities, “monopolize code authorship and simultaneously de-legitimize the kinds of social ties necessary to build mechanisms for women’s inclusion” (Nafus 2012). In general, cultures that describe themselves as meritocracies, such as OSS, have been found to be male-dominated environments that seem unfriendly to women (Turkle 2005).

Differences in motivation

for contributing to OSS projects have also been reported as a barrier for women to remain active contributors in OSS communities (as reported by P6, F4, and F6). P6, for example, stated that “for men it’s more their job to contribute to OSS, but women want to do it because they find it exciting.” Later she added: “It’s more difficult for women to stick around also, the top reason is that it’s not their job – they’re not being paid to do it.” During our follow-up, the topic appeared again. F6 believes that “the world has many barriers against female people and it is due mainly to the fact that women are often less ambitious and competitive than men.” The literature has found that motivations for women to use technology relate to accomplishments, whereas men are more motivated by their own interest and enjoyment of technology (Burnett et al. 2010, 2011; Cassell 2003; Hou et al. 2006; Kelleher 2009; Fisher and Jane Margolis 2002; Simon 2000; Singh et al. 2013). These differing motivations might also explain why some women do not stay involved in the community.

Women’s departures from OSS have also been attributed to style of communication. In fact, Nafus et al. found that acrimonious talk about which code piece should be incorporated leads to the system being “pushyocracy,” instead of a meritocracy, and is a prime reason why women leave OSS communities (Nafus 2012). This was reflected in P6’s comment: “Some communication styles that are used are occasionally more awkward, and men can come off as creepy.”

During our follow-up, another topic that emerged relates to the influence of peer-parity in OSS projects. According to Ford et al. (2017), the presence of peers increases activity from underrepresented users in unfamiliar spaces. One of the mentors (F3) mentioned that “not seeing a lot of people like oneself in a community is always a challenge. It’s lonely.” This was also brought to light by F5, when responding to the question about what strategies help reduce gender-related barriers in OSS: “Encouraging more women to participate in OSS projects… so that they feel a sense of attachment towards it.”

When we analyzed the male mentors’ answers to our interviews, we noticed that they did not seem to perceive differences in behavior when comparing men and women newcomers, and typically focused only on the newcomers’ contributions. For example, only two of our male participants even explicitly touched upon these differences: P3 stated, “I have only had two female contributors. I did not feel any difference so far.” When asked whether he has ever observed any differences in the behavior of women and men newcomers, P5 first said, “No and all I care about is good code,” but then added: “… 90% of the contributors are males, so there is underrepresentation of women. I have noticed that women were more proactive actually.” While women newcomers report a harsh onboarding experience or OSS environment, men do not seem to notice this phenomenon. This might indicate that women find the community less welcoming, perhaps since there are fewer women, and as a result may feel the need to prove themselves by working extra hard.

4.4.2 Gender-specific challenges for mentors in OSS projects

When we looked at the challenges faced by women mentors in OSS projects, we also found some gender-specific issues. One common theme discussed by our women interviewees was the underestimation of their skills and abilities. P10 stated that

“It’s easy for others to say I’m the mentor or I’m the community organizer, and not see me as an engineer, for example. That can be frustrating. It’s more likely to happen to women because people associate us more with nurturing, teaching roles.”

During our follow-up interviews, F1 mentioned that she sometimes experiences such issues with mentor colleagues as well: “Sometimes I feel my feedback is not taken as seriously as feedback from a male co-mentor.” Additionally, we found that some newcomers underestimate women mentors, as reported by F4, who faces difficulty in “getting my students to listen to my advice/take me seriously.” This reflects recent literature on stereotyping, in which women were seen to be warm and men as competent (Otterbacher et al. 2017).

The upshot of this stereotyping was that women mentors were seen as more approachable (stereotyped as warm Otterbacher et al. 2017). A woman participant mentioned that, in general, people find women mentors more accessible than men mentors, which is in line with a previous study (Ragins 1989). Along these lines, P10 stated that “being a mentor, my gender actually makes me more approachable.” In essence, although the stereotype that women are nurturing can distort how they are viewed as contributors, it can also make people feel more comfortable asking them for help as a mentor. Moreover, for women mentors it makes no difference mentoring men or women, as P10 stated, “I’m equally comfortable mentoring men and women and non-binary people.” Therefore, while in OSS women mentors are an asset to the OSS environments, and help make OSS a more desirable place to join for newcomers, all mentors (men and women) should recognize the need to improve mentor-mentee relationships to make OSS welcoming to all.

During our follow-up, we also found that influence of peer-parity, differences in motivation, and low self-efficacy, which we identified as challenges for newcomers (Section 4.4.1), are also challenges for women mentors. Interestingly, F3 discussed the influence of peer-parity as the only challenge she faces as a mentor: “I find it difficult when I go long stretches without working with other women either as newcomers or mentors for myself. I love mentoring young men but I’d really like to work with more women.” For low self-efficacy, F9 pointed out that a challenge would be “the constant feeling of maybe not being good enough.”

5 Discussion

In this section, we discuss the implications of this study for research and practice from the point of view of different stakeholders.

Researchers

As can be seen in Table 7, there are many gaps in mapping the strategies and barriers faced by newcomers, which can be explored in future research. In particular, social barriers are challenging and sparsely covered by the reported strategies (only 9 out of the 29 reported). Traditional socialization techniques (Griffin et al. 2001) could be investigated in this context. Besides, more research is necessary to investigate how to overcome mentor’s barriers, and how the shared barriers presented in Section 4.2.5 can differently impact newcomers and mentors.

Mentoring already occurs in some well-known summer of code programs (e.g., Google Summer of Code, Julia Summer of Code, and Outreachy) (Silva et al. 2017), and in formal mentorship programs like the Apache Mentoring Programme.Footnote 3 It would be of great interest to analyze how mentoring takes place in such kind of programs, and how it influences newcomers’ onboarding and retention. In particular, it would be interesting to understand the motivation and demotivation factors influencing mentors in these cases.

Mentors

We found evidence of 12 interpersonal barriers that impact mentors’ work while onboarding newcomers. Thus, it is important to make it clear to mentors that mentoring is not an entirely technical duty, as it involves an enormous amount of social skills (including friendship, coaching, and other psychosocial support (Baranik et al. 2010)), which can be decisive for the newcomers’ onboarding success. P3 states this as a problem with mentors: “… the biggest pitfalls of the mentor are: […] engaging in other ways than just coding. These projects are about community effort and more than just the code.” The mentoring literature shows that a mentor can help shield a mentee from flaming wars with more senior members and intervene in certain situations to help them resolve it appropriately (Kram 1988). Thus, helping newcomers with interpersonal barriers and making them feel supported potentially reduce the challenges faced while interacting with the community. Additionally, mentors can take advantage of the strategies presented in Section 4.3, employing them to support newcomers. Ultimately, mentors can benefit from the set of barriers uncovered in this paper (Figure 3), becoming more aware of what they can expect when dealing with newcomers, and better prepare themselves for supporting those willing to contribute to or join the community.

Online communities

We found that newcomers face barriers related to community atmosphere, micro-climate, and reception. Thus, a community can make newcomers feel welcome by treating them as potential contributors and showing them that the community cares about them. Sending thankful and welcoming messages helps in dealing with cultural differences and misunderstandings. In addition, not involving newcomers in unnecessary discussions and avoiding harsh/rude review messages helps keep newcomers motivated. Given the number of barriers mentors face (19), it is important that communities provide adequate support to those who volunteer or act as mentors, since the duties of mentoring can be challenging.

Specifically for OSS communities, we would like to highlight that our results reinforce previous work that suggests newcomers’ orientation is a barrier affecting both newcomers and mentors. Most significantly was the difficulty related to finding an appropriate task (as per Table 6), which was reported as a barrier for both mentors and newcomers. We reinforce that tagging the tasks (and keeping them up-to-date) showed to be effective, and this strategy is already in place in some well-established projects, like LibreOffice, Apache Open Office, Mozilla, Gnome, Media Wiki, and Ubuntu. In addition to “difficulty in setting up development environment,” previously evidenced in the literature (Steinmacher et al. 2015b), we found that “difference in the devices that mentors and mentees use” is also a barrier. Thus, providing ways to make it easier to build the system locally is of great benefit to the onboarding process. A potential solution would be a pre-configured environment, by means of a Virtual Machine with a built environment (Wolff-Marting et al. 2013), or a container management tool, such as Docker.Footnote 4

Education and Training Personnel

People interested in Education and Training can make use of our findings to better understand the barriers faced by both mentors and newcomers to OSS. We showed that the mentor position is challenging. When asked whether they had been trained to act as a mentor, all of our participants answered “no.” Therefore, it is important to offer training on the skills needed to be a mentor, either in undergraduate level or even in a professional environment. Moreover, given the number of social barriers revealed by the participants, it is important that (future) professionals acquire the proper soft skills that will better prepare them to mentor. For newcomers’ education and training, the barriers evidenced here serve as a starting point for making instructors aware of what to expect when making use of OSS projects as part of their teaching approach, which is becoming more common (Pinto and Igor Steinmacher 2017; Nascimento et al. 2013; Bishop et al. 2016).

6 Limitations

Although we collected data from mentors with different backgrounds and we kept interviewing until reaching saturation in the identification of barriers, we likely did not discover all possible barriers or provide full explanations of the barriers. We are aware that the OSS universe is huge, meaning the barriers and strategies can differ according to the projects.

Another threat to the results’ validity is the subjectivity of the data classifications. To avoid this threat, we used an approach in which all analysis was thoroughly grounded in the data collected. Additionally, we exhaustively discussed the analysis and results with the whole team to reach agreement.

Since we employed a snowballing approach to sample our participants, we acknowledge that sampling bias affects our interviewees’ selection, namely self-selection and social desirability biases. However, we counteracted this effect by inviting people with different profiles, from various projects, and with a diverse background, seeking out different perspectives.

We understand that some barriers that we have identified may exist (and have already been identified) in other types of online communities and other types of users. Here we chose to keep our focus on OSS settings to have a deeper understanding of this specific community. Future research should focus on analyzing the commonalities and differences among barriers faced in different domains to build generalized models and theories about onboarding and mentorship in open collaboration communities.

Finally, we acknowledge that we used the model proposed by Steinmacher et al. (2015b) as a baseline to compare our findings and this model may not encompass all the barriers reported in the literature. However, this model was built using data from multiple sources, including a systematic literature review conducted in 2014 to identify barriers faced by newcomers to OSS projects (Steinmacher et al. 2015a). In our additional searches, we could not find additional barriers reported in other studies.

7 Conclusion

OSS communities frequently rely on mentors to guide newcomers to become long-term, active contributors. In this paper, we relied on data collected via interviews with mentors of varying experience levels in OSS communities to identify 44 barriers faced by newcomers and mentors in OSS projects. In addition to analyzing the barriers faced by newcomers, we identified challenges faced by mentors while supporting newcomers. As a result, we found that, while some barriers affect only newcomers (25) or only mentors (10), other barriers affect both newcomers and mentors (9).

In addition to this perspective, we observed that most of the barriers identified (29 out of 44) relate to personal and interpersonal issues. This fact demonstrates the importance of soft skills for mentoring. In addition, we also uncovered strategies used by the interviewees to help newcomers overcome some of the barriers, and found a gap in how to help newcomers dealing with social barriers.

Moreover, in this study we identified some factors that influence the onboarding and retention of women contributors in OSS community, including: 1) differences in the viewpoint of men and women mentors about gender personalities; 2) underestimation of women’s capabilities by both OSS community and women newcomers themselves; 3) male mentors’ ignorance about the community being harsh to women; 4) differences in motivation when joining OSS projects; and 5) lack of peer-parity.

Our results provide insights regarding newcomer onboarding process and how it can be improved. By presenting strategies to overcome newcomer barriers, we aim to foster a new understanding of how to engage newcomers while enhancing the onboarding process as a whole.

While understanding what barriers affect newcomers is important, there are many future directions that can follow this research. In addition to the implications presented in Section 5, a potential next step would be to look at how mentors assign tasks, delving into how mentors assess newcomers’ skills, and how they match tasks to fit a newcomer’s interests and skill level. Another future step is to understand what motivates developers to work as a mentor in open collaboration communities and conceive strategies to attract more volunteers to this important role.