Background

Direct-to-consumer genetic testing (DTC GT) has been controversially discussed since its introduction. Decreasing sequencing costs have made genetic testing accessible to the average consumer, enabling 23andMe, a leading company in the field, to offer its services for $99 (23andMe 2017a). There are no definite numbers concerning how many commercial genetic testing companies exist that market genetic testing to consumers and how many consumers have actually used such services.Footnote 1 However, there appears to be a significant public interest in DTC GT in general (Goldsmith et al. 2012; Sherman et al. 2015). For example, 23andMe announced in June 2015 that it has genotyped more than one million customers and, according to its website, this number has recently increased to two million (23andMe 2017b). These numbers indicate that DTC GT business has grown considerably during recent years.

Ethicists have anticipated potential ethical problems with DTC GT quite early (Hoedemaekers and ten Have 1998) and scholars have expressed a variety of concerns regarding the commercial provision of genetic testing since then:

  1. (1)

    Informed consent An important ethical issue has been that some companies do not comply with ethical standards ensuring that consumers give proper informed consent to research conducted with their data and samples. This may have a negative effect on trust in genetic research as such (Niemiec and Howard 2016; Sterckx et al. 2013).

  2. (2)

    Clinical and personal utility Other ethical issues center on insufficient evidence for validity and clinical utility of test results (Janssens and van Duijn 2010; Saukko 2013). This includes questioning the clinical utility of genetic test results provided by companies, the extent to which commercial tests should be considered in genetic counseling and also whether marketing as a means for consumers to access their genetic information is justified (Kohler et al. 2017; Chung and Ng 2016). Lately, personal utility of DTC GT has become a topic of ethical inquiry. It refers to individual experience and use in line with personal values outside the realm of medicine (Chung and Ng 2016; Kohler et al. 2017; Bunnik et al. 2015).

  3. (3)

    Data protection Another major issue is the concern about non-transparent data protection policies of DTC GT companies (Laestadius et al. 2017). Economically, the data generated seems very valuable for DTC GT companies for selling and patenting even beyond the actual sale of tests (O’Doherty et al. 2016; Stoeklé et al. 2016). In a wider scope, it is possible that the data processed and stored will be used in unforeseen ways in the future, raising further questions regarding informed consent to such activities in the present. This is because consumers might not expect such uses and give superficial consent while not understanding the consequences. (O’Doherty et al. 2016; Obar and Oeldorf-Hirsch 2016).

  4. (4)

    Unnecessary use of health service Further worries have been raised that DTC GT might lead to a futile increase in use of public health services because of consumers who take subjectively alarming results to their doctors for clarification and seek treatment, even though it is not necessary (Plöthner et al. 2017).

  5. (5)

    Possibility of harm Another important aspect of the debate is the possibility of psychological harm as a result of knowing about personal genetic risks. Deficits in the current provision of appropriate counseling of consumers may fail to prevent such harm (Salm et al. 2014; Howard and Borry 2013).

However, there are also positive aspects pointed out by scholars: DTC GT may be a valuable option for individuals seeking access to personal health information that would otherwise not be able to get this kind of information and now do so independently from their doctors (Hogarth et al. 2008). It has also been suggested that DTC GT services could motivate consumers to adopt healthier lifestyle habits and make better health-related decisions, even though evidence for the latter is lacking (Covolo et al. 2015; Roberts et al. 2017; Bloss et al. 2011; Gordon et al. 2012). Surprisingly, there is to date little knowledge about the specific role of DTC GT for taking responsibility in the context of family life (e.g. the decision whether or not to alarm genetic relatives about their ‘potential’ genetic risks or decisions on reproductive behavior) and therefore its implications for intra-familial relationships (Anderson and Wasson 2015). Corpas (2012) illustrates that, from a consumer perspective, a decision to undergo DTC GT may appear as an individual rather than socially embedded decision.

DTC GT might also serve on a larger societal level as an educational tool raising awareness and understanding of genetics and disease in the population (Su 2013). Covolo et al. (2015) conclude in a systematic review, that the overall awareness of DTC GT in the public is currently low (Covolo et al. 2015). While some data exists for other European countries (Cherkas et al. 2010; Vayena et al. 2012; Mavroidopoulou et al. 2015; Oliveri et al. 2016), lay perspectives on DTC GT are still unexplored in Germany. This knowledge gap is particularly interesting since Germany is among the countries with a highly developed genetic research sector and diagnostic facilities and with a theoretically high market potential for DTC GT. However, according to the German law for genetic diagnostics, genetic testing for medical purposes may only be carried out by a specialized physician and with mandatory (personal) counseling.

Research question

Against this backdrop, we explored and analyzed lay people’s attitudes, perceptions and their assumed ethical implications towards DTC GT, for the purpose of contributing to the debate the perspective of potential users from a western industrialized country with restrictive regulations and a well-established publicly financed healthcare sector. For this, we used a qualitative exploratory approach with focus group discussions in Germany.

Method of data collection and analysis

We used the qualitative approach with focus groups in order to develop new hypotheses concerning laypersons’ perceptions of new developments in genetic diagnostics (Berg 2007; Barbour 2007; Krippendorf 2013). We conducted seven focus groups with laypeople (n = 43) from June to November 2016 in different German cities (Göttingen, Berlin, Frankfurt (Main), Cologne, thus, covering different cities in the east and west of Germany). We developed a structured discussion guideline according to which, after a short thematic introduction, we first presented screenshots of two DTC GT websites in German and then two fictional sample test reports displaying a variety of disease risk predictions to the participants as a stimulus. The websites were selected to display different examples in terms of quality in terms of web design and different degrees of commercial appearance. The guideline was pre-tested twice, with laypeople and with academic staff, to improve it and make minor adjustments. The test run allowed us to reduce redundancies and improve the flow of discussion. The focus group guideline consisted of five main scenarios, including predictive genetic testing for breasts cancer and Alzheimer’s Disease, genetic testing in biomarker research for stratification in neo-adjuvant colorectal cancer therapy and the possibility of becoming a research subject in a study that uses whole-genome sequencing and the DTC GT case vignette. This article focuses on a detailed analysis of the DTC GT scenario, while the others are analyzed and published elsewhere. This selection is due to the fact, that the different scenarios were designed to touch different ethical questions in the field of genetics and genomics. Of course, the general discussion flow and each group dynamic were also considered for this focused analysis.

Recruitment

For the purpose of the study we recruited people who were interested in sharing their views by means of flyers and posters at a variety of local public institutions (libraries, vocational schools, tech colleges, sports colleges) and also institutions providing genetic testing/counseling (clinics, physicians, genetic counseling practices) in the above mentioned cities. Additional recruiting was done online using social media (Facebook), mailing lists (e.g. sports or nutrition groups) and online ads (e.g. both print and postings on virtual bulletin boards unrelated to medicine). The highest response was achieved via online small ads. Online forms for registration were set up on our department’s website to be sent back via e-mail or on paper via mail or fax. A link to the forms was spread in various Facebook groups with local relevance to the mentioned cities (group topics including but not limited to self-help and advice, flea markets, giveaways, but also various sports clubs) and various, mostly local, small ad platforms.

Procedure

Participants were selected in order to achieve variation in terms of age, gender and educational background (Table 1). The large majority (72%) had no experience with genetic testing whatsoever.Footnote 2 Along with the invitation, participants received a pre-session info sheet briefly describing the topics the discussion would cover, in order to ensure basic understanding. This measure proved useful as the guideline was relatively extensive regarding the five different scenarios. All discussions were audio-recorded for later transcription. Each group was moderated by two researchers (1 male/1 female) experienced in the method. The group discussions took about 90–120 min each. Participants received an expense allowance of 25 €. The study was approved by the local ethics committee in March 2016 (Ref. Nr. 16/10/14).

Table 1 Sample characteristics: participants by sex, age, educational background, prior experience with genetic testing and average time spent online per day in total numbers and percentages and focus group composition. *FG = focus group
Full size table

Sample composition

All participants (n = 43) were 18 years of age or older, group size varied between three and nine participants. Written consent was obtained by all participants after reviewing an information form and receiving verbal instructions before the discussion. Reflecting the overall sample composition in terms of gender (17 male/26 female) five of the seven groups had a slight gender bias, including more women than men. Only one group covered all age groups. Participants between 36 and 50 and 71+ were less represented than other age groups, while the majority of participants fell into the categories of 26–35 and 51–70. The sample was highly educated with about 75% high school graduates and 50% holding an academic degree. Since DTC GT is marketed mostly online we also asked for the daily internet use of the participants. The majority of participants (n = 28) spent 2–4 h/day using the Internet, while about half of the remaining 15 spent less than 1 h or more than 6 h online per day. One participant had no internet access. 12 of the 43 participants had personal experience with some (non-commercial) form of genetic diagnostics.

Data analysis

We conducted a qualitative content analysis and focused on five basic codes (see Supplement material) (Krippendorf 2013). Coding and analysis was supported by the analysis software Atlas.ti™. All focus groups were audio-recorded and transcribed verbatim for the analysis in German and the quotes were translated into English for this article. Transcripts were pseudonymized (Metschke and Wellbrock 2002). We focused on comparing the statements in each group and compared the individual statements as well as the discussions’ flow and topics of the groups. The aim was to detect differences and similarities regarding attitudes and perceptions on DTC GT. The reporting of participants’ positions follows the scheme: many/most = > 50%, some = 10–50%, few = 0–10% of participants. “Majority” refers to absolute majority unless otherwise specified while “minority” means a very small number of participants.

Empirical results

Most participants were unfamiliar with DTC GT and even unaware of its existence, even though a smaller number of participants with experience in genetic testing were present in each discussion. However, we noticed that persons with experiences of genetic testing did not have noticeably special impact on the group discussions compared to the other participants. The general response was critical regarding trust in companies and utility of test results. Participants were in favor of availability of DTC GT if there was sufficient quality control and oversight. Table 2 shows the varying themes across the focus groups. We can distinguish three overarching topics within the findings that include several aspects of these main ethical themes: (a) questioning the utility of health risk information; (b) (un)trustworthiness of companies in this field, and (c) critical attitudes towards banning DTC GT. We will discuss the relatedness of these topics after the presentation of the main findings.

Table 2 Ethical themes across focus groups by frequency, distinguishing main, common and minor themes emerging in the discussion
Full size table

Questioning the utility of health risk information by DTC GT

Participants made a clear distinction between commercially offered tests for lifestyle purposes and for disease prediction. Lifestyle tests were seen by some participants as an interesting product with a personal benefit that is not related to genetic risk.Footnote 3 Especially tests related to nutrition and diet were named repeatedly. The underlying premise of the respective statements was that the data has no negative impact on personal lives and no practical consequences arise if this information is passed on to third parties.

Ms. A.: “[…] I wouldn’t get tested for diseases […], but all these other things, diabetes, overweight, blah, I would do that.” (Life-partnership, 26–35 years old, no kids, unexperienced with genetic testing)

Few participants saw a connection between DTC GT and self-tracking and self-optimizing and consequently criticized it. They stated that it creates an illusion of control that is incompatible with their view on the unpredictable reality of life as a human being.

Ms. F.: “[…] I’m reluctant to deal with these kinds of topics much. Not because […] I’m not interested […], but this idea of wanting to control everything, and […] to become a little obsessed with fears, and […] control life […]. Because I also think it’s a fallacy. Maybe in very specific cases, it might somehow be an option.” (Single, 26–35 years old, unexperienced with genetic testing)

Testing for genetic predisposition to diseases, especially serious diseases such as cancer, was seen much more critically by nearly all participants. When confronted with a sample report showing risk estimates for a variety of diseases including obesity, coronary artery disease, lung cancer, psoriasis and others with both realistic and mock numbers,Footnote 4 participants were insecure in handling the information and making sense of it. The discussions on risk estimates were rather reserved and participants questioned the utility of risk information.

Mrs. N.: “I don’t see any added value in these numbers either. […] In my opinion, those are no results […] about diseases which couldn’t be detected without this genetic test as well.” (Married, 36–50 years old, two kids, previous experience with genetic testing)

Skepticism was also expressed regarding the accuracy of results, implying that companies might not be competent or reliable enough to provide such information. The lack of capability to interpret probabilistic information and its meaning was paired with a strong emphasis on the importance of professional counseling. Many insisted that they would not want to be left alone with disease risk information and would want to have the opportunity to consult a physician to help understand and deal with consequences of test results. Participants frequently emphasized that personal interaction was important to them.

Mr. L.: “Yes, well, I’m also of the opinion that you should seek […] medical consultation there. […] Yes […], medical consultation should be placed special emphasis on […].” (Single, 18–25 years old, no kids, unexperienced with genetic testing)

Mrs. P: “When it comes to things like that […] you have to be able to communicate with a person.” (Widowed, 70+ years old, three kids, unexperienced with genetic testing)

Thus, they would prefer not to receive test results indirectly via commercial providers but via (their) physicians. Some participants expressed a complementary position: they emphasized that DTC GT would enable them to obtain information independently without having to consult their physicians, which would take the pressure from them to justify their wish for a genetic test. In their eyes, avoiding their physicians seemed a good option since they felt that they are not always taken seriously or got the medical attention they wanted.

The physician as an authority could then be bypassed in the process. However, few participants assumed that physicians would not accept if their patients brought test reports by companies.

(Un)trustworthiness of DTC GT companies

The discussion about DTC GT companies focused on their trustworthiness and reliability. Almost all participants agreed that they would not trust commercial providers and the discussion on this point were very lively. Three motives underlay this point of view:

First, companies were regarded critically simply because they are for-profit enterprises that presumably pursue primarily monetary goals, a position that even was the dominant point of discussion in one group. The motive here was a rejection of commercial exploitation of medical-scientific progress. Nearly all participants felt that it would be more suitable to consult physicians or medical institutions for disease-related genetic information and that the latter were to be trusted more when handling “their” data.

Ms. B.: “Yes, I think that should be taken then […] into medical hands and […] not into some private companies […], you would also be scared what else they can do with the data - whether they really are that reputable.” (Single, 36–50 years old, no kids, unexperienced with genetic testing)

Second, there was consensus in almost all groups that the companies are not reliable. Participants feared they are not run by qualified personnel, might deliver inaccurate or fake results, have unfavorable terms and conditions regarding hidden costs or data protection or that the service is a simple money-making scheme. The concern was frequently expressed that data could be passed on to third parties such as insurances or employers causing a disadvantage for the consumer.

Mrs. R.: […] It looks so petty. As if you were being petty about your data […] But I’m […] very sure, that these data have a high value the second you have lots of data and trends. And you can deduce a development from it, by controlling people. By persuading people, it’s really, really great and you all have to do it - like lemmings now… And I think… our genome is one of the most individual and most personal things we have. To my mind, it doesn’t belong to the internet. I hold on to it firmly, right? And see […] that no one else […] gets their hands on it, except now for a doctor I really trust. (Widowed, 51–70 years old, no kids, unexperienced with genetic testing)

This cautious stance may have had two reasons, the first being that participants did not have any background information about the companies, which gave room for speculation, and the second being the way the companies presented themselves online: Notably, in most groups at least one participant referred to one of the example websites presented to them as looking poorly designed and therefore suspicious. It was expressed that the websites were too prominently displaying payment information, resembling an insurance website or its content sounding like it was a religious cult. One person thought the design was “unbelievably clever” in that they make their selling point very clear.

Third, a minority of older participants was skeptical about conducting business online altogether. This, however, seemed to be a general resentment not specific to this type of business. Three women in the age groups above 50 opposed DTC GT because they felt it was part of a development undermining freedom and autonomy of future individuals. They claimed that not everyone might like genetic testing to become a common practice or even the norm, fearing it could bring discriminatory practices.

Mrs. R.: “I am afraid of accustoming to such things, that they appear so normal, all of a sudden. There is a genetic test, some idiots will definitely become customers there. And that is then so completely normal and then I see the development, then it slips. It starts slipping. With things like that it starts slipping. Then it will perhaps be completely normal perhaps, when you apply you have to submit an excerpt of your genomes [sic!], if you are suitable for the job and the devil know what else. How sick you are. What risks you have. […] So I… I am terrified of that.” (Widowed, 51–70 years old, no kids, unexperienced with genetic testing)

Pro-regulation, against ban of DTC GT

The discussion about legal restrictions on DTC GT revealed a mixed picture. Despite the critical stance outlined above, the majority of participants opposed a ban and some proposed regulatory oversight by the state to ensure quality and confidentiality.

Mr. M.: “So I am principally for a liberal economy but I think […] important information […] must be protected by the state. If we conduct something like that, we can talk about whether we do it or not. But if we do it, then the whole thing [should] please be institutionalized, that there, that we don’t have any charlatans there that gain a fortune with some sort of sensitive data of which many do not have any idea what exactly they are doing.” (Single, 18–25 years old, no kids, unexperienced with genetic testing)

Two reasons became apparent in the discussions: First, it is up to the individual to decide whether it is a good option for him or her. Second, participants stressed that the service would remain available over the Internet. A ban would thus be unlikely to prevent people from using such services via mail order. Among the opponents, it was mostly younger participants who explicitly expressed that a ban would put restrictions on consumers’ freedom of choice.

Ms. A.: “But if I say as a whole man now: I simply want to know it for myself. Then it’s my decision, nobody is forced to do it online, it’s only offered as an alternative. It’s not as if doctors were no longer allowed to do it, that’s not the case.” (Life-partnership, 26–35 years old, no kids, unexperienced with genetic testing)

Discussion

Quantitative studies conducted in other countries such as UK, Greece, Switzerland and Italy showed a relatively high willingness among respondents to use DTC GT. Obtaining personal health information was reported as a main motive in laypeople as well as in early adopters (Cherkas et al. 2010; Mavroidopoulou et al. 2015; Vayena et al. 2014; Oliveri et al. 2016; Gollust et al. 2012). However, German laypeople in our focus groups were very reserved about DTC GT and were overall less informed of its existence. The apparent main motive for uptake made out by other studies (obtaining health-related information) was seen with concern by our participants. The differences in results may be due to data collection methods, as other studies from the UK, Greece, Switzerland and Italy used non-representative surveys, while we used qualitative methodology. The methods also rely on different recruitment strategies: The surveys were mainly conducted with specific groups (university students, twins), while our sample consisted of laypersons ‘interested’ in genetic testing. Moreover, focus groups allow for more reflective and argumentative dynamics, while surveys often ask for spontaneous preferences and attitudes.

Participants’ clear distinction between well-accepted lifestyle-tests and only partly accepted health-related tests revealed a moral difference regarding genetic test results. Some results they can handle and others they do not fully understand or find confusing and are critical and sensitive. The distinction between the two types, however, was not always consistent. While the former appealed to curiosity and fun (i.e., has personal utility) the latter was preferred to be discussed with a ‘real’ physician. After all, genetic information remained for them sensitive information that should not be shared with third parties such as employers or insurances.

Laypeople’s principal distinction between medical and commercial services was also found in a quantitative study by Critchley et al. (2015) for an Australian population. As in the Australian case, a possible explanation other than the negative perception of DTC GT companies and services is that the healthcare system impacts laypeople’s assessment of DTC GT. Germany has a long tradition of a public health care system where out-of-pocket payments play a minor role compared to many other countries (e.g. USA, Switzerland). Most Germans are used to receiving state-of-the-art medical services covered by their health insurance. They are used to being treated as patients, deserving of the treatment most suitable to them, which implies a purely beneficent approach in a classic physician-patient relationship. They are less used to a consumerist attitude and perceive medicine and the market as two separate spheres of action. Commercial health-related genetic testing therefore causes insecurities.

Hence, we interpret the skepticism towards companies as a result of a hybrid commercial-medical practice placing people in a hybrid consumer–patient position they are rather unfamiliar with in the German setting. We can also understand this skepticism embedded in expectations held for the medical profession. The participants’ ideal of a physician is threefold in this context:

  1. 1.

    He/she is supposed to work as a competent expert who is able to interpret genetic (risk) information (someone with the ability to explain things).

  2. 2.

    He/she is as a trustworthy instance (someone who is on the patient’s side) who will not pursue any goals that are not in line with or counter the patient’s interest and who is bound by professional code to medical secrecy.

  3. 3.

    He or she is a social instance (someone to talk to) who is available for face-to-face interaction.

Participants therefore apparently perceive companies and online communication as an uncanny way to deal with sensitive medical information since these important aspects are not or not fully covered in their eyes. Conversely, these are the criteria that in their perception make physicians (or more broadly, medicine) trustworthy. There is a perception of the physician-patient relationship as a special type of social relationship that is not comparable or reducible to a mere business relationship. Our results thus show that trustworthiness of the providing party is an important factor when it comes to individual preferences of whether or not to undergo genetic testing and this is a property that such companies do not necessarily have.

As O’Neill (2002) points out, there is a moral function of trust that allows institutions as well as individuals to form relationships they can rely on and consequently perform their respective roles and functions properly. In maintaining a critical stance toward commercial provision of health-related genetic testing and upholding the strong medical-ethical standards (esp. beneficence, non-maleficence, but also non-directive counseling standards), the medical profession can prevent an undermining of trust that might affect it in the long run as well. The public trust in medicine depends on the ability of its institutions to maintain a strong profile of trustworthiness, i.e., being competent, reliable and honest regarding its tasks. More specifically, it needs to be an ethically consolidated field of practice focused on patients’ needs and well-being without moving monetary profits to the forefront, or, as O’Neill (2002) puts it: “good legislation, good regulation, good policies and consistent professionalism are a beginning” to create and consolidate trustworthiness—and trustworthiness is an important resource for medicine and its institutions.

Some scholars have stressed that multiple dimensions of personal utility may be attached to the use of genetic tests that go beyond clinical utility and fall more in line with a consumerist approach to medicine (Bunnik et al. 2015; Kohler et al. 2017; Turrini and Prainsack 2016). As long as harms are prevented and quality standards are sufficiently high, it is difficult to argue why consumers should not have direct access to such services and be granted autonomous decision-making (Loi 2016). However, we suggest that the implications for the physician-patient relationship are another important consequence of DTC GT to keep in mind. Scholars have claimed that blurred lines between medicine and commercial provision of medical goods change those relationships, when patients increasingly demand treatments and services exceeding traditional curative medicine (Karsch 2015; Siegrist 2012). Skepticism and distrust may be adequate approaches between contractual partners in a market environment, where participants seek to maximize own benefits and keep expenses low, but this approach is rather toxic to a trustful physician-patient relationship. Therefore, we stress the general category of trust in medicine as another important dimension in the medical-ethical discussion about DTC GT. Our results suggest that a consumerist approach may come with the disadvantage of also indirectly affecting the reputation of medicine negatively, particularly when lifestyle testing (mainly of personal, non-clinical utility) is marketed alongside tests with medical value (especially tests for risk prediction for common complex diseases or monogenic disorders).

Unlike findings by Bollinger et al. (2013) for a US population of DTC GT customers, our participants had very low trust in companies offering DTC GT. However, both studies find similarities in the public’s wish for more regulation but against a total ban. This implies support for a liberal approach where state paternalism is being rejected.

Topics not mentioned by laypeople also can be indicative of future ethical analysis: We found it intriguing that participants in our focus group study did not discuss family issues or related responsibility (e.g. arguing that for reproductive decisions or later life planning it would be morally good to know about potential health risks) when discussing DTC GT There was also no discussion about whether the results might have an impact on one’s family or partners or how the decision to undergo such tests may be influenced by a feeling of responsibility towards others as it was reported in many qualitative studies about genetic testing (Leefmann et al. 2017). Instead, we observed a strict focus on individual decision making with personal consequences. Nonetheless, responsibility aspects were very present in other sections of the focus groups, where clinical predictive genetic tests for several diseases like breast cancer were discussed. However, these were not analyzed for this article. So the absence of it in regard to DTC GT does not necessarily mean that the topic was irrelevant for the participants. It rather indicates that the skepticism was so profound that they did not get to the point of talking about whether there may be cases in which it would be important in a family context to have commercially offered genetic testing available (e.g. for carrier status). This shows that the overall focus on individual consumer autonomy ignores implications of genetic risk information for a family. By this, a strong individualistic focus dismisses notions of responsibility other than for oneself (Arribas-Ayllon et al. 2012). DTC GT perceived as an “empowering” option here reflects the idea of a socially disconnected individual managing his or her own health. In turn, this indicates that the participants feel addressed by the companies (at least the ones from our examples) accordingly.

Limitations

The results of our study must be regarded in the light of the methods used. Sample size and compilation might have had an impact: more women than men took part in the focus groups and people between 30 and 50 were less represented (the typical age of people who are working day jobs and are thus harder to recruit) who may have had different attitudes. Since the method applied here is qualitative in nature, we cannot claim representativeness of the group and generalize results for the broader population. However, the aim was to explore lay understandings to get a first idea of what potential users in Germany think of DTC GT. The study was embedded in a more comprehensive study design investigating laypeople’s attitudes toward recent developments in genetic diagnostics. Some more general aspects of the research subject will therefore be discussed in another publication.