Introduction

If the question “What is a wearable?” was posed just 20 years ago, it is likely most people would have responded with a list of (power-less) clothing such as shoes or jackets. However, the same question posed today could produce a very different list—one that includes powered devices used to monitor and quantify the body rather than merely protect it from weather. This new understanding “wearable” now includes a category of computing artifacts that make use of sensor driven self-quantification to implicitly or explicitly direct user behaviors. This genre of technological application has a strong presence in healthcare as a means to monitor health, modify behavior, improve health outcomes, and reduce medical costs (Boulos et al. 2014; Lewis et al. 2005; Swan 2012a). However, the commercial sector is quickly adopting this technology as a means to monitor and/or modify consumer behaviors as well (Swan 2013; Berglund et al. 2016). Actors within this growing industry make claims that sensor data from wearables can be used to guide and improve numerous aspects of user behaviors by providing users with more information about the self through which to make decisions. This paper explores ways in which self-quantification, particularly through use of consumer wearable technologies as they are currently being regulated, designed, and deployed today, may inadvertently contribute to negative outcomes directly opposed to the positive outcomes that they claim to afford.

Lisanne Bainbridge’s well known “Ironies of Automation” (1983) laid out a set of fundamental criticisms surrounding the promises of automation that, even 30 years later, remain both relevant and, in many cases, intractable. For example, Bainbridge points out that designers often automate as much of a task set as they have the intellectual and technological capacity to do at the time, leaving the rest (and often most difficult part) of the task for humans to do. This can lead to loss of valuable manual skills and/or situational awareness that, ironically, can make the human worker’s job more difficult rather than easier. Bainbridge’s work drew attention to potential problems in the design, implementation, and control of automation that began an important dialogue among stakeholder about best practices in automation. A similar set of ironies in technologies for self-quantification (often referred to simply as wearables) is laid out here, spanning from instrumental problems in human factors design (such as disagreement over physiological norms) to much broader social problems (such as loss of freedom). As with automation, these ironies stand in the way of many of the promised benefits of these wearable technologies. The paper begins by providing background on how self-sensor data (SSD) are implemented in self-quantification and what regulation currently exists, then moves to describe four key ironies of sensor driver self-quantification: (1) know more, know better versus no more, no better; (2) greater self-control versus greater social control; (3) well-being versus never being well enough; (4) more choice versus erosion of choice. It is argued here that without addressing these ironies now, the promises of wearables may not come to fruition, and instead user may experience outcomes that are opposite to those which the designers seek to afford, or, at the very least, those which consumers believe they are being offered.

Background

Self-sensoring Technologies Ends Versus Means

Self-sensor data (SSD) can be used as a means or as an end in a technological system. In some wearables, such as prosthetic limbs, cochlear implants, and speech to text devices, SSD is used as a mere means of communication between the body and a device in order to control the action of that device. In this category of wearable, the SSD it makes use of may not necessarily be biologically relevant or intellectually meaningful to the user. For example, a myoelectrically controlled prosthetic arm acts through the SSD collected from electrical activity in the residual muscle of the amputated limb as the user directs the actions of this muscle. The user need not regard the SSD objectively for the data to have value; it is merely a means through which to move the arm. On the other hand, for some wearables, SSD are treated more as ends in that the purpose of the wearable is to inform the user of the objective quantity of SSD—the ends are the quantifications of the self and value of these ends are derived from the user’s desire to know what these data are and what they mean. In this case if a wearable is designed to monitor myoelectric activity of a body part so the wearer can identify strengths and weakness of the muscle tissue, the value of this device is driven by what the user believes these SSD tell themFootnote 1 about their body and by how important it is to know this information. Granted, one could argue if the user intends to use these self-quantifications to improve their health, the SSD becomes a means yet again. However, for the purposes of this paper, the distinction is made to clarify that wearables that only use SSD as a mere means to operate the technology (such as a wearable eye tracker used to control a video game) are not subjects to the ironies that will be laid out. Rather the ironies are relevant to technologies that act as a means to the end of self-quantification, it is through this commodification of SSD that unique ethical concerns arise.

When consumer wearables engage SSD as ends, success (when measured as market share and profits) rests in the ability of the manufacturer to convince the user these ends have value—in other words, the user must perceive some need to know what these data represent. The underlying theme in all such attempts to construct a value for these devices (whether accurate or not) is that SSD inform the user to act. This is the prescriptive quality of the device. For example, the value in monitoring heart rate is derived from the belief that it ought to be some particular value, and that certain actions or inactions can help achieve or maintain this value. If heart rate is within a normal range the prescription is to maintain current activity or habits, if it is outside a normal range, the prescription is to act is such a way as to get to the normal range. If there is no norm and/or the user has no control over it (say for example if heart rate were random) prescriptions cannot be derived and the value in monitoring it would be low. For the remainder of this paper, unless otherwise noted, the term “wearable” refers to artifacts marketed to consumers where SSD are the ends and these prescriptive qualities are imbued in the design and marketing.

Ironies and Regulation

A number of the ironies of self-quantification laid out in this paper will be better understood by first looking at how consumer wearables are (or more accurately, are not) regulated. Although the healthcare industry has so far been relatively transparent in terms of ethical standards, outcome goals, and behavioral impacts of wearable design (Lewis et al. 2005), research and development policies and regulations for consumer wearables are nebulous and far less transparent. The United States Food and Drug Administration (FDA) exercises oversight over some wearables, specifically those deemed to be medical devices for diagnosis and treatment or certain health conditions. However, in 2015, the FDA released an unofficial, nonbinding set of guidelines for what they call “low-risk general wellness” products, which would apply to the type of consumer wearables of interests in this paper (US FDA 2015). These guidelines define what the FDA considers to be a low-risk, general wellness products and, critically, specify that such products will not be subject to FDA compliance or regulatory requirements. What follows is an examination of how the language and implications of these guidelines shapes the landscape of the consumer wearable industry and associated risks.

The characterization of “low risk” concerns how the physical components of the device interact with the human body. Most of the best-selling wearable devices available today, such as the Fitbit and Apple watch, would be considered “low-risk” under the guidelines because they are non-invasive (do not breach the skin or mucous membrane of the body), are not known to cause harm if the controls malfunction, and do not damage the skin due to biocompatibility issues. Classifying a device as a “general wellness” product is much more subjective and is primarily linked to what a manufacturer claims the device can do rather than what it actually does. The guidelines describe “general wellness” products as those with intended use claims solely related to:

  1. 1.

    sustaining or offering general improvement to conditions and functions associated with a general state of health that do not make any reference to diseases or conditions, OR

  2. 2.

    promoting, tracking, and/or encouraging choice(s), which, as part of a healthy lifestyle, may help to reduce the risk of certain chronic diseases or conditions; and may help living well with certain chronic diseases or conditions (US FDA 2015)

These guidelines only address “intended use” claims, not the affordances of a product. While a device might be marketed as a low-risk, general wellness product, the capabilities of the device may still afford users access to in-depth symptomatic information unrelated to the intended use claims, and which would normally be used by a physician to diagnose disease or chronic illness. For example, wearable device manufacturer Empatica produces two products that collect similar information but are marketed differently (Empatica 2019). Their E4 Wristband is marketed as a consumer-available product that provides the wearer with continuous real-time monitoring of blood volume pulse, electrodermal activity, skin temperature, and motion-based activity. Marketed as a data collection device with no explicit claims related to diseases or medical conditions, it does not require FDA approval. Meanwhile, the company also offers the Embrace Watch which collects identical information, but is marketed as an epilepsy monitoring and prevention tool for Sudden Unexpected Death in Epilepsy (SUDEP). Marketed in this manner, the Embrace Watch falls under FDA purview because the intended use claims are directly related to diagnosis of a medical condition.

To push the issue of affordance even further, the FDA’s suggested guidelines for low-risk, general wellness wearables also do not address the issue of compatibility and data sharing with third-party, FDA approved, self-sensor technologies that can transmit data from within the body. This approach could vastly expand the scope of real-time data available to these wearable devices and would in effect circumvent the “low risk” aspect of the guidelines so long as the compatible third-party device has its own FDA approval and the connected low risk device doesn’t make claims beyond general wellness.

Indeed, the notion of general wellness can be difficult to narrow down as well. The guidelines state “disease-related general wellness claims should only contain references where it is well understood [emphasis added] that healthy lifestyle choices may reduce risk or impact of chronic disease or medical condition,” but leaves unsaid from whom this type of consensus must come and what standards establish it. When it comes to disease-related risk and treatment, the notion that things are ever “well understood,” seems overly idealistic. Perceptions of what is and what is not normal or healthy continually change and things we consider “well understood” as contributing to a healthy lifestyle (and even what we consider to be a “healthy lifestyle”) are intimately tied to changes in culture and power. The individuals and organizations that control medical knowledge are able to shape these perceptions; however, some of those considered to be “part of” the medical community, such as the pharmaceutical industry, are motivated to create, or selectively make available, knowledge that most benefits their economic interests, rather than the best interests of the patient (Poitras and Meredith 2009; Moynihan and Henry 2006; Goldacre 2013).

A final criticism of the FDA’s unofficial guideline is related to a psychological phenomenon referred to as the gatekeeper effect. This phenomenon occurs when an authority (the gatekeeper) exercises some form of implicit or explicit information filtering before releasing information to a group. Research suggests that information not excluded by an authority may be perceived as more persuasive even though it is not specifically endorsed (Schweitzer and Saks 2009). In a legal setting, a judge, or the judicial process more broadly, may imbue scientific evidence with a higher degree of credibility merely by allowing or not disallowing access to the information—regardless of the intellectual merit of the information itself (Schweitzer and Saks 2009). Thinking of the FDA as the gatekeeper, by publishing guidelines through which a manufacturer can avoid review of their product (by making only certain kinds of claims about intended use) products that monitor health related information are in some sense “approved” by a process of omission. When thinking about determinations of product safety, not having FDA approval is quite different from not needing FDA approval. The former is more ambiguous and could imply either a product has yet to seek approval or has been rejected. The latter, however, implies a degree of risk so low that it could be misinterpreted by laypersons as essentially having FDA approval insofar as the perceiver believes the product ought to be within the purview of the FDA. The draft guidelines associate the FDA with product safety for these low-risk, general wellness products in a way that did not previously exist. Because the affordances of the wearable are not considered in the draft guidelines, through careful wording of intended use claims a manufacturer could claim a product, which might otherwise be required to endure clinical testing, is so safe it does not even require FDA approval.

In addition to the FDA, the United States Federal Trade Commission (FTC) could also play a role in regulation of consumer wearables. Although the FTC has been involved in public discussions of health and fitness tracking, so far, it has been just as reluctant to offer official guidance for consumers or manufacturers. Although the FTC’s stated mission is to protect consumers against unfair or deceptive business practices (US FTC 2015), their focus for wearables has mainly been on ensuring data security and not on the impact devices have on consumer health (US FTC 2013, 2016). In a prepared statement released March 22, 2016, the FTC acknowledged that products and services that collect and store health information raise serious privacy and security concerns for consumers, especially when these activities take place outside traditional medical contexts not subject to the Health Insurance Portability and Accountability Act (HIPAA). However, the statement characterized consumer protection only in terms of preventing fraudulent access to personal data, with no allusions to protections against negative outcomes related to, for example, bogus health claims (US FTC 2016).

Given the relatively small market penetration of wearables like the Fitbit and Apple’s smart watch (Patterson 2016), the existing FDA draft guidelines and the FTC’s limited involvement may not seem terribly problematic. However, expectations for all of these technologies are that their use, relevance, and permeation will grow exponentially in the coming decades, such that ubiquitous, robust data collection will soon become more pervasive and more powerful (Patterson 2016; Gibbs 2015; Swan 2012b). Efforts to bring this future to a reality are clear; with government and industry actors working together to pave the way, so it is critical to consider current policies and standards. The complex network of actors involved in wearable design, which is discussed throughout this paper, and the reluctance of critical government agencies to get more involved in risk assessment and prevention for consumer wearables sets the stage for a number of ironic outcomes, those in which the promises made to consumers about the value of the self-quantification may backfire for end users.

Ironies

Know More, Know Better Versus No More, No Better

Perhaps one of the most salient promises of wearables is that they will afford an increased depth and breadth of knowledge about one’s body. However, the ways in which knowledge is derived from wearable data streams and the manner in which humans process information may stand in the way of this promise. First, it is important to recognize that this affordance of knowledge is achieved not merely through passive awareness of a wider variety and greater volume of raw data points, but through active interpretation of the meaning of changes and interactions within these data. Consumers must have some understanding of the how to interpret the SSD stream in order to “know” anything. How these understandings are created for consumers and who has control over such activities is not an objective, value-neutral process.

There are many actors and stakeholders involved in the design cycle of wearables and many points in this cycle at which ethical standards of knowledge production are either unclear, unarticulated, or conflicting. Figure 1 illustrates key aspects of the wearable design cycle and factors that require designers to make value-based judgments influenced by personal beliefs and goals. For example, the decision to use a pressure sensor that operates by measuring changes in the volume of a gas substrate rather than one that uses a liquid substrate is based on the designer’s beliefs about efficiency, costs, availability of materials, etc. in combination with their goals for the “user,” which may be a manufacturer, a regulatory body, peers, an end-user, and so on. In its most basic form, engineering related standards apply to the mechanics of wearables, science and healthcare related standards apply to the construction of a knowledgebase from which rules about sensor data can be interpreted, and private industry standards are used to pick and choose which of these rules to use.

Fig. 1
figure 1

Key interactions between major value drivers, actors, and processes that shape wearables design factors. This is not meant as an exhaustive list, but instead as illustrative of the numerous points at which value-based decision making toward wearable design is pushed and pulled by often competing and mutually exclusive needs

Full size image

The algorithms that define the wearable’s behavior are at the heart of knowledge making through SSD and they are shaped by interactions across these domains. For example, the task of coding a set of commands based on input from the sensor data (e.g. when heart rate reaches > 90 bpm, “X” should appear on screen) into syntactic and semantic symbols that can be understood by both the technology and the user, is a computer engineering task (e.g. software engineers). But before that can happen, interpretation of the biological meaning of sensory data must be derived from a body of knowledge primarily under the domain of natural sciences. These interpretations are rarely definitive even within fields (e.g. disagreement among doctors and scientist in the same specialty), let alone across fields (e.g. disagreements among doctors and scientist in different specialties). So, when agreement is shallow or even contradictory, wearable designers must choose which interpretations to use and which to ignore in order to generate a set of “truths” for rendering to the user. These decisions may be made, for example, by an engineer, a scientist, or an entrepreneur with access to the internet. Decisions associated with algorithm design for wearables are at the heart of identifying and understanding the social implications of wearables and assigning responsibility for those outcomes, this is discussed further in the context of the other potential ironies presented later.

The user interface for wearables is another design aspect of knowledge making that falls under multiple actor domains. The codification of the interface software behavior is primarily within the domain of software engineering, but the aesthetic, messaging, and instrumental goals actualized through the work of the engineer are based on the social, cultural, and economic knowledge (and needs) of the actors who are driving market production. Their efforts together shape which knowledge is accessible to the user, which is most salient or made to seem important and actionable. Users may be unaware this knowledge has been filtered and shaped for them by a series of actors who are influenced by values or goals that may or may not reflect the user’s own values and goals.

Finally, user behaviors must also be considered as a key aspect of knowledge making in the wearable design cycle; this factor also falls under multiple domains of ethical and regulatory design considerations. Actors in the designer cycle can opt to restrict the degree to which users can modify or interfere with a user interface. In order to do this, designers must imagine the various ways in which a user could use a technology in a different manner from which it was intended and must then decide whether to allow this alternate use or to prevent it. Whether this practice ought to be more formalized as a required responsibility of the design engineer and the extent to which this type of analysis should be pursued is part of ongoing debate regarding computer engineering ethics and ethics in technology more broadly (see Herkert 2001; Basart and Serra 2013; Association for Computing Machinery 2018). Users themselves make choices about how and when the interface is used, but whether these choices are fully informed or not depends on how transparent and salient the technology’s purpose, operating instructions, and risks are. In other words, just as a pharmacy may attempt to control user behavior by supplying childproof caps on certain prescription bottles, the patient to whom that bottle is issued can choose whether to use the childproof cap or not by altering it or moving the pills to a different container.

In terms of generating more knowledge about the self through wearables, these gaps and ambiguities in responsible design threaten to derail this promise and amplify the risk that this increased information stream becomes nothing more than noise or propaganda. But addressing this concern may not guarantee a positive outcome. Interestingly, even if the knowledge generated through such technologies could be deemed reliable by some regulatory process, numerous studies in the field of human information processing demonstrate that human decision making can actually be deteriorated by information overload (Eppler and Mengis 2004; see also Simon 1956). That is, when faced with too much information about the self (i.e., more than can be held in the limited capacity of short-term memory during a decision-making event), users may actually use less information than they would in a relatively lower information situation, which can negatively impact decision making (Cook 1993). In other words, more information does not always result in more “knowing.” Without methods for ensuring that biosensor data are relayed to users in a safe, unbiased manner, and in the right quantity (so as not overload and effectively reduce the amount of information users process), the promises to know more and know better may manifest a future in which the knowledge accessed by the user is no more and no better.

This irony can be illustrated by looking at consumer sleep monitors. There are a number of different devices on the market with myriad claims about what can be made known about sleep using SSD (for a review see Peake et al. 2018). A growing number of wearable sleep monitors claim to use SSD to assess the percentage of time spent in different sleep stages such as light, deep, or REM sleep. However, there is a great deal of disagreement and ongoing debate within scientific communities about how to measure sleep stages and sleep quality, and questions have been raised about whether these devices actually measure what they claim to (de Zambotti et al. 2019). Indeed, there is some debate about whether these devices can even reliably distinguish between sleep/awake states (Peake et al. 2018). Designers of these wearables are left to make their own choices about which SSD will be monitored (e.g. heart rate, motion, blood oxygen, sound, brain waves), what parameters or threshold values of these SSD streams will indicate being in one stage or the other, what other factors will be considered in making these determinizations (e.g. gender, age of users, retrospective appraisal inputs), and finally, what sleep quality norms to present to the users (e.g. what ideal amount of REM sleep to strive for). Given the high degree of uncertainty about the relation between SSD and sleep, it seems these choices are necessarily dubious. It is perhaps not surprising very few of these devices have been validated in experimental settings (Peake et al. 2018). The irony is that although these devices undoubtedly generate a greater volume of information for the user about sleep, there is little evidence to suggest this information is reliable, meaningful, or helpful in terms of knowing more about one’s sleep quality (de Zambotti et al. 2019; Peake et al. 2018). Users may end up knowing no more about themselves, in terms of objective truths about the body, than if they didn’t have the device at all, worse still, they may be less-well if they feel stress, worry, or confusion about perceived abnormalities or imperfections in their sleep quality.

Greater Self-control Versus Greater Social Control

The promise that an era of self-quantification will afford increased knowledge about the body is made relevant by a second promise; that this knowledge will lead to greater control of the body. However, the persuasive capacities of SSD and the imbalance of power over defining what users are being persuaded to do may stand in the way of this promise. While wearables have the potential to act as the ultimate tool in self-surveillance of the body—from its location in time and space, to the location of electrons in its brain, and from the construction of muscles to the construction of identity, ubiquitous low-risk, general wellness wearables can also act as a modality of social control that users may not be aware of, and may in some respects reduce self-control in so doing.

The emerging Internet of Things and Big Data economies offer unprecedented opportunities to connect, share, and control information about people and objects moving in the world, and to infer patterns and knowledge from these interactions. Wearables offer this network unprecedented access to the human body qua body, and the individual objects it subsumes (such as blood, heart, brain and so on). From that knowledge emerges the ability to calculate, organize, and direct both objects and bodies at scale and frequency previously unattainable. While some argue the ability to monitor and observe every aspect of the body creates an opportunity for individuals to control their own health outcomes (Hofmann 2016), philosopher and social theorist Michael Foucault’s writing about “the political technology of the body” suggests that this ability to know and control the body can be used as a form of subjugation and as a tool for discipline (Foucault 1977). He argues:

“subjection is not only obtained by the instruments of violence or ideology; it can also be direct, physical, pitting force against force, bearing on material elements, and yet without involving violence; it may be calculated, organized, technically thought out; it may be subtle, make use neither of weapons nor of terror and yet remain of a physical order. That is to say, there may be a ‘knowledge’ of the body that is not exactly the science of its functioning, and a mastery of its forces that is more than the ability to conquer them: this knowledge and this mastery constitute what might be called the political technology of the body.” (p. 26)

Foucault argues constitution and control of this political technology of the body is complex, diffuse, and “not localized in the relations between the state and its citizens” (p. 27). However, as Lurie and Mark (2016) argue, increased complexity of and dependence on computing artifacts is correlated with an increased gap in user understanding of how such technologies work; this could result in excluding users from this process of constitution and control, putting designers (and their organizations) in the role of conqueror. User are further excluded from this process through issues of access. Wearable designers prevent access to the algorithmic rules that are used to calculate, organize, and direct the body (and the specific scientific principles the rules are based on) by simply not publishing this information. For now, there is no legal imperative or industry norm that motivates companies to disclose this information, which means it cannot be easily scrutinized.

Foucault also describes the notion of docility, which is ultimately tied to subjection of the body, and institutional interests in techniques that exploit the body as an object and target of power. Foucault argues that this exploitation of the body has taken place throughout recorded history, but unique changes to approaching docility emerged around the eighteenth century (p. 134). What is chilling in his descriptions of these changes is how the affordances of wearables are so well suited to the same purposes in terms of “scale of control” and “modality.”

  1. 1.

    …there was the scale of the control: it was a question not of treating the body, en masse, ‘wholesale’, as if it were an indissociable unity, but of working it ‘retail’, individually; of exercising upon it a subtle coercion, of obtaining holds upon it at the level of the mechanism itself—movements, gestures, attitudes, rapidity: an infinitesimal power over the active body. (p. 136)

  2. 2.

    …there is the modality: it implies an uninterrupted, constant coercion, supervising the processes of the activity rather than its result and it is exercised according to a codification that partitions as closely as possible time, space, movement. (p. 137)

Foucault referred to these methods of “meticulous control of the operations of the body” as “disciplines” and noted their use and legacy in not only penal institutions, but in schools, hospitals, and the military, as well as in non-governmental and commercial organizations such as factories and workshops (p. 137). These techniques of discipline are omnipresent and habitualized in our cultural institutions today, and so may go unnoticed. One can see this scale of individuality and the modality of constant surveillance in times of disease outbreaks and terrorist threat when citizens fully expect the state to seek docility, and therefore the application of ambient sensors, SSD, and other monitoring technologies are intentionally transparent. When this same approach to docility is used to limit economic choices such as determining which health insurance options are presented to a person based on SSD, it is likely to be much less transparent. A challenge with not only wearables, but with much of the monitoring technologies employed today is that citizens often enter into these technological platforms with an expectation of personalized benefit, so rather than docility, it can be “experienced” as empowerment and control. The difference between the two may not be merely perceptual, but again, to the extent that wearables are not currently scrutinized as a political technology of the body, they offer an exceptional opportunity for exploitation by those in positions of power with the privilege of knowledge. For example, McCrea and Farrell (2018) recently proposed a conceptual model for integrating continuous SSD into health risk models in order to “enhance pricing of health or life insurance.” The model uses SSD to generate individualized risk scores. In order for such a model to be relevant in the real-world, insurance companies will need access to user SSD. One avenue to obtain these data would be through offering to pay employers for employee data. To entice employees to provide SSD, employers could offer free tracking devices and encourage participation in wellness programs, informing the employees that the data “belong” to the company. If employers share what is now “their” data with insurance companies, it could then be used to control employee bodies, either overtly or covertly. Overt actions would include something like explicitly limiting insurance options unless the employee demonstrates a change in behavior (such as losing weight, being “less sad,” or avoiding high risk areas of a city). Covert actions might include introducing gamified incentives that encourage the employee to engage in a targeted behavior (such as earning points for reaching a daily activity goal or sending a coupon for a retailer outlet that falls along a safer driving route than the employee traditional drives). With expanding capabilities of wearables self-quantification and Big Data integration, this could afford the private insurance company control over meticulous operations of employee bodies.

Foucault argues that in addition to exerting power over life through subjection and docility, a second form of power began to emerge in the late 17th century through the rapid introduction of interventions and regulations to manage populations through quantification and control of biological processes (such as control of birth rates, life expectancy, living conditions, and level of health) (Foucault and Hurley 1978). Foucault referred to the ability to control these forces as “bio-power;” and argued that as western nation states develop so does the breadth and depth of control over human biological functions and its use as a tool for the maximization of productivity. While discipline is used to direct human behavior at the individual level, “bio-power” is used to direct a population of people (e.g., the residents of a city, state, or nation) to behave in a desired fashion. Wearables present new methods for quantification and control; with biological and behavioral data constantly being collected then transmitted across networks.

The economic success of consumer wearables relies on convincing users that their physiological states should not only be monitored, but also controlled. While it is possible that a user could in some sense be empowered to control bodily processes solely on their own terms using wearables, the current trajectory of design and regulation pose significant threat to that end. Instead, lack of oversight and transparency have greatly increased the risk that these technological systems become not a tool of self-governance, but rather, a tool of social control, where understanding the “self” becomes a process of corporate and state construction rather than personal reflection.

Well-Being Versus Never Being Well Enough

One of the side effects of the FDA guidelines limiting their involvement in assessing “low-risk general wellness” is that this effectively drives consumer wearable producers to problematize wellness to avoid costly and time-consuming FDA scrutiny. As discussed earlier, in order to sell SSD as ends, users must perceive some need to know what these values are. This need can be derived by constructing normative degrees of wellness, or a spectrum of wellness, for individual aspects of the body that users can then ascribe to à la carte. By creating a perception that certain values of wellness are “better” than others, users may be moved to act even if sub-division of the spectrum are not meaningful. Further, this type of subdivision provides a method of ordering bodies or aspects of the body from least to most well, an affordance that could lead to discriminatory or other harmful outcomes for those who fall on the low end of a potentially dubious scale. For example, some estimates of the optimal range of vitamin D in the body are as wide 25–80 ng/mL (Kennel et al. 2010). It is not clearly established whether within the optimal range there are optimal–optimal ranges (i.e. whether 60 ng/mL is better than say 40 ng/mL). A wearable designed to monitor a user’s vitamin D levels could present the SSD within this optimal range using a color scheme placing higher values in green (suggesting a positive result), middling values in orange, and lower values in red (suggesting less positive results). In reality any level in the range would be considered optimal by current medical standards, but the wearable creates a sense of less-than-well for user within the red levels, and might spur them to action to rectify their newly constructed problem; such as laying out in the sun or buying a vitamin supplement sold by the wearable manufacturer. This is not to say a user could not or should not strive to improve wellness, but rather to point out that, without careful ethical considerations, wearables offer a unique and unprecedented form of self-correction toward a norm that may or may not be benevolent, fair, or even benign. As described in the previous section, users may not be allowed access to the information needed to make informed decision regarding these norms, and, as will be discussed in the next section regarding choice and persuasion, they may not even be cognitively equipped to do so.

Problematizing wellness is sometimes referred to as “medicalization” by social sciences to critically describe the process of expansion of medical authority “beyond a legitimate boundary,” (Rose 2007a) “into the domains of everyday existence,” (Metzl and Herzig 2007) and “over our bodies through the reduction of social phenomena to individual biological pathologies” (Fainzang 2013). Medicalization occurs through an exploitation of knowledge and power. The social control afforded by medicalizations “comes from having the authority to define certain behaviors, persons, and things,” (Conrad 1979) and subjection of the body (Foucault 1977). This encompasses the ability to define who can work, who can go to school, who can move around, who can reproduce, who can be held responsible for actions, and who can be considered ‘good.’ One of the earliest and most prominent scholars to formalize the notion of medicalization was Ivan Illich. In his book Medical Nemesis (1976), he attributed the problem of medicalization primarily to physicians. More recent discourse in the field however, argues that private pharmaceutical companies are responsible for the continued expansion of medical authority (Clark 2014; Goldacre 2013; Moynihan and Henry 2006; Poitras and Meredith 2009) through their efforts to culturally manufacture illness for existing drugs. This same mechanism of medicalization is being used to expand the consumer wearable industry by establishing medical narratives for normativity that do not explicitly claim to diagnose illness, but rather seek to quantify wellness medically. The economic gains come not through prescriptions for drugs or medical tests, but through the consumption of lifestyle products aimed to modulate degrees of wellness. For example, the healthy user whose wearable monitors heart rate, skin conductivity, blood volume, etc. may be advised that their readings are in the 20th percentile for their matched peers and that following a particular set of actions (defined by the manufacture) will help elevate them to status quo. This might be an excellent idea; perhaps the wearable will advise the user to think about exercising more and eating healthier. But, the healthy user has given the wearable manufacturer the opportunity to evaluate, categorize, and advise their level of healthiness which, even if it is well and good, may have just became subpar based on standards and knowledge that are not transparent. In this case, medicalization is used to make something real or relevant by identifying it as a biological phenomenon (Clark 2014), but this may or may not benefit the recipient of this newly constructed reality. Medicalizations and the technological solutions they offer are appealing to consumers because they offer simple and objective explanations and responses focused on the individual for complex, subjective phenomenon.

An intersection of medicalization and wearable design that is particularly problematic is within the domain of mental health. While the most highly visible examples of wearables are in the fitness industry, developers are making inroads to mental health and acuity. There are already a number of low-risk, general wellness wearables that claim to monitor and trace biometric data associated with mood. For example, using primarily electroencephalography (EEG) readings the Emotiv Insight headset claims to monitor “6 different cognitive states in real time—Excitement (Arousal), Interest (Valence), Stress (Frustration), Engagement/Boredom, Attention (Focus) and Meditation (Relaxation).” (Emotiv 2019). However, the quantitative boundaries of mental states and mental wellness are far from well-defined (Beaulieu 2002; Ulman et al. 2014; Uttal 2012). There are strong economic interests for pharmaceutical companies to expand the boundaries of mental “illness” through the medicalization of social problems, and this is a critical issue in 21st century bioethics discourse (Poitras and Meredith 2009; Moynihan and Henry 2006; Goldacre 2013) that is beyond the scope of this paper. However, huge economic gains could also exist for “lifestyle” related retailers who are able to use the language of self-quantification to implicitly medicalize and quantify mental “wellness” and, in turn, prescribe treatments that control or exploit users by guiding their purchases and activities, but still are subsumed under the FDA “safety zone” of general wellness.

Not every social scientist believes that medicalization is negative, and some see ubiquitous monitoring of physiological data as a way to democratize healthcare. Hofmann (2016) writes that this type of pervasive medicalization, “making ordinary life experiences subject to medical attention to measuring every aspect of life, and thereby making it subject to ‘experience,’ attention and control … makes persons themselves control their own lives.” However, this is a technocentric view of pervasive ubiquitous self-sensoring data—one that ascribes a value-neutral quality that does not currently exist. It ignores the situated and embedded nature of how understandings of these data are constructed as described in the previous two sections. It assumes a level of individual understanding, critical thinking, and attention on behalf of the user that is not in line with understandings of information processing. Further, as soon as a technological solution intervenes to assist with this processing of raw biosensor data (as wearables would do), the designers of that technology automatically, even if inadvertently, infuse their own cultural, political, and economic interests into that interpretation. Hoffman acknowledges that the epistemic challenge in a new frontier of ubiquitous SSD is in this process of validation and selection of relevant data to make sense of it for a particular user. But at least right now, this is most definitely not in the control of the users. Rather, the definitions and characteristics of the new norms to define general wellness are validated and selected by those who develop wearables behind closed doors both figuratively and literally. Without an understanding of the real impacts these norms might have on consumers it is also difficult to know whether there can ever be an optimal level of wellness. For example, could chronic stress about not being the “well-est” counteract the benefits related to healthy activities the user is trying to engage to get there? In the meantime, nothing is in place to prevent wearable developers from keeping these norms always ever so slightly out of reach, so that there is always a “problem” that needs to be fixed; leaving users in a perpetual state of never well enough.

More Choice Versus Erosion of Choice

Another implied promise of wearables is that they afford more choice about how to manage and control the body. For example, for a person struggling with unexplained fatigue during the day, a wearable sleep monitor affords them a method of quantifying different aspects of sleep, comparing those quantifications against some norm or standard, and then exercising behavioral choices to maintain or change them, ostensibly to control the body’s state of fatigue. This section examines the “persuasive” qualities of wearables that might influence these behavioral choices, and argues current practices of wearable design and implementation may employ these persuasive qualities to limit choice rather than broaden it.

The degree to which SSD could be unduly persuasive can be investigated empirically, as some researchers have begun to do (see Matthews et al. 2016 for a review), but much more work is needed to understand how the manner and style in which SSD are presented to the user can influence persuasion, for example, whether variations in the external representation of SSD (e.g. gamified, medicalized, highly branded, text based, iconographic, and so no) will impact decision making differently. While direct examination of the persuasive impact of SSD is needed, in the meantime, mapping the qualities of SSD (as they are being used today) onto existing research about persuasion and decision making can help with risk assessment.

The study of “message tailoring” offers some interesting insight into what we might expect from SSD in terms of persuasion and coercive behavior change. Tailoring a message means that the content, context, or method of delivery is partially determined by specific information about the individual for whom the message is intended. This is found to be more effective than group-targeted messaging or mass messaging (Hawkins et al. 2008; Noar et al. 2007). While there is no universally accepted theory to explain the persuasiveness of message tailoring, a number of researchers suggest that attitude change through this method may be the results of activating personal relevance, which increases attention and depth of processing (e.g. Hawkins et al. 2008; Ho and Chua 2013; Rimer and Kreuter 2006). Arguably, SSD offer an unprecedented avenue of highly specific, real-time message tailoring data. So, what might this mean in terms of persuasion? The answer is likely determined by whether a user has the ability to understand what the SSD actually mean. When this ability is high, SSD may act as a central cue, eliciting thoughtful and critical thinking about the message (Petty and Cacioppo 1986). However, ability can be hindered when information is complex, missing, obfuscated, or when distractors are present (e.g. Osterhouse and Brock 1970; Eagly 1974; Eagly and Warren 1976). While it is certainly a possibility that consumers could understand SSD in a meaningfully accurate way if it were presented in an objective and transparent manner and with sufficient background information, the current state of wearable design and regulation (as described in the previous sections) means their ability to do so will almost certainly be hampered by algorithmic complexity and secrecy. This changes the role of SSD in decision making.

To illustrate this point consider again the person struggling with daytime fatigue; suppose they purchase a wearable sleep monitor to help quantify their sleep. The persuasive message of this wearable is two-fold: that restful sleep can be measured and that some particular amount of restful sleep is better than other amounts of restful sleep. Say the wearable is similar to those offered on the market today and it tracks things such as nightly heart rate, heart rate variability, pulse volume variations, and arterial blood oxygen saturation, and then provides the wearer with a “measure” of total restful sleep time. As it stands today, it would be difficult for the user to think critically about whether this measure of total restful sleep time and the ideals it is compared to is meaningful with respect to the SSD being collected by the wearable. In part because the science on how these data should be used to inform something called “restful sleep” is unclear, and in part because exactly how (or even whether) these SSD were used to generate this measure are not made transparent to the user. If that’s the case, why would the user even want to be informed of this information, and why be concerned about whether this information would be persuasive at all let alone unduly persuasive? When the highly tailored SSD draws the user’s attention, but it cannot be easily understood, rather than activate critical thinking about the message, it may instead serve as a peripheral cue, activating heuristics or mental shortcuts about the general sophistication or “scienciness” of the message or the messenger. Research suggests that the use of scientific rhetoric in this way can be unduly persuasive.

The language used in marketing materials to describe how wearables collect and produce information, such as “quantify,” “measure,” “track,” and, perhaps most importantly, the word sensor itself, are all meant to imply knowledge of some reality (or “ground truth” if it were). This language allies itself with the language used by many cultures to explain how humans collect “factual” data about the self, others, and the environment; that is, we rely on our own sensors such as the eyes, nose, and ears, to measure and to perceive some truth. While we also rely on context, the environment, and others to interpret the meaning of these sensory observations, so long as we believe our sensors to be in good working order, we perceive their data to be a “reality.” These are also the languages and credibilities of engineering and science. Table 1 shows examples of consumer wearable taglines using this type of rhetoric and connecting it to a persuasive message that such data can be used to control physical or mental well-being. Research suggests that framing SSD to users as a “scientific” measure of an internal state may boost its credibility, whether warranted or not. Dumit (1999) found in a number of court cases laypersons tended to put more faith in information when they believed it was obtained from a “scientific mechanism” than from a more subjective source. Other research has shown that biological explanations of behavior are perceived to be more complex and scientific than psychological explanations (Keil et al. 2010) and, in some circumstances, more persuasive (Baker et al. 2017). This bias in thinking may be because insights derived from mechanisms of hard science are perceived to be “culturally objective” (Dumit 1999). If SSD are afforded this characteristic of objectivity, it raises the question of whether this would bias a user to incorrectly believe that prescriptions derived from SSD (i.e. the persuasive message) are also culturally objective.

Table 1 Examples from wearable websites using language to situate scientific, objective measurement with knowledge and control of the self, well-being, and choice
Full size table

By promoting the languages and credibilities of engineering and science, marketing and design efforts in the field of consumer wearables attempt to construct a perception of a speaker who possesses wisdom, virtue, and goodness. Aristotle argued that an essential factor in any persuasive attempt is the perceived trustworthiness of the speaker (Aristotle and Kennedy 1991, pp. 27–35) and that a speaker perceived to possess all three of these qualities would be “necessarily persuasive” (p. 113). As described earlier, however, there are many actors involved in the design of wearables, with many different social and economic goals motivating their decision, and the actual trustworthiness of the messenger may be dubious. Without a requisite for transparency in wearable design, the complexity of how and by whom a persuasive message is constructed may be impossible for users to know.

The potential mechanisms through which wearables might erode choice described so far have implied that SSD may be unduly persuasive as a peripheral cue, leading users to be persuaded that certain aspects of the body can and ought to be measured, when in reality their measurement may not be meaningful or necessary. Another avenue through which SSD might be unduly persuasive is related to research suggesting individuals have a general preference for reductive explanations (Hopkins et al. 2016; Craver 2007; Garfinkel 1981; Trout 2007) and that they tend to seek mental shortcuts to decision making (Tversky and Kahneman 1974) especially in times of uncertainty or high cognitive load. Wearables are likely to appeal to consumers who seek to reduce the cognitive load related to careful, critical decision making about complex phenomena. In theory, adding SSD as a new factor in decision making about the body should represent a new layer of complexity especially if those data require special skills to understand. However, if SSD is presented to a user as a simplification of a number of other complex decision processes, it represent a short-cut to decision making. For example, returning again to the person struggling with daytime fatigue: to make judgments about the cause of their fatigue and how to control it would require observing and considering numerous factors and interactions such as how their body feels through the day, retrospection assessment of sleep quality and habits, physical activities, diet, stress, potential illness, and so on. This is labor intensive, so the appeal of a wearable offering to do one these jobs, (for example, measure “sleep quality”) may depending on how it impacts this labor. If the wearable monitors five or six more factors and interactions that now need to be considered this is unlikely to appealing (because it would not reduce the cognitive load of the original task). On the other hand, if it monitors these five or six more factors, but then only presents the user with a summary result, such as a simplified “restful sleep” score, an avatar’s facial expression, or a simple thumbs up or down, this is likely to be appealing. The risk of erosion of choice here is if the user believes this reductive summary is more comprehensive when it is in fact less so and then limits their behavioral choices to only those that directly address this incomplete summary. For example, if our tired user incorrectly perceives their sleep monitor to be an objective and comprehensive measure of sleep quality, and the device reports that they are getting “enough” restful sleep, they may stop considering sleep quality as the cause of their fatigue when it actually might be.

The degree to which SSD could be unduly persuasive is unknown, but considering research on message tailoring, scientific rhetoric, the appeal of reductive explanations, and the manner in which SSD is likely to be used in consumer wearables, it seems the potential risk is high. Further, given the amount of resources being invested in developing wearables to influence user behaviors, it is critical to understand the extent to which such a risk exists, and if warranted, to protect consumers from related harms. Without such knowledge and protections users may provide highly detailed, intimate knowledge of themselves in hopes of broadening their choices about how to maximize certain wellness goals, without realizing they have been unduly influenced to adopt a limited or misinformed choice set that could be potentially harmful to those same wellness goals.

Conclusion

Wearables represent a new opportunity for governing individual behavior. Whether this will manifest primarily as self-governance as some have argued (Rose 2007a, b; Topol 2011) or merely as a transfer of individual sovereignty to industry and/or the state, will rest on how these technologies are designed and regulated, and how persuasive they are. The discourse surrounding standards for avoiding risk through design and regulation has so far focused on data security, privacy, accuracy, and to some degree, physical harm; while standards for avoiding risk related to construction of knowledge, social control, medicalization of wellness, and erosion of choice, have not been as widely considered. This may be in part because it is unclear how responsibility for addressing these social implications should be assigned, whose moral authority can be trusted to do it, and what values should play a role. Although engineers are centrally located in reflecting on and responding to ethical implications of wearable design and deployment, it is also critical that government regulatory bodies such as the FDA and the FTC place a higher emphasis on these types of risks, especially for consumer wearables.

As a final note, it will be tiresome to some and critical to others to point out that the research and arguments presented here, though generally critical, do not, in fact, constitute a luddite call to ban wearables, just as Bainbridge’s “Ironies in Automation” (1983) did not call for a ban on automation. Rather, it seeks to lay out some key areas of risk that ought to be addressed and does so by unifying numerous criticism and concerns from multiple scholarly domains that have individually raised concerns about wearable use and implementation, both inside and outside of the healthcare industry. Further, it draws attention to social implications and highlights deficiencies in current ethical and regulatory practices that ought to be addressed in order to avoid unwanted or unintended consequences; before, or at least at the same time as, a future of ubiquitous surveillance of bodies and actions is being feverishly constructed. Finally, it is meant to underscore that materialization of the promises of user empowerment and personalized wellness through self-quantification does not rest merely in expansion of engineering capabilities toward variation and volume of SSD data made available to users; but rather in the nature of the design and deployment of the wearables that use these data to define the self and influence behavior.