Introduction

Depressive episodes typically begin with the experience of distress or dysphoria and the way young people experience and manage these symptoms of dysphoria is likely to have a major influence on either healthy adjustment or the development of a fully fledged depressive episode [8]. Cole et al. [2] suggest that internalising disorders such as depression are likely to result from an inability to down-regulate negative emotions and/or up-regulate positive emotions, yet capturing and demonstrating this phenomenon has proven difficult.

Although the lives and experiences of adolescents occur on a moment to moment basis, research seeking to understand adolescent mood and coping has predominantly used retrospective questionnaires assessing global concepts such as mood, stress, and coping in the previous 2 weeks to 1 month [9, 27]. Evidence suggests that recall of past emotions, experiences, and attributes are often biased because experiences are not simply encoded in memory and later retrieved [24]. For example, retrospective measures of coping (a) tend to reflect personal conceptual models of coping, (b) are affected by current mood, and (c) have poor concordance with actual behaviours and thoughts that occur in the situation [19, 24]. Thus, retrospective questionnaires fail to reflect this shifting pattern of time-varying coping responses [19]. Methods of data collection that can omit retrospective recall error and capture the experience of mood, stressful events, and coping responses as they occur are imperative to progress our understanding of the factors associated with the development of depression in young people.

Methods and new technologies for measuring coping and depression

Momentary sampling (MS) is a research method that assesses participants in their natural environment a number of times a day, by using either paper-and-pencil diaries or hand-held computers to capture the information. Assessment of stressful events, mood, and coping strategies (both behavioural and cognitive) is possible with MS, and has been shown to reduce the systematic recall bias of questionnaires [19, 28]. In addition, research has shown that, when compared to MS, retrospective recall leads to under-reporting of (a) fluctuations in mood, (b) factors surrounding stressful events [16], and (c) cognitive coping strategies [17], and to over-reporting of behavioural strategies [24]. Importantly, sampling in real-time allows researchers to prospectively evaluate the success of the actual behaviours and thoughts that young people have in response to distress. Subsequently, research has begun to investigate the daily mood-related experiences of young people [16, 21, 22]. To date, however, the widespread use of MS in understanding time-varying constructs such as mood, coping, and stress has been hampered by the limitations of the tools used to collect such data.

The first MS studies provided participants with paper-and-pencil diaries designed to record multiple closed-ended questionnaire-like entries, and participants were cued to complete an entry by a beeping wrist-watch or pager [9]. In MS research, diary entry times tend to be randomly allocated throughout the day, and the frequency of diary entries ranges from once per day [26] to ten times per day [6]. A major limitation of using paper-and-pencil diaries for MS is the absence of means to confirm the entries were made at the scheduled or prompted time. Using an unobtrusive photosensor in MS paper-and-pencil diaries to assess when they were being completed, Stone et al. [25] demonstrated that on many occasions participants back-filled and forward-filled the diary.

Capitalising on improvements in technology, MS methods have been developed on personal digital assistants (PDAs) or “PalmPilots”. Audible “beeps” emitted from the PDA prompt participants to complete on-the-spot assessments that are timed, coded, and saved for later download as electronic data. The limitations of PDAs are that they are (a) expensive to purchase, (b) often time consuming to program and can require specialist training or support, and (c) foreign or novel to many participants, especially young people [30], and (d) in many research studies other functions of the PDA are disabled [21]. The novelty and lack of usability may lead to a failure to carry the PDA at all times or a lack of engagement with the tool.

Momentary sampling on mobile phones

With more recent developments in technology, interactive MS monitoring programs can now also be run on java-enabled MIDP 2.0 mobile phones. This is an exciting development as across most developed countries, adolescent mobile phone ownership has increased exponentially in the last 10 years. Ownership rates have been estimated at 83% of Australian teenagers [13], more than 85% of Finnish teenagers [12], 73% of Hungarian teenagers [14], and over 90% of British teenagers [5]. Further, research suggests that mobile phones are an integral aspect of young people’s social networks [13]. Whilst some research has captured MS data by calling participants on their mobile phones and used either automated interactive voice response systems to conduct interviews [3] or researcher-lead interviews over the phone [1], to date mobile phones have not been used in any study as an electronic means of gathering MS data [16].

The aim of this study was to develop, pilot, and evaluate a youth-friendly mobile phone program to monitor, in real-time, young people’s everyday experiences of mood, stress, and coping behaviours. More specifically, the study aimed to design and evaluate a mobile phone program that would

  1. 1.

    be completed at four randomly chosen times a day and elicit responses from young people,

  2. 2.

    be easy to use and limited in its invasiveness into a young person’s day-to-day activities,

  3. 3.

    capture data in a systematic manner that gave meaningful information about

    1. (a)

      locations, activities, and companions,

    2. (b)

      mood,

    3. (c)

      stress,

    4. (d)

      responses to distress/coping,

    5. (e)

      alcohol and cannabis use.

Method

Participants

Participants were recruited from two independent secondary schools in Melbourne, Australia. One class of Year 9 students from the first school and one class of Year 11 students from the second school were invited to participate in this study. These year levels were chosen to provide a sample from mid to late adolescence. A total of 40 students were invited to participate, and 29 returned parent consent forms allowing them to participate (20 females, 9 males). Eleven students (7 females, 4 males) were randomly allocated to the focus group. A total of 13 female students aged 14–17 years (M = 15.9, SD = 1.10) and 5 male students aged 14–17 years (M = 15.8, SD = 0.84) were randomly allocated to participate in the pilot study. All participants spoke English as their primary language at home. One participant reported having Greek heritage (3%), two (7%) reported Dutch heritage and the remaining sample indicated their ethnic background as Australian. All participants had parental consent to take part in the study and the study was approved by the Royal Children’s Hospital Human Research Ethics Committee, study number #24116A.

Development of the mobile tracking young people’s experiences program (mobiletype)

For the mobiletype program to successfully capture data about young people’s experiences, the following rule was mandatory and adhered to in the development:

Participation in the study and use of the mobile phone program must not cost the young person: That is, data collection must not rely on the young person having credit in their mobile phone account.

The mobiletype program was designed using open source software (J2ME). The mobiletype program was transferred to the phone via Bluetooth, data captured by the mobiletype program was stored on the phone, and all data uploaded at the end of the monitoring period via Bluetooth. The program was designed to be completed four times per day at random times between 8 and 11 am, 11 and 3 pm, 3 and 8 pm, 8 and 10 pm. When initiated, the mobiletype program ran through a list of closed-ended and open-ended questions assessing the following:

Current activity Four closed-ended questions were adapted from [10] assessed current activity, location, companionship, and enjoyment of current activity.

Current mood Current mood was assessed using a measure developed by [28] in which participants indicated the extent to which they felt angry, sad, tired, stressed, anxious, happy, and alert, together with their self-perceived level of well-being, on a 6-point Likert scale in which six adjectives indicating an increasing degree of the relevant mood were displayed on the phone screen rather than numbers (with the six adjectives then being assigned rating of 0–5). A negative mood score was constructed by summing responses to the angry, sad, tired, stressed, and anxious questions, and the possible scores therefore ranged from 0 to 25 with a higher number indicating more negative mood. An algorithm was constructed in the program, such that should the young person score 3 or above on any one of the individual anxiety, stress, sadness or tiredness questions, they were then asked the open-ended question “Looks like you’re feeling pretty sad/anxious/tired, what do you think you might do now?”

Stress In each electronic diary, the participants were asked about their experiences of stress via branched questions:

Step 1:

“Has something stressful happened since the last electronic diary” with yes/no response options. If no was their response, no further stress questions were asked.

Step 2:

If they responded yes to Step 1, then “How stressful was it?” was asked and response options were: Not at all, kind of bad, pretty bad, or very bad.

Step 3:

If the response was pretty bad or very bad, then the following two open-ended questions were asked: “What happened?”, and “Why do you think it happened?”, to which participants replied using the written text function on the phones to write their answer. The closed-ended question “how much do you feel you can control it?” was subsequently asked, with the response options being not at all, kind of, somewhat, and definitely, followed by the open-ended question: “What, if anything, did you do about it?” which was again answered using the written text functional on the phones.

Alcohol and cannabis use The previous evening’s alcohol and cannabis use was captured in the morning diary. A series of branched closed-ended questions assessed what type of alcohol was consumed (i.e., beer, wine, spirits, wine cooler or alcoholic cider/lemonade), the container the alcohol was served in (i.e., 30 ml shot, 285 ml “middy” of beer) and number of containers, main companions in drinking, and the main reason for drinking. The questions assessed alcohol and cannabis use were adapted from Shrier et al. [21] to the Australian context.

Procedure

Focus groups

Three focus groups were conducted: one with three Year 9 students, and two with four Year 11 students. Each student was given a study mobile phone with the mobiletype program on it and asked to complete the diary in the focus group. The researcher asked targeted questions aimed at: the open-ended questions in the electronic diary (i.e., What would you type in response to this question? Would recording your answer verbally be easier or harder than texting?), the response lists (i.e., Is there anything that you would regularly answer that is not on the list?), the amount of information captured (i.e., Is there anything important about your daily routine that is not being captured?), and their general likes and dislikes about the diary. The information from these focus groups assisted the fine-tuning of the wording and response options in the mobiletype program.

Study trial

Participants met the researchers at school during class at a pre-arranged time to complete baseline assessment. At this time, participants were lent a Nokia 6630 mobile phone for the duration of the study. Participants were given the choice of using their own SIM card in the study mobile phone (and receiving a $25 gift voucher) or using one of the study’s $25 pre-paid SIM cards. Use of the mobile phone and the mobiletype program was explained in detail and participants completed a practice diary entry. A study manual that described the research procedure, offered trouble shooting tips and included contact information for the study coordinator, was provided to all participants. Participants were called at a pre-arranged time on the first day of assessment to discuss the study procedure and have any queries answered.

Participants then completed the mobiletype program for 1 week. Four SMS messages were sent to the participants’ mobile phones per day at random times between 8 and 11 am, 11 and 3 pm, 3 and 8 pm, 8 and 10 pm requesting that they complete the program. There were four versions of the diaries referred to as the morning, noon, afternoon, and evening diaries, which were completed in the above listed corresponding time periods. Participants were provided with the researchers’ phone numbers and advised to call if they had any questions or problems with the mobile phone or the mobiletype program. At the end of the 1-week period, participants met with the researcher to complete a follow-up feedback assessment, and return the mobile phone.

The feedback questionnaire asked young people to indicate what time of day, and day of the week, was the easiest and most difficult time to complete a diary. Young people were asked if there were times when they did not answer truthfully, what they like most and least about the diaries, and to what extent did they feel that the diaries captured their day-to-day situations, thoughts, and feelings.

Analyses

This study conducted a formative evaluation [20] in two stages, using (a) a focus group, and (b) a week-long prospective-design MS trial. The evaluation was conducted by assessing qualitative feedback from participants of their experience using the mobile phone program in stages (a) and (b), and by collecting quantitative and qualitative data from stage (b) about the manner of responses to the program. To examine the type and quality of the data captured by the program we use the term “meaningful” rather than valid as this study is not designed as a validation study. To investigate the structure of variance in negative mood both within a participant, and between participants, a three-level variance components model was fitted to the variance in mood scores. Particular care was taken around maintaining confidentiality of records given the sensitivity of the data. Only de-identified records were used in data analysis.

Results

As this was a formative evaluation of the feasibility of using mobile phones to obtain MS data, the results of the study are reported directly in relation to each of the functional aims.

  1. 1.

    Were the mobile phone diaries completed at four randomly chosen times a day and did this completion elicit responses from participants?

Collapsing across participants, 382 (76%) of a potential total of 504 diaries were completed during the week (there was a maximum of 28 diary entries per participant). On inspection of the rate of responding, 72% of participants completed 80% or more of the diary entries. Females completed 78% of the possible diaries, whilst males completed 71% of the possible diaries. Response rates to particular times of the day were, 74% of morning diaries, 75% of noon diaries, 77% of afternoon diaries, and 78% of evening diaries, suggesting that response rates to the program varied little across the different time points during the day. There did appear to be some respondent fatigue in responding for 7 days, with lower response rates by day 7 (67%) compared with day 1 (91%). Participants started the study on different days of the week. On inspection of the response rates for days of the week, there appeared to be no discernable trend in responding based upon day of the week, suggesting that the program response rates were not likely to be associated with day of the week.

  1. 2.

    Were the mobile phone diaries easy to use and limited in the invasiveness into a young person’s day-to-day activities?

In anonymous questionnaire-based feedback received after completing the diary for 1 week, 83% of participants reported that the electronic diary was good-to-excellent in its ease of use. Seventeen out of 18 participants reported that the electronic diary adequately captured the types of situations young people were in, the types of thoughts they had, and the moods they were experiencing.

In terms of invasiveness in a young person’s day-to-day activities, 33% of participants reported that morning was the most difficult time to complete the diaries, 50% reported the middle of the day, and 17% reported the night-time was the most difficult time. It is interesting to note that these results are not reflected in the actual response rates to the diaries which were even distributed across times. Two-thirds (67%) reported that it was more difficult to complete the diary on weekends than weekdays although again this was not reflected in the actual response rates. Students reported that it was difficult to complete the diaries during exams (22%), sport (17%), and when they were at work (22%). One-third of the sample reported that they occasionally answered untruthfully to avoid the branching of the questions (stressful occurrences, alcohol and cannabis use), which results in increased number of questions answered.

In response to the open-ended question “what did you like about the diaries?” the primary theme in responses was about understanding oneself, e.g., “the fact that I had to understand what I felt everyday,” “the way they made me more aware of my feelings and how I could address my problems,” and “a better way of doing a survey.” Other comments were that the phones and diaries were easy to use, and that the language was relevant to young people.

In response to the open-ended question “what did you not like about the diaries?”, only three participants reported that they felt the questions were repetitive across diary entries, and two participants reported that they found it somewhat inconvenient.

  1. 3.

    Did the mobile diaries capture data in a non-random manner that was meaningful about:

    1. (a)

      Locations, activities and companions?

The two most common locations in which young people reported completing the diaries were at home (41% of occasions) and at school (32%). Other locations were in a public place (6%), in a car (5%), or at a friend’s house (4%). The majority of activities participants were doing when diary entries were recorded were schoolwork (28%) or “hanging out” (14%). Overall, the majority of activities (85%) were rated as either neutral or enjoyable. Few activities were rated as “a bit of a pain” or “hating it”. The activities that were rated as negative were schoolwork (24%), working (35%), thinking (31%), and travelling (22%).

  1. (b)

    Negative mood?

Average negative mood for each participant was calculated by summing participants’ responses across all time points (Table 1).

Table 1 Negative mood (mean, SD, and range) as captured by the electronic diary
Full size table

There was considerable variation in average negative mood amongst the participants, and the standard deviations for some participants indicated that their negative mood fluctuated quite considerably. To investigate the structure of this variance, a three-level variance components model (diary entry number nested within day nested within individual) was fitted by restricted maximum likelihood estimation using Stata 9.2’s [23] xtmixed procedure to examine the amount of total variation in negative mood scores that could be attributed to (a) within-individual variation, (b) variation between days within individuals, and (c) between-individual variation. The variance estimates obtained at the three levels were 11.65, 2.51, and 4.62, respectively. Therefore, 27.5% of the total variance (i.e., 4.62/16.78) in negative mood could be attributed to individual differences between adolescents, and 15.0% of the variance (i.e., 2.51/16.78) could be attributed to differences in participants’ day-to-day fluctuations of negative mood. This means that 42.5% of total variance [i.e., (4.62 + 2.51)/16.78] in negative mood was found to be occurring both between individuals and between days within individuals. Therefore, the mobiletype program meaningfully captured systematic variability in young people’s moods over the 1-week trial, both between individuals and within individuals across their day-to-day experiences.

  1. (c)

    Stress?

Out of a possible 382 valid responses to the mobiletype program over the week, there were 64 reports (17%) of stressful occurrences being experienced since the previous diary entry. Forty-five (70%) of these reports were rated by the participants as “not too bad” or “kind of bad.” Nineteen stressful occurrences were rated as “pretty bad” or “very bad.” Participants’ open-ended responses about “pretty bad” and “very bad” experiences reflected themes of exams, conflict with parents, friends and boyfriends/girlfriends; only 2 out of 19 responses were incomprehensible. Determining cognitive style or responses to stress was not possible from the responses, as participants' responses to “why do you think it happened” tended to be repetitive of the “what happened” responses. Participants’ perceived control over the 19 “pretty bad/very bad” stressful experiences, varied with 7 occurrences of “not at all,” 3 of “kind of,” 4 of “mostly,” and 5 of “definitely.”

  1. (d)

    Responses to distress/coping?

Out of a possible 384 responses, the question “looks like you’re feeling sad/anxious/tired what do you think you might do now?” was asked 201 times (53%). There were 180 comprehensible responses to this questions, with 21 (11.6%) of the 180 indicating that the young person did not feel anxious/stressed or that there was nothing wrong. The responses were coded by two researchers into the following categories, with high inter-rater agreement (kappa = 0.94).

Upon analysis of the data in Table 2, it is apparent that the question of “looks like you’re feeling sad/anxious/tired what do you think you might do now” captured active or behaviourally based responses to feeling distressed, with little data being provided on the cognitive strategies they may have used to cope with the way they were feeling at that moment. There were only two reports from the same participant indicating some cognitive content.

Table 2 Coded responses to current elevated distress levels by frequency
Full size table
  1. (e)

    Alcohol and cannabis use?

There were 16 MS reports of alcohol use from 12 of the participants. Two reports were invalid and omitted from these analyses. The reported quantities consumed ranged from 1 standard drink to 12 standard drinks. The young people predominantly drank with their boyfriend or girlfriend (7 instances), or with their family/parents (5 instances). They drank alone on only two occasions. The general reasons for drinking were: to get drunk (3), to relax (4), to feel good (2), to distract themselves (1), and “other” (4). Only two participants reported using cannabis.

Discussion

The aim of this study was to develop and pilot a novel, youth-friendly mobile phone program to monitor young people’s everyday experiences of mood, stress, and coping behaviours as these occur, and to evaluate the feasibility of using this approach for MS data collection. Overall, the mobiletype program met the aims of functionality, it was youth-friendly, non-invasive, acceptable, and the qualitative reports of young people’s experiences using the program supported this assessment. Further, the mobiletype program captured meaningful data about young people’s moods, the stresses they experienced, their responses both to these stressful experiences and to feeling distressed, and their alcohol and cannabis use.

Meaningfulness of data captured

To examine the type and quality of the data captured by the program, we use the term, “meaningful” rather than “valid.” This study did not intend to, nor had the statistical power or design, to examine the validity of the data. In terms of the meaningfulness, usefulness, and appropriateness of the mood data captured by the program, the multi-level modelling indicated that the mood data reflected meaningful variability both between and within individuals, and that as in other studies [6, 21, 28], this data may readily be analysed in a useful manner in larger more representative samples.

The mobiletype program captured interesting and unexpected data on the things young people do when they are feeling distressed, in particular that “nothing” and “eat” were the two most common responses. These are not responses found in other MS studies which have used closed-ended and often coping-type response options [4, 22, 24, 27] and this difference may be attributable to the use of an open-ended question in the current study. Closed-ended, theoretically-driven coping scales may encourage respondents to think about or conceptually reframe their behaviours into the underlying coping framework of the scale and elicit more desirable or apparently appropriate responses than open-ended questions. It was interesting to note that the open-ended question predominantly elicited behavioural reports from young people, such as eat, study and sleep, with only two reports of cognitive responses. This is likely to be a factor of asking young people “what do you think you will do now” rather than an indication that young people are not cognitively responding to their current distress. Similarly, the stress-related open-ended questions targeting cognitions and attributions about stressful occurrences failed to capture such data, and the responses instead tended to be general descriptions of the stress. Nevertheless, this was the first MS study in which young people were alerted to immediate elevated levels of distress and asked what they would do next. Further modifications to the program that specifically target young people’s thoughts about feeling distressed, and their hot cognitions in response to stressful situations, are needed to capture the breadth of both behavioural and cognitive responses to distress and stressors.

Logistics of capturing daily data using a mobile phone

The response rates to the mobiletype program were similar to Henker et al. [6] and Whalen et al. [28] and better than some studies using PDAs with adolescents [21]. The variability of individual differences in response rates, i.e., some individuals completing very few responses, and some completing all responses was also similar to previous research with adolescents [6, 21]. Interestingly, our study is one of the few MS studies that did not entice responding via remuneration, with some other studies offering up to $100 for more than 80% completion [15, 28]. It is likely that using an accessory that is an integral part of young people’s social networks and day-to-day activities led to increased response rates. Whilst the mobiletype program appeared to sample regularly and without bias from a young person’s day-to-day experiences, there was evidence of reduced responding by day 7, suggesting similar respondent fatigue to that found in previous research [9, 11] and, more commonly, not reported in other research [6, 22, 28, 29].

Limitations

As the aim of this study was to develop and pilot a novel youth-friendly mobile phone program to monitor young people’s everyday experiences, the sample selection was non-representative and small. The participation rate in the study was 72.5% suggesting that using a mobile phone program to track mood, stress, and coping appeals to many young people, but not all. Both factors limit the generalisability of the findings from this study, but generalisability was not a primary research aim of such a formative evaluation. Further, the small sample size limited our ability to examine the validity and reliability of the items of the mobiletype program.

It is important to acknowledge that self-monitoring, as is required in MS, can sometimes lead to changes in the behaviours being monitored and increased awareness of moods and behaviours. This is referred to as “reactivity” and has been shown to range from none to a small amount in the case of reporting on alcohol consumption [7]. Some of the young people in our study reported that monitoring their moods, stressors, and coping responses increased their awareness of their problems and of their problem solving strategies in a positive manner. Interestingly, harnessing the “reactivity” produced by MS research may potentially have clinical utility, especially if researchers are able to provide individual feedback to patients and to health professionals and doctors. The utility and application of electronic MS to the clinical setting has recently received attention [16]. Subsequent developments of the mobiletype program have lead to the development of a clinical prototype and current trial of it as a clinical tool, for use in primary care settings in which young people and their doctors are provided with individualised feedback from the mobiletype program [18].

In summary, the present study sought to develop and pilot an innovative youth-friendly mobile phone program to monitor young people’s everyday experiences of mood, stress, and coping behaviours as they occur and evaluate the feasibility of using this approach to MS data collection. Results from the implementation of the mobiletype program and analysis of the data captured suggest that the mobiletype program was acceptable to young people, and captured meaningful, reflective, and possibly valid data on young people’s day-to-day experiences.