1 Introduction

In 2013, researchers at Hadley Centre for Adoption and Foster Care Studies (University of Bristol) were asked to identify local authorities in England who were providing good experiences for children in care (known as looked after children) and to identify and promote the practices and processes that made that possible. These apparently simple research questions could not be answered. Although each local authority collects data on broad outcomes such as children’s educational qualifications and teenage pregnancies there are no national measures of the well-being of looked after children. Indeed, the National Audit Office (2014) noted that although £2.5 billion was spent during 2012–13 on the care of children in foster and residential homes, there are no indicators that measure the efficacy of the care system. So began an action research project that set out to identify what children in care thought was important to their well-being and to devise a survey that children would find meaningful and be willing to complete. The project is on-going and funded by the Hadley Trust with researchers from the Hadley Centre, School for Policy Studies, University of Bristol working in partnership with Coram Voice a children’s rights charity. A young people’s project advisory group developed and contributed to materials used in the research. The work was situated within a children’s rights based perspective that acknowledges children as social agents with an active and critical role in their own well-being (O’Neill and Zinga 2008).

We began the project by conceptualising care experiences as a journey and considering transitions and movement in care. However, we quickly realised that by emphasising stability, which is just one aspect of care, many other important elements were omitted. Therefore, we have used a framework of well-being to guide the research, accepting that it is a multifaceted concept with different domains identified and emphasised (e.g. Ryff 1989; Seligman 2011). Concepts of resilience and prosocial behaviour are also closely related. In the study and in this article we use the term well-being to mean how children feel (e.g. happiness, life satisfaction, life has meaning) and how they are functioning and flourishing (e.g. relationships, self–efficacy, life getting better). The concept of flourishing is particularly apt for children in the care system. There has been five phases in the development of the well-being survey: literature reviews, expert group roundtable, focus groups with young people, survey development and piloting.

2 Looked after Children in England

In England at 31st March 2015, there were 69,540 children and young people in the care of the state. However, during the previous 12 months a further 31,100 children had ceased being looked after. Most children (approx. 75 %) were looked after because of parental abuse or neglect and lived with foster carers in family based care (Department for Education 2015). A strong evidence base (e.g. Jones et al. 2011; Jaffee and Christian 2014) shows that the impact of child maltreatment can be long lasting. Consequences include mental and/or physical disabilities resulting from the initial injuries; psychological problems related to experiencing trauma such as post-traumatic stress disorder, hyperarousal, anxiety, depression; cognitive problems such as lack of curiosity and short attention span; and later health problems such as the development of diabetes and heart conditions. The difficulties are evident in school with 68 % of looked after children having a special educational need in comparison with 19 % of the school aged population (Department for Education 2014). Low attainment often leads to unemployment and poor adult outcomes with an estimated quarter of homeless people sleeping on the street having come from a care background (Reeve and Batty 2011). Overwhelming stress and complex trauma interferes with normal development. Importantly, resilience research (e.g. Ungar 2013) suggests that children who have been subjected to traumatic experiences are less able to use their own personal resources (e.g. self-esteem, health, optimism) to maintain well-being and rely much more on external factors to foster positive development. Therefore, the quality of substitute care the child receives makes a large contribution to a child’s well-being (Sroufe et al. 2005). The Association of Directors of Children’s Services (ADCS 2012), recognising the impact of early trauma, set out the purpose of the care system as being: to protect children from further harm, address a child’s need for good parenting, improve their outcomes and enable them to recover from traumatic experiences.

To date measurement of the care system has not focused on the ADCS aims but on process issues such as timeliness of decision-making, movement within the care system and the numbers of children entering and leaving the care system (Axford and Little 2006). There is some information on broad outcomes (e.g. educational results, proportions in employment or education after leaving care) but re-abuse rates, quality of substitute parenting and children’s well-being are not collected. The data are primarily there to compare local authority performance against national standards and for central government to assess the impact of policy changes. Comparisons on education performance are made against the general child population and not against similarly disadvantaged groups nor does analyses consider improvements after entering care.

Despite the large sums of money spent on the care system and the importance of knowing whether what is spent makes a difference, the only measure that that is used to examine individual children’s well-being is the Strengths and Difficulties Questionnaire (SDQ). The SDQ (Goodman 1997) is used throughout the world as a brief behavioural screening questionnaire but it was never intended to be a measure of children’s well-being (www.sdqinfo.com). In England, every child aged between 5 and 16 years old and who has been looked after for at least 12 months, is required to have a SDQ completed by their carer each year. The scores are returned to the Department for Education to provide a national picture of the emotional and behavioural difficulties of looked after children. Results show that about 60 % of looked after children have scores that suggest they have mental health difficulties in comparison with 10 % of children in the general population (Ford et al. 2007). However, the information is rarely used at the individual level to improve a child’s mental health or to plan the demand for mental health services within a local authority area (Axford et al. 2013).

Local authorities run ‘children in care councils’ with the intention of listening to children’s opinions on the services they receive. However, these small groups of young people usually have little or no impact on local authority decision-making (Wright et al. 2006). The Children’s Commissioner for England also has a role in promoting and protecting all children’s rights. From 2014, the Children’s Commissioner took over responsibility for an annual survey of children in care and care leavers (Children’s Commissioner 2015). Survey questions were developed by a looked after young people’s expert group. The survey aimed to ask about children’s feelings at the time they came into care, their experiences in care and their feelings about the future. However, it is a very long survey and over the years has had a poor response rate; typically of less than 4 %. About 2,000 young people respond each year with most responses coming from young people aged between 15 and 24 years old. In 2015, children and young people reported that: being listened to, having good and supportive relationships with caring adults, making progress, having contact with birth family, being ordinary, not being moved around and feeling a sense of belonging in their placements all contributed to their well-being. Tools to enable professionals to communicate with looked after children have also been developed such as Talking Mats http://www.talkingmats.com/ or computer assisted programmes e.g. www.inmyshoes.org.uk or materials to help talk about feelings e.g. http://www.incentiveplus.co.uk/. Some individual local authorities have also devised their own surveys for their looked after children. These are not publicly available but those we have seen have been poorly devised, very long and with potentially upsetting questions that are conceptually confused.

None of these surveys appears to have taken account of the modern psychological study of well-being (e.g. Seligman 2011) or the extensive international research and development that has taken place to produce measures of child well-being (e.g. Ben-Arieh et al. 2001; Gabhainn and Sixsmith 2005; Cheevers and O’Connell 2013). There is increasing interest throughout the world in how to measure the subjective well-being of children (e.g. Ben-Arieh et al. 2014). That interest can be seen in the successful Children’s World survey where to date over 90,000 children from 21 countries have completed well-being surveys (http://www.isciweb.org/).

2.1 National Measures of Child Well-Being

The Children’s Society and the University of York’s annual surveys and reports on the well-being of the nation’s children are one of the largest of their kind in the world (Children’s Society 2014). Their survey developed with children has a measure of overall well-being and ten domains: family, friends, health, appearance, time use, the future, home, money and possessions, school and amount of choice (Rees et al. 2010). The centrality of interpersonal relationships with family and friends, as well as the value of ‘activities’ and ‘things to do’ are a recurrent theme across research on young people and well-being (Fattore et al. 2009; Fernandes et al. 2012). The work (Rees et al. 2010) that underpinned the development of the surveys has strongly influenced the Office of National Statistics (ONS) in their National Measuring Well-Being programme (http://www.ons.gov.uk/ons/guide-method/user-guidance/well-being/index.html). The ONS programme has developed seven domains for children and young people: personal well-being, relationships, health, what we do, where we live, personal finance, education and skills. Apart from finance, all the ONS domains have subjective measures and contain a provisional set of 24 measures for children under 15 years old and 28 measures for the age group 16–24 years old (ONS 2014).

The ONS (Newton et al. 2011) reported that in comparison with adults, children and young people more frequently mentioned the importance of accessing technology such as the Internet and thought of pets as part of the family. There were also age differences. Young children placed more emphasis on the importance of family and parents (including the need for physical contact), pets, toys and celebrating festivals (e.g. Christmas) than did teenagers. Adolescents were more likely to emphasise the importance of appearance to their well-being - including having the right clothes and shoes, and wearing make-up. Gender differences remain largely unexplored in England, although there is some evidence that girls give greater priority to friendships (Children’s Rights Director 2010). It is interesting to note that the domains selected by ONS, whilst of importance to children, are measured in ways that address government’s priorities – in other words, this is far from a “user-centred” design process. It is unclear whether children and young people contributed to the wording of questions. For example, a health question only measures obesity and does not ask about pre-occupation with being slim or body image.

3 Looked after children’s Well-Being

UNICEF (2009) has developed a set of 15 indicators for children in formal care (institutional or foster) for use throughout the world. The 15 core indicators are divided into 12 objective indicators (such as the number of children entering care, ratio of children in residential and foster care, number of child deaths) and 3 indicators that show a) the existence of policies and a framework for dealing with children’s complaints, b) registration and regulation of providers and c) a legal and policy framework for children in formal care. However, there are no subjective measures of looked after children’s well-being.

In many countries there are well-being frameworks that are intended to hold local government accountable for children in their care and are completed by adults. For example in the US, the child well-being framework has four domains of well-being: 1) cognitive functioning, 2) physical health and development, 3) behavioural/emotional functioning and 4) social functioning (http://www.acf.hhs.gov/sites/default/files/cb/im1204.pdf and www.chapinhall.org/feb-18-forum-video.). In Scotland, child well-being domains are 1) safety, 2) healthy, 3) achieving, 4) nurtured, 5) active, 6) respected, 7) responsible and 8) feeling included (http://www.gov.scot/Topics/People/Young-People/gettingitright/wellbeing). However, looked after children and young people’s own perspective and subjective well-being are rarely considered.

Research in other countries has begun to examine the subjective well-being of children in care: for example Spain (Llosada-Gistau et al. 2015), Australia (http://www.australianchildwellbeing.com.au) and New Zealand (Fulcher and Garfat 2013). However, the care system in other countries is so different that comparisons with England are difficult. In England, the majority (76 %) of looked after children are in foster care: residential care is rarely used (9 %). Many other countries make much more use of residential care and care by relatives (e.g. in Spain 42 % of looked after children are in residential care and 41 % are cared for by relatives). In England, most foster carers are not relatives but are professionals who have been assessed and trained for a paid foster carer role.

A systematic review of research in the UK reporting looked after children’s views (Dickson et al. 2009) grouped children’s response into nine domains of outcomes that mattered to them: love, a sense of belonging, being supported, having someone to talk to, contact with birth parents, stigma and prejudice, education, relationships with professionals, preparation and support for leaving care. Some of the domains identified by Dickson and colleagues (2009) fit well with those identified for children in the UK general population but others do not. For example, frequent changes in social workers and placements are key concerns for looked after children that leave children feeling even more vulnerable (Dex and Hollingworth 2012). It is important to emphasise that the literature reviewed was not focused specifically on well-being but on other aspects of the care experience and was mainly concerned with outcomes. ‘Outcomes’ are often used in social care to encompass well-being but the term ‘outcomes’ also implies a process of change and an emphasis on results. Outcome stars are used by many social care agencies (http://www.outcomesstar.org.uk/childrens-star/) and are mainly completed by professionals.

Holder and colleagues (2011) conducted one of the few studies that asked looked after children and young people in England their views but on a predefined set of well-being domains. Based on their review of the literature they developed nine possible domains: physical care; feeling safe and secure; school support; help and encouragement; communicating (including getting the information you need); feeling understood and being involved; recreation and leisure; relationships with family; relationships with friends. The study was a small involving 12 children aged 13 years and above, whose views on the domains were heard in a focus group. Holder and colleagues (2011) reported that children had difficulty with some of the adult terminology. For example, physical needs were associated in young people’s minds only with bodily health and the researchers concluded that more research was needed to examine age differences, clarify the wording of questions and undertake testing of psychometric properties. Some ‘extra’ domains (e.g. looking good) were identified in the focus group but the researchers stated that they were unsuitable for taking forward in the measures because they were not directly linked to Children’s Services. That assumption is debateable, as ‘looking good’ involves having the right sort of clothes and trainers that would be provided by Children’s Services. It is noticeable that there is a tendency to omit from indicators, items that children think are important but ones that do not easily fit into the adult defined domains. The lack of knowledge of looked after children’s own views of the domains that matter to their well-being was striking, as was the focus on negative and objective measures.

4 Method

The aims of the first stage of this on-going study were: to identify how the subjective well-being of looked after children could be measured and to understand which domains were held in common with children in the general population or were unique to looked after children. The theoretical underpinning of the study and of the daily work of Coram Voice (a children’s rights charity) was from a children’s rights perspective (Pona and Hounsell 2012). This perspective focuses on children’s agency, seeing children as individuals who have important valid views on their lives, have rights to have their voices heard (UNCRC) and are ‘experts’ in their experiences of the care system. However, we were also aware of the power inequalities between adults and children especially for children in care where the power of court decisions and those of social workers can restrict children’s agency (Carnevale et al. 2015). As researchers we had responsibilities to support children, in a way consistent with their age, interests and understanding (Woodhead 2005; Watson et al. 2012) to enable them to participate fully in the study. Five stages have been completed in the development of the well-being surveys: literature reviews, expert group roundtable, focus groups with young people, survey development and initial piloting. Further piloting is on-going in 2016.

4.1 Literature Review and Expert Roundtable

The sparse published literature on measuring looked after children’s well-being led to the research team undertaking a desk review of published UK research. The focus was on identifying research that reported the voices of looked after children and young people and not simply adult’ perspectives on children’s lives. Ninety-seven UK studies were identified and from these four primary themes emerged:

  • Relationships Trusting relationships with family members including siblings, friends, teachers, carers, and social workers were of central importance. However, many children reported that maintaining relationships was not prioritised by professionals. Frequent placement moves, changes of social worker, lack of contact with birth family members disrupted relationships.

  • Respect Children and young people wanted adults to challenge the negative stereotypes associated with care. Respect was also shown by the way professionals spoke about the birth parents and kept personal information confidential.

  • Rights Children and young people wanted their views to be listened to, to be involved in their care planning, to be given information and choices. Being encouraged and having the resources and opportunities to have fun and taking part in activities also gave their lives a sense of normality and a chance to fit in with peers. Young people did not want to ‘stand out’. In comparison with children in the general population, there was greater emphasis on ‘having a say’ and being able to participate in decisions.

  • Responsibility Children and young people wanted to be given the opportunities to be / to practice becoming responsible and have the opportunity to take on roles and identities other than that of a ‘looked after child’. They also wanted to feel that they would be given a second chance after making a mistake. They wanted opportunities to learn life skills such as cooking meals and using ATMs outside banks.

The literature reviews were followed by a roundtable event with senior social care professionals to check whether the domains identified through the literature reviews rang ‘true to them’ and to get ideas on how they could be measured. The expert group provided thought-provoking responses and fresh ideas. A press release was issued at the start of the study, which led to nine local authorities volunteering to become ‘early adopters’ of the survey. In each local authority, a ‘project champion’ was identified whose role was to ensure clear communication between the project team and the social work and education professionals.

4.2 Focus Groups with Children and Young People

The nine local authorities enabled the project team to recruit interested children and young people. Ethical approval was granted by the ethics committee at the School for Policy Studies, University of Bristol. Eighteen focus groups were held, including a specialist one for asylum-seeking young people, and attended by 140 looked after children and young people aged between 6 and 18 years old. Individual interviews were undertaken with two children with disabilities covering the same range of activities as the focus groups. The focus groups were led by participation workers from CoramVoice and the same researcher attended every focus group, which were recorded. Most of the children in these groups knew each other well, as the groups were well established. The groups undertook a range of activities including being asked: Imagine you’re in a foster home and there is a small boy and girl being looked after and they are extremely happy. You are an investigator what clues or evidence could you find that proves they are happy? There were further activities that focused on the themes identified in the literature review especially relationships, contact with family, trust, stigma and transitions. The activities were not intended to shape the discussion on well-being but as a kick-start. The focus groups were analysed using a thematic framework approach (Ritchie and Spencer 2002). The focus groups were transcribed by the researcher and then read and re-read to establish key themes and sub themes. These themes along with themes established in the literature review formed a coding framework in Excel. The information was analysed looking for dominant and minor themes.

4.3 Domain and Indicator Development

From the focus groups and the literature, review it was apparent that, although there were domains that were held in common with children in the general population, looked after children identified other domains and their emphasis differed (Table 1). The four domains that had emerged from the literature review were revised and became relationships, rights, resilience and recovery.

Table 1 Indicators identified by looked after children and young people (n = 140)
Full size table

Unlike children in the general population, material goods were barely mentioned by looked after children. This is likely to be because the vast majority of looked after children would have experienced a rise in their living standards on entering care, after coming from a background of neglect. Possessions and material goods were not so important in themselves but enabled children to keep in touch with friends and often provided links back to their birth families and early memories. Children complained about important objects (e.g. teddy bears, photos) being lost as they changed placements or of being stolen by other children where they lived. Looked after children and young people thought that one of the most important contributors to their well-being was having an understanding of why they came into care and their early history. Evidence suggests (e.g. Cook-Cottone and Beck 2007; Ward 2011) argue that having a coherent account of one’s early history is associated with recovery from abuse and the development of an integrated identity. Conversely, gaps in information can result in children who blame themselves for abuse or removal from home and who continue to show symptoms of PTSD and trauma.

Relationships had a central significance, as they do for all children. Children and young people emphasised that their key relationships were not only with birth parents but also with siblings (who they may have been separated from), their current carer and with professionals such as their social worker and teacher. However, the emphasis for looked after children differed, as the most frequently used word in the focus groups was ‘trust’. Research (e.g. Zeanah et al. 2011) has shown that early maltreatment interferes with the normal development of trust and insecure attachment patterns can develop. Once looked after, children’s placements are often unstable. Children move placement and have frequent changes of social workers resulting in lack of trust in carers and professionals. It is not surprising that trust was mentioned so often by looked after children and the feeling that they needed a second chance if things went wrong. Fears of rejection by current carers and being moved again were common.

A trusting relationship also meant that key adults (social worker, carer, teacher) kept personal information confidential and that young people were not marked out as ‘a looked after child’. Looked after children were also very concerned that they were prepared and were learning life skills because they knew that many of them would be leaving care between 18 and 19 years old and would be living on their own. We found, as had Fattore and colleagues (2009), that the domains were not discrete but interconnected. A child who experienced multiple foster placements, did not feel that they belonged anywhere, did not have a trusted adult in their lives, had few friends, did poorly at school and was unhappy. For some looked after children there were also internal conflicts as they recognised that it was not safe for them to have contact with a birth parent but the importance of that relationship led to them making contact and putting themselves at risk.

Children thought that the questions used in the Children Society/Children’s World 2014 surveys on happiness, life satisfaction, life worthwhile and being positive about the future were important questions to ask, but with an additional question about whether life was improving. From the Children’s World surveys, children and young people did not like the question that asked about frequency of bullying. In their view they wanted a question that asked about the impact of bullying, as even one bullying event might lead to truancy. They also did not like the question, “How much do you agree with each of these statements: My parents (or people who look after me) listen to me and take what I say into account.” Some young people thought that the onus of responsibility seemed to be on the child to “speak out” and they would be blamed if they had not. They wanted the responsibility to be with the adult to ensure children’s views were sought.

The analysis of all the material resulted in over 200 possible questions being identified: far too many for a successful self-completion online survey. Survey fatigue and dropout rates are high within the normal population, and we were concerned that this would be even more of an issue for looked after children. For example, we were mindful of what is known about how children answer questions especially the importance of taking into account brain maturation. The pre-frontal cortex, the site of processing and decision-making skills, is not fully developed until early adulthood (National Institute of Mental Health 2012). The difference in processing skills is shown in research examining the speed of survey completion. De Leeuw (cited in Smith and Platt 2013) found that teenagers took 1.5 times longer than adults did to process information needed to answer questions. In addition, we already knew that not only are there high rates of special education needs in the care population but also the effects of trauma can leave children’s developmental age several years behind their chronological age. We wanted to collect the views from pre-teens, and hence were further concerned to create a survey that was fit for purpose and would avoid cognitive overload and a negative experience. We became more aware during the focus groups that many looked after children were angry and disillusioned at the large number of surveys and forms they are required to complete by professionals, but that they thought made no difference to their lives. All children are likely to find questionnaires intimidating (Barker and Weller 2003) but in addition looked after children were wary and weary of form filling.

An expert in survey design and a web designer joined the research team and the number of questions was reduced by removing similar questions. The reduced list of questions was taken back to three of the focus groups for their opinions on the content. The children and young people approved all the questions but said it was still far too long. Further work by the research team, who were joined by a care leaver, reduced the questions to a point where we felt the surveys (age 4–7 years, 8–11 year and 11 year +) were of an acceptable length for each age group. The process was not easy and may have resulted in important questions being omitted.

Primacy and scale effects were also considered in designing the survey. Fuchs (2005) found that when presented with a long list, children aged 10–13 years were twice as likely as older children to tick the first item. Therefore, we needed to ensure a clear lay out and question and answers had to appear on the same page. The survey needed to be meaningful, short, simple, with no negative phrasings and be easy to navigate with no distractions. A decision was made not to use any photographs on the survey pages.

Borgers and colleagues’ (2004) work suggest that the number of response options should be kept to four with no midpoint for children. However, the eleven point happiness scales used in the Children Society/ONS surveys were required to provide a comparison with general population scores. The scales have been tested with young people aged 8–16 years with a refusal rate of less than 1 % and no problems identified in completion, although some children preferred five point scales (Rees et al. 2010; NatCen 2012). Eleven point scales also provide greater sensitivity to subpopulation differences such as age and gender (Casas and Wold 2014). Therefore, three 11-point scales were kept to allow national comparisons.

4.4 Survey Administration and Piloting

The team met and took part in a paper exercise designed to map a ‘typical’ looked after child’s day with the intention of identifying places and times when a survey might be completed. It became clear that the children were very busy and were involved in meetings with professionals, contact visits with members of the birth family and other processes that children in the general population would not experience. It appeared that the best place to complete the survey was in school, as it was a familiar and neutral environment (Scott 2008) as many of the questions were about children’s relationship with their carer and social worker. School could also provide a ‘trusted adult’ in case of distress or more likely if the child had difficulty logging on, reading or understanding the survey. Every school has a ‘designated teacher’ with special responsibility for looked after children and, because of the high level of special educational needs, provide learning mentors or helpers who regularly work with the child on a 1:1 basis. The regular sessions provided an opportunity for a short online survey to be completed on an IPad without the looked after status of the child being flagged up in front of peers and could be completed in a private space in a low-key way.

Previous research (e.g. Borgers et al. 2004; Scott 2008) has asserted that children under the age of 7 years old are unable to complete surveys because young children are more susceptible to bias, answers may be given to please adults and are more likely to lack the necessary cognitive skills. It is argued that Piaget’s theory of cognitive developmental suggests that children under the age of 7 years do not have the language skills or the verbal memory to understand survey questions. This assertion did not match our experience of talking with young children about their experiences in care. Indeed young children had not learnt to be so guarded in their answers in comparison with the teenagers who had been in the care system for many years and who knew what they were expected to say. Neither was it acceptable to the researchers or the managers within local authorities to omit looked after children simply based on chronological age. Punch (2002) argues that treating children like adults can result in exacerbating power relations and poor data but so can treating children as completely different. She proposes that researchers can perceive children as similar to adults but with different competencies. From this perspective, children are capable and it is researchers who lack the methods and flexibility to hear the voice of the child. Therefore, having a trusted adult close by, who could read out the survey questions if needed and answer children’s queries allowed young children and those with reading difficulties to participate. Most importantly, it gave them the opportunity to have their views heard.

Three online surveys were created: age 4–7 years (16 questions), 8–11 years (31 questions), and 11 year + (46 questions). All the questions were optional and were completed anonymously. Through anonymity, we thought that we were more likely to get honest answers but of course, the drawback was that individual children’s progress could not be tracked. However, the high levels of churn in the care system with about a third entering and leaving each year meant that tracking progress over time would be difficult. One of the nine local authorities agreed to pilot the survey over a 2 week period in April 2015 on their looked after children (total population of 183 individuals aged 4–18 years old) and gave permission for some children to be involved (with their consent) in cognitive interviewing.

4.5 Cognitive Interviewing

Cognitive interviewing has developed over the last 30 years and is a useful tool in exploring participants’ thought processes. It focuses on the usually hidden mental processes that participants use in answering survey questions (Collins 2015). Five children were observed completing the survey and nine were interviewed later. The children were purposively selected to represent a range of ages (5–16 years) and types of placement (living with relatives, foster care and residential). The observations were focused on the environment and the interactions between the child and trusted adult if one was present (Table 2). Interviews were intended to answer whether children had understood the question in the way we had intended (comprehension), were able to recall the answer (retrieval), that the response options offered were adequate (responses), and whether any of the questions were too sensitive and caused distress or anxiety (sensitivity). Observations and interviews also provided an opportunity to assess the length of time it took to complete the survey and to ask children if thought any key questions were missing.

Table 2 Pilot observational checklist
Full size table

Time to complete

Most children and young people took 10 min to complete the survey. In most circumstances, the teacher was on hand to answer any questions but did not take an active role. However, we observed one young person who understood little English taking 30 min to complete the survey. The teacher’s role in these circumstances was much more intense, as she had to translate and explain each question.

Comprehension

Most of the questions were easily understood. However, the youngest age group (4–7 years) did not understand the demographic question at the start of the survey that asked about their ethnicity. They were unfamiliar with ethnic categories. One child ticked ‘mixed ethnicity’ and when asked why he had selected that category replied that he had done so because his father lived in London. Other children and young people did not know their ethnicity as the identity of their father was unknown. The youngest children also had difficulty deciding what kind of placement they were living in. Looked after children who were being brought up by grandparents did not think the option ‘living with family or friends’ applied to them.

We also wanted to probe whether children would understand a question that asked ‘Do you feel settled in the home you live in? The intention was to tap into whether children felt secure in their placements and had a sense of belonging without raising any fears that they might be about to move. Children and young people understood the meaning of the word ‘settled’ and said it was used by their social workers when asking about whether things were going well. An autistic young person did not understand the question and asked his teacher for further clarification. The teacher understood the word differently and defined it as “everything feeling calm and like that every day.’

Retrieval

The 11 year + survey included two questions that needed a calculation 1) How many placements have you lived in the last 12 months? 2) How many social workers have you had in the last 12 months? We were expecting that children and young people might think about 12 months in different ways calculating from either September (the start of the school year) or January. Three young people when asked individually about how they had arrived at their answer said that they counted all the placements they had had since entering care and divided it by their years in care. The number of social workers was also calculated in a similar way for young people who had experienced many changes. This gives a very practical example of Punch’s (2002) comment regarding the need for adults to think differently when listening to the child’s voice: here the children’s perspective could be considered inaccurate, as an adult researcher would be seeking an exact numerical data point. Yet, whilst the answers were estimates rather than exact numbers, they are valid in reflecting how children and young people felt about stability.

Measurement

Most of the measurement options worked well except for the options available to answer questions about contact with the birth family. The questions asked, ‘Do you see your mother too much, just the right amount, too little?’ The question was repeated asking about contact with fathers and siblings with a text box available for those who wanted to add more information. The question and options had been kept simple, as we did not want to distress children, as we were expecting contact to be the most sensitive question. However, children were frustrated at the lack of options. For example, one young person who was living with her full sister, but separated from half-siblings, and thought of the grown up birth children of her foster carer as siblings too, wanted to see some siblings more and some siblings less. Other children said that contact was prohibited because of safeguarding issues or a parent had died or had rejected and refused to see the child and therefore contact could not take place.

Changes in the way the survey questions were laid out on a page also did not work well. We thought that variety might keep children’s interest but instead the change, from questions posed as a sentence to ones that were in a grid, just made children sigh and wonder how much longer the survey would take. There was a particular dislike of grid questions, and a preference for a longer page that presented one question at a time.

Sensitivity of Questions

The questions about contact with birth family members were sensitive but did not prevent completion of the survey, whereas questions about asylum-seeking status did affect completion. The focus group with asylum-seeking looked after children had identified specific areas that detrimentally affected their well-being: disputes about their actual age, lengthy delays in deciding the outcome of their case and children feeling that they were not kept informed. Therefore, we had included three questions in the survey and asked: Are you an asylum seeker? (If yes) Are you kept up-to-date with your asylum claim? At the moment, are there any disagreements about your age? Although the survey was anonymous an observation of a child who was seeking asylum, and feedback from teachers of other asylum-seeking young people, found that children became very anxious when faced with these questions. Children may have wondered how the information might be used and not trusted the confidential nature of the survey.

Role of ‘Trusted adult’

Safeguarding concerns were raised by trusted adults who had witnessed children completing a question stating that they did not feel safe where they lived. This highlighted the difference between how adults and looked after children interpret and use the word ‘safe’. Adults were unsure of what to do in such circumstances. Should they intervene immediately and ask more questions or should they let the child complete the survey? Whom should they tell? Professionals feared that they might make a mistake and be blamed if child abuse was later discovered. Concerns were also raised in respect of older young people who would be less likely to have a trusted adult present and therefore no one would know how they had responded to the question. Questions about safety are complex for maltreated children, as children may continue to feel unsafe even though they are safe. However, having experienced maltreatment places children at greater risk of further abuse.

4.6 Edits to the Surveys

As a result of the cognitive interviewing and analyses of pilot data, a number of changes were made to the surveys. Questions that had not been well understood were clarified, more examples added, and those that asked about asylum-seeking status and ethnicity were removed. Demographic information would no longer be completed by young children but by the trusted adult before the child began the survey. Further guidance was written for trusted adults about what to do if they had safeguarding concerns and telephone helpline numbers were added for children. The layout of the survey was made more consistent and visual smiley/sad faces were added to anchor the 11-point scales.

5 Key Findings and On-Going Work

The pilot survey was successfully completed by children of all ages (overall response rate was 40 % of 183 individuals) and we were particularly pleased to get a 54 % response rate from the 4–7 years age group. Survey data were explored in SPSS using frequencies, cross tabulations and Chi-square tests and the Mann–Whitney U were used to test differences by age and gender. Key findings were: that the majority of children and young people thought their lives were improving, were as satisfied with life as children in the UK general population but expressed more unhappiness. Half of those aged between 8 and 18 years old said they were dissatisfied with the amount of contact they had with either their mother or father. Most children made positive comments about being looked after but about a quarter of young people over 11 years old were concerned about the stigma of being in care and of adults who drew negative attention to their care status. Nearly half thought they did not get a ‘second chance’ if they made a mistake.

Younger children (4–11 year) were less settled in their placements, less confident of the identity of their social worker and lacked an understanding of why they were being looked after in comparison with older young people. More than a third of young people over 11 years old reported changes of placement and only 23 % had kept the same social worker in the previous year.

The majority of all children had a trusted adult on whom they could rely and a good trusting relationship with their carer and social worker. Most children liked school, had at least one good friend and a carer who helped with learning. But 36 % of young people over 11 years old did not have access to a computer in their foster home and a quarter thought they were not being taught independence skills.

We were keen to close the circle and feedback findings to the Local Authority and to their looked after children. The results of the pilot survey were therefore presented at an event chaired by the Director of Children’s Services attended by approximately 60 social workers and teachers and presented at other key meetings in the local authority. We have been extremely fortunate to be working with a local authority with such a high commitment to improving the well-being of children in their care. A senior manager stated, “You have enabled all of us … to have a very thorough understanding of what we do well and what we need to change from our children in care’s perspective. I have enjoyed the whole process, but to be in a packed room of school, social care, and senior leaders all listening to our children’s views of their care journey and then discussing what they do differently was a particular highlight. Lots to do now to make sure we embed this valuable information into all areas of our improvement planning.”

The key findings (“You said”) were sent to every looked after child in the authority (not just those who completed) along with the local authority’s statement of how they planned to address their concerns (“We will”). The first pilot local authority intends to re-run the survey and new local authorities will run the surveys for the first time in 2016. With more completed surveys and lessons learnt from the pilot (especially ensuring everyone in the authority is informed about the survey) some of the limitations of the first pilot should be addressed. The limitations included piloting in a small rural Local Authority, few surveys completed by children excluded from school, in residential care or placed many miles away from their home area. We intend to address these difficulties in the second phase of pilots working with a senior professional (the survey champion) in each local authority. This work is on-going.

The next stage of the research will be to use the findings of the well-being surveys to identify those local authorities whose children report higher well-being so that we can begin to answer the original research question: Which local authorities provide a good care experience and what practices and policies do they have in place to make that happen?