Introduction

The internet has become an important tool for people to access health information as up to 7 million health-related searches are performed on Google daily [1]. It is established that when parents are well-informed of their child’s medical condition(s), they are better able to engage in the plan of care and this leads to better outcomes [2]. Measures of improved parental/family engagement include increased compliance with medications, planning of post-operative rehabilitation needs, and improved post-surgical quality of life [2]. However, how parents and families can best become informed on the myriad of surgical conditions that can befall a child to ensure improved outcomes is not well defined.

Health literacy refers to the basic reading and numerical skills that allow a person to function within a health care environment. Those with a high health literacy are better able to participate in the care of their child, whether by making scheduled appointments, adhering to medication regimens, or by following the instructions of the treatment team [3]. The opposite is true for those who have low health literacy. Health literacy can be improved by providing disease-specific education materials that are readable by patients [4], where readability refers to the ease in which written text can be understood by the population. The National Institutes of Health recommends that medical information should be presented at a 4th–6th grade reading level and should not exceed that of an 8th grade reading level, which encompasses 80% of Americans [4].

Multiple pediatric surgical associations exist worldwide, with goals of connecting their membership and serving the patients/families in their geographic regions. As an example, the American Pediatric Surgical Association (APSA) website presents patient and family education materials on an extensive list of common and rare pediatric surgical conditions. The site contains information about the condition, including signs and symptoms, diagnostic modalities, medical treatments, when surgery is needed, and general post-operative care. However, this information is curated by medical professionals, and may not be presented in a format suitable for the general population to understand. The purpose of this project was to assess the readability of pediatric surgical association websites’ We hypothesized that the patient/family education materials presented on the pediatric surgical association websites would be above the reading level that would make it the most useful for the general population.

Methods

Ethics statement

The Institutional Review Board determined that this study qualified for Not Human Subjects Research status in that it does not involve “human subjects” as defined in 45CFR46.102(e)(1).

Identification and analysis of pediatric surgical association patient/family education materials

The publicly accessible portions of the American Pediatric Surgical Association (APSA), American Pediatric Surgical Nurses Association (APSNA), British Association of Paediatric Surgeons (BAPS), Canadian Association of Paediatric Surgeons (CAPS), European Paediatric Surgeons' Association (EUPSA) and Pan African Paediatric Surgery Association (PAPSA) websites were searched for patient/family education materials. APSA and BAPS websites contained education materials for patients/families while APSNA, CAPS, EUPSA and PAPSA did not.

All patient/family education materials on the APSA and BAPS websites were accessed and downloaded as separate documents for analysis. Online readability formula tools (https://readabilityformulas.com/) were used to perform multiple grade-level assessments and readability assessments on each topic. To ensure accurate assessment of each of the conditions mentioned on the website, the full text was used to calculate the grade-level and readability scores.

The Flesch Reading Ease Formula, which uses the total words, sentences, and syllables in a text to come up with a readability score between 1 and 100, is considered as one of the oldest and most accurate readability formulas [5]. The Flesch–Kincaid Grade Level Readability Test also uses total words, sentences, and syllables of a text for its calculation, but improves upon the Flesch Reading Ease Formula by converting the results into a US grade-level equivalent.

The Coleman–Liau Index, based on the average number of letters and average number of sentences per 100 words, was designed to approximate the usability of a text, originally to calibrate the readability of all textbooks for the US public school system. The Coleman–Liau Index has been previously applied to medical texts because of its utility in evaluating the length of words [6].

The Simple Measure of Gobbledygook (SMOG) Index assesses the number of polysyllabic words in a passage to present a grade reading level equivalent. The SMOG Index is best applied to text that has at least 30 sentences and was designed specifically for the healthcare field. It is measured by taking 10 sentences from the beginning, middle, and end of a text, counting every word that has 3 syllables or more, taking the square-root of that number, rounding to the nearest tenth, and adding 3 to get the final score [6].

Finally, the Lasbarhetsindex Swedish Readability Formula (LIX) is designed to measure the difficulty of reading text in a foreign language. LIX utilizes the percentage of words greater than six characters and the average number of words per sentence to provide a readability index (lower values indicate greater readability).

Statistical analysis

Summary statistics were calculated in GraphPad Prism v9 (GraphPad Software, LLC).

Results

Availability of patient/family education materials on pediatric surgical websites

The publicly accessible portions of the American Pediatric Surgical Association (APSA) and British Association of Paediatric Surgeons (BAPS) websites contained patient/family education materials. The American Pediatric Surgical Nurses Association (APSNA), Canadian Association of Paediatric Surgeons (CAPS), European Paediatric Surgeons' Association (EUPSA) and Pan African Paediatric Surgery Association (PAPSA) websites did not contain patient/family education materials.

American Pediatric Surgical Association (APSA) website

The American Pediatric Surgical Association (APSA) website contained patient/family education materials for 74 conditions, ranging alphabetically from Achalasia to Wilms tumor (Table 1). Articles had a median word count of 1053 (IQR 850-1339). Three separate grade-level assessments were performed. The Flesch–Kincaid Readability Test indicated that the articles were written at a median grade level of 9.6, although five topics were written below the 8th grade level (Adrenal tumors, Burns, Inguinal Hernia/Hydrocele, PDA, Sacrococcygeal Teratoma) (Table 1, Fig. 1A). In contrast, the SMOG Index demonstrated that topics were all above the 10th grade level, with many assessed to be at a college level (Table 1, Fig. 1B). Finally, the Coleman–Liau Index similarly demonstrated topics to be above the 8th grade reading level, with a moderate number at the college level as well (Table 1, Fig. 1C). The Flesch Reading Ease test demonstrated a median score corresponding with above the 9th grade reading level, although six topics were below the 8th grade reading level (Adrenal tumors, Alpha-1 Antitrypsin Deficiency, Burns, Inguinal Hernia/Hydrocele, Pectus Carinatum, and Thyroglossal Duct Cyst) (Table 1, Fig. 1D).

Table 1 List of all conditions with patient/family education materials on the APSA website and assessment of readability using multiple measures
Full size table
Fig. 1
figure 1

Visual representation of readability tests from APSA. Three grade-level assessments were performed. A The Flesch–Kincaid Readability Test indicated that most articles were written at a median grade level corresponding with 8th–12th grades (black dots), although five topics were written below the 8th grade level (green dots; details in Table 1), and two topics were written at the college level (red dots; details in Table 1). B The SMOG Index demonstrated that topics were all above the 10th grade level, with many assessed to be at a college level (red dots; details in Table 1). C The Coleman–Liau Index demonstrated topics to be above the 8th grade reading level, with a moderate number at the college level (red dots; details in Table 1). D The Flesch Reading Ease test demonstrated a median score corresponding with above the 9th grade reading level, although six topics were below the 8th grade reading level (green dots; details in Table 1). E The LIX readability score was used to assess the difficulty of reading a foreign text and indicated a median readability score corresponding with 9th grade or medium difficulty, with five topics assessed as easy to read (green dots; details in Table 1) and ten topics assessed as being above medium difficulty (red dots; details in Table 1)

Full size image

British Association of Paediatric Surgeons (BAPS) website

The British Association of Paediatric Surgeons (BAPS) website contained patient/family education materials for 6 conditions, ranging alphabetically from Acute Appendicitis to Umbilical Hernia [Table 2]. Articles had a median word count of 351 (IQR 304–386). Three separate grade-level assessments were performed. The Flesch–Kincaid Readability Test indicated that the articles were written at a median grade level of 7.5, with 5/6 topics written below the 8th grade level (acute appendicitis, circumcision, hydrocele, inguinal hernia, orchiopexy) (Table 2, Fig. 2A). In contrast, the SMOG Index demonstrated that topics were all above the 10th grade level, but were below a college level (Table 2, Fig. 2B). Finally, the Coleman–Liau Index similarly demonstrated topics to be above the 8th grade reading level but below the college level (Table 2, Fig. 2C). The Flesch Reading Ease test demonstrated a median score corresponding with above the 10th to 12th grade reading level, although one topic was below the 8th grade reading level (Hydrocele) (Table 2, Fig. 2D).

Table 2 List of all conditions with patient/family education materials on the BAPS website and assessment of readability using multiple measures
Full size table
Fig. 2
figure 2

Visual representation of readability tests from BAPS. Three grade-level assessments were performed. A The Flesch–Kincaid Readability Test indicated that most articles were written below the 8th grade level (green dots; details in Table 2). B The SMOG Index demonstrated that topics were all above the 10th grade level but below the college level (black dots; details in Table 2). C The Coleman–Liau Index also demonstrated topics to be above the 8th grade reading level but below the college level (black dots; details in Table 2). D The Flesch Reading Ease test demonstrated a median score corresponding with above the 10th to 12th grade reading level, although one topic was below the 8th grade reading level (green dot; details in Table 2). E The LIX readability score was used to assess the difficulty of reading a foreign text and corresponded to the 7th grade or easy difficulty, with three topics assessed as easy to read (green dots, details in Table 2)

Full size image

Readability of patient/family educational materials for non-English speakers

No articles were available on the APSA or BAPS websites in languages other than English. Therefore, we used the LIX readability score, which assesses the difficulty of reading a foreign text. Materials on the APSA site indicated a median readability score corresponding with 9th grade or medium difficulty, with five topics assessed as easy to read (adrenal tumors, anal fissure, burns, PDA, and sacrococcygeal teratoma) (Table 1, Fig. 1E). Materials on the BAPS site indicated a median readability score corresponding to the 7th grade or easy difficulty, with three topics assessed as easy to read (acute appendicitis, hydrocele, and umbilical hernia) (Table 2, Fig. 2E).

Discussion

In this study, we assessed the readability of multiple pediatric surgical associations’ patient/family education materials and found that (1) the majority of associations do not present these materials to the public, (2) the APSA materials are predominantly presented at a high school grade reading level, and (3) the BAPS materials are often presented at an easier reading level. These findings were consistent across all the readability and grade-level assessments utilized. Further analysis of the APSA materials identified that the average grade level for the topics were in high school for the Flesch–Kincaid and Coleman–Liau assessments and college level for the SMOG Index, respectively.

Multiple prior studies have assessed the readability of medical texts. One of the earliest studies in the literature, from the 1970s, analyzed health education materials that were found both in medical pamphlets and lay text [6]. That study used the SMOG Index, which was designed specifically for health-related texts, and identified averages above a 9th grade reading level. Similar studies from that period, utilizing multiple readability formulas, demonstrated similar results [7]. Over time, as medical texts have transitioned from written pages to computer screens, readability formulas have demonstrated increasing levels of complexity. This is also not limited to pediatric surgery, as assessments of other surgical subspecialties, have shown similar levels of complexity [8]. This may be related to the rapid escalation of technology and techniques for diagnosis and treatment, resulting in complex writing.

Further analysis of the results from APSA showed that despite what may be considered “easier” topics, the content remained above the NIH recommended 8th grade reading level. For example, acute appendicitis, one of the most common pediatric surgical conditions, is described using 1487 words and had grade-level assessments of 9.9 from Flesch–Kincaid, 12.5 using the SMOG Index, and 11.4 using the Coleman–Liau Index. Conversely, conditions that may have non-surgical treatment options such as gastroesophageal reflux disease (GERD) were described using 913 words and had grade-level assessments of 10.8, 13, and 13.1, respectively. Interestingly, while BAPS presented fewer topics on its website, the materials presented were generally more readable than those on the APSA website. This may be due in part to the short word length of each topic (median ~ 350 words vs. > 1000 words). Readability of a text has health literacy implications and potential impact on the health of the person or population to which the text applies [3].

While our study focused on counseling materials for surgical conditions, all documents, ranging from discharge instructions, vaccination information, and survivorship care plans for cancer patients, are impacted by the complexity of the information they contain [9, 10]. Additionally, as care plans become increasingly complex and multidisciplinary in nature (e.g., multimodal care for cancers), it becomes incredibly important that information be presented at an appropriate level for patients and families to understand. It is equally important that the providers also take an active role in improving the health literacy of patients, which includes increased use of lay language, slowing down to the pace of discussions with patients, and asking patients to repeat back information that was presented to them during the encounter [11].

Potential limitations to this study relate to the measures used to assess readability. This study used five different assessments to measure readability; however, there are dozens of different tools available to assess readability with varying levels of complexity. The measures chosen for this study include some of the most used and easy to interpret (Flesch) as well as those specifically created to assess medical texts (e.g., SMOG Index and Coleman–Liau). It was important to use different assessments to measure readability to establish ways in which text can be improved. For example, the Coleman–Liau Index is helpful for reducing the number of words in a text. Furthermore, the SMOG Index may assist with changing longer words into smaller words which could improve readability. In addition, we only identified materials on the APSA and BAPS websites available in English, and so we can only infer how the information would be interpreted if it was in a different language. To mitigate this limitation, we employed the LIX formula in our analysis. In addition, readability assessments only measure written text and, therefore, cannot account for how visual aids may improve or detract from comprehension. Finally, materials on the multiple association websites were not publicly accessible and, therefore, not able to be analyzed.

Conclusions

APSA’s patient/family education materials aim to help families “better understand the problem, what can be done and what to expect when considering pediatric surgery.” Our analysis indicates that the patient/family education materials available on pediatric surgical association websites may not be written at a level that is comprehensible by the general population. Potential solutions include re-writing the materials with an emphasis on less, more precise wording, and presenting materials in other languages.