Introduction

Abnormal vaginal discharge is the commonest reason why women of reproductive age consult their general practitioner for a gynaecological complaint [1]. The most common cause is bacterial vaginosis (BV), which accounts for 22–50% of vaginal infectious morbidity [2]. BV is a polymicrobial syndrome of unknown aetiology, characterised by a shift from Lactobacillus-dominated vaginal microbiota to a more diverse microbiota dominated by anaerobes such as Gardnerella vaginalis and Atopobium vaginae. BV is associated with a number of adverse sequelae in obstetrics and gynaecology, including increased susceptibility to sexually transmitted infections and preterm birth [3]. In 2017, the FDA recognised BV as a serious or life-threatening condition, which permitted “Qualified Infectious Disease Products” to treat BV for “Fast Track Designation” through the 2012 US Gain Act [4].

European guidelines recommend to base diagnosis on clinical symptoms and signs supported by additional test findings [5]. Often, Amsel’s clinical criteria [6], Nugent score [7], or culture-based techniques are used. According to Amsel, diagnosis of BV is based upon the presence of three out of four of the following clinical criteria: (i) vaginal pH > 4.5; (ii) homogenous white/grey adherent vaginal discharge; (iii) the presence of clue cells (vaginal epithelial cells covered in bacteria), and (iv) a positive whiff test (fishy odour after addition of potassium hydroxide). Although useful clinically as an immediate office-based test, assessment of the Amsel criteria is subjective, irreproducible, time-consuming, and unpleasant to perform [8, 9]. Nugent score is a Gram stain scoring system, based on the quantitative assessment of Lactobacillus, Gardnerella, and Mobiluncus morphotypes. It is more objective and reproducible than diagnosis based on Amsel criteria but requires a certain level of experience [9]. Using culture-based techniques, BV is often diagnosed when G. vaginalis is isolated, but the sensitivity and specificity of this method is poor [10].

Recently, molecular-based assays became available for the diagnosis of BV, including two CE-IVD marked multiplex, quantitative (q)PCR assays [11,12,13,14]. One is the AmpliSens® Florocenosis/Bacterial vaginosis-FRT PCR kit of InterLabService (henceforth referred to as AmpliSens assay), which uses the relative concentration of Lactobacillus spp., G. vaginalis clades-1 and -2, A. vaginae and total bacteria to diagnose BV. The other is the BD MAX™ Vaginal Panel of BD Diagnostics (henceforth referred to as BD MAX assay), which targets Lactobacillus crispatus and Lactobacillus jensenii, G. vaginalis, A. vaginae, Bacterial Vaginosis–Associated Bacteria-2 (BVAB-2) and Megasphaera-1 for the diagnosis of BV. Both qPCR assays are fast and have a high sensitivity and specificity [15,16,17].

Of these additional tests, the Nugent score is considered as the gold standard for the diagnosis of BV. Another reference method is required to compare all conventional methods and qPCR assays with each other, such as 16S ribosomal RNA (rRNA) gene amplicon sequencing (microbiota analysis). This method enables accurate characterisation of complex microbial communities in terms of membership and their relative abundance to one another. Investigation of the vaginal microbiota has shown that < 50% relative abundance of Lactobacillus is associated with BV [18,19,20,21,22]. Based on statistical analysis of the vaginal microbiota data, BV has been defined as ≤ 47% relative abundance of Lactobacillus and increased presence of anaerobes [23]. Although recommended by some, microbiota analysis is currently too laborious and expensive to be used in the routine clinical setting [24].

The aim of this study was to compare Amsel criteria, Nugent score, culture, the AmpliSens assay, and the BD MAX assay with microbiota analysis for the diagnosis of BV. First, diagnostic methods were (individually) compared with microbiota analysis using microbiota analysis as reference test. Subsequently, the vaginal microbiota profiles of the cases with discrepant results between microbiota analysis and at least one of the diagnostic methods were evaluated.

Materials and methods

Study design

The study was approved by the local ethics board (METC Zuidwest Holland, The Hague, The Netherlands) and written informed consent was obtained from all subjects. Sixty women complaining of abnormal vaginal discharge (increased in volume, “thick or cheesy” in consistency, malodorous, itchy causing irritation, or a different colour from the norm of that woman), visiting the Gynaecology outpatient clinic of the Haaglanden Medical Centre (The Hague, The Netherlands) between January and July 2015 were recruited to the study. To obtain a sufficient number of BV negative swabs, 20 women visiting the outpatient clinic for either a routine cervical cytology follow-up, insertion of an intra-uterine contraceptive device or a first-trimester ultrasound in pregnant women were included. Postmenopausal women or those who had received antibiotics in the previous 3 months were excluded.

At visit 1, a standardised interview and gynaecological examination were performed. Samples were collected in the following order: (i) vaginal secretions for vaginal pH; (ii) three microscopy slides (for detection of clue cells, whiff test and Gram stain); (iii) a charcoal swab for culture, and (iv) an eSwab for the AmpliSens assay, the BD MAX assay and microbiota analysis. At visit 2, approximately 4 weeks after visit 1, the gynaecological examination and sample collection were repeated.

Amsel criteria

A woman was categorised as BV positive when three out of four of the following clinical criteria were present: (i) vaginal pH > 4.5 measured using pH indicator strips with a pH range from 4.0 to 7.5 (Johnson Test Papers, Oldbury, UK); (ii) homogenous white/grey adherent vaginal discharge; (iii) the presence of clue cells detected by wet-mount microscopy, and (iv) a fishy odour after addition of 10% potassium hydroxide to a microscopic slide of vaginal secretions [6]. If one of the tests could not be performed, the slide was classified as indeterminate.

Culture

Culture was performed in the routine laboratory setting. Swabs were inoculated onto chocolate agar, blood agar and blood agar with polymyxin B (BD, New Jersey, USA) and incubated at 35 °C in 5% CO2 for 24 and 48 h. A culture was reported as BV positive if G. vaginalis was present as a monoculture.

Nugent score

The Gram stains were analysed in a double-blind manner by two experienced cytology technicians. For the discrepancies, consensus was achieved. The Nugent score was calculated by assessing the numbers of Lactobacillus morphotypes (scored as 0 to 4), G. vaginalis morphotypes (scored as 0 to 4), and Mobiluncus morphotypes (scored as 0 to 2) [7]. A score of 0–3 was categorised as normal flora, 4–6 as intermediate flora, and 7–10 as BV. If the quality of the slide was poor, the slide was classified as indeterminate.

DNA extraction

DNA was extracted from 200-μL sample and eluted in a final volume of 100 μL with the MagNA pure 96 instrument using the MagNA pure 96 DNA and Viral NA Small Volume kit and the Viral NA Plasma protocol (Roche Diagnostics, Basel, Switzerland).

CE-IVD marked assays

Both the AmpliSens and the BD MAX assay were performed according to the manufacturer’s instructions. For the AmpliSens assay, a predefined algorithm of the manufacturer categorised the swabs as BV negative, BV positive, intermediate, unspecified dysbiosis or indeterminate, and for the BD MAX assay as BV negative, BV positive or indeterminate.

Microbiota analysis

Microbiota analysis was performed as described elsewhere [25]. Briefly, a fragment of ~ 464 bp of the V3–V4 regions of the 16S rRNA gene was amplified. Nextera XT and MiSeq Reagent Kits v2 500-cycles (Illumina, San Diego, USA) were used for library preparation and sequencing with the MiSeq desktop sequencer (Illumina), respectively. Data was processed with the Metagenomics workflow of the MiSeq Reporter v2.3 software. A sample was considered positive for a specific genus when more than 1% of the classified reads were assigned to that genus.

Based on the microbiota profiles, samples were categorised as normal vaginal microbiota (> 47% relative abundance of Lactobacillus), microbiota associated with BV (≤ 47% relative abundance of Lactobacillus and mainly anaerobes) or microbiota associated with a different vaginal infection (≤ 47% relative abundance of Lactobacillus and mainly aerobes) [23]. For the figures containing microbiota profiles, a limited number of genera were selected representing the microbiota composition of each sample, which included genera (i) involved in one of the diagnostic methods if detected, (ii) associated with BV and dominating microbiota profiles or (iii) involved in aerobic vaginitis. The remaining genera formed the other genera category.

Data availability

The datasets generated and analysed during the current study are available in the NCBI Sequence Read Archive (https://www.ncbi.nlm.nih.gov/sra) repository with the accession number PRJNA524112.

Statistical analysis

For the determination of the test characteristics, cases categorised as intermediate (Amsel criteria, AmpliSens assay), unspecified dysbiosis (AmpliSens assay), or microbiota associated with a different vaginal infection (microbiota analysis) were interpreted as BV negative. Statistical analysis was performed using the software package SPSS. To compare the sensitivity between the first and second visits, we selected at each time point the measurements which were positive according to the reference test and performed a logistic regression, with test result as dependent and visit as independent variable. Generalised estimation equations were used to estimate the coefficients and standard errors, to account for the fact that some women provided more than one sample for the study. Test characteristics of the different diagnostic methods were compared using the McNemar Test.

Results

Study population

The age of the 80 women ranged from 18 to 52 years (mean 34.1 ± 8.6 years), the majority of the women were of European origin and 25 of them were treated for BV based on clinical information at visit 1 (Supplementary Table S1). Of the 80 women, 14 failed to attend visit 2, and data of 31 visits were excluded because of an insufficient sample volume or indeterminate outcome by at least one of the methods, resulting in 115 complete datasets (63 from visit 1 and 52 from visit 2). Based on the microbiota profiles, 73/115 (64%) cases were categorised as normal vaginal microbiota and 36/115 (31%) as microbiota associated with BV (Fig. 1 and Supplementary Table S2). The microbiota profiles of the remaining six (5%) cases were dominated by aerobes, which is associated with a different vaginal infection, namely aerobic vaginitis (AV) [26].

Fig. 1
figure 1

Microbiota profile of 115 vaginal swabs categorised as normal vaginal microbiota, microbiota associated with bacterial vaginosis or microbiota associated with a different vaginal infection

Full size image

Comparison of the different diagnostic methods with microbiota analysis

Amsel criteria, Nugent score, culture, the AmpliSens assay and the BD MAX assay were individually compared with microbiota analysis (Supplementary Table S3), resulting in a sensitivity of detecting BV of 61.1% for Amsel criteria, 63.9% for Nugent score, 38.9% for culture, 80.6% for the AmpliSens assay, and 63.9% for the BD MAX assay (Supplementary Table S4). The specificity of all methods was ≥ 92.4%. The sensitivity of the AmpliSens assay was significantly higher than the sensitivity of the other methods (p ≤ 0.031; McNemar Test). There was no significant difference between test characteristics based on data of visit 1 and visit 2 for any of the methods, confirming that data of both visits could be used for calculation and comparison of the test characteristics.

Comparison of all five diagnostic methods with microbiota analysis showed that 57/73 (78%) cases with a normal vaginal microbiota profile were BV negative by all five diagnostic methods (Fig. 2a). For the remaining 16 cases, at least two diagnostic methods were in agreement with microbiota analysis. Of the 36 cases with a microbiota profile associated with BV, seven cases (19%) were BV positive by all five diagnostic methods (Fig. 2b). The remaining 29 cases showed variable results between the five diagnostic methods. For 24 cases, at least one diagnostic method was in agreement with microbiota analysis, whereas none of the five diagnostic methods was BV positive for the other five cases.

Fig. 2
figure 2

Venn-diagram of the number of cases categorised as a negative or b positive for bacterial vaginosis by the five different diagnostic methods and microbiota analysis

Full size image

Discrepancies between microbiota analysis and the different diagnostic methods

Microbiota profiles of the swabs with discrepant results between microbiota analysis and at least one of the diagnostic methods were evaluated (Fig. 3). Variable microbiota profiles with various dominating Lactobacillus spp. were observed for each diagnostic method, but all five methods missed BV positive cases that had a relatively high abundance of the genus Alloscardovia, Bifidobacterium, or Dialister. Three of these five cases were categorised as unspecified dysbiosis by the AmpliSens assay due to the complete depletion of Lactobacillus spp., and the absence of G. vaginalis and A. vaginae. The remaining two cases were categorised as BV negative due to the relatively high abundance of Lactobacillus spp. and/or not detecting G. vaginalis. Furthermore, cases categorised as intermediate by the AmpliSens assay or Nugent score had variable microbiota profiles, leaving the clinical importance of this category unknown.

Fig. 3
figure 3

Microbiota profiles of the discrepancies between the five diagnostic methods and microbiota analysis

Full size image

Discussion

To our knowledge, this is the first study to compare Amsel criteria, Nugent score, culture, the AmpliSens assay and the BD MAX assay with microbiota analysis for the diagnosis of BV. Based on microbiota analysis, Amsel criteria, Nugent score, culture and the BD MAX assay each had a very low sensitivity (≤ 63.9%) compared to the AmpliSens assay (80.6%). Microbiota profiles of the cases with discrepant results between microbiota analysis and the diagnostic methods were variable, but all five diagnostic methods missed BV positive cases that had a relatively high abundance of the genus Alloscardovia, Bifidobacterium or Dialister.

In the present study, microbiota analysis was used as reference test because it allowed independent analysis of the performance of the different diagnostic methods, including the current golden standard; Nugent score. Compared to microbiota analysis, the sensitivity of the Nugent score was low and the clinical importance of the intermediate category remains unknown. Based on these data, microbiota analysis should be considered as a serious alternative for the current golden standard to evaluate new diagnostic methods.

When all five diagnostic methods were compared to microbiota analysis, the AmpliSens assay was most in agreement with microbiota analysis. The sensitivity of 80.6%, however, remains low. One BV positive case missed by the AmpliSens assay, had a high relative abundance of G. vaginalis, which was probably G. vaginalis clades-3 or -4. Addition of these clades as targets would increase the number of BV positive samples by 3% [15]. The remaining missed BV positive cases had high relative abundances of anaerobic species not targeted by the assay. Since these cases were categorised as unspecified dysbiosis, the sensitivity of the AmpliSens assay would improve if this category was interpreted as BV positive. Specificity would, however, decrease because cases with a microbiota profile dominated by aerobes are also included in this category. This is a characteristic of AV, which requires different treatment than BV [27, 28]. Others obtained a sensitivity of 100–96.9% for the AmpliSens assay, but a combination of Amsel criteria and Nugent score rather than microbiota analysis was used as reference test or the definition of BV was different [15, 16].

A limitation of our study is that the focus was on diagnosis of BV and therefore the diagnosis of AV was not evaluated. However, there is ongoing discussion if AV is a separate identity from BV. In this study, microbiota profiles dominated by aerobes were treated as a separate identity, which was supported by the data of the evaluated diagnostic methods.

In conclusion, compared to Amsel criteria, Nugent score, culture and the BD MAX assay, the AmpliSens assay was most in agreement with microbiota analysis. A positive or unspecified dysbiosis result is indicative of a shift in vaginal microbiota from a normal vaginal microbiota to a more diverse microbiota characterised by potentially pathogenic microorganisms. If the outcome is unspecified dysbiosis, subsequent culture should be considered to avoid missing the diagnosis of AV, which requires a different treatment than BV.