Introduction

Anterior cruciate ligament reconstruction (ACLR) surgery is the treatment of choice for physically active individuals who injure their anterior cruciate ligament (ACL) [1]. The treatment objective is to improve the function and stability of the knee to enable the patient to return to their pre-injury sports activities [2].

The quadriceps strength deficit in the injured limb can range from 5 to 40% up to 7 years postoperatively [3, 4]. Asymmetry in the limbs strength of > 15% after ACLR is a predictor of re-injury and a sports performance parameter [5, 6]. In addition, adequate quadriceps strength is an important criterion for the progression of rehabilitation phases and an indicator for return to sport (RTS) [4, 7, 8].

The isokinetic dynamometer is currently the main and most accurate method with established validity and reproducibility used for the assessment of muscular strength [9,10,11,12,13]. However, there are certain disadvantages of this method, including a high cost of acquisition and maintenance, requirement of considerable amount of time for evaluation, and lack of portability of the equipment [13,14,15,16]. Another potential method for quantitative assessment of strength is that using a hand-held dynamometer (HHD) [14, 17]; this method is cheaper and faster and the instrument is portable, compared to the isokinetic dynamometer [13, 15, 16, 18, 19].

Several studies on different populations have reported the validity of HHD against the isokinetic dynamometer for strength assessment in subjects with dysfunctions in the lower limbs [16, 19, 20], healthy individuals [15, 21], elderly [22], and athletes [23]. However, validity and diagnostic accuracy of HHD for assessing quadriceps torque asymmetries in patients who have undergone ACLR on return to sports was not verified by any study. Our initial hypothesis is that HHD is a valid instrument for assessing the torque of the quadriceps muscle in individuals after ACLR with adequate levels of sensitivity, specificity and positive and negative likelihood ratio. Thus, in favour of the ease and low cost in quality assessments in clinical practice, the objective of the present study was to verify the validity and diagnostic accuracy of the HHD with the isokinetic dynamometer (gold standard) to evaluate the quadriceps muscle torque following ACLR.

Materials and methods

This was a validity and diagnostic accuracy study conducted prospectively; the participants were initially evaluated using the HHD and then using the isokinetic dynamometer. Data collection was conducted at the Laboratory of Analysis of Human Movement of the Department of Physical Therapy of the Federal University of Ceará (UFC) from 2015 to 2017. The present data are reported as per the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) and Standards for Reporting Diagnostic Accuracy Studies (STARD) [24, 25].

Participants

Seventy-four consecutive participants (16–45 years old) who had undergone ACLR at least 6 months previously with a single-band technique with patellar or semitendinosus/gracilis graft and had completed rehabilitation or still in rehabilitation were recruited using the convenience sampling method from the university hospital; outpatient clinics; and orthopaedics, trauma, and sports clinics. Six months was taken into account as an intermediate period in the evaluation process in ACLR, considering that 3 months is very early and 9 months a safer time for discharge, since the period from the sixth to the ninth month is related to a sixfold higher rate of ACL re-injury [26,27,28], being the period chosen to observe the rehabilitation process with time for optimization in the path to discharge. Even so, based on empirical data, some authors still suggest returning to the sport approximately 6 months after surgery [29,30,31,32]. The rehabilitation process of the participants did not controlled; only the RTS criteria after ACLR were evaluated. Individuals with other associated ligament lesions, fracture, history of contralateral ACL injury, knee pain, impaired range of motion on knee flexion or extension, edema, or any other post-surgical complication that would disallow the performance of the tests were excluded.

Data collection

Initially, an evaluation form with questions about clinical and anthropometric characteristics was administered. Subsequently, the participants answered the following two questionnaires for sample characterization: the International Knee Documentation Committee (IKDC) and the Anterior Cruciate Ligament-Return to Sport after Injury Scale (ACL-RSI). The IKDC is a questionnaire comprising 10 items, ranging on a scale of 0–100, with 100 representing optima knee function. This questionnaire is widely used in individuals after ACL reconstruction to assess the symptoms, function, and sports activities [7, 33]. The ACL-RSI is a questionnaire with 12 items, developed according to the following three psychological responses identified as associated to RTS: emotions, confidence in sports performance, and re-injury risk assessment. The score ranges from 0 to 100. Both questionnaires were culturally adapted and validated for Brazilian Portuguese [34, 35]. Next, the isometric strength of the femoral quadriceps was evaluated using the HHD; thereafter, the evaluation was performed using the isokinetic dynamometer. The two evaluators in the study had 5 years’ experience in measuring the muscular strength of lower limbs using the HHD and the isokinetic dynamometer. The evaluators were blinded to the results obtained using of the isokinetic dynamometer (reference standard) and the HHD (index test).

Index test: hand-held dynamometer

For strength measurement using the HHD (Nicholas Manual Muscle Tester, Lafayette Instrument Company, Lafayette, Indiana, USA), the participant was positioned in the sitting position on the stretcher, with hands crossed over the trunk. Two stabilization belts were placed, one on the thighs to reduce compensations and the second at the ankle of the evaluated limb to maintain hip and knee flexion at 90° [17, 21, 36]. The HHD was positioned 2 cm proximal to the lateral malleolus midpoint [21, 36]. Maximum isometric strength was then requested for knee extension for 5 s wherein continuous verbal incentives were given. They performed two practice trials, rested for 30 s and then performed the two measure trials. The interval between consecutive measurements of the same limb was 30 s and that between the limbs was 60 s. In case of a difference > 10% between the repetitions in the same limb, the measurement was redone. The length of the lever arm was measured as the distance in metres from the knee joint and the HHD application point. The torque peak (Nm) of the quadriceps was measured by multiplying the force, acceleration of gravity and lever arm: [force (kg) × 9.81 × lever arm (m)]. Quadriceps torque (Nm) data were normalized by the body mass (kg) of each participant: [torque (Nm)/body mass (kg)) × 100. The limb symmetry index (LSI) was calculated according to the following formula: 100 − [(injured member/non-injured member) × 100].

Reference standard: isokinetic dynamometer

After 20 min of rest, all participants were evaluated using the isokinetic dynamometer (Biodex Multi-Joint System Pro, Biodex Medical System, Shirley, New York, USA) by a blinded evaluator in relation to the results of the index test. The positioning was performed as follows: when sitting in a chair, the popliteal fossa was positioned 2 cm from the end of the seat, the hip was positioned at 85° flexion, the axis of movement of the device was aligned with the intercondylar line of the knee and the lever arm held 2 cm above the lateral malleolus. Belts were placed to stabilize the trunk, abdomen, and thigh of the assessed limb. Thereafter, the limits of range of motion of maximum extension and flexion were established, the adequacy of the initial positioning of the knee at 90° flexion and weighing of the lower limb to be evaluated. The participant was registered in the system of the device, with information, such as dominance and injured limb. The protocol used involved five concentric repetitions with maximum intensity at 60°/s for force evaluation. Continuous verbal incentives were also given during the test [10, 37]. The isokinetic expresses its peak torque values and calculates the LSI, as described above for the HHD. The isokinetic dynamometer is considered a gold standard method for evaluating the muscle strength [19, 38] and is widely used as RTS criteria after ACLR [39].

For both the isokinetic dynamometer and the HHD, the cutoff point for considering the test as positive was asymmetries > 10% between the limbs because asymmetry values > 10% at the time of RTS are related to poorer performance and function in young active individuals with a higher risk of re-injury after ACLR [6, 40].

The study was approved by the Institutional Review Board (IRB) of Federal University of Ceará/PROPESQ (number 1.000.404), and all participants provided informed written consent before study participation.

Statistical analyses

Normality of the data was determined using the Kolmogorov–Smirnov test. Descriptive statistics (mean and standard deviation) were used to describe the anthropometric, clinical, and outcome variables.

Initially, the clinimetric properties of the HHD to evaluate patients who had undergone ACLR with respect to RTS were analysed. The test–retest reliability of the HHD to evaluate the femoral quadriceps strength in patients with ACLR was assessed using the degree of consistency for the ICC2,1, considering the following values: < 0.69 indicated weak reliability, 0.70–0.79 indicated reasonable reliability, 0.80–0.89 indicated good reliability, and 0.90–1.0 indicated excellent reliability [41]. In addition, the t test for paired samples was used to compare the average test and retest values.

Three agreement measures were used: the Bland and Altman graphs, for agreement analysis between the test–retest strength evaluation of the HHD [42]; the standard error of measurement (SEM), to verify the absolute error of the instrument; and the minimum detectable change (MDC), that reflects the smallest change considered significant, above the error of measurement of an individual [43]. The SEM was calculated by dividing the standard deviation (SD) of the mean differences between the two measurements by the square root of 1 minus the ICC (SD differences ÷ √1-ICC) and the MDC was calculated using the MDC formula = 1.96 × √2 × SEM. The agreement limits (LOA) were calculated as the SD of the differences between the evaluations and multiplied by 1.96. Both SEM and MDC are presented as percentages, dividing the SEM and MDC by the mean score.

The validity was analysed using the Pearson correlation coefficient to verify the relationship between the strength results measured with the HHD and isokinetic dynamometer, with values of coefficients established as follows: < 0.5 indicated weak validity, 0.5–0.75 indicated moderate to good validity, and > 0.75 indicated excellent validity [41].

The diagnostic accuracy of the HHD with the isokinetic dynamometer to identify quadriceps torque asymmetries after ACLR was verified using the calculation of sensitivity (SN), specificity (SP), positive likelihood ratio (+ LR), and negative likelihood ratio (− LR). For this, the participants were classified as true positives, false positives, false negatives, and true negatives on the index test.

Sensitivity was defined as the percentage of people with index test for non-RTS from among those with reference standard for non-RTS (presence of asymmetry > 10%). Specificity was the percentage of favourable RTS according to the tests included in the model among those who reported having returned to the sport at the same level. A + LR is the ratio of the true positive to the false positive patients (+ LR = SN/[1-SP]), and -LR is the ratio of the true negative to the false negative patients (− LR = [1 − SN]/SP). Analysed independently, the tests with high SN and low-LR are useful to exclude the deficit (screening), to find those with strength favourable to the RTS. While tests with high SP and high + LR are useful to confirm the non-RTS, to find those with strength no favourable to the RTS.

The area under curve (AUC) of receiving operator curves (ROC) was developed to determine the inherent ability of the test to discriminate between those who may and those who may not RTS. An AUC of 0.5 indicates no predictive power, while an AUC of 1 indicates perfect prediction [44].

A total 46 subjects were needed in order to detect a minimum correlation of 0.5 between isokinetic and hand-held quadriceps torque, with 5% of alpha and 95% of power by sample size calculations. Another 24 participants were included due to great demand [45].

Statistical analyses were performed using SPSS version 17.0 for Windows (Statistical Package for the Social Sciences Inc., Chicago, IL, USA).

Results

The characteristics of the 70 study participants are presented in Table 1. Four participants were excluded due to pain when performing HHD, with a numerical scale score for pain > 3. The flowchart of the study design is shown in Fig. 1.

Table 1 Characteristics of the participants (n = 70)
Full size table
Fig. 1
figure 1

Patient flowchart

Full size image

Reliability and validity

HHD presented excellent test–retest reliability for measuring the femoral quadriceps strength in patients with ACLR (ICC2,1 = 0.98, 95% CI 0.98–0.99). The LOA ranged from − 18.7 to 17.9, as shown in the Bland and Altman plot (Fig. 2). The SEM was 1.02 N m/kg (0.6%), and the MDC was 2.8 N m/kg (1.7%).

Fig. 2
figure 2

Bland and Altman plots (test–retest) for hand-held dynamometer

Full size image

The HHD presented moderate to good validity with the isokinetic dynamometer to evaluate the peak torque of the femoral quadriceps in patients with ACLR (r = 0.62, p < 0.001) (Fig. 3).

Fig. 3
figure 3

Validity between hand-held dynamometer with isokinetic dynamometer for quadriceps strength evaluation

Full size image

Diagnostic accuracy

Diagnostic accuracy of HHD and isokinetic dynamometer was accessed for asymmetry in the quadriceps muscle torque in the RTS after ACLR (Table 2), HHD exhibited a sensitivity of 63.4% (95% CI, 48.9%-76.3%), specificity of 100% (95% CI 81.4%–100%), as well as + LR infinity and − LR 0.3 (95% CI 0.2–0.5). The result of the AUC of the ROC was 0.835 (95% CI, 0.743–0.927; p < 0.001) (Fig. 4).

Table 2 Diagnostic accuracy of the hand-held dynamometer with the isokinetic dynamometer (gold standard)
Full size table
Fig. 4
figure 4

Area under the curve (AUC) of receiving operator curves (ROC)

Full size image

Discussion

The most important findings of our study were that the HHD proved to be valid for the evaluation of quadriceps and LSI, obtained excellent test–retest reliability and moderate to good correlation with the isokinetic dynamometer, and diagnostic accuracy with perfect specificity and + LR, capable of accurately identifying deficits of quadriceps strength > 10% between limbs.

The equipment is confirmed as a useful tool for identifying femoral quadriceps strength asymmetries in individuals with ACLR, indicating that the HHD has high diagnostic value for identifying deficits ≥ 10% [6, 40]. In addition to its relevant clinical utility, HHD is also cheaper and is easy to perform.

To our knowledge, the observation of the utility of HHD in a population that undergone ACLR was first verified through the present study. HHD has been an alternative method for the evaluation of muscular strength because other studies have reported results similar to ours, with good to excellent reliability [15, 21, 23, 36, 46, 47] and moderate to strong validity [13, 15, 17, 21,22,23, 36, 46, 47] when HDD was compared to the isokinetic dynamometer. Despite the heterogeneity among the evaluation protocols, such as the different criteria for data collection, number of repetitions and positioning during the test, as well as the use of different manual dynamometer models and reference standards [15, 21, 46], all studies aimed to use the HHD as a cheaper and more practical measure for assessing the lower limbs.

There are contradictory reports regarding the ability of HHD to detect the differences between the limbs [47, 48] and in the evaluation of strength in each quadriceps [14]. Deones et al [47] and Reinking et al [48] observed the use of HHD in a population with knee dysfunctions, such as patellofemoral pain, patellar subluxation, patellar tendinopathy, and meniscal lesions and found no significant differences in the symmetry between the limbs. However, their sample sizes were small (21 and 23 subjects, respectively) with non-standardized assessments for reducing the bias risk of the test. Our study used a methodology that minimized the bias risks by standardizing the stabilization and reducing the possible discomfort while performing the test, measures taken to avoid compensation and reduce the effect of external influences; similar reasoning was also present in previous studies that have shown significant results, similar to ours [17, 21].

The study by Kim et al. [14] performed on healthy women found large variations between the correlations for the quadriceps strength. The most plausible explanation for this imprecision would be that the possible fatigue caused by the high demand of the evaluation protocol, in addition to the possible contamination of the tests results wherein it used the force of the evaluator to support the torque of the quadriceps femoris, altered the study results, resulting in substantial variation in the correlation.

When compared with the results reported by Sinacore et al. [49] the identification of asymmetries was superior in the findings of the present study. This was attributable to the fact that Sinacore et al. [49] did not use the gold standard as a comparative measure and there was a tendency of the study population to have a high rate of symmetrical quadriceps with respect to the force.

To our knowledge, this is the first study to validate the use of the HHD for individuals who have undergone ACLR, but the study has certain limitations: the non-evaluation of the knee flexors, owing to the fact that it was not possible to observe the agonist/antagonist ratio; logistics involving the use of HHD because there persists some disagreement about the optimal time of its use as a reference for the criteria of RTS in the evaluation between the limbs [50]. Further, the postoperative rehabilitation control was not performed; the type of graft was not considered for symmetry evaluation, since the postoperative rehabilitation is independent of the type of graft [51, 52]; the inter-rater reliability was not assessed; and it was not determined whether the greatest HHD deficit was related to an increased injury risk. The influence of quadriceps asymmetry assessed by HHD with functional capacity and risk of re-injury after ACLR should be the focus in future studies.

Conclusions

HHD is a valid and reliable evaluation method in individuals who have undergone ACLR. In addition, this method is easy-to-use and cheaper than the isokinetic dynamometer and can be used as the first choice for identifying quadriceps strength asymmetries in patients undergoing ACLR, presenting an effective tool for the analysis of the ability to assess quadriceps extension torque and LSI with high specificity for identifying deficits > 10%, measures that make up the criteria for safe rehabilitation progression and RTS.