Validation of the Ottawa Ankle Rules at a Tertiary Teaching Hospital
Tharao MK1, Oroko P1, Abdulkarim A1, Saidi H1,2
Department of Surgery, Aga Khan University Hospital
School of Medicine, University of Nairobi
Correspondence to: Dr. Mark Tharao. P.O Box 30867-00100 Nairobi. Email: email@example.com
Background: Ankle joint and foot injuries are among the most common injuries seen at the accident and emergency (A&E) department of any hospital. Radiographs are ordered in over 95% of cases yet the prevalence of fractures is in the range of 15-20%. The Ottawa ankle rules have been designed to reduce the need for radiographs in these patients and associated healthcare costs. This study aimed to validate the Ottawa ankle rules within our local setting and assess the impact of introduction of the rules. Methods: This was a cross sectional study at the Aga Khan University Hospital A&E department and the orthopedic outpatient clinics. Consenting patients with ankle trauma were examined based on the criteria set out in the Ottawa rules and subsequently sent for radiographs to confirm the presence or absence of a fracture. Results: The study recruited 175 patients over a six month period. There were 27 fractures with an incidence of 15%. The decision rule had a sensitivity of 96.3% and specificity of 57.4%. The negative predictive value was 98.8%. Application of these rules showed a potential of reducing the requested radiographs by 46%. Conclusion: The results have shown that implementation of the rules will result in significant savings in cost, time and unnecessary radiation exposure.
Keywords: Ottawa Ankle Rules, Radiographs, Predictive Value, Healthcare Costs.
Ann Afr Surg. 2015; 12(2): 77-80.
The ankle joint complex is a mortise and tenon joint comprising the tibia and fibula proximally and the talus distally (1). The medial and lateral malleoli form the medial and lateral articular surfaces of the mortise respectively. The primary medial stabilizer is the deltoid ligament while lateral malleolus is supported by the anterior and posterior talofibular ligaments and the calcaneofibular ligament. The mortise of the ankle joint is completed by the distal tibiofibular syndesmosis (2).
Injuries to the ankle joint and mid-foot are common. It is estimated that about six million ankle radiographs are done each year in the United States (3). Despite all these radiographs being done, studies have shown that only 15% of patients sustain any fractures of the ankle or mid-foot after injury (4-6). This exposes the need for more efficient utilization of resources while maintaining effectiveness of care. The development of clinical practice guidelines to assist in clinical decision making (7, 8) has grown from this need.
Stiell and his colleagues evaluated thirty two standardized clinical variables to determine reliability in clinically assessing ankle injuries. Based on this, they came up with the Ottawa ankle rules (9). The rules are designed to be used in assessing and predicting the possibility of fractures of the ankle and foot. In this way unnecessary radiographs of the ankle and foot can be avoided hence reducing the cost to the patient. Although the rules have been validated in certain countries in the world including France and Hong Kong their role in our set-up is yet to be determined (10,11). In addition, other studies done in Asia showed a lower sensitivity hence questioning the effectiveness of the rules in clinical practice (12). This study thus aimed to determine the accuracy and applicability of the Ottawa ankle rules within Aga Khan University Hospital (AKUH).
This was a cross sectional study at a 254-bed private, not-for-profit tertiary referral university hospital which also serves as a secondary level health facility for inhabitants of the city of Nairobi. The hospital has a 24 hour accident and emergency department (A&E), several specialist inpatient wards, outpatient clinics and a family medicine centre. There is also a well-equipped and fully staffed radiology department. There is no defined protocol for managing patients with ankle and foot injuries within the hospital. Each joint is inspected, palpated and moved for assessment of injuries as per the physician’s understanding of ankle and foot injuries. Majority of the patients would routinely get X-ray examinations to rule out fractures and then referred for follow-up in the orthopedic outpatient clinic. A few patients present at the clinic for their initial evaluation. The study population included all patients over 18 years with ankle and foot injuries seen at the A&E and the orthopedic outpatient clinics.
Excluded were all patients with injuries over seven days old, pregnancy, altered mental status at consultation, revisits, those with x rays before consultation, patients with other distracting major injuries and those with gross ankle deformity. The sample size calculated was 172 with the following set variables: fracture prevalence of 20%, target OAR sensitivity of 90%, precision of 10%, power set at 80% and P < 0.05. The physicians underwent two training sessions coordinated by authors MMK and PO. Here, the concept of the Ottawa ankle rules was explained and techniques of examining the ankle and foot in these patients demonstrated. Once an injured patient reported to the A&E, informed consent was sought and each patient was then assigned a study number filled in the questionnaire. The trained physicians administered the questionnaire asking specific questions as per the OAR specifications (Appendix 1). The physician then examined the patient by checking for specific points of tenderness and ability to bear weight as shown. The physician, after filling the questionnaire, indicated the possibility whether the patient has a fracture or not. The patient was then sent for a radiograph of the ankle and foot region to confirm the findings of the clinicians.
All radiographs were interpreted by qualified radiologists who were blinded to the contents of the data collection questionnaire and the patient clinical features. The radiological findings were then compared with the clinical diagnosis and the information fed into a database. After every thirty patients interviewed, five questionnaires were sampled randomly by taking every sixth questionnaire. They were inspected to ensure that the physicians adhered to the protocol as described.
All questionnaires had a unique identification number so as to avoid any confusion. The collected data were coded numerically to allow for data entry and statistical analysis. The data collected were then entered regularly into a database created using Microsoft Excel TM. Analysis was done using SPSS version 16.0.
The clinical variables described in appendix three were assessed separately by univariate analysis using X2 for nominal data. These included patient age, gender, mechanism of injury and number of days since injury.
The study also assessed for the sensitivity, specificity, positive predictive value and negative predictive value for using the Ottawa ankle rules in detecting ankle and mid-foot fractures. The cost reduction per patient was calculated and based on the number of patients seen over the study period, we calculated the total reduction in costs per year to patients seen at Aga Khan University Hospital when utilizing the Ottawa ankle rules.
A total of 175 patients were recruited into the study over a period of six months, from September 2012 to February 2013. Male patients comprised 56.6% of the study population with females making up 43.4%. The average age of recruited patients was 36 years with a minimum of 16 years and a maximum of 80 years. The average delay in presentation was 1.6 days. Based on the questionnaire, twisting of the ankle was the most common mechanism of injury accounting for 58.3% followed by road traffic accidents which accounted for 16.6% (Table 1).
Use of the Ottawa ankle rules in the study population resulted in positive identification of fractures in 26 patients. This translated to a sensitivity of 96.3% (CI 79.1-99.8%). Additionally, the rules were shown to have a specificity of 57.4% (CI 49.0-65.4%) (Table 2).
Based on the above figures, the study also revealed a negative predictive value of 98.8% (CI 92.7-99.9%) and a positive predictive value of 29.2% (CI 20.2-39.9%) (Table 3). In comparing the performance of individual Ottawa ankle rule criteria, the ability to bear weight on examination was found to have a significantly high specificity at 98.3% (CI 93.4-99.7%) in ruling out a fracture (Table 3).
This cross sectional study found that in the 175 patients who were examined using the Ottawa ankle rules, the incidence of fractures was 15.4% in keeping with other emergency departments as reported in literature (9, 13-16). The sensitivity of the rules was high at 96.3% with a lower specificity of 57.4%. The results confirm that the rules are applicable with implementation of the rules identifying all but one of the patients with fractures before performing the x-ray. Some previous studies that assessed the Ottawa ankle rules had found lower sensitivities; however these have been criticized as not implementing the original rules including a pictorial representation when collecting patient information (12,17). In this study the rule were used as originally described by Stiell and has reproduced findings similar to other such studies (10,11,18).
From the study findings, the rules have a high negative predictive value of 98.8% and a positive predictive value of 29.2%. Before introduction of the Ottawa ankle rules, over 95% of patients presenting with injuries had a radiograph performed. By introducing the rules there is a potential for reducing the number of radiograph by 46%. This rate is much higher than in most studies (10,13,16,19-21). A possible explanation for our higher rate might be our patient profile. AKUH is a private tertiary hospital where most patients are insured. Cost considerations may not deter request for radiograph. Further, some of the previous studies used much larger samples than the current study.
During the analysis of data, the study compared the individual components of the Ottawa ankle rules with the final results. The analysis revealed that the ability of the patient to bear weight and ambulate for four steps during examination had a high specificity of 98.3% (P<0.05). This single factor was able to rule out fractures in all but two of the cases. However the sensitivity was low at 44.6%. This finding replicates what has been found before in America and in Asia with specificities for weight bearing and four step ambulation ranging between 90-95% (12,21). At AKUH a single radiograph costs 2000 shillings ($23.50). This study has shown that with implementation of the Ottawa ankle rules there is a potential of reducing the number of x rays by 46% and the rules in this study were truly negative in 85 out of 175 patients involved. Based on these findings the implementation of the rule would result in a total saving of 170 000 shillings ($2 000) from this population alone.
This study only reviewed patients presenting to the accident and emergency department and the orthopedic outpatient clinic. This leaves out patients who are referred for radiographs for similar injuries by private physicians and Aga Khan satellite clinics. An analysis of the total radiographs done including other patients from the radiology department revealed there are a total of 210 radiographs done for isolated ankle injuries over a 6 month period reported as normal. This would mean a potential saving of 420 000 shillings ($4 900) over six months or 840 000 shillings ($9 800) per annum. Projected over many hospital and clinics in Kenya, reducing ankle radiographs by 40% could save tens of millions of healthcare shillings per year. Such similar cost reductions were observed in America and Canada (21,22).
The uptake of the Ottawa ankle rules within the study period showed that if implemented the rules would significantly reduce costs and time. Uptake of the rules has been challenged by some authors (3). However from this study, it was shown that with effective training physicians were able to fully implement the rule. This study was sufficiently powered to show that implementation of the Ottawa ankle rules will result in significant savings in cost, time and avoid unnecessary radiation exposure in patients with ankle injuries. This is especially beneficial in the resource limited setting environment in Kenya.
The Ottawa ankle rules have been validated within an urban teaching hospital in Kenya and introduction of the rules into daily practice will reduce costs, time and unnecessary radiation exposure to patients. The rules should be introduced as part of the management protocol in patients presenting with ankle injuries in the hospital.
Appendix: Ottawa Ankle Rules Rules for an Ankle Radiograph
Pain around the ankle plus one or more of the following:
a. Age 55 or greater
b. Inability to bear weight both immediately and in the emergency department and walk for four steps c. Bone tenderness at the tip or distal 6cm of either medial or lateral malleolus Rules for a Foot Radiograph
Pain around the foot region plus one of the following a. Bone tenderness at the navicular or cuboid
b. Bone tenderness at the base of the fifth metatarsal
Snell RS, editor. Clinical Anatomy by Regions. 8th ed: Lippincott Williams and Wilkins; 2007.
Bruce D, Browner JBJ, Allan M, et al. Skeletal Trauma. Philadelphia: Elsevier’s Health Sciences 2008.
Holroyd BR, Wilson D, Rowe BH, et al. Uptake of Validated Clinical Practice Guidelines: Experience with Implementing the Ottawa Ankle Rules. Am J Emerg Med. 2004;22(3):149-55.
Brooks SC PB, Rainey JB. Inversion Injuries of the Ankle: Clinical Assessment and Radiographic Review. Br Med J (Clin Res Ed). 1981;282:607-8.
Dunlop MG, Beattie TF, White GK, et al. Guidelines for Selective Radiological Assessment of Inversion Ankle Injuries. Br Med J (Clin Res Ed). 1986;293:603-5.
Lloyd S. Selective Radiographic Assessment of Acute Ankle Injuries in the Emergency Department: Barriers to Implementation. CMAJ.1986;135:973-4.
Brehaut JC1, Stiell IG, Visentin L, et al. Clinical Decision Rules in Everyday Practise. Acad EmergMed. 2005;12:948-57.
Patel VL, Arocha JF, Diermeier M, et al.. Methods of Cognitive Analysis to Support the Design and Evaluation of Biomedical Systems: The Case of Clinical Practice Guidelines. J Biomed Inform. 2001;34(1):52-66.
Stiell IG, Greenberg GH, McKnight RD, et al. A Study to Develop Clinical Decision Rules for the Use of Radiography in Acute Ankle Injuries. Ann Emerg Med. 1992;21(4):384-90.
Auleley GR, Kerboull L, Durieux P, et al. Validation of the Ottawa Ankle Rules in France: A Study in the Surgical Emergency Department of a Teaching Hospital. Ann Emerg Med. 1998;32:14-8.
Yuen MC, Sim SW, Lam HS, et al. Validation of the Ottawa Ankle Rules in a Hong Kong ED. Am J Emerg Med. 2001;19:429-32.
Tay SY, Thoo FL, Sitoh YY, et al. The Ottawa Ankle Rules in Asia: Validating A Clinical Decision Rule For Requesting X-Rays In Twisting Ankle and Foot Injuries. J Emerg Med. 1999;17(6):945-7.
Stiell IG, Greenberg GH, McKnight RD, et al. Decision Rules for the Use of Radiography in Acute Ankle Injuries: Refinement and Prospective Validation. JAMA. 1993;269:1127-32.
Stiell IG, McKnight RD, Greenberg GH, et al. Implementation of the Ottawa Ankle Rules. JAMA. 1994;271:827-32.
Stiell IG, Wells G, Laupacis A, et al. Multicentre Trial to Introduce the Ottawa Ankle Rules for Use of Radiography in Acute Ankle Injuries: Multicentre Ankle Rule Study Group. BMJ. 1995;311:594-7.
Verma S, Hamilton K, Hawkins H, et al. Clinical Application of the Ottawa Ankle Rules for the Use of Radiography in Acute Ankle Injuries: An Independent Site Assessment. AJR Am J Roentgenol. 1997;169:825-7.
Stiell IG, Greenberg GH, McKnight RD, et al. The “Real” Ottawa ankle Rules. Ann Emerg Med. 1996;27:103-4.
Broomhead A, Stuart P. Validation of the Ottawa Ankle Rules in Australia. Emerg Med (Fremantle). 2003;15:126-32.
McBride K. Validation of the Ottawa Ankle Rules. Experience at a Community Hospital. Can Fam Physician. 1997;43:459-65.
Lucchesi G, Jackson RE, Peacock WF, et al. Sensitivity of the Ottawa Rules. Ann Emerg Med. 1995;26:1-5.
Leddy J, Kesari A, Smolinski RJ. Implementation of the Ottawa Ankle Rule in a University Sports Medicine Center. Med Sci Sports Exerc. 2002;34(1):57-62.
Anis A, Stiell IG, Stewart BG, et al. Cost-effectiveness Analysis of the Ottawa Ankle Rules. Ann Emerg Med. 1995;26:422-8.