Original article
Reliability of the Sedation-Agitation Scale between nurses and doctors

https://doi.org/10.1016/j.iccn.2007.11.004Get rights and content

Summary

This study determined the inter-rater reliability of the Sedation-Agitation Scale (SAS) when used by staff in a tertiary level general intensive care unit (ICU). The study was designed to answer the question in the ‘real world’, with minimum patient exclusion criteria, do nurses and doctors rate ICU patient's sedation levels using the SAS similarly? A convenient sample of 35 nursing and seven medical staff and a randomly selected sample of 69 patients were used. A nurse and a doctor rated each patient simultaneously using the SAS, with a systematic five-stage arousal process. The results showed that there was exact agreement between the nurses’ and doctors’ scores in 74% of assessments. The weighted kappa finding of 0.82 indicates very good agreement (reliability). The mean SAS scores recorded for nurses (2.33 ± 1.21) and doctors (2.36 ± 1.35) were similar. Intraclass correlations for single measures (r = .921, p < .001) and average measures (r = .959, p < .001) indicated individuals who completed multiple ratings did not introduce bias. Where there was a difference between the paired ratings, these were only one level of the SAS away from each other. This research indicates nurses and doctors rate patients’ levels of sedation similarly using the SAS. It also provides support for the use of the instrument in general ICUs outside the USA. Research is now needed to determine the value of the SAS in guiding clinical decision-making related to sedation management.

Introduction

The management of sedation in critically ill patients has been recognised as a complex issue for intensive care units (ICUs) worldwide. Although non-pharmacological techniques (e.g. correct positioning, minimising noxious environmental stimuli, and re-establishing sleep cycles) are essential, they are often not enough to ensure patient comfort. Sedation is therefore an integral part of the management of many critically ill patients. Its use can minimise agitation, promote synchronised breathing with the ventilator, and reduce the anxiety and discomfort associated with the highly technological environment of the ICU. It has been estimated that 90% of critically ill patients require sedation and analgesia for at least part of their stay in an ICU (Devlin et al., 2001). Consequently notable complications of sedation practices, such as over sedation, have been identified (Cook et al., 1998, De Jonghe et al., 1998, Kollef et al., 1998, Kress et al., 2000, Meade et al., 1997) and systems to better manage sedation practices in ICUs are now being developed and utilised. These advances have resulted in a number of sedation scoring instruments being developed (De Jonghe et al., 2000). Many of these instruments however have not been established as reliable instruments for use clinically. This does not mean they are unreliable, only that formal reliability testing has not been carried out. One instrument which has reliability established in certain ICU contexts and with some health practitioners is the Sedation-Agitation Scale (SAS) (Brandl et al., 2001, Riker et al., 1994, Riker et al., 1999).

The SAS originated in, and has only undergone face validity and inter-rater reliability testing in the United States of America (USA) intensive care context (Brandl et al., 2001, Riker et al., 1999). Given there are differences in how ICUs are organised from those with a specialised patient group such as cardiac ICU to those catering for a generalist patient group it is unclear whether the USA reliability studies which were undertaken in specialist units are applicable to generalist units outside the USA. No published studies conducted outside the USA which confirmed its reliability were identified.

Section snippets

The Sedation-Agitation Scale

The SAS uses a seven-point scale (Table 1) to assess patient behaviour and define where the patient lies within the spectrum of sedation from unrousable to dangerously agitated. The scale is designed to be used by nursing and medical staff to inform the need for and level of sedation.

There are several potential limitations of the SAS. The SAS relies on an understanding of the English language. In the multicultural society of New Zealand it is common to come across patients with little

Method

The design was that of an inter-reliability study that distinguished between the raters by identifying them as nurses or doctors. In contrast, the earlier studies on the SAS considered how similarly people were rated without taking into account the category of the rater.

The study was undertaken in a 12-bed tertiary general ICU in New Zealand. The ICU covers most specialties including surgical, medical, paediatric, trauma, cardiothoracic, neurosurgical and renal intensive care. The study design

Samples

All staff except those who were agency and casual resource (pool) staff or who had worked in the unit for less than 6 weeks were eligible to participate in the study. All patients, except those under 16 years of age (defined in New Zealand as a child), those who were deaf or could not speak English, and those who had already had a rating assessment were eligible for the study. In addition, patients who had procedures performed on them (such as turns, airway suctioning, or line insertion) or had

Patient assessment

The nurse–doctor ratings were carried out simultaneously. There was no set pairing of nurses and doctors, the nurse–doctor pair was determined on the day by who was caring for the randomly selected patient. The raters were observed to ensure they did not communicate with each other during the ratings process and were asked not to discuss their ratings after. The rating process replicated Riker et al. (1999) (Table 2). Prior to ratings clinical data were collected from patient notes related to

Analysis

The weighted kappa statistic was used to examine inter-rater reliability. The weighted kappa estimates the proportion of staff agreement between two administrations of the SAS. Intraclass correlation (using a one-way random effects model) was undertaken to examine whether staff who completed multiple ratings biased results. Data were analysed using the Statistical Package for the Social Sciences (SPSS) versions 10 and 14 and Kw was calculated using software Analyse-it® for Microsoft Excel

Staff

Sixty percent of ICU staff, consisting of 48 nurses and seven doctors consented to participate in the study. Of these a convenient sample of 35 nursing and seven medical staff actually performed paired ratings on 69 randomly selected adult ICU patients. The study was completed over an 8-week period.

The mean number of years the nursing participants had worked in any ICU was 6.9 years (±5.0, range 1–18) and 31% had worked in any ICU for 3 years or less. Sixty-six percent said they held a

Discussion

The aim of this study was to determine whether nurses and doctors rate patients similarly using the SAS ratings. It established there was a high level of agreement that is reliability of the SAS by doctors and nurses. The study also provides confirmation of the applicability of the SAS outside the USA. In this research the weighted kappa score compares favourably with earlier studies. This study, performed in a natural setting, has reported a w score of 0.82. Nurses and doctors in the general

Conclusion

For a sedation scale to be useful in guiding therapy it should be able to reliably indicate the degree of sedation or agitation within defined categories, be simple to use, be mutually understood by all relevant clinicians and to be validated to the patient group. The SAS has all these features. This research has confirmed the SAS as an appropriate instrument for use in general ICUs and nurses and doctors using the SAS do provide consistent scores and have a mutual understanding of the SAS and

References (18)

  • M. Kollef et al.

    The use of continuous IV sedation is associated with prolongation of mechanical ventilation

    Chest

    (1998)
  • W.A. Knaus et al.

    The APACHE 11 prognostic system: risk prediction of hospital mortality for critically ill hosptialized adults

    Chest

    (1991)
  • Analyse-it®. Analyse-it®-the most popular statistical software add-in for Microsoft Excel (for windows). Retrieved...
  • D.G. Altman

    Practical statistics for medical research

    (1991)
  • P. Benner

    From novice to expert. Excellence and power in clinical nursing practice

    (1984)
  • K.M. Brandl et al.

    Confirming the reliability of the sedation-agitation scale administered by ICU nurses without experience in its use

    Pharmacotherapy

    (2001)
  • T. Bucknall

    Critical care nurses decision-making activities in the natural clinical setting

    J Clin Nurs

    (2000)
  • T. Bucknall

    The clinical landscape of critical care: nurses’ decision-making

    J Adv Nurs

    (2003)
  • D.J. Cook et al.

    Incidence of and risk factors for ventilator-associated pneumonia in critically ill patients

    Ann Intern Med

    (1998)
There are more references available in the full text version of this article.

Cited by (20)

  • Opioid-Induced Sedation and Respiratory Depression: Are Sedation Scales Enough to Prevent Adverse Drug Events Postoperatively?

    2020, Pain Management Nursing
    Citation Excerpt :

    A concern with these four scales is that the intention of the scales was to capture sedation in the intensive care setting, where intentional sedation is common practice and ventilator compliance is an outcome indicator. None of the studies regarding the Ramsay, RASS, SAS, or VICS examined pain management with opioids as a consideration (Ramoo et al., 2014; Rassin et al., 2007; Ryder-Lewis & Nelson, 2007; Sessler et al., 2002; Varndellet al., 2014). The POSS has been found to be most commonly adopted as the scale of choice within the current literature; however, its reliability, although significant at α = .903, has only been tested in one study (Nisbet & Mooney-Cotter, 2009).

  • The validity, reliability, responsiveness and applicability of observation sedation-scoring instruments for use with adult patients in the emergency department: A systematic literature review

    2015, Australasian Emergency Nursing Journal
    Citation Excerpt :

    A similar relationship was demonstrated by Deogaonkar et al.58 when comparing RASS and SAS to BIS (r2 = .81 vs. r2 = .725, respectively). The RASS demonstrated greater (κw = .91)59 inter-rater reliability between clinical staff compared to RSS (κw = .87)55 and SAS (κw = .82).77 Further, the RASS showed high (>8.08/10) levels of reliance and ease of use in scoring and communicating (7.72/10) sedation, agitation and intuitiveness, compared to the RSS, MAAS and SAS.44

  • A Review of Sedation Scales for the Cardiac Catheterization Laboratory

    2014, Journal of Perianesthesia Nursing
    Citation Excerpt :

    For this reason, the ATICE was selected for in-depth review instead of the COMFORT scale. The Richmond Agitation Sedation Scale (RASS),3,31-38 Bloomsbury Sedation Score,39,40 Harris Scale,41,42 Luer Sedation Scale,43,44 Motor Activity Assessment Scale,45-48 New Sheffield Sedation Scale,49-51 Nursing Instrument for Communication of Sedation,52 and Sedation Agitation Scale53-63 all measure consciousness and agitation in one item with the deepest depression of consciousness rated at one end of the scale and the most severe ranking for agitation at the other. Of these, the RASS has been used and tested the most in the clinical setting.

View all citing articles on Scopus
1

Tel.: +64 4 463 6138; fax: +64 4 463 5442.

View full text