The Safety Attitudes Questionnaire (SAQ) is a self-administered survey that measures six safety attitude constructs. The performance of the ambulatory version of the survey (SAQ-A) has not been evaluated in primary care offices. Objectives: The objectives of this project were to (1) test the internal consistency-reliability of the SAQ-A in primary care offices, and (2) develop a cybernetic model to help clarify the terms, culture, climate, and attitude. Methods: Internal consistency-reliability for each safety attitude construct was estimated using Cronbach’s alpha. A literature review informed the development of the cybernetic model. Results: For all respondents combined, Cronbach’s alphas for the six safety attitude constructs ranged from 0.58 to 0.77. The lowest alphas were for perceptions of management (especially for nursing staff) and working conditions (especially for administrative staff). Conclusion: The instrument appears to have good overall consistency-reliability in primary care offices, but it performed poorly with some subgroups. Further work is needed to evaluate and refine the SAQ-A for use in primary care settings.

Introduction

Of the various strategies available for improving patient safety, creation of a culture of safety is widely considered to be the most effective and sustainable.^{1, 2} This approach has been embraced by the National Quality Forum,³ and The Joint Commission has appropriately included an annual assessment of safety culture in its 2007 Patient Safety Goals.⁴

The term “safety culture” made its first appearance in the literature in 1987. In safety-critical industries, such as health care, safety culture is the prime facet of overall organizational culture. Definitions of organizational and safety culture abound in the literature.^{5, 6} These constructs aspire to help analytic reasoning and practical research. Taking a holistic view, we accept the United Kingdom Health and Safety Commission’s 1993 definition of safety culture. The safety culture of an organization is …

“…the product of individual and group values, attitudes, competencies, and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation’s health and safety programmes. Organisations with a positive safety culture are characterised by communications founded on mutual trust, by shared perceptions of the importance of safety, and by confidence in the efficacy of preventative measures.”⁷

The causes and effects of an organization’s safety culture are intertwined. A highly reliable organization has a safety culture and is a safety culture, wherein the objective of cultivating this culture is to continuously enhance safety, advisedly, with self-empowered and motivated teams.⁸

Our functional implicit definition is adopted from Guldenmund.⁴ Accordingly, we define safety culture as, “those aspects of the practice’s culture that have impact on attitudes and behaviors related to enhancing patient safety” It is also helpful to acknowledge and treat each ambulatory practice (organization) as a unique and complex adaptive microsystem.⁹ In the paradigm of complex adaptive systems, a culture of safety not only functions as a conceptual model but also as a central attractor, bringing order to disorder (i.e., reliability where there was risk).¹⁰ It creates stability and continuity, reducing anxiety and performance variation in its members.⁴

The term “safety culture” is often used interchangeably with “safety climate” and occasionally with “attitudes” We believe this is unfortunate and misleading; important and useful distinctions can be made between these concepts. In broad terms, climate can be seen as the observable/measurable part of culture. Safety attitudes, in turn, are a subset of safety climate; they are the part of the climate that resides in individuals and, therefore, can be most readily and conveniently measured via surveys.

Figure 1 depicts a framework that attempts to clarify the contributors to safety culture and the relationship of culture to climate and attitudes. The figure displays eight factors that, in our view, contribute to a safety culture.⁷ Although the relative importance of these contributors (and potentially others not identified in this framework) is not well understood at this time, it is important to note that each of these contributors interacts with and influences every other contributor (see narrow arrows); they work synergistically to create a culture of safety. This culture is complex and hyperdimensional and is the result of complex interactions between multiple players and their attitudes, beliefs, and behaviors over time.

Figure 1

Framework of interactive contributors to the construct of culture of patient safety: Manifesting as safety climate, which expresses itself (partly) in measurable attitudes and perceptions with numerous cybernetic loops with the culture.

Some aspects of safety culture are not observable or measurable because they are subconscious or rarely manifest themselves. The manifest, or observable, aspects of safety culture are referred to as the safety climate and are in a cybernetic loop with overall safety culture (via the wide arrows). Cybernetics can be defined in many different ways. A recent definition, attributed to Louis Kauffman, president of the American Society of Cybernetics, is “the study of systems and processes that interact with themselves and produce themselves from themselves.” It refers to the complex interactions of goals, predictions, actions, feedback, and response within systems.¹¹ In this context, climate is seen as a primary manifestation of culture which, in turn, influences and nourishes culture.^{12, 13, 14, 15, 16}

However, even climate is difficult to measure because, like culture, it exists largely not in individuals but in the interactions among them. Anthropologists use complex and labor-intensive methods, such as participant observation, to try to characterize climate qualitatively. These methods, while informative, are not suitable for widescale adoption outside of the research context.

Self-administered questionnaires have been developed in a variety of industries as a practical and convenient means of measuring quantitatively some of the important aspects of safety climate. These surveys—whether referred to as safety attitude, safety culture, or safety climate questionnaires—can examine only those aspects of climate that are quantifiable and expressible by individuals. In addition to perceptions about individual and group behaviors, these include individual attitudes and beliefs.

It is the apparent ability to quantify safety climate or culture, albeit in a limited way, that has driven the development of self-administered safety attitudes questionnaires over the last 30 or more years for the expressed purposes of measurement, description, diagnosis, and design of interventions for safety. These measures should be seen in light of the fact that the mere process of measurement influences the measured (i.e., there is a cybernetic loop here also).

In attempting to describe safety climate quantitatively, a large number of variables can be identified. A number of these measurable variables are interrelated and measure aspects of the same underlying dimension of safety climate. A number of these dimensions, in turn, capture different aspects of the same underlying (unobservable) latent factor,¹⁶ domain,¹⁷ or group/category.¹⁸ It is, therefore, possible to reduce/transform, successively, these variables to manageable dimensions and factors. This transformation is usually done by using factor analyses.^{16, 17, 19, 20, 21, 22} Capturing the climate in terms of these factors helps provide a clearer view of climate changes within, and variations between, different health care settings.

The Safety Attitudes Questionnaire (SAQ)¹⁶ is a 60-item, self-administered survey tool that was derived from a questionnaire used in commercial aviation, namely, the Flight Management Attitudes Questionnaire.²³ In a 2005 study comparing published health care safety attitude/climate/culture surveys, the SAQ appeared to be the most robust psychometrically.²⁴ The SAQ has been successfully used in inpatient and ambulatory clinics.^{16, 25} It elicits attitudes through the following six scales (or factors): (1) teamwork climate, (2) safety climate, (3) job satisfaction, (4) perceptions of management, (5) working conditions, and (6) stress recognition. These scales (encompassing 30 of the 60 questions in the SAQ) were developed through multilevel factor analysis using data from 10,843 respondents from 203 clinical areas in three countries (United States, United Kingdom, and New Zealand). The 203 clinical areas included 179 intensive care units (ICUs), 11 inpatient settings, 11 ambulatory clinics, and 2 operating rooms.

A further study, using the ambulatory version of the SAQ (SAQ-A) in a single large multispecialty academic outpatient practice (282 respondents), demonstrated good internal consistency-reliability for the same six factors (Cronbach’s alphas ranging from 0.68 to 0.86).¹⁵ Although the outpatient care environment is very different from the intensive care hospital settings where the factors were primarily developed, the same six factors appeared to be robust in this setting. This may be due, in part, to the fact that this study was conducted in a large academic practice that in some ways is organizationally similar to a hospital, with a centralized administrative infrastructure.

The typical primary care outpatient setting, where the majority of outpatient care is provided, is very different from this. For example, the organizational structure is typically flatter, roles are sometimes less clearly defined (with more cross-coverage), and relationships among staff are usually closer. These and other differences might have significant effects on the performance of the SAQ-A in this type of setting. A recent study that attempted to identify dimensions of safety culture, specifically in primary care settings, listed dimensions that overlap considerably with those shown in Figure 1 and with the SAQ-A.²⁶ The goal of the work presented here is to test the internal consistency-reliability of the SAQ-A in primary care offices.

Methods

Data Collection

The SAQ-A was administered voluntarily and anonymously to all eligible staff at eight primary care offices within the Upstate New York Practice-Based Research Network. To be eligible, staff had to have worked at the office (full- or part-time) for at least one month prior to survey administration. Table 1 shows the characteristics of the practices, which ranged from a rural solo practice to an urban academic residency practice site.

Table 1

Characteristics of the eight primary care study sites.

The main part of the survey consisted of a series of statements that respondents rated according to a 5-point Likert scale (1 = disagree strongly; 2 = disagree slightly; 3 = neutral; 4 = agree slightly; and 5 = agree strongly). Respondents also could indicate that an item was “not applicable.” The survey took approximately 10 to 15 minutes to complete. Most surveys were distributed in person at brief informational meetings accompanied by a concise explanation of the purpose of the survey, instructions for completion, and assurances of anonymity. For those employees unable to attend the informational session, materials were left with brief written instructions. To help maintain anonymity and confidentiality, participants were instructed to refrain from placing any identifying information on the survey; a secure drop-box was provided for completed questionnaires. Surveys returned within 2 weeks were included in the analysis.

Results

A total of 160 questionnaires were returned. Response rates for each site are shown in Table 1. The overall response rate was 63 percent. Table 2 summarizes the results for all eight practices, by factor. Analysis of variance (ANOVA) showed significant differences among sites (P <0.001) on all subscales except stress recognition.

Table 2

SAQ-A scores by site for each factor: Percent with positive attitude, mean, and range of scores.

For example, Site 5, an urban hospital-based clinic with 12 staff, appeared to perform well overall; on four of the six scales, greater than three-quarters of staff had a positive attitude. Site 7 performed similarly well. In contrast, at Site 3 (another urban hospital-based clinic), less than one-third of staff had a positive attitude on five of the scales. Interestingly, this site is much larger than the others and is also a residency practice site—two factors that might contribute to the difference.

Table 3 shows the same data arranged by respondent type for all practices combined. ANOVA found no significant differences among respondent groups on any subscale, both overall and in post hoc comparisons of all pairs of groups.

Table 3

SAQ-A scores by job category for each factor: Percent with positive attitude, mean, and range of scores.

Results of Cronbach analysis for each factor broken down by respondent type are shown in Table 4. For all respondents combined, alphas ranged from 0.58 to 0.77. Perceptions of management had the lowest alpha, while the other five factors all had alphas of ≥0.70, generally taken to indicate good internal consistency-reliability. Among nursing staff, perceptions of management again had the lowest alpha (0.40). Safety climate and stress recognition scales performed better but still had alphas <0.70. Among administrative staff, working conditions and stress recognition had the lowest alphas. Physicians and physician extenders produced alphas that were consistently good except in the area of perceptions of management, which was slightly low at 0.63.

Table 4

Cronbach’s alpha (n) for office staff by position: All sites.

Discussion

The SAQ-A proved to be practical and convenient to administer in a variety of primary care office settings. As expected, attitude scores varied considerably among sites, helping to identify each site’s strengths and weaknesses and highlighting the potential for this kind of instrument to be used as a means for driving climate/culture change in response to identified weaknesses.

Underlying reasons for the observed variations in safety attitudes are not well understood. Safety attitudes, as a reflection of culture, are intrinsically complex and unpredictable, resulting from the complex interactions among unique individuals and unique circumstances over time. However, there may be some measurable moderating factors, such as practice size and type, location, residency program affiliation, and others. Further study with a larger number of practices is suggested to continue to explore these potential relationships.

The data presented in this article are part of a larger study in which the authors presented SAQ-A results to each office’s staff (along with other data specifically related to medication safety) as a means of initiating discussions around change. Staff then worked together to design and implement change. Repeat measures of safety climate using the SAQ-A and correlations between safety attitudes and outcomes are underway and will be reported separately.

For SAQ-A results to be used meaningfully to drive change and monitor safety attitudes over time, the instrument must first demonstrate satisfactory psychometric properties. In prior studies, the SAQ has demonstrated good internal consistency-reliability for each of its six factors in multiple hospitals and in a large multispecialty ambulatory setting. In this study, in eight primary care offices, we found that overall; the six constructs were robust, with the possible exception of “perceptions of management.” Other than with administrative staff, this scale performed suboptimally. Given the different and varied management structures found in most primary care offices compared to hospital-based facilities, this is, perhaps, not surprising. In most of the sites in our study and in most primary care offices, management generally consists of one or two people (perhaps a practice manager and a medical director, at most) who work closely on a day-to-day basis with the rest of the staff and are not perceived as a separate or distinct department.

This contrasts with the hospital settings in which this scale was developed, where management was often a separate and sometimes amorphous group of people who were less intimately involved in the practice team. More recently, the authors of the SAQ have developed a version that differentiates among levels of management. This distinction might also prove helpful in primary care settings. Further work is needed to identify the most suitable measures for this construct or perhaps even to identify a more relevant alternative construct to capture management issues in primary care settings.

Interestingly, in earlier studies of the SAQ in other settings, administrative staff, in general, had low response rates across multiple factors and so were excluded from the factor analysis. In our primary care settings, the administrative staff category included receptionists, schedulers, referral coordinators, medical records staff, practice managers, and (importantly) staff who performed a variety of these functions. In primary care settings, it is highly desirable to include these staff as part of a safety climate assessment because they are typically integrated very closely into practice teams (more so than in other settings).

It is encouraging to note that other than for working conditions and stress recognition, greater than 75 percent of administrative staff responded fully to the questions that made up the various factors. The two factors that had low response rates (i.e., “working conditions,” 53 percent; and “stress recognition,” 57 percent) also had low alphas, although these should be interpreted with caution since they were based on small numbers.

The four factors that had good response rates all demonstrated satisfactory internal consistency-reliability for administrative staff (Table 4), even though this class of respondents was not included in the original analyses that identified these factors. Therefore, our findings support the inclusion of administrative staff when administering the SAQ-A in primary care, with the caveat that the “working conditions” and “stress recognition” scales might not apply in their current forms, in light of their low response rates. The missing responses might indicate that some of those questions are less relevant to the primary care setting.

Among nursing staff, response rates to all scales were good. However, there were suboptimal alphas for “safety climate” and “stress recognition,” in addition to the “perceptions of management” scale discussed earlier. These and other differences (discussed earlier) in the internal consistency among different groups of respondents listed in Table 4 raise the question of whether a single questionnaire can reliably capture the safety attitudes of a diverse group of respondents, or whether tailored questionnaires are needed for each subgroup. Our experience suggests that common elements in surveys are helpful. For example, when we share practice-specific results with staff (showing scores on each scale for each group of respondents), we have observed that staff find it helpful to note similarities and differences of opinion between themselves and other staff groups. It helps them build a common vision (where there are similarities) and prompts them to explore the reasons for differences that are revealed. Further work is needed to explore these issues.

Note that, in the SAQ-A used to date, only 30 of the 60 items are included in the six climate scales evaluated in this and prior studies; the remaining items are retained only because individual organizations may find the responses useful. Shortening the questionnaire to include only the items that make up the six climate scales could help improve compliance and might also influence (probably favorably) the psychometric properties.

The theoretical framework described in the introduction (Figure 1) includes several contributors to safety culture. Some of these contributors can be mapped to measurable attitudes, many of which are covered by the SAQ and SAQ-A. For example, creation of a learning environment maps very closely to the SAQ’s safety climate scale; creation of nonhierarchical teams corresponds to teamwork climate, job satisfaction, and stress recognition in the SAQ; and prioritization of safety by leadership is an area that is well addressed via the perceptions of management and working conditions scales. The framework reveals some areas that could be addressed further—such as design of the system for recovery and adoption of a proactive approach—that are not explicitly covered by the SAQ. These eight contributors manifest in complex multidimensional ways, some of which are either immeasurable or require methods of assessment other than self-administered surveys.

Conclusion

The SAQ-A holds promise as a convenient tool for assessing certain safety climate domains in primary care offices, with fairly good internal consistency-reliability across most of these domains. Weaknesses were found in the perceptions of management scale overall and in specific scales for nursing and administrative staff. Further study is warranted, preferably with a larger sample size, with the goal of developing a more robust instrument tailored to this setting.

It is hoped that the conceptual framework presented in Figure 1 can make a contribution to guiding further work in assessment of safety climate by various means, including qualitative and quantitative analyses.

Acknowledgments

Funding: The Agency for Healthcare Research and Quality (R21HS014867).

References

1.

Kohn LT, Corrigan JM, Donaldson MS, editors. Institute of Medicine. Washington, DC: National Academies Press; 1999. To err is human: Building a safer health system. [PubMed: 25077248]

2.

Sagan SD. The limits of safety organization, accidents, and nuclear weapons. Princeton, NJ: Princeton University Press; 1993.

3.

Safe practices for better health care: A consensus report. Washington, DC: National Quality Forum; 2003.

4.

2007 National patient safety goals. Oakbrook Terrace, IL: The Joint Commission; 2007. [Accessed April 14, 2008]. Available at: www​.jointcommission.org​/PatientSafety/NationalPatientSafetyGoals/07_npsgs.htm.

5.

Guldenmund FW. The nature of safety culture: A review of theory and research. Safety Sci. 2000;34:215–257.

6.

Cooper MD. Towards a model of safety culture. Safety Sci. 2000;36:111–136.

7.

Health & Safety Commission. 3^rd Report: Organising for safety. London: HMSO; 1993. ACSNI Study Group on Human Factors.

8.

Singh R, Singh A, Taylor JS, et al. Building learning practices with self-empowered teams for improving patient safety. J Health Manage. 2006;8:91–118.

9.

Plsek P. Crossing the quality chasm: A new health system for the 21^st century. Washington, DC: National Academies Press; 2001. [PubMed: 25057539]

10.

McDaniel RR Jr. Strategic leadership: A view from quantum and chaos theories. Health Care Manage Rev. 1997;22:21–37. [PubMed: 9058085]

11.

Wiener N. Cybernetics: Or control and communication in the animal and the machine. 2nd ed. Cambridge, MA: The MIT Press; 1965.

12.

Schein EH. Organizational culture and leadership. 2nd ed. San Francisco: Jossey-Bass; 1993.

13.

Glick WH. Conceptualizing and measuring organizational and psychological climate: Pitfalls in multilevel research. Acad Manage Rev. 1985;10:601–616.

14.

Keenan V, Kerr W, Sherman KW. Psychological climate and accidents in an automotive plant. J Appl Psychol. 1951;35:108–111.

15.

Hofstede GR. Cultures and organizations: Software of the mind. London: McGraw-Hill; 1991.

16.

Sexton JB, Helmreich RL, Neilands TB, et al. The Safety Attitudes Questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6:44. [PMC free article: PMC1481614] [PubMed: 16584553]

17.

Pronovost P, Sexton B. Assessing safety culture: Guidelines and recommendations [editorial] Qual Saf Health Care. 2005;14:231–233. [PMC free article: PMC1744052] [PubMed: 16076784]

18.

Singla AK, Barret K, Weissman JS, et al. Assessing patient safety culture: A review and synthesis of the measurement tool. J Patient Saf. 2006;2:105–115.

19.

Guttman L. A general nonmetric technique for finding the smallest coordinate space for a configuration of points. Psychometrika. 1968;33:469–506.

20.

Borg I, editor. Multidimensional data representation: When and why. Ann Arbor, MI: Mathesis Press; 1981.

21.

Williamson AA, Feyer A, Cairns D, et al. The development of a measure of safety climate: The role of safety perceptions and attitudes. Safety Sci. 1997;25:15–27.

22.

Vogus TJ, Sutcliffe KM. The safety organizing scale: Development and validation of a behavioral measure of safety culture in hospital nursing units. Med Care. 2007;45:46–54. [PubMed: 17279020]

23.

Helmreich RL, Merritt AC, Sherman PJ, et al. NASA UT/FAA Technical Report 93-4. Austin TX: The University of Texas; 1993. The flight management attitudes questionnaire (FMAQ).

24.

Colla JB, Bracken AC, Kinney LM, et al. Measuring patient safety climate: A review of surveys. Qual Saf Health Care. 2005;14:364–366. [PMC free article: PMC1744072] [PubMed: 16195571]

25.

Modak I, Sexton JB, Lux TR, et al. Measuring safety culture in the ambulatory setting: The Safety Attitudes Questionnaire – ambulatory version. J Gen Intern Med. 2007;22:1–5. [PMC free article: PMC2227589] [PubMed: 17351834]

26.

Kirk S, Parker D, Claridge T, et al. Patient safety culture in primary care: Developing a theoretical framework for practical use. Qual Saf Health Care. 2007;16:313–320. [PMC free article: PMC2464946] [PubMed: 17693682]