Table 3 shows the matrix of main factors, obtained by the varimax rotation method and the Kaiser normalization, where the factor weights of the different items in each factor are indicated with reference to the commonalities. In the factor analysis, no item was excluded (all presented a correlation value of higher than 0.30). As a result, it was concluded that the 3 factors are sufficient to describe the underlying structure of the data.
After performing the rotation and before a satisfactory factor solution, meanings were assigned to the factors, taking into account the theoretical reference. Thus, the dimensions that make up the scale were named: assessment of the quality of care; evaluation of the processes; and control systems.
“Assessment of the quality of care” was the first dimension obtained. It consists of 6 items that refer to quality assessment methods, namely, construction of indicators, health gains, satisfaction assessment, complications, and return to the operating room. This first factor explains 53.33% of the total variance.
The second dimension, “evaluation of the processes,” is composed of 4 items that explain 7.28% of the variance. These items refer to the work processes, namely, the evaluation of waiting times, teamwork, and the profitability of the operating room.
“Control systems” was the last dimension obtained. Composed of 3 items, it explains 6.93% of the variance. This dimension is composed of items that refer to the importance of audits to monitor the results and the issues related to the performance monitoring of the professionals working in the operating room.
The matrix of Spearman correlations between the factors is shown in Table 4. All correlations are significant and positive, with a strong correlation between the factors of “assessment of the quality of care” and “evaluation of processes.”
To evaluate the quality of the factor model, we observed differences between the simple correlations and the correlations estimated by the factorial model with the 3 factors retained. There are 26 residues (33%) with an absolute value greater than 0.05, which indicates a good quality of the adjustment. In addition, the adjustment quality index or goodness-of-fit index is 0.913, which indicates a good quality of adjustment.
Finally, internal consistency was assessed using Cronbach's α coefficient. The value of the entire scale was 0.925, which is a very high value and shows a very strong internal consistency of the instrument. The Cronbach's α for each dimensions is very good (assessment of the quality of care: 0.901; evaluation of the processes: 0.819; control systems: 0.747).
The operating room is a complex environment that comprises multiple interactions, where it is completely unacceptable if performance does not approach perfection and therefore needs metrics to evaluate its efficiency.1,4,10–12 In the present study, the RAQBO scale, composed of 13 items, was validated to measure quality results in the operating room as perceived by a population of Portuguese health professionals. The value obtained from the KMO measure was 0.94, which is considered excellent.
As for the study of internal consistency through Cronbach's α coefficient, the value was excellent (0.925). Cronbach's α is indicated to calculate the internal consistency of a measuring instrument in the case that Likert or multiple-choice scales are adopted and whose categories have an ascending or descending order of values.13 Cronbach's α reflects the degree of covariance among the items in a scale; values greater than 0.7 are ideal.9,14,15 From the exploratory factor analysis with varimax rotation, the 13 items of the scale were grouped into 3 dimensions that account for 67.55% of the total variance.
The dimension “assessment of the quality of care,” consisting of 6 items, refers to the definition of some indicators and their importance. One of the main challenges in the development of an instrument for measuring and monitoring the performance of the operating room is to define which indicators should be included.4 The balance between cost reduction and efficiency ensures patient safety is a continuous challenge, and it is important to develop strategies that reduce inefficiencies and provide personalized metrics in real time.16 Monitoring through indicators presents benefits to the respective institutions, measuring the quality of care at the time of surgery, the shorter length of stay for specific procedures, and better patient perception of quality of care.17,18 However, Fixler and Wright4 contested some of these qualitative measures, as in the case of satisfaction assessment, since most of them are not validated.
The dimension “evaluation of the processes” integrates indicators of process evaluation that refer, as the name indicates, to the monitoring of the processes of the operating room, among them the waiting times between the surgical procedures. Some authors are more specific in relation to these metrics, namely, the number of cases cancelled, the average time of rotation, the average time of occupation, the time of beginning of the first case, the mean time of induction, the mean time of extubation, and the mean time until recovery, among others.16
Another topic addressed in this dimension is the monitoring of teamwork. Effective teamwork is critical to the quality of patient care and safety in the operating room. Teamwork behaviors below the ideal often cause adverse events or near misses.19,20 Monitoring the dynamics of the team in the operating room, the way the teams function, and the factors that facilitate or impede teamwork are essential for obtaining high-quality care.20
The last dimension addresses issues related to “control systems,” integrating audits and performance evaluation. Surgical audits were developed with the objective of evaluating and facilitating quality improvement. Audit is a quality assessment method that collects detailed clinical data that are used to improve the quality of care through feedback on its results, as well as to facilitate benchmarking among participating hospitals.21 These audits make it possible to evaluate the efficiency of the resources used and to identify areas for improvement.17
Despite its important social and economic positions, the health industry has lagged behind in the availability of key data on the process and outcomes of care.21 The assessment of care costs and their outcomes should be a priority, and this will only be possible through the incorporation of very precise metrics.
This effort in the development and validation of an instrument to measure the Quality of Care Results in the Operating Room (RAQBO scale) integrates a set of essential perceived performance indicators to measure, monitor, evaluate, and compare performance and efficiency.
The scale comprising 13 items is easy to apply, has good conceptual properties, and presents good reliability and validity. The instrument has 3 dimensions, which are as follows: quality of care assessment; process evaluation; and control systems; that allow us to evaluate the results in their entirety.
The indicators under evaluation provide managers with an assessment tool to ensure the safety and quality of care. It also allows identifying areas in need of improvement and establishing corrective measures, constituting a valuable tool for use in a hospital environment or in research to replicate in future studies. As a limitation of this instrument, we consider the fact that it is restricted to the questions of the result, as well as being tested in the Portuguese context.
1. Rothstein D, Raval M. Operating room
efficiency. Semin Pediatr Surg. 2018;27(2):79–85. doi:10.1053/j.sempedsurg.2018.02.004.
2. Sartini M, Spagnolo A, Panatto D, Perdelli F, Cristina M. Improving environmental quality in an operating room
: clinical outcomes and economic implications. J Prev Med Hyg. 2013;54(2):75–79.
3. Gomes JA, Martins M, Tronchin D, Fernandes CS. A técnica de grupo focal na validação de conteúdo para avaliação da qualidade assistencial em bloco operatório. Braz J Surg Clin Res. 2017;21(2):88–93.
4. Fixler T, Wright J. Identification and use of operating room
efficiency indicators: the problem of definition. Can J Surg. 2013;56(4):224–226. doi:10.1503/cjs.020712.
5. Lin Q, Liu L, Liu H, Wang D. Integrating hierarchical balanced scorecard with fuzzy linguistic for evaluating operating room
performance in hospitals. Expert Syst Appl. 2013;40(6):1917–1924. doi:10.1016/j.eswa.2012.10.007.
6. Wu Q, Huang L, Xing M, et al Establishing nursing-sensitive quality indicators for the operating room
: a cross-sectional Delphi survey conducted in China. Aust Crit Care. 2017;30(1):44–52. doi:10.1016/j.aucc.2016.04.003.
7. Childers C, Maggard-Gibbons M. Understanding costs of care in the operating room
. JAMA Surg. 2018;153(4):e176233. doi:10.1001/jamasurg.2017.6233.
8. Gomes JA, Martins M, Fernandes CS. Instrumentos para avaliar a qualidade e segurança no bloco operatório—revisão integrativa. Cogitare Enferm. 2016;21(5). doi:10.5380/ce.v21i5.45640.
9. Mokkink LB, Terwee CB, Patrick DL, et al The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19:539. doi:10.1007/s11136-010-9606-8.
10. Tanaka M, Lee J, Ikai H, Imanaka Y. Development of efficiency indicators of operating room
management for multi-institutional comparisons. J Eval Clin Pract. 2013;19(2):335–341. doi:10.1111/j.1365-2753.2012.01829.x.
11. Weerakkody RA, Cheshire N, Riga C, et al Surgical technology and operating-room safety failures: a systematic review of quantitative studies. BMJ Qual Saf. 2013;22(9):710–718. doi:10.1136/bmjqs-2012-001778.
12. Civil I. Operating room
culture affects patient outcomes, and we should operate accordingly. ANZ J Surg. 2018;88(4):264–265. doi:10.1111/ans.14390.
13. Echevarría-Guanilo ME, Gonçalves N, Romanoski PJ. Propriedades Psicométricas de instrumentos de medidas: bases conceituais e métodos de avaliação. Texto Contexto—Enferm. 2017;26(4):e1600017. doi:10.1590/0104-07072017001600017.
14. Souza AC, Alexandre NC, Guirardello E. Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade. Epidemiol Serv Saúde. 2017;26(3):649–659. doi:10.5123/s1679-49742017000300022.
15. Cunha CC, Netob OP, Stackflethc P. Principais métodos de avaliação psicométrica da confiabilidade de instrumentos de medida. Rev Aten Saúde. 2016;14(49):98–103. doi:10.13037/rbcs.vol14n49.3671.
16. Gabriel R, Gimlich R, Ehrenfeld J, Urman R. Operating room
metrics score card-creating a prototype for individualized feedback. J Med Syst. 2014;38(11):144. doi:10.1007/s10916-014-0144-8.
17. Perkins J, Chiang T, Ruiz A, Prager J. Auditing of operating room
times: a quality improvement project. Int J Pediatr Otorhinolaryngol. 2014;78(5):782–786. doi:10.1016/j.ijporl.2014.02.010.
18. Cardoen B, Demeulemeester E, Beliën J. Operating room
planning and scheduling: a literature review. Eur J Oper Res. 2010;201(3):921–932. doi:10.1016/j.ejor.2009.04.011.
19. Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room
? A systematic review. Ann Surg. 2013;258(6):856–871. doi:10.1097/SLA.0000000000000206.
20. Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N. Assessment of stress and teamwork in the operating room
: an exploratory study. Am J Surg. 2011;201(1):24–30. doi:10.1016/j.amjsurg.2010.07.039.
21. Govaert J, van Bommel A, van Dijk W, van Leersum N, Tollenaar R, Wouters M. Reducing healthcare costs facilitated by surgical auditing: a systematic review. World J Surg. 2015;39(7):1672–1680. doi:10.1007/s00268-015-3005-9.
Keywords:© 2019Wolters Kluwer Health | Lippincott Williams & Wilkins
health care quality assurance; surgical procedures; process and results evaluation; operating room