Skip Navigation LinksHome > August 2009 - Volume 84 - Issue 8 > Social Network Analysis as a Method of Assessing Institution...
Academic Medicine:
doi: 10.1097/ACM.0b013e3181ad16d3
Assessing Institutional Culture

Social Network Analysis as a Method of Assessing Institutional Culture: Three Case Studies

Lurie, Stephen J. MD, PhD; Fogg, Thomas T. MS; Dozier, Ann M. RN, PhD

Free Access
Article Outline
Collapse Box

Author Information

Dr. Lurie is director of assessment, Office of Curriculum and Assessment, University of Rochester School of Medicine and Dentistry, Rochester, New York.

Mr. Fogg is senior administrator, University of Rochester School of Medicine and Dentistry, Rochester, New York.

Dr. Dozier is associate professor in community and preventive medicine, University of Rochester School of Medicine and Dentistry, Rochester, New York, and lead evaluator of their Clinical and Translational Science Institute.

Correspondence should be addressed to Dr. Lurie, Office of Curriculum and Assessment, University of Rochester School of Medicine and Dentistry, 601 Elmwood Ave., Box 601, Rochester, NY 14642; telephone: (585) 273-4323; fax: (585) 273-3290; e-mail: (Stephen_Lurie@urmc.rochester.edu).

Collapse Box

Abstract

Purpose: To describe the basic concepts of social network analysis (SNA), which assesses the unique structure of interrelationships among individuals and programs, and introduce some applications of this technique in assessing aspects of institutional culture at a medical center.

Method: The authors applied SNA to three settings at their institution: team function in the intensive care unit, interdisciplinary composition of advisory committees for 53 federal career development awardees, and relationships between key function directors at an institution-wide Clinical Translational Sciences Institute (CTSI). (Key functions are the major administrative units of the CTSI.)

Results: In the ICU setting, SNA provides interpretable summaries of aspects of clinical team functioning. When applied to membership on mentorship committees, it allows for summary descriptions of the degree of interdisciplinarity of various clinical departments. Finally, when applied to relationships among leaders of an institution-wide research enterprise, it highlights potential problem areas in relationships among academic departments. In all cases, data collection is relatively rapid and simple, thereby allowing for the possibility of frequent repeated analyses over time.

Conclusions: SNA provides a useful and standardized set of tools for measuring important aspects of team function, interdisciplinarity, and organizational culture that may otherwise be difficult to measure in an objective way.

Medical research is becoming an increasingly interdisciplinary enterprise. Multidisciplinary teams, a key feature of the National Institutes of Health (NIH) Roadmap, are thought to be essential for effective translational research.1 Collaboration across disciplines has also been proposed as a necessary component of health services research and integration of medical research into primary care.2,3 Similarly, the Accreditation Council for Graduate Medical Education has highlighted the idea of “the competent institution,”4 in which multidisciplinary members of health care teams are aligned to provide the best possible care for each patient. Extramurally, institutions are increasingly finding that collaboration across geographical distances is also beneficial to patients.5

Nonetheless, one of the largest obstacles to encouraging team-based care is a culture that does not reward individual investigators for such collaboration,6 as well as a resulting lack of measures that assess collaboration. Although Holmes et al7 described several metrics for assessing research productivity, they cautioned that current measures may not capture important collaborative aspects of research activity.

Unlike traditional evaluation methodologies, which typically assess program effects by averaging and comparing outcomes across individuals or programs, social network analysis (SNA) assesses the unique structure of interrelationships among individuals or programs.8–10 In addition to providing measures of team cohesion, information flow, and degree of hierarchical organization, SNA can also identify each individual's degree of centrality and connectedness to other members. Beyond the graphic displays of networks, SNA also can compute measures of team cohesion, information flow, and degree of hierarchical organization both for the entire network and also for each of its constituent members.

SNA has been found to be an efficient method to describe dynamics of networks in settings ranging from medical offices,11 interracial friendship choice among teenagers,12 kinship patterns in Central America,13diffusion of medical innovation,14 and labor negotiations.15 Recent studies found that SNA could track the spread of obesity through social networks16 as well as patterns of communication among primary care physicians.17 SNA has been widely used in organizational settings to reveal patterns of communication that would not otherwise have been evident.9 The Agency for Healthcare Research and Quality recently described its intention to use SNA to study patterns of communication among researchers and, ultimately, to help improve patient safety.18

The basic data unit for SNA is a measure of the strength of the connection between all possible pairs of members of a defined network. In practice, each member of the network may be queried about his or her relationship to all other members. These connections may be self-reported, in the form of individuals' self-reported frequency of contact with one another or level of trust and knowledge about one another. Data may also be archival, such as rosters of shared committee memberships, key personnel on grant applications, coauthorship, or frequency of e-mail contact. Because most social networks are open (i.e., members of groups have contacts with other individuals who are not necessarily directly involved in the group, but who may influence group function), the evaluator must always make a deliberate decision about how to define the network for purposes of analysis. When using archival data, the network is limited to those individuals for whom data exist.

SNA is a largely visual technique that is supported by a number of quantitative indices of both group and individual function. The main output is the sociogram, which provides information about the number and strength of connections between members of a network. Each member of a network is depicted by a node, or vertex, in the sociogram. Although there are a number of algorithms for determining optimal placement and arrangement of the nodes, ultimately it is up to the analyst to decide how best to arrange these data points to make the interpretation most clear.

Furthermore, SNA can produce some basic descriptive statistics relating to the overall pattern of interactions among nodes in the network as a whole, as well as additional descriptive statistics that characterize each member of the network. These numbers generally serve as further aids to interpretation of the sociogram. Although it is possible to perform statistical tests to compare individuals in a network, or to compare networks with one another, the purpose of SNA is generally to characterize particular groups rather than to test inferential hypotheses. Thus, in the present report, we do not present P values or other aspects of statistical testing—rather, we present our results as a way of demonstrating how SNA can yield insights into structure and dynamics of specific groups.

Back to Top | Article Outline

Method and Results

In this section of our report, we describe three examples—using actual data from within a medical center—of how SNA may be used to understand network dynamics in institutions. In the first example, we describe patterns of team functioning in the intensive care unit. In the second, we describe relationships among academic departments as evidenced by memberships in advisory committees by federal career development awardees. Finally, we demonstrate how SNA can be used to describe patterns of relationships between the leadership of a large, institution-wide, translational-research enterprise.

Back to Top | Article Outline
Team function in the intensive care unit

As part of a pilot project, we used SNA to describe relationships among teams of caregivers in our medical intensive care unit (MICU). We began by observing the teams on rounds and then identifying all the individuals who were responsible for caring for an index patient. These caregivers could include attending physicians, residents, nurses, pharmacists, respiratory therapists, and medical students. We also identified the patient's key family member. We then created a team-specific survey that asked how often each team member interacted with all the other team members (and key family member) about the index patient. The family member was also given a copy of the survey to complete. We found that, under optimal conditions, it required less than an hour to identify the team, create the survey, and collect the data. Response options ranged from “never” to “three or more times per day.” A critical decision for the analysis is how to define a “connection.” A loose definition (e.g., once a day) risks creating a network with large numbers of relatively superficial connections and, thus, may make it difficult to discern a pattern of important relationships in the network. On the other hand, an overly strict definition may yield a network with too few connections to interpret. In practice, the investigator may look at networks produced by several definitions of connectedness and select the one that strikes the best balance. For purposes of this analysis, we assumed that a connection existed between team members if they said that they spoke at least twice a day with one another.

We present data from two teams for illustrative purposes. Team 1 (Figure 1) was caring for an elderly woman who had been recently admitted to the MICU for progressive dyspnea of unknown etiology. Team 2 (Figure 2) was caring for a patient who had been in the MICU for nearly two weeks with slowly improving respiratory status and who was about to be weaned from mechanical ventilation. In both networks, a connection was assumed to exist between individuals if they said that they interacted with one another at least twice per day. In these two figures, the individuals are represented by nodes. The gray shadings and degrees of gray distinguish individuals carrying out one role (e.g., the role of nurse) from those carrying out another role. The size of each individual's node is proportional to the number of other team members with whom that individual has at least twice-daily contact. With the exception of the pharmacist, the two teams had no individuals in common.

Figure 1
Figure 1
Image Tools
Figure 2
Figure 2
Image Tools

From a visual inspection, it is apparent that the teams are quite different in structure. Team 1 has a large number of connections between individuals of different roles. The attending physician has frequent interactions with more members of the team than does anyone else. The family has frequent contact with both nurses, both of whom have connections with several other members of the team, including the attending physician, the residents, and the pharmacist. With the exception of the respiratory therapist, all team members have contact with at least four other team members across various roles.

By contrast, Team 2 is characterized by a relatively small number of daily connections between team members. In this team, it is the nursing staff, rather than the medical staff, who seem to be central. As in Team 1, the patient's family has frequent contact only with the nursing staff. In this case, however, the patient's family reports they have contact only with a nurse who is relatively isolated from the rest of the team. Thus, this family has a relatively greater distance from the rest of the team than does the first patient's family.

It is unclear whether different team structures are more appropriate to some clinical situations than others. Although the relatively unconnected, nurse-centered structure of Team 2 might be more appropriate for stable patients with little day-to-day change in their clinical status, it remains unclear whether such a structure may nonetheless be more prone to errors in handoffs of information. Such differences in team structure might also affect family members' perceptions of the quality of care. SNA provides indices of team functioning that could be used as predictor variables in later studies of quality of care.

Back to Top | Article Outline
Departmental structure of research-mentoring advisory committees

We assembled lists of the advisory committees for 53 federal career development awardees at our institution. This represented all such awardees who were active at our institution at some point during the period 2004 through 2006, where the awardee continued to serve on our faculty at the beginning of 2007. A total of 157 faculty served as committee members. Such membership not only allows the awardees to benefit from a greater diversity of expertise, but also provides an opportunity for faculty from different departments and disciplines to meet and interact with one another. Furthermore, if an awardee's committee members sit on multiple other committees, this provides the awardee with greater opportunity to indirectly network with other awardees.

Such archival data contain large amounts of information, which would be challenging to interpret by simply examining all 53 mentorship-committee membership lists. SNA, however, can provide depictions of these data that are readily interpretable and quantifiable. In fact, this sort of committee-membership data permits examination of two different kinds of networks. First, the network of connections between awardees with one another via shared committee members, and second, the network of connections between faculty via shared committee membership. Both types of networks can be coded by departmental affiliation. This permits inferences about the degree to which different departments, as a whole, are associated with one another via their respective members' committee work. Important to understanding the relationships in this type of network is that the individuals are not connected directly to one another by individual choice but, rather, through their joint memberships in formally convened groups.

Figure 3 displays the network of awardees, with different degrees of shading or no shading to distinguish each academic department. Each circle represents an awardee. Each line in the diagram represents a shared committee member between pairs of awardees; thicker lines indicate more than one shared member. The size of the nodes is proportional to the total number of committee members that awardees share with other awardees. Note that this network does not contain information about the committee members' departmental affiliations.

Figure 3
Figure 3
Image Tools

Several general conclusions can be drawn from this diagram. First, awardees in departments 4 and 19 have the greatest diversity in their committee memberships, as evidenced by their awardees' committee members' frequently shared membership in other committees and generally larger circle sizes. Second, awardees in these departments frequently share committee members with awardees in other departments. By contrast, department 10 represents the opposite situation. Several awardees are not at all connected to the network, indicating that none of these individuals' committee members serve on any other committees. Furthermore, at least a few of these awardees share more or less the exact same committee, as indicated by the thick line connecting them. Thus, it seems that awardees in this department have committees comprising individuals who are largely chosen from within their own departments, and that many of these committee members did not have the experiences of serving on other committees during this time. Those that do seem to have more committee memberships were largely serving on other committees within their own departments. Finally, department 18 is heterogeneous, with some awardees' committees composed of individuals who serve on committees of several other departments, while other awardees have committees composed largely of individuals who do not have other committee experience.

Figure 4 complements Figure 3 by providing information about committee members' affiliations with one another. Because a network of 157 separate individuals is difficult to interpret, committee members have been collapsed within departments, which are represented by circles in the diagram. The thickness of lines between departments is proportional to the number of committees that shared members from that pair of departments. This view provides an overall summary of the number of times that members of each department jointly served on awardee committees.

Figure 4
Figure 4
Image Tools

These results mirror those of the awardee network. There are large numbers of faculty connections between departments 4 and 19. In general, department 19 seems to be the most well connected, with large numbers of frequent connections with several other departments. Furthermore, department 13, although having few connections with the rest of the network, brings together the relatively isolated departments 14 and 22, thus forming a small subnetwork within the larger network. This depiction not only provides an economical way of looking at all the committee memberships simultaneously but also suggests an inherent methodology for tracking efforts to improve interdisciplinarity; if such data were collected and analyzed yearly, this would provide an index by which institutional change could be measured. These data also help identify departments that may need some additional effort to become more interdisciplinary in their support of new researchers.

Back to Top | Article Outline
Leadership of a multidisciplinary research institute

As part of a multiyear NIH award to establish a Clinical Translational Sciences Institute (CTSI), our institution defined a series of 11 key functions (i.e., the major administrative units within the CTSI), each with its own directors (from one to four) and staff. The shared purpose of these key functions is to promote translational research by giving investigators access to institution-wide services such as funding opportunities for pilot studies and for the development of novel research methodologies; research support services such as study design consulting, informatics support, and a clinical research center; and education and training in the evolving disciplines of clinical and translational research.

As part of an initial evaluation of the structure of the CTSI, we asked all the key function directors to rate their current state of knowledge of the role of the other 10 key functions, as well as their anticipated frequency of interaction with the other key functions. We then averaged these responses, creating a mean level of knowledge and anticipated interaction of each possible pair of key functions. This allowed us to produce a customized report for each key function that described the strength of their relationships with each of the other ten key functions.

To produce a network diagram of the key function directors' levels of knowledge about one another's key functions (Figure 5), we assessed a number of possible cutoffs for the means mentioned in the preceding paragraph. An arrow was drawn between key functions if the mean response for that pair was greater than the cutoffs. Note that these connections are thus bidirectional; if directors of a pair of key functions both indicate a high level of knowledge or anticipated interaction with one another, then the connection between them is represented by a two-headed arrow. On the other hand, the relationship would be one-way if one key function gave a high response about the second, but if that response were not reciprocated.

Figure 5
Figure 5
Image Tools

In the figure, the size of the nodes is proportional to the number of incoming arrows (i.e., for the respective key function, the degree to which other key function directors report a high level of understanding of the role of that key function within the CTSI). Figure 6 represents the same data, but with the size of the nodes proportional to the number of outgoing arrows (i.e., the degree to which the directors of the respective key functions report that they understand the roles of the other key functions). From inspection of Figures 5 and 6, it is apparent that although key functions 2, 3, 4, 6, and 9 are well understood by many other key function directors, their own directors report relatively less understanding of the other key functions. By contrast, key functions 1 and 10 are relatively less well understood, although their directors report a relatively high level of understanding of the role of many other key functions. Furthermore, the diagrams also illustrate potential weaknesses in relationships between specific pairs of key functions. Thus, these results reveal specific gaps in knowledge that could be addressed early through targeted interventions.

Figure 6
Figure 6
Image Tools

Figure 7 presents data about key function directors' anticipated interactions with one another. The size of the nodes is proportional to each key function's betweenness, which is a measure of the degree to which each key function is on a pathway between all other possible pairs of key functions. Thus, those with high betweenness values are more critical to connecting other key functions that would otherwise have little contact with one another. Key functions 9 and 12 owe much of their high betweenness to the fact that they connect the larger network to the smaller subnetwork comprising key functions 2, 3, 6, and 8. Key function 6 is also prominent because of its exclusive connections with key functions 2 and 3. Finally, key function 11 also uniquely connects numerous other key functions, although this is not immediately apparent from simply inspecting the arrows between nodes. These analyses can thus be used to identify deficits and strengths in the network. If efforts are made to promote different interrelationships between key functions, these diagrams can serve as a baseline from which to assess institutional change.

Figure 7
Figure 7
Image Tools
Back to Top | Article Outline

Discussion and Conclusions

The examples presented in the previous section highlight some ways that the tools of SNA can shed light on a range of a medical center's organizational dynamics. In addition to these purely visual representations of data, SNA also permits computation of a range of numerical indices, both for individuals and for networks as a whole, and thus helps identify trends in network structure over time.

The results of SNA must be interpreted in light of other sources of information about organizational structure and function, which are typically more qualitative and nuanced. Nonetheless, with its standardized methodology and reporting formats, SNA provides a useful complement to such subjective impressions and provides standard metrics for assessment of change in organizational structure over time. Furthermore, SNA data are relatively simple and rapid to collect and analyze, thus offering an appealing complement to more labor intensive methods involving observers or structured interviews of network members. SNA is also relatively free of observer bias. Thus, SNA provides a promising addition to the set of tools currently used to assess and evaluate organizational structure.

Back to Top | Article Outline

Acknowledgments

The authors thank Tina McCoy, Nicole O'Dell, Kathleen Holt, and Nikki Murray for their help in data collection.

The research reported here was made possible by Grant Number UL1 RR024160 from the National Center for Research Resources (NCRR), a component of the NIH and the NIH Roadmap for Medical Research.

Back to Top | Article Outline

Disclaimer

This report's contents are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH. Information on NCRR is available at (http://www.ncrr.nih.gov). Information on Reengineering the Clinical Research Enterprise can be obtained from (http://nihroadmap.nih.gov/clinicalresearch/overview-translational.asp).

Back to Top | Article Outline

References

1 National Institutes of Health. NIH Roadmap for Medical Research. Research teams of the future. Available at: (http://nihroadmap.nih.gov/researchteams). Accessed April 20, 2009.

2 Carey TS, Howard DL, Goldmon M, et al. Developing effective interuniversity partnerships and community-based research to address health disparities. Acad Med. 2005;80:1039–1045.

3 Kindig DA, Dunham NC, Eisenberg JM. Needs and challenges for health services research at academic health centers. Acad Med. 1999;74:1193–1201.

4 Gibson R. The competent institution. ACGME Bull. April 2005:4–6. Available at: (http://www.acgme.org/acWebsite/bulletin/bulletin04_05.pdf). Accessed April 20, 2009.

5 Sostman HD, Forese LL, Boom ML, et al. Building a transcontinental affiliation: A new model for academic health centers. Acad Med. 2005;80:1046–1053.

6 Cohen JJ, Siegel EK. Academic medical centers and medical research: The challenges ahead. JAMA. 2005;294:1367–1372.

7 Holmes EW, Burks TF, Dzau V, et al. Measuring contributions to the research mission of medical schools. Acad Med. 2000;75:303–113.

8 Wasserman S, Faust K. Social Network Analysis. New York, NY: Cambridge University Press; 1994.

9 Durland MM, Fredericks KA, eds. Social Network Analysis in Program Evaluation. San Francisco, Calif: Jossey-Bass; 2006.

10 de Nooy W, Mrvar A, Batagelj V. Exploratory Social Network Analysis With Pajek. New York, NY: Cambridge University Press; 2005.

11 Scott J, Tallia A, Crosson JC, et al. Social network analysis as an analytic tool for interaction patterns in primary care practices. Ann Fam Med. 2005;3:443–448.

12 Hallinan MT, Williams RA. Interracial friendship choices in secondary schools. Am Sociol Rev. 1989;54:67–78.

13 Loomis CP, Morales JO, Clifford RA, Leonard OE. Turrialba: Social Systems and the Introduction of Change. Glencoe, Ill: The Free Press; 1953.

14 Burt RS, Minor MJ, eds. Applied Network Analysis: A Methodologic Introduction. Beverly Hills, Calif: Sage; 1983;195–222.

15 Michael JH. Labor dispute reconciliation in a forest products manufacturing facility. Forest Prod J. 1997;47:41–45.

16 Christakis NA, Fowler JH. The spread of obesity in a large social network over 32 years. N Engl J Med. 2007;357:370–379.

17 Keating NL, Ayanian JZ, Cleary PD, Marsden PV. Factors affecting influential discussions among physicians: A social network analysis of a primary care practice. J Gen Intern Med. 2007;22:794–798.

18 Introcaso DM. Who you know. FedTech. November 2005. Available at: (www.fedtechmagazine.com/article.asp?item_id=152). Accessed April 20, 2009.

Cited By:

This article has been cited 2 time(s).

Scientometrics
Doctoral dissertations of Library and Information Science in China: A co-word analysis
Zong, QJ; Shen, HZ; Yuan, QJ; Hu, XW; Hou, ZP; Deng, SG
Scientometrics, 94(2): 781-799.
10.1007/s11192-012-0799-1
CrossRef
Nursing Outlook
Using social network analysis to examine collaborative relationships among PhD and DNP students and faculty in a research-intensive university school of nursing
Merrill, JA; Yoon, S; Larson, E; Honig, J; Reame, N
Nursing Outlook, 61(2): 109-116.
10.1016/j.outlook.2012.08.001
CrossRef
Back to Top | Article Outline

© 2009 Association of American Medical Colleges

Login

Article Tools

Images

Share