The Challenge of Integration Social Network Analysis to Evaluate an Interdisciplinary Research Center Sally W. Aboelela PhD* Research Associate, CIRAR Assistant Professor of Physiology Columbia School of Nursing 630 W 168th St. NY, NY 10032 [email protected] Phone: 212-342-3651 Fax: 212-305-0722 Jacqueline A. Merrill RN, MPH, DNSc Associate Research Scientist Department of Biomedical Informatics Columbia University School of Nursing 630 W 168th St. NY, NY 10032 [email protected] Phone: 212-305-3659 Fax: 212-305-0722 Kathleen M. Carley PhD Professor of Computation, Organization and Society Institute for Software Research International Carnegie Mellon University Pittsburgh, PA 15213 Phone: 412-268-6016 Fax: 412-268-1744 [email protected] Elaine Larson RN, PhD Professor of Pharmaceutical and Therapeutic Research Columbia University School of Nursing Professor of Epidemiology Mailman School of Public Health, Columbia University 630 W 168th St. NY, NY 10032 Phone: 212-305-0723 Fax: 212-305-0722 [email protected] *corresponding author Authors’ Note This review was written in collaboration with The Center for Interdisciplinary Research on Antimicrobial Research, CIRAR, http://www.cumc.columbia.edu/dept/nursing/CIRAR/, funded by The National Center for Research Resources, P20 RR020616. The Journal of Research Administration Volume XXXVIII, Number 1, 2007 61 The Challenge of Integration Abstract We sought to examine the growth of an interdisciplinary center using social network analysis techniques. Specific aims were to examine the patterns of growth and interdisciplinary connectedness of the Center and to identify the social network characteristics of its productive members. The setting for this study was The Center for Interdisciplinary Research on Antimicrobial Resistance (CIRAR) at Columbia University. Periodic surveys and social network analysis comprised the study design. The data for this study included a relational survey taken by all members of the Center at three time points over one year. Respondents confirmed whether or not they had “heard of,” “met,” or “know the work of,” or had “worked with” each of the other Center members. Data were analyzed using the social networking software program Organizational Risk Analyzer (ORA). Over time the social network increased in size, density, centralization, and complexity. The density of connections among and between different disciplines in the Center varied from Time 1 to 2 to 3; some increased, some decreased, while others stayed the same. Finally, the total degree centrality and the betweeness centrality of Center members were highly correlated to productivity. The study shows that a number of characteristics of an interdisciplinary research center can be quantified and described using social network techniques. Data from these analyses can be used to evaluate a center’s progress, identify important indicators of leadership, identify areas of strength and need for improvement, and inform decisions on strategic direction. Key Words: Interdisciplinary, Social Networking, Collaboration, Research, Growth Introduction In a recent survey, more than 2,000 fulltime academic researchers ranked Despite this nation’s potential to deliver their collaborators above salary and job the finest health care in the world, the security as their highest priorities for job translational blocks from basic science to satisfaction (Grimwade & Park, 2003). human studies and from clinical research to Nevertheless, academic environments practice and policy clearly “impede efforts generally have established incentives for an to apply science to better human health in entrepreneurial, independent approach to a expeditious fashion.”(Sung et al., 2003) research. It has been suggested, in fact, that One way to expedite the translation of the academic culture hinders collaboration research to health care delivery is through and, hence, slows translational research interdisciplinary research, which crosses (Pober, Neuhauser, & Pober, 2001; Sung the traditional boundaries of profession, et al., 2003). Thus, data suggest that an department, or institution. Indeed, much interdisciplinary culture must be well has been written in recent years about the planned and executed before success value of interdisciplinary collaboration, is possible. Despite this, there is little to the extent that it has become one of the empirical evidence of a change in the academic bandwagons of the day, and the traditional departmental academic systems National Institutes of Health (NIH) has and networks, with many initiatives identified interdisciplinarity as an explicit identified as interdisciplinary actually being priority in its recent Roadmap, a strategic reconfigurations of traditional modes of plan for future funding priorities http:// multidisciplinary research (Rhoten & Parker, nihroadmap.nih.gov/interdisciplinary/index. 2004). asp). 62 Volume XXXVIII, Number 1, 2007 The Journal of Research Administration The Challenge of Integration The ultimate purpose of interdisciplinary related to each other (Aviv, Erlich, Ravaid, research is to develop new knowledge & Geva, 2003). Network methods focus or solve a relevant human problem by on the relational linkages between entities combining the skills and perspectives (i.e., individuals or groups of individuals of multiple disciplines. This requires or “things,” such as electronic message a realistic understanding of the nature boards, citations, or computer stations), of disciplinarity. Although academic using techniques based on graph theoretic disciplines are often thought of as “bodies methods (Wasserman & Faust, 1994). A of teachable knowledge” (Woollcott, 1979) graph is a finite set of dots called nodes or as “conceptually specific structures” that are connected by edges that represent (Robertson, Martin, & Singer, 2003), these links. To create a social network graph, dehumanized descriptions do not capture individuals are represented as nodes in a the entire domain. Disciplines are also network and the relationships that connect “organized social groups,” “sets of social them (such as “heard of” or “worked with”) relationships” (Lattuca, 2002), and “isolated are represented as edges that connect the domains of human experience possessing nodes. Each edge indicates an information its own community of experts” (Nissani, link between two individuals. Graphs are 1997). Many of the challenges inherent in often notated in the form of a matrix thus interdisciplinary research emanate from the allowing quantitative calculation using isolation of disciplinary experts, resulting operations from matrix and linear algebra to in knowledge silos. Viewed in this way, mathematically define characteristics of the accomplishing interdisciplinary research network members and structure (Scott et al., becomes, at least in part, an issue of social 2005). interaction and the creation of integrated There is a growing body of literature on social networks. the application of network methods in the Social Network Analysis study of organizations (Borgatti & Foster, 2003; Brass, Galaskiewicz, Greve, & Tsai, Social network analysis involves a unique 2004; Lin & Carley, 2003). Although these set of tools capable of revealing the patterns methods have been used in business as of human interactions. Social networking well as in the social and basic sciences can be used to track the extent to which a to describe interdisciplinary interactions network grows and also answer questions (Barabasi, 2005; Cott, 1997; Girvan & regarding how it grows: What is the Newman, 2002; Newman. 2001; Singer& disciplinary composition of the team? Is the Kegler, 2004), there has been minimal team all connected or are there subgroups? application of social network analysis within Are there central players crucial for creating health care research, and little is known connections between people? Social network about how an interdisciplinary research analysis can elucidate many patterns of team center develops after its establishment. assembly, such as team size, membership composition, and tendency to repeat Specific Aims previous collaborations that can determine The purpose of this project was to the performance of creative teams (Guimera, evaluate the growth of an interdisciplinary Uzzi, Spiro, & Amaral, 2005). research center using social network A “social network” is defined as a group of analysis. Specific aims were threefold: to collaborating (or competing) entities that are understand the patterns of growth over The Journal of Research Administration Volume XXXVIII, Number 1, 2007 6 The Challenge of Integration time (e.g., did members join as individuals of, had met, or had worked with each of the or in subgroups?); to evaluate the extent others. The same survey was administered at and patterns of connectedness among 6 and 12 months after the formation of the center members across disciplinary and Center. As individuals joined and departed departmental boundaries and over time; from the Center their names were added and finally to determine the network or removed from the survey. Individuals characteristics of productive center members who left the Center were primarily students and subgroups based on work products and whose period of study had ended (6), or emerging research teams. faculty members who left the University or whose interests were peripheral to the Methods purpose of the Center (3). Sample and Setting Team Building and Expanding Efforts The Center for Interdisciplinary Research We employed several tactics to build on Antimicrobial Resistance, CIRAR (P20 connections among existing members of the RR020616, National Center for Research Center and expand the team. To facilitate Resources, NIH), was funded as a planning interactions among members, the core grant in 2004 to develop interdisciplinary team met monthly, and several smaller research aimed at reducing antimicrobial working groups met at regular intervals. resistance. The core team of researchers Within the first few months of the Center’s and staff included 15 individuals from 12 establishment, each core team member different academic departments or divisions: made a presentation describing his/her work four nurses and four physicians as well as during part of the monthly team meetings. experts in epidemiology, microbiology, The smaller working groups gave members higher education, biostatistics, dentistry, a chance to work together; each group was health policy, informatics, economics, responsible for carrying out one aspect organizational systems, and behavioral of the Center’s mission (e.g., identifying sciences. Student liaisons from the various gaps in the field and planning educational health professions schools were also seminars). The Center also held a team- included as full members of the team. This building half-day retreat composed of short social network study included these core talks from core team members followed by team members and others added to the “brain-storming” breakout groups to identify team over time as they became involved in ideas for collaborative projects. activities of the Center. Students, postdoctoral fellows, and junior Data Collection faculty were recruited by the Center through requests for applications for small At the first general meeting of the Center, pilot grants. Eight grants were awarded. core team members completed a survey We also increased our exposure through in which they were asked to indicate, periodic seminars and guest speaking for every other team member as well as events that were extensively advertised. for the external advisors and University The Center convened two major events oversight group (which provides input aimed at expanding potential collaborative regarding direction and goals of the center partnerships: a meeting of interdisciplinary but generally does not directly contribute to center directors across the university and work products), four levels of relationship: a discussion forum with pharmaceutical whether he/she had heard of, knew the work 64 Volume XXXVIII, Number 1, 2007 The Journal of Research Administration The Challenge of Integration company researchers working in the field 2) co-authoring a publication, 3) giving a of infectious disease. An informational presentation, 4) participating in the CIRAR pamphlet about the Center was available at retreat, and 5) participating in a grant all events. Any new contacts made at these application. Each individual received a events were maintained through an e-mail productivity score based on activity in each database that was continuously updated. of these categories. Descriptions and photos of all events were posted on the Center’s website http://www. cumc.columbia.edu/dept/nursing/CIRAR/), along with announcements for future events and minutes from all meetings. Data Analysis We selected a set of network measures to address the three specific aims for evaluating the Center. To evaluate patterns of growth over time, we examined the size, density, complexity, and centralization of the full network at the beginning and 6 and 12 months after formation of CIRAR (Times 1, 2, and 3). We also examined the average numbers of cliques that developed among the members in the network, and the average effective network size. See Table 1 for definitions of all network measures. To evaluate cross-disciplinary collaboration, we examined the network densities of “worked with” interactions within and among three disciplinary subgroups (Medicine, Nursing, and “Other,” which encompassed Public Health, Microbiology, Dentistry, Sociology, and Education). Members affiliated with more than one discipline were grouped with their primary affiliation (e.g., a nurse epidemiologist was grouped with Nursing). To reflect growth of network by discipline we examined the change in each discipline as percent of the network at Times 1, 2, and 3. Finally, to determine characteristics of productive CIRAR members, we examined the relationship between network position and productivity. Productivity was measured in 5 categories: 1) leading a workgroup, The Journal of Research Administration Volume XXXVIII, Number 1, 2007 6 The Challenge of Integration Table 1 Definitions and Interpretation of Network Measures Used in this Study NetworkLevelMeasures Measure Definition Interpretation Network Thedensityofanetworkisequaltothe Representstheextentofcommunication Density totalnumberofconnectionsdividedbythe withinthenetwork.Highernumbers(above numberofpossibleconnections.The .03)suggestfasterinformationpropagation numberofpossibleconnectionsassumes andgreatergroupcohesion. thateachpersoncanhavealinkwitheach otherperson.Normalizedtherangeis0-1. Overall Combineddensityofallrelationalgraphs Thismeasureisapredictorofnetwork Complexity ateachtimeperiod(i.e.,heardof,know performance. Ascomplexityincreasesan work,met,workedwith). Theratioofthe organizationperformsbetteruptoa numberoflinksversusthemaximum (unknown)pointwheretoomuch possiblelinksforthemeta-matrix. complexityresultsinexcessivecoupling Normalizedtherangeis0-1. andthepotentialforerrorcascades. Network Thecentralizationofthenetworkbasedon Indicateswhetherornotthereisasymmetry Centralization, theextenttowhichthemajorityofthe inthedistributionofconnections.It TotalDegree connectionsaretoasmallsetofnodes. indicatesthedegreetowhich Expressesinequalityorvarianceinthe communicationiscentralizedarounda networkasapercentageofthemost singleagentorsmallgroup. More unequalnetworkpossible. Normalizedthe centralizedgroupstendtobemore rangeis0-1. hierarchicalinnature. CliqueCount Theaveragenumberofmaximally Ameasureofsocialintegrationandnetwork connectedsubgroups.Acliqueisdefined cohesion.Membersofacliquecanusetheir assubgroupofpeoplewho arealldirectly strongrelationstodrivetheprocessof linkedtoeachother. constructingknowledge. Effective Theaverageoftheobservednumberof Indicatestheaveragereachofthe NetworkSize eachindividual’spersonallinkswithinthe individuals:i.e.,onaverage,foreach network,minusredundantlinks(i.e., person,howmanyothersarelikelytoget connectionstothesameindividualthrough informationfromthemortosend morethanoneperson). informationtothem,evenifthat informationhastogothroughan intermediary. IndividualMemberLevelMeasures Measure Definition Interpretation Centrality, Numberofdirectconnectionsthataperson Indicatesthelevelofextroversion. Higher TotalDegree hastoothersinthenetwork.Normalized numbersindicatemoreconnectivity therangeis0-1. Centrality, Measurestheextenttowhichflowsof Indicatestheextentthatapersonisa Betweeness informationbetweendiverseotherspass conduitforinformation.Peoplehighinthis throughthisperson.Normalizedtherange measureofteninfluencewhatflowsinthe is0-1. network,andoftenserveasgatekeepersand brokersofinformation. 66 Volume XXXVIII, Number 1, 2007 The Journal of Research Administration The Challenge of Integration We then used two measures to assess Results network position. Total degree centrality Aim 1: Patterns of Growth over Time measured the number of ties each member had to others in the network. Individuals Network Size and Centralization with many ties are most likely to receive or The network increased steadily in size from generate whatever information is flowing 22 members at Time 1, to 39 members at through the network. Betweeness centrality Time 2, and to 47 members at Time 3 -- an measures the extent to which an individual overall increase of approximately 113%. The connects those persons who may not be “worked with” network in CIRAR showed directly connected to each other, thus steady increase in network centralization, serving as a link between unconnected from 0.21 at Time 1 to 0.41 at Time 2 and people (Freeman, 1979). Individuals 0.50 at Time 3. Network centralization who rank highly on this measure serve expresses inequality or variance in the as intermediaries who are in a position to network structure as the degree to which control information flow in the network the network connections gather around a (e.g., what information is received and few central individuals (Scott, 2000). It can how it is perceived). Spearman’s rho be equated with coordination in the sense was calculated to determine if individual of “command and control.” Lower scores productivity in CIRAR was associated with indicate distributed connections and higher these measures of network prominence. scores suggest a more cohesive group. That The relational data collected by survey is, the higher the centralization the greater at the three time points in the Center’s the likelihood that there is one person, or a development were analyzed with the small set of people, to whom everyone is software program Organizational Risk connected. Thus, over time as the CIRAR Analyzer (ORA: http://www.casos.cs.cmu. is maturing, an increasingly centralized edu/projects/ora/index.html). ORA is unique and perhaps hierarchical organization is among network analysis programs because emerging. it can be used to analyze multiple networks These patterns are displayed in Figure 1; collectively. This allows calculation of note the more tightly centralized core at measures that reflect the complexity found Time 3. in organizational systems. Analysis in ORA is based on formal logic, matrix algebra, and discrete and continuous equations (Reminga& Carley, 2005). The results are index numbers that convey aspects of the distribution of relational ties within the network (Hanneman, 2001). The Journal of Research Administration Volume XXXVIII, Number 1, 2007 67 The Challenge of Integration FFigiguurree 11.. SSiizzeea annddC eCnetrnatlrizaalitzioantioofnt hoef “tWheo r“kWedoWrkiethd” WNietthw”o NrkeOtwveorrkT iOmveer Time Time1, N=22 Time2, N=39 Time3, N=47 Centralization0.21 Centralization0.41 Centralization0.50 Network Density and Complexity The network complexity measure calculated all of the links recorded in the four networks Network density and complexity over time we measured (heard of, know work, met, are compared in Figure 2. Network density and worked with) in relation to all the links describes the extent to which individuals possible. The organizational complexity are connected by measuring the number of CIRAR increased steadily over time, of connections present in relation to the from 0.05 to 0.26, a sign of a more tightly number possible (i.e., as if everyone were knit organization with broadening interests completely connected to everyone else). and goals. The pattern of falling density Density for CIRAR’s “heard of” and “met” and increasing complexity suggests that, networks increased between Times 1 and 2 on average, the typical person knew/ and fell between Times 2 and 3. Density in was connected to fewer people in the the “know work” network remained steady overall group, but the overall group was over time. Between Times 2 and 3 the becoming more complex as members “worked with” network showed increasing became associated with more knowledge density. In other words, as the organization and activities. In general, as complexity matured, members tended to retain a certain increases, to a point, an organization level of understanding of what others did will perform better due to increased (know work), even though they were less connectedness (and the associated awareness likely to have actually met these others. of what others are doing) among sub-groups This suggests that the group may be moving and processes (Carley, 2002) and sufficient to role-based interactions predicated on redundancy to enable adaptivity. generic knowledge of what others did. At the same time, new members tended to join based on extant collaborations with current members, while current members increased collaboration, resulting in an overall increase in who “worked with whom,” despite the growth in membership. 68 Volume XXXVIII, Number 1, 2007 The Journal of Research Administration The Challenge of Integration FiFgiugruer e2.2 C. Comomppaarirsisoonn ooff NNeettwwoorrkk DDeennssiittyy aannddC CoommplpelxeixtyityO OvevreTr iTmime e NetworkDensityandComplexity Heardof Knowwork 1.0 Met 0.9 Workedwith OverallComplexity 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 Time1 Time2 Time3 Clique Count belonged at Time 1 was only 3. The average increased to 34 at Time 2 and fell slightly Cohesion in the structure of a network to 31 at Time 3 as research teams began contributes to the creation of knowledge to coalesce. However, there was wide through shared reasoning and perspective variance in the clique counts, with standard (Burt, 2000). One indicator of cohesion deviations ranging from 0-22 at Time 1, and is the presence of cliques – subgroups of 0-128 at Times 2 and 3 (see Table 2). This participants within the network for which all suggests that the CIRAR was becoming possible links are present. In collaborative more cohesive. However, whether there organizations cliques can drive the process is a natural cap on cohesion or an optimal of constructing knowledge by taking number of cliques (e.g., people do not have advantage of their strong inter-relations the cognitive resources to be in more than (Aviv et al., 2003). The average number 20-40 cliques) is a point for future research. of cliques to which a CIRAR member TablTea b2l e2 MeasuresofNetworkCohesion Measures of Network Cohesion Measure Time1 Time2 Time3 MeanCliqueCount 3.1(5.48) 34.1(41.15) 31.1(35.60) (+/-standarddeviation) Minimum/Maximum 0-22 0-128 0-128 MeanEffectiveSize 1.38(2.67) 5.14(5.58) 6.93(6.90) (+/-standarddeviation) Minimum/Maximum 0-10.2 0-18.4 0-2.6 The Journal of Research Administration Volume XXXVIII, Number 1, 2007 69 The Challenge of Integration Effective Network Size Aim 2: Cross Disciplinary Collaboration While individuals in a network may have Disciplines as a Percent of Network redundant links to each other through several Over time, the number of members in network members, the effective network size each disciplinary group increased. At indicates the size of each person’s network Time 1 individuals in the physician group without this redundancy, and so gives a comprised 35% of the network and were the better sense of the actual number of people dominant group. Nursing, public health and to which a network member is effectively microbiology comprised 23%, 18%, and 9% linked. (Burt, 2001) On average, CIRAR’s of the network, respectively, and individuals members increased the size of their personal in “other” disciplines comprised less network of connections from less than 2 at than 15%. At Time 3, CIRAR had a more Time 1 to nearly 7 at Time 3, as displayed balanced membership: physicians comprised in Table 3. Hence, on average, over time, 26% of the network and no longer those who join CIRAR are likely becoming dominated the disciplinary makeup. Instead, increasingly linked into CIRAR related individuals in the “other” disciplinary activities by interacting with other CIRAR group comprised about one-third of the members. network. Nursing remained at 23%, public health decreased to 11%, and microbiology increased to13% (Figure 3). Essentially, over time, participation from various CIRAR sub- groups was becoming more democratic. Figure . Change in Each Discipline as a Percent of the Network Figure3. ChangeinEachDisciplineasaPercentof theNetwork 0.40 0.35 0.30 0.25 Medicine Nursing 0.20 PublicHealth Microbiology 0.15 Other 0.10 0.05 0.00 Time1,N=22 Time2,N=39 Time3,N=47 70 Volume XXXVIII, Number 1, 2007 The Journal of Research Administration