Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Filter by Categories
BOOK OF ABSTRACTS | HEALTH EQUITY
BOOKS OF ABSTRACTS | HEALTH EQUITY
Brief Research Article | HIV and AIDS
COMMENTARY
COMMENTARY | CHLAMYDIA
COMMENTARY | COVID-19 AND HIV/AIDS
COMMENTARY | COVID-19 VACCINE
COMMENTARY | DIGITAL HEALTH
COMMENTARY | EMERGENCY PRACTICE
COMMENTARY | HIV Testing
COMMENTARY | MALARIA
COMMENTARY | MATERNAL MORTALITY
COMMENTARY | MONKEYPOX
COMMENTARY | PERINATAL HIV TRANSMISSION
Editorial
LETTER TO THE EDITOR | COVID-19 PANDEMIC
LETTER TO THE EDITOR | COVID-19 TEST
METHODOLOGY ARTICLE
METHODOLOGY ARTICLE | HIV
NOTES FROM THE FIELD | COVID-19 PANDEMIC
NOTES FROM THE FIELD | MCH STUDENT ASSOCIATION
Original Article
ORIGINAL ARTICLE | ADOLESCENT HEALTH
ORIGINAL ARTICLE | BIRTH INTERVAL
ORIGINAL ARTICLE | CAFFEINE AND CHILDHOOD OBESITY
ORIGINAL ARTICLE | CHILD HEALTH
ORIGINAL ARTICLE | CLUBFOOT
ORIGINAL ARTICLE | COMPLEMENTARY FEEDING PRACTICES
ORIGINAL ARTICLE | CONTINUUM OF CARE
ORIGINAL ARTICLE | COVID-19 DISEASE
ORIGINAL ARTICLE | COVID-19 IN PREGNANCY
ORIGINAL ARTICLE | COVID-19 OUTCOMES
ORIGINAL ARTICLE | DEPRESSION & HIV
ORIGINAL ARTICLE | DIETARY MICRONUTRIENTS AND HIV
ORIGINAL ARTICLE | HIV
ORIGINAL ARTICLE | HIV AND PREGNANT WOMEN
ORIGINAL ARTICLE | HIV INFECTION
ORIGINAL ARTICLE | HIV PREVENTION
ORIGINAL ARTICLE | HIV SCREENING
ORIGINAL ARTICLE | HIV TESTING
ORIGINAL ARTICLE | HIV Viral Load
ORIGINAL ARTICLE | HIV/AIDS
ORIGINAL ARTICLE | HIV/AIDS IN ECUADOR
ORIGINAL ARTICLE | HOME DELIVERY
ORIGINAL ARTICLE | HOUSING AND HOUSING INEQUALITIES
ORIGINAL ARTICLE | IMMUNIZATION
ORIGINAL ARTICLE | INFLAMMATORY BOWEL DISEASE
ORIGINAL ARTICLE | LEISHMANIASIS
ORIGINAL ARTICLE | LIFE EXPECTANCY
ORIGINAL ARTICLE | MALARIA & HIV
ORIGINAL ARTICLE | MATERNAL HEALTH
ORIGINAL ARTICLE | MATERNAL MORTALITY
ORIGINAL ARTICLE | MATERNAL OBESITY
ORIGINAL ARTICLE | MORTALITY
ORIGINAL ARTICLE | NEONATAL MORTALITY
ORIGINAL ARTICLE | OBSTETRIC FISTULA REPAIR
ORIGINAL ARTICLE | OXYTOCIN VS MISOPROSTOL IN PPH
ORIGINAL ARTICLE | PEDIATRIC HIV
ORIGINAL ARTICLE | PREGNANCY
ORIGINAL ARTICLE | PRENATAL CARE
ORIGINAL ARTICLE | PUERPERAL SEPSIS
ORIGINAL ARTICLE | REPRODUCTIVE HEALTH
ORIGINAL ARTICLE | TEENAGE PREGNANCY
ORIGINAL ARTICLE | VACCINATION
ORIGINAL ARTICLE | VACCINE
ORIGINAL ARTICLE | VACCINE EQUITY
ORIGINAL ARTICLE | VIRTUAL PRENATAL CARE
ORIGINAL ARTICLE | WOMEN HEALTH
ORIGINAL ARTICLE | WOMEN’S HEALTH
ORIGINAL ARTICLE | YOUTH MORTALITY
Original Research | Article Healthcare
PROTOCOL | PREGNANCY
PUBLIC HEALTH PRACTICE | ACADEMIC DETAILING
PUBLIC HEALTH PRACTICE | CHILD DEVELOPMENT
PUBLIC HEALTH PRACTICE | CHILD HEALTH
PUBLIC HEALTH PRACTICE | HIV
PUBLIC HEALTH PRACTICE | OPT-OUT APPROACH IN HIV TESTING
PUBLIC HEALTH PRACTICE | PREGNANCY WHEEL
PUBLIC HEALTH PRACTICE | SURVEILLANCE
RESEARCH COMMENTARY
Review Article
SHORT RESEARCH COMMUNICATION
SHORT RESEARCH COMMUNICATION | COVID AND MENTAL HEALTH
SHORT RESEARCH COMMUNICATION | COVID MCH RESEARCH AGENDA
SHORT RESEARCH COMMUNICATION | COVID-19
SHORT RESEARCH COMMUNICATION | COVID-19 AND CHILD VACCINATION
SHORT RESEARCH COMMUNICATION | COVID-19 AND MATERNAL MORTALITY
SHORT RESEARCH COMMUNICATION | COVID-19 AND REMOTE WORKERS
SHORT RESEARCH COMMUNICATION | COVID-19 PANDEMIC DISPARITIES
SHORT RESEARCH COMMUNICATION | HEALTHCARE PROVIDER TRAINING
SHORT RESEARCH COMMUNICATION | MALARIA
SHORT RESEARCH COMMUNICATION | SINGLETONS, TWINS, MULTIPLE BIRTHS
SHORT RESEARCH COMMUNICATION | SURVEILLANCE SYSTEM FOR COVID-19
SYSTEMATIC REVIEW
WAME STATEMENT
Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Filter by Categories
BOOK OF ABSTRACTS | HEALTH EQUITY
BOOKS OF ABSTRACTS | HEALTH EQUITY
Brief Research Article | HIV and AIDS
COMMENTARY
COMMENTARY | CHLAMYDIA
COMMENTARY | COVID-19 AND HIV/AIDS
COMMENTARY | COVID-19 VACCINE
COMMENTARY | DIGITAL HEALTH
COMMENTARY | EMERGENCY PRACTICE
COMMENTARY | HIV Testing
COMMENTARY | MALARIA
COMMENTARY | MATERNAL MORTALITY
COMMENTARY | MONKEYPOX
COMMENTARY | PERINATAL HIV TRANSMISSION
Editorial
LETTER TO THE EDITOR | COVID-19 PANDEMIC
LETTER TO THE EDITOR | COVID-19 TEST
METHODOLOGY ARTICLE
METHODOLOGY ARTICLE | HIV
NOTES FROM THE FIELD | COVID-19 PANDEMIC
NOTES FROM THE FIELD | MCH STUDENT ASSOCIATION
Original Article
ORIGINAL ARTICLE | ADOLESCENT HEALTH
ORIGINAL ARTICLE | BIRTH INTERVAL
ORIGINAL ARTICLE | CAFFEINE AND CHILDHOOD OBESITY
ORIGINAL ARTICLE | CHILD HEALTH
ORIGINAL ARTICLE | CLUBFOOT
ORIGINAL ARTICLE | COMPLEMENTARY FEEDING PRACTICES
ORIGINAL ARTICLE | CONTINUUM OF CARE
ORIGINAL ARTICLE | COVID-19 DISEASE
ORIGINAL ARTICLE | COVID-19 IN PREGNANCY
ORIGINAL ARTICLE | COVID-19 OUTCOMES
ORIGINAL ARTICLE | DEPRESSION & HIV
ORIGINAL ARTICLE | DIETARY MICRONUTRIENTS AND HIV
ORIGINAL ARTICLE | HIV
ORIGINAL ARTICLE | HIV AND PREGNANT WOMEN
ORIGINAL ARTICLE | HIV INFECTION
ORIGINAL ARTICLE | HIV PREVENTION
ORIGINAL ARTICLE | HIV SCREENING
ORIGINAL ARTICLE | HIV TESTING
ORIGINAL ARTICLE | HIV Viral Load
ORIGINAL ARTICLE | HIV/AIDS
ORIGINAL ARTICLE | HIV/AIDS IN ECUADOR
ORIGINAL ARTICLE | HOME DELIVERY
ORIGINAL ARTICLE | HOUSING AND HOUSING INEQUALITIES
ORIGINAL ARTICLE | IMMUNIZATION
ORIGINAL ARTICLE | INFLAMMATORY BOWEL DISEASE
ORIGINAL ARTICLE | LEISHMANIASIS
ORIGINAL ARTICLE | LIFE EXPECTANCY
ORIGINAL ARTICLE | MALARIA & HIV
ORIGINAL ARTICLE | MATERNAL HEALTH
ORIGINAL ARTICLE | MATERNAL MORTALITY
ORIGINAL ARTICLE | MATERNAL OBESITY
ORIGINAL ARTICLE | MORTALITY
ORIGINAL ARTICLE | NEONATAL MORTALITY
ORIGINAL ARTICLE | OBSTETRIC FISTULA REPAIR
ORIGINAL ARTICLE | OXYTOCIN VS MISOPROSTOL IN PPH
ORIGINAL ARTICLE | PEDIATRIC HIV
ORIGINAL ARTICLE | PREGNANCY
ORIGINAL ARTICLE | PRENATAL CARE
ORIGINAL ARTICLE | PUERPERAL SEPSIS
ORIGINAL ARTICLE | REPRODUCTIVE HEALTH
ORIGINAL ARTICLE | TEENAGE PREGNANCY
ORIGINAL ARTICLE | VACCINATION
ORIGINAL ARTICLE | VACCINE
ORIGINAL ARTICLE | VACCINE EQUITY
ORIGINAL ARTICLE | VIRTUAL PRENATAL CARE
ORIGINAL ARTICLE | WOMEN HEALTH
ORIGINAL ARTICLE | WOMEN’S HEALTH
ORIGINAL ARTICLE | YOUTH MORTALITY
Original Research | Article Healthcare
PROTOCOL | PREGNANCY
PUBLIC HEALTH PRACTICE | ACADEMIC DETAILING
PUBLIC HEALTH PRACTICE | CHILD DEVELOPMENT
PUBLIC HEALTH PRACTICE | CHILD HEALTH
PUBLIC HEALTH PRACTICE | HIV
PUBLIC HEALTH PRACTICE | OPT-OUT APPROACH IN HIV TESTING
PUBLIC HEALTH PRACTICE | PREGNANCY WHEEL
PUBLIC HEALTH PRACTICE | SURVEILLANCE
RESEARCH COMMENTARY
Review Article
SHORT RESEARCH COMMUNICATION
SHORT RESEARCH COMMUNICATION | COVID AND MENTAL HEALTH
SHORT RESEARCH COMMUNICATION | COVID MCH RESEARCH AGENDA
SHORT RESEARCH COMMUNICATION | COVID-19
SHORT RESEARCH COMMUNICATION | COVID-19 AND CHILD VACCINATION
SHORT RESEARCH COMMUNICATION | COVID-19 AND MATERNAL MORTALITY
SHORT RESEARCH COMMUNICATION | COVID-19 AND REMOTE WORKERS
SHORT RESEARCH COMMUNICATION | COVID-19 PANDEMIC DISPARITIES
SHORT RESEARCH COMMUNICATION | HEALTHCARE PROVIDER TRAINING
SHORT RESEARCH COMMUNICATION | MALARIA
SHORT RESEARCH COMMUNICATION | SINGLETONS, TWINS, MULTIPLE BIRTHS
SHORT RESEARCH COMMUNICATION | SURVEILLANCE SYSTEM FOR COVID-19
SYSTEMATIC REVIEW
WAME STATEMENT
View/Download PDF

Translate this page into:

ORIGINAL ARTICLE
7 (
1
); 207-216
doi:
10.21106/ijma.265

The CI Index (CII): A New Instrument to Evaluate and Foster Collaborative Partnership in Public Health

Baylor College of Medicine, Department of Family and Community Medicine, 3701 Kirby Drive, MS: BCM700, Houston, TX 77098, USA
University of South Florida, College of Public Health, 13201 Bruce B. Downs MDC 56, Tampa, FL 33612, USA
REACHUP, Incorporated, 2902 N Armenia Ave #100, Tampa, FL 33607, USA
Corresponding author email: Hamisu.Salihu@bcm.edu
Licence

This is an open-access article distributed under the terms of the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background or Objectives:

Collective Impact (CI) is the sum of collaborative actions through coalition building to examine and address issues that threaten public health. We sought to develop an instrument to measure CI across diverse health-related organizations.

Methods:

The study was performed in 5 phases initiated with a comprehensive literature review through the generation of candidate items using a deductive instrument development approach. A matrix was then created to map assembled items onto related CI domains. An one-hundred and nine (n=109) item questionnaire covering multiple items per domain was administered to 200 health-related organizations selected randomly throughout the United States. Survey data were evaluated using Principal Component Analysis and Kaiser criterion or eigenvalue-greater-than-one rule was the factor retention method utilized.

Results:

Based on Kaiser’s eigenvalue criteria and communality estimates, the number of items across the five domains of CI was reduced from 109 to 20, with 4 items populating each subdomain. All communality estimates in the final instrument had values > 0.6, which was sufficiently adequate as per Kaiser’s criterion.

Conclusion and Global Health Implications:

This pilot study demonstrates CI represents a multi-factorial concept with domains that are multi-dimensional capturing diverse aspects of a construct. We developed the first measure of CI for public health practice. The index bears potential utility for assessing and monitoring areas of strengths and weaknesses within collaborative partnerships across the spectrum of population health.

Keywords

Collaborative Partnerships
CI
Federal Healthy Start
Program Evaluation
Index
Public Health Practice

1. Introduction

The healthcare environment is increasingly becoming a complex macrosystem that envelopes several inter-related factors which impact the health of individuals and communities at various levels.1,2 Many of these factors cannot be addressed or controlled through isolated efforts, but they demand the consolidation of diverse arrays of community assets, strengths, resources and know-how from each member in order to maximize CI.3,4 The CI (CI) framework proposed by Kania & Kramer5 represents a guiding framework for structuring collaborative efforts of coalitions and comprehensive community initiatives. The framework consists of five theoretical constructs or domains that indicate the degree of broad cross-sector coordination demonstrated by a particular collaborative group (multiple organizations partnering toward a common set of goals), which will ultimately result in the desired effects on the public health issues of interest. The five domains are: 1) common agenda, 2) shared measurement, 3) mutually reinforcing activities, 4) continuous communications, and 5) backbone organization (table 1).

Table 1 Core Pillars of CI
Common agenda Members possess a shared vision for change that encompasses understanding the issue of interest and the need for a collaborative approach toward a solution
Shared Measurement systems Gathering data and measuring stated outcomes uniformly across the collaborative partnerships to ensure alignment of efforts and accountability
Mutually reinforcing activities Coordinating diverse partner activities through mutually reinforcing action plans
Continuing communications Regular and open communication across partners to strengthen trust, buttress mutual objectives and generate common motivation
Backbone support Creating and managing a supportive organization that is separate from the partnering collaborative entities with a focus on sustaining and coordinating the entire initiative

Source: Adapted from 13

The CI framework is based on social action premises such as: 1) no single organization is capable of effectively shouldering the responsibility for any major social problem, nor can any single institution address social problems effectively in silo; and 2) complex problems can be effectively solved only through cross-sector coalitions that engage those outside the sector including community stakeholders. The framework is gaining importance and acceptance as funders of health-related projects seek to maximize the impact of resource infusion at various levels.6 For instance, the Federal Healthy Start program, located and implemented throughout the United States (U.S.), is an example of a federally funded program with established effectiveness that promotes the adoption and integration of CI across the performance platforms of its grantees.7,8 The purpose is to create a culture that embraces coalition partnerships and collaborative work as integrative and essential elements of Federal Healthy Start programs with the expectation of enhanced efficiency and heightened impact. It is increasingly becoming clear that program accountability, an atmosphere of limited funding, and the need to be selective among competing programs of variable effectiveness constitute justifiable factors that warrant a CI index. In this context, a CI framework comes at a right time to meet a long-standing need to capture collaborative processes among community-based organizations partnering to improve maternal and child health (MCH) outcomes. Although the CI framework appears to be potentially useful in improving effectiveness of inter-organizational collaborations, currently, an effective instrument for measuring CI in the healthcare setting is lacking. To address this gap in public health practice, we conducted this study to create and pilot an instrument that measures CI across health-related organizations throughout the U.S. Based on the CI framework proposed by Kania & Kramer,5 we hypothesized as follows:

  1. That CI will represent a multi-factorial concept with multi- dimensional domains that capture diverse aspects of a construct; and

  2. That as a multi-dimensional concept encompassing multi-faceted constructs, an index scale rather than a traditional psychometric instrument will be more appropriate measure for overall CI in public health.

2. Methods

The study was conducted from January 2016 through November 2017 using survey data collected across the U.S. The initial step involved the generation of candidate items through a comprehensive literature review. We began with a deductive instrument development approach by identifying a theoretical definition of the constructs for “CI.” First, we searched PUBMED, CINHAL, and Scopus using the following search terms: CI, collective efficacy, collective action, collaborative action, community mobilization, comprehensive initiatives, collectivism, community coalition action, community engagement, community participation, inter-organizational collaboration, history of collaboration, shared leadership, and community readiness. We also conducted open internet searches for gray literature using Google, Bing, and Yahoo search engines. Our search revealed a definition frequently used for CI from a seminal gray document by Kania & Kramer.5 This definition had been operationalized into a distinctive framework encapsulating five domains or conditions for collective success: 1) Common Agenda, 2) Shared Measurement, 3) Mutually Reinforcing Activities, 4) Continuous Communication, and 5) Backbone Support. We then conducted a narrowly focused literature search to identify measures related to the conceptual domains of the referent framework and other previously used measures that attempted to capture related content areas for CI. To focus the search on measures or instruments, we ran secondary searches using the following combined terms “questionnaire” OR “inventory” OR “survey” OR “scale” OR “measure” OR “instrument” OR “assessment.” We also employed the same search terms combined with “Common Agenda” OR “Shared Measurement System” OR “Mutually Reinforcing Activities” OR “Continuous Communication” OR “Backbone Organization.”

2.1. Literature review and item generation

Articles abstracts and full texts were reviewed, and then categorized by relevance. To achieve understanding of the relevant literature and the CI construct, two faculty members with expertise in social science and community health reviewed full-texts of retrieved articles independently, with the goal of identifying topics that were related to CI. The findings were discussed weekly to achieve consensus. A matrix was developed that mapped the potential candidate-items onto related CI domains. Candidate items were derived from questions previously used by other researchers, as well as identified statements that could fit into specific CI domains. Next, based on examples reported in the literature, our research staff compiled items that corresponded to the conceptual map, which represented the first item pool. For this first set of preliminary items, we followed guidelines to ensure that the items were properly constructed. First, items were designed to address only a single issue (avoiding “double-barreled” items). We created items predominantly positive to keep consistency and reduce question burden. One or two negatively worded items were included per domain to control for agreement bias and social desirability. Second, items were ordered in a consistent manner in terms of sequential concepts to make sure that we did not mix items that assess one concept with items that assess other responses or outcomes of behaviors. Content redundancies were desirable at this stage, so we included similar items measuring similar ideas (but still differently worded). Finally, a list of multiple items per domain was framed and formed the template for evaluative analysis subsequently.

2.2. Content assessment of candidate items

To assess the adequacy of our initial item pool, two experts in community cognitive assessment & epidemiology, held a series of six-weekly virtual meetings to review the relevancy of items relating to CI. Through this collaborative effort, item wording was refined and some items dropped. The remaining item pool was then formatted into an online survey using the Qualtrics data collection software to gather additional expert insights. Federal Healthy Start and other community partners were invited to participate in online nominal groups (3 groups of 5-6 participants each) to assess face validity and content validity of the subdomain items. Participants were asked to assess the content adequacy (yes/no) and clarity of language (open-ended), as well as describe insights or experiences about CI within their organizational settings and assist in identifying aspects of CI not captured by the deductive approach. Any input regarding comprehension issues, needs for re-wording, and/or readability of items were noted and incorporated into the final survey. Further, comments/responses to open-ended statements were classified into several categories using thematic analysis. From these categorized responses, second-wave items were then derived and organized using a Likert Scale (5: Strongly Agree to 1: Strongly Disagree). The entire set of questions is presented in Supplement A.

2.3. Instrument administration

This initial scale was comprised of 109 items, and was administered to 200 healthcare or health-related organizations in the US. The organizations include community health centers, hospitals, public health departments, and non-profit health care organizations. Efforts were made to randomly select at least one organization from each state in the U.S. Individuals, including public health nurses, social workers, psychologists, administrators, and healthcare executives, were nominated by their organizations to respond to the survey. A trained staff interviewer contacted the organizations via telephone, provided a brief overview of the research project, and administered the questionnaire. Responses were voice-recorded and documented on paper. The collected data were then organized in a single Excel document for analysis.

2.4. Statistical analysis

We performed a principal component analysis (PCA) on the 109 items across the five CI domains. Initially, we employed both the scree plot and Kaiser’s rule9 of eigenvalue-greater-than-one criterion to select the number of components (i.e., number of factors to retain). The Kaiser rule is to drop all components with eigenvalues under 1.0 – this being the eigenvalue equal to the information accounted for by an average single item. With this approach, we retained and interpreted any component with an eigenvalue greater than 1.00. Our approach also involved striking a balance between “reduction” and “meaningfulness” of the correlations that existed within variable groups. We employed oblique rotation (promax) to optimize configuration on factors, allowing for maximum restriction of orthogonality (Delta = 0). The promax option allows for correlations of factors, and is computationally fast. When items loadings were >0.4 on more than one factor, one factor was selected based on clinical judgement regarding the most plausible solution. We then evaluated and compared the pattern and structure of matrices to determine the influence of shared variance. We applied the Barlett-test (test of sphericity) which assesses the following hypothesis: H0: Variables are uncorrelated; H1: Variables are correlated. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (KMO) was computed to determine adequacy of sample size. Communality was calculated to show the proportion of variance explained by the extracted factors, with initial set at 1.00. The generated extraction communalities would then represent estimates of the variance in each variable accounted for by the components. The data were analyzed with the statistical package SPSS version 22.0 and by two statisticians working independently. Periodic meetings were held to discuss areas of discrepancies, and results were only adopted when the two statisticians concurred. Since this was a phone interview, verbal consent was recorded and archived. The study was considered exempt by Baylor College of Medicine Institutional Review Board.

3. Results

The instrument (Supplement A) is comprised of ordinally-framed variables on a 5-point Likert Scale that were recoded as numerical types (scores of 1 to 5) during analysis. In all, 109 items covering the five CI framework domains were administered with the following item distribution: 1) Common Agenda (A) = 22 items; 2) Shared Measurement Systems (B) = 23 items; 3) Mutually Reinforcing Activities (C) = 22 items; 4) Continuous Communication (D) = 24 items; and 5) Backbone Support (E) = 18 items. The mean score for each domain varied across the items. For Common Agenda, the highest mean score (Mean ± SD = 4.95 ±0.22) was observed for item A22 (Our partners’ individual work is increasingly aligned with the common agenda), while the lowest score (Mean ± SD = 3.90 ±0.49) was for item A19 (People in our community do not work together to solve their problems). For shared measurement systems, the highest score (Mean ± SD = 4.99 ±0.12) was for item B18 (Members of our collaborative group have clear expectations about the confidentiality of the data used by the collaborative), while the lowest score (Mean ± SD = 2.36 ±0.72) was observed for item B14 (There is sufficient funding to improve the information system for our collaborative group). Item C1 (In our collaborative, there is a process in place to share successes with each other and the larger community; Mean ± SD = 4.80 ±0.40) and C20 (The collaborative uses data as the basis to determine which activities should be stopped; Mean ± SD = 4.80 ±0.60) shared the greatest score for mutually reinforcing activities, while the lowest score was observed for item C21 (Funders are redirecting funds to support the collaborative goal; Mean ± SD = 2.36 ±0.74). A score of 4.96 (SD±0.20) and 2.75 (SD±0.94) represented the highest and lowest mean value for the domain assessing continuous communications respectively. The corresponding items were D6 (highest mean score; The collaborative group provides opportunities to share our different points of views) and D12 (lowest mean score; We have a common language of communication among partners in this collaborative). Finally, for backbone support, we observed the highest mean score of 4.93 (SD±0.36) for item E14 (This collaborative uses a supporting backbone infrastructure to build public will) and the lowest mean score of 1.70 (SD±0.75) on item E12 (The activities of this collaborative are established by a senior-level committee with ultimate decision-making power). It is noteworthy that the latter item scored (E12; see Supplement A) the lowest for all domains combined.

The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was greater than 0.40 for all domains, while the Barlett-test (test of sphericity) was significant for common agenda (p-value = 0.03) and marginally significant for shared measurement systems (0.06). This could be due to the assumption that each domain mirrored a specific dimension of CI. However, the significant correlations among fewer items might have been blunted by the presence of and dominance by uncorrelated variables. For this reason, we proceeded with a reduction procedure and generated the eigenvalues after rotation for each component in the data. The scree plots for all the domains exhibited a number of small inflexions. Under the criteria of eigenvalue of >1 or Kaiser’s criterion, 10, 11,10,12 and 9 components emerged from the sub-set of items belonging to common agenda, shared measurement systems, mutually reinforcing activities, continuous communications and backbone support respectively. Table 2 presents our results with the percentage of variance accounted for by the components.

Table 2 The CI components satisfying Kaiser’s criteria and the percent variance explained
% of variance explained 60% 63% 59% 65% 61%
CI domain Common agenda (communalities) Shared measurement systems (communalities) Mutually reinforcing activities (communalities) Continuous communications (communalities) Backbone support (communalities)
A3: Everyone in our collaborative group works on the same community issue B14: There is sufficient funding to improve the information system for our collaborative group C15: The leaders in our collaborative are responsive to change that will impact their organizations D6: The collaborative group provides opportunities to share our different points of views E5: For this collaborative group or coalition, an existing organization took on the lead in coordinating the strategies of the collaborative group
A5: Together we developed a plan of action to outline the way in which the identified community problems will be addressed B9: In our collaborative, we routinely participate in learning forums or continuous improvement discussions C16: The efforts of our partners build upon existing activities in the community D13: When communicating internally, member organizations use various channels to communicate more effectively, such as online platforms, facetoface meetings, and teleconferences E16: Board members and other community leaders look to our backbone organization for initiative support, or strategic guidance and or leadership
A7: If there were a problem that affected the whole or most of our community, agencies in our community would work together to deal with the problem B16: Our collaborative group effectively uses webbased technology for data collection C22: Our partners continue to explore new resources D19: We involve community members to identify priority areas of needs E2: In our collaborative, there is a leading organization that guides the vision of the collaborative
A1: Community leaders who are part of our collaborative group are very committed to solve problems through collaboration B2: Our results are measured using the same metrics or indicators C3: Everyone supports the activities of others D16: Leaders of the group take into account all members’ points of view when making important decisions E1: As part of our collaborative, there is a separate organization dedicated to coordinating the various activities of the collaborative
A22: Our partners’ individual work is increasingly aligned with the common agenda B22: Our community partners lack the capacity to implement a shared data system C10: People in this collaborative group know each other very well D15: All partners are equally involved in the decisionmaking of the collaborative group E3: We currently are supported by one funder who initiated the collaborative group (serves as planner, financier, or convener)
A13: Participants in this collaborative group have a lack of commitment to work together B10: We are able to use the data we collect to rapidly implement solutions to the problems found C8: There are persons in this collaborative that others can look up to D9: This collaborative provides a platform for concerns to be addressed E15: In this collaborative, we have a supporting backbone infrastructure which mobilizes funding or seek new resources from public or private sources
A4: Agencies in our community are all working together to achieve significant community change in our locality B13: In our collaborative, there is strong leadership to improve the information systems of the group C11: I feel my voice counts in this collaborative group D8: We have a strong capacity to solve problems together E10: As part of our backbone support we have a facilitator for meetings
A17: People in our community can successfully stand up for their rights B11: Our collaborative group is engaged in ongoing evaluation and quality improvement C17: There is duplication of services among partners in our collaborative D5: Yes, communication is one of our strengths E18: There is a backbone organization that provides support and recognition to partners for their work and leadership
A11: People in our collaborative group can mobilize many supporters in our community B18: Members of our collaborative group have clear expectations about the confidentiality of the data used by the collaborative C9: There are people in this collaborative that I can look up to D10: Ideas discussed in our collaborative are transmitted throughout our organizations E17: All partners coordinate efforts regularly with the backbone organization
A21: All collaborators or partners who are part of our collaborative group publicly discuss/advocate for common agenda goals B3: In our collaborative, we all have common outcomes for which we collect data C4: It is very easy to initiate or coordinate any activities with the help of members of the collaborative D1: As a member of this collaborative, I feel that my interests are valued and protected
B12: If a member of our collaborative group needs help with data collection or analyses, there is ongoing support available D2: People in this collaborative attend meetings to get to know each other better
D21: Partners report increasing levels of trust with one another

Further analysis of the components demonstrated that there were generally minor differences in communalities when these components were arranged in a descending array ranging from the highest to the lowest estimates. For almost all the domains, a plateau was observable after extraction of the top four components (Table 3). This suggests that fewer components could offer the same information. Consequently, a decision was reached to retain the top four components (based on the communalities and relevance) as adequate items that could explain the same magnitude of component variance as the ones in Table 2. A reduction of the number of components using Kaiser’s eigenvalue criteria and communality estimates greatly enhanced the sub-domain correlations which now became statistically significant except for backbone support (Common Agenda, p = 0.045; Shared Measurement Systems, p = 0.038; Mutually Reinforcing Activities, p = 0.028; Continuous Communication, p = 0.020). A notable observation (Table 3) is that all the communalities had an individual value >0.6, which, as per Kaiser’s criterion, was indicative of sufficient communality.

Table 3 Communalities of extracted components (Initial Set at 1.00)
Common agenda Communality (Sum of Squared Factor Loading)
A3: Everyone in our collaborative group works on the same community issue 0.69
A5: Together we developed a plan of action to outline the way in which the identified community problems will be addressed 0.69
A7: If there were a problem that affected the whole or most of our community, agencies in our community would work together to deal with the problem 0.68
A1: Community leaders who are part of our collaborative group are very committed to solve problems through collaboration 0.66
Shared measurement systems
B14: There is sufficient funding to improve the information system for our collaborative group 0.81
B9: In our collaborative, we routinely participate in learning forums or continuous improvement discussions 0.77
B16: Our collaborative group effectively uses webbased technology for data collection 0.75
B2: Our results are measured using the same metrics or indicators 0.69
Mutually reinforcing activities
C15: The leaders in our collaborative are responsive to change that will impact their organizations 0.75
C16: The efforts of our partners build upon existing activities in the community 0.75
C22: Our partners continue to explore new resources 0.71
C3: Everyone supports the activities of others 0.68
Continuous Communications
D6: The collaborative group provides opportunities to share our different points of views 0.76
D13: When communicating internally, member organizations use various channels to communicate more effectively, such as online platforms, facetoface meetings, and teleconferences 0.75
D19: We involve community members to identify priority areas of needs 0.74
D16: Leaders of the group take into account all members’ points of view when making important decisions 0.73
Backbone support
E5: For this collaborative group or coalition, an existing organization took on the lead in coordinating the strategies of the collaborative group 0.79
E16: Board members and other community leaders look to our backbone organization for initiative support, or strategic guidance and or leadership 0.73
E2: In our collaborative, there is a leading organization that guides the vision of the collaborative 0.69
E1: As part of our collaborative, there is a separate organization dedicated to coordinating the various activities of the collaborative 0.68

4. Discussion

Our study results demonstrated that the five tenets of the CI framework do not subscribe to a unidimensional interpretation. We observed that each domain embodies a lens that captures some specific construct of CI. Given this multi-dimensional structure, the instrument may not be suitable for use as a scale but rather should be used as an index. An index is a set of items (questions) that structures or focuses multiple yet distinctly related aspects of a dimension or theoretical domain of behavior, attitudes, or feelings into a single indicator or a score.6 A scale is typically a cluster of items (questions) that syncs into a single domain. Our results showed this was not the case for CI items. Hence, the findings in this study are in agreement with our first hypothesis which posits that CI represents a multi-factorial concept with domains that are multi-dimensional capturing diverse aspects of a construct. Considering that coalition building and successful collaborative partnerships are characterized by a variety of activities at multiple levels framed to maximize utilization of community assets to attain shared objectives, this finding is not surprising. Therefore, in support of our second hypothesis, the instrument we developed more appropriately represents an index rather than a traditional psychometric instrument that captures a simple construct.

The new instrument, referred to as “CI Index” or CII for short, is framed on a Likert scale from 5 to 1 with four items in each CI domain. All the scores are numerically summed across the five domains to yield a maximum score of 100 and a minimum score of 20. Subsequent studies should attempt to compute the subdomain scores to ascertain areas of strength and weakness. Subdomain scores will then potentially range from a maximum of 20 to a minimum of 4. At this pilot stage of instrument development, it is difficult to advise cut-offs correctly without information from future studies that will assess predictive validity of the index in diverse settings. Nonetheless, the following may be recommended for evaluators and practitioners interested in using the tool to assess the impact of collaborative endeavor. For subdomains, we recommend: ≥15 indicates strength; 10-14 indicates promise and <10 indicates weakness. The corresponding total score recommendations for CI are: ≥75 indicates strength; 50-74 indicates promise and <50 indicates weakness of the partnership. These values could change as utilization experiences are gathered through large-scale studies to test the performance and predictive ability of the CII.

5. Conclusion and Global Health Implications

A merit in our study is that we were able to demonstrate the feasibility of measuring CI based on a comprehensive process that entailed a rigorous literature review, expert consensus analysis, stakeholder input, nationwide data collection from a variety of health-related organizations and a meticulous data analysis and evaluation process. Although the CI framework has assumed popularity in health promotion through coalition building,10-12 we are unaware of an evaluated measurement instrument that coalition partners could easily utilize to assess baseline efforts and measure progress over time. Our instrument could be employed to identify specific segments of the collaborative that need more attention in order to attain desired goals.

A limitation of this study is that our search probably missed some non-English publications on the subject matter. However, this is very unlikely given that we employed the phrase “CI” as part of key phrases which could have captured relevant articles in other languages as well. Another limitation of the study is the likelihood that the US population may differ considerably from other settings in terms of the healthcare delivery system. It is therefore, to be recommended that this instrument be tested and validated in other settings.

Constructed as an index, the CII yields a summative score of collaborative efforts across the themes of a common agenda, shared measures, continuous communication and mutually reinforcing activities. Further, if the collaborative group has identified an organization that provides a cohesive leadership and administrative structure to the partnership, the instrument can be utilized to assess the functionality of the backbone organization as well. The inventory can be utilized to identify strengths and weaknesses in the collaborative group, or could be employed to provide an overall score of CI. The utility of the CII is vast, and potential end users include health-related organizations, social service providers, and other agencies with vested interest in public health.

Supplementary

Acknowledgements

We thank all the organizations that took part in the survey.

Conflicts of Interest: The authors have no conflicts of interests to declare.

Funding/Support: This work was supported in part by project H49MC12793 from the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration (HRSA), US Department of Human and Health Services. The funding agency did not play any role in any aspect of the analyses.

Ethics Approval: Approved by Baylor College of Medicine IRB as non-human research.

References

  1. , . A Complex Adaptive Systems Model of Organization Change. Nonlinear Dynamics, Psychology, and Life Sciences. 1997;1(1):69-97.
    [Google Scholar]
  2. , , . Understanding health care delivery as a complex system Achieving best possible health outcomes for individuals and communities by focusing on interdependencies. Journal of Evaluation in Clinical Practice. 2014;20(6):1005-1009.
    [Google Scholar]
  3. , , . Project CHARGE:Building an Urban Health Policy Advocacy Community. Progress in Community Health Partnerships-Research Education and Action. 2012;6(1):17-23.
    [Google Scholar]
  4. , , . Building a consensus for expanding health coverage. Health Aff (Millwood). 2001;20(1):40-48.
    [Google Scholar]
  5. , , . Stanford Social Innovation Review 2011(Winter 2011) :36-41.
  6. , , . Cross-National Indices with Gender-Differentiated Data:What Do They Measure?How Valid Are They? Social Indicators Research. 2013;111(3):801-838.
    [Google Scholar]
  7. , , , . Effectiveness of a federal Healthy Start Program on HIV/AIDS risk reduction among women in Hillsborough County, Florida. South Med J. 2015;108(4):235-241.
    [Google Scholar]
  8. , , , , , . Healthy start program and feto-infant morbidity outcomes:evaluation of program effectiveness. Maternal and Child Health Journal. 2009;13(1):56-65.
    [Google Scholar]
  9. , . The Application of Electronic-Computers to Factor-Analysis. Educational and Psychological Measurement. 1960;20(1):141-151.
    [Google Scholar]
  10. . LiveWell Colorado 2016 https://livewellcolorado.org/
  11. . . Lets Go!The Barbara Bush Children's Hospital at Maine Medical Center. http://www.letsgo.org
  12. . . Community Health Improvement Partners (CHIP). http://www.sdchip.org/initiatives/san-diego-county-childhood-obesity-initiative/
  13. , , , . Stanford Social Innovation Review 2012 https://ssir.org/articles/entry/channeling_change_making_collective_impact_work
Show Sections