HRSA Transplant Center Growth and Management Collaborative: Best Practices Evaluation Report - September 2007

This study employs a qualitative, case study approach to identifying transplant center practices that are associated with high organ transplantation rates, while maintaining expected or higher than expected patient and graft survival outcomes. In order to identify and spread these best practices to transplant programs across the country to help them effectively grow, a sample of 15 transplant centers and 34 organ programs that are among the National leaders in number of organs transplanted with expected or higher than expected outcomes was selected for this study. The primary sources of data on the factors that contribute to success in high organ transplantation rates were face-to-face interviews with staff of transplant centers. In total, more than 450 people were interviewed for this study. Following the data collection phase of the study, the findings were analyzed, and best practices were assembled.

A. Site Selection

Transplant centers were selected for this study using data from the Organ Procurement and Transplantation Network (OPTN) provided by HRSA. Data for 275 institutions were provided, which included deceased donor transplant volume from 2000 to the first 6 months of 2006 for heart, kidney, liver, lung, and pancreas transplants. Six Veterans Affairs transplant centers were excluded from the analysis, resulting in a final sample size of 269 centers. The transplant volume data were combined with quality measures for each center and type of organ transplanted to rank each center and program.

To rank and select the high-performing transplant centers and organ programs, the following four criteria were used to measure volume, growth, and quality:

• High Volume: Centers in the top 10 percent for number of transplants performed in 2005 and in the top 10 percent for average number of transplants performed from 2000 to 2006.
• High Growth: Centers in the top 10 percent for average annual absolute change from 2000 to 2005. Absolute change is the number of transplants from one year to the next.
• Low Graft Failures: Centers with lower than expected graft failures 3 years post-transplant. The methodology utilized by the Scientific Registry of Transplant Recipients (SRTR) was followed.
• Low Patient Mortality: Centers with lower than expected patient mortality 3 years post-transplant. The methodology utilized by the SRTR was followed.

The following additional criteria were also considered when selecting the high-performing centers and programs:

• Donor information (percent SCD/ECD/DCD)
• Whether the center performs pediatric transplants
• Geographic diversity
• Percent imported organs
• Waitlist mortality
• Organ type representation

The data used for the site selection process had several limitations. First, pancreas quality data, including observed versus expected graft failures and patient mortality, were not available. Therefore, the pancreas transplant programs were only analyzed in terms of volume and growth in number of transplants performed from 2000-2006. Another limitation was missing data on number of transplants. If a center’s data on number of transplants were missing for any year between 2000 and 2006, the center was excluded from the analysis. These centers were excluded because no trend data were available for the center to determine if it was a high volume or high growth center. Lastly, given that recent data may best reflect current practices, we included the incomplete 2006 data and doubled the number of transplants in an attempt to estimate volume for the entire year.

Exhibit 1 lists the high-performing transplant centers and organ programs that were selected for the study based on the data analysis. Appendix C includes data on organ transplantation rates and patient and graft survival outcomes for these centers and programs.

Exhibit 1:
Selected High-Performing Transplant Centers and Organ Programs

B. Data Collection and Synthesis

HRSA extended invitations by email to transplant center leadership to participate in the study. All of the selected transplant centers accepted. Background information about the transplant centers was collected through the centers’ Web sites.

Site visits were used to collect data on best practices of transplant centers that have achieved high rates of organ transplantation, while maintaining expected or higher than expected patient and graft survival outcomes. These site visits involved extensive series of in-person discussions with transplant center staff. Transplant centers recommended key informants for these discussions. Depending on the preferences and availability of key informants at each transplant center, discussions were conducted either individually or in a group. Individual discussions ranged from 30 to 60 minutes in length; group discussions lasted as long as 2 hours.

Discussions with informants did not follow a strict format. The purpose of these discussions was to determine what the informants perceived, from their various perspectives, to be the factors that contribute to the high rates of organ acceptance and transplantation, while maintaining expected or higher than expected patient and graft survival outcomes. Informants usually identified what they considered to be the most important factors or practices associated with accepting and transplanting more organs. More extensive probing of informants’ initial observations and inquiring about general areas of potential that they did not cite initially (which varied across informants), provided the opportunity to identify other relevant factors.

Probes addressed transplant center policies, procedures, management, administrative, and other clinical, behavioral, cultural, organizational, and financial practices associated with high performance in organ acceptance, transplantation, and outcomes. The following are examples of areas probed:

• Overall success factors
• Transplant center leadership and commitment
• Business/financial arrangements and issues
• Operations and communications
• Social, behavioral, and cultural factors
• Staffing issues
• Clinical practices in organ recovery, transplant, and post-transplant management
• Planning and evaluation activities
• Information technology and data collection capacity
• Interaction with OPO

Standards for answers included concrete, descriptive language, consistency, and evidence,
where relevant. Wherever possible, opinions were grounded with specific examples; when they
were not, they were recorded as ungrounded for study purposes.

In total, 465 individuals were interviewed for this study, including transplant surgeons, physicians, nurse coordinators/managers, social workers, transplant administrative staff, leadership/administration, other physicians, and non-physician clinicians. Exhibit 2 shows the distribution of interviewees by staff type and by site visited.

Qualitative data from on-site interviews were analyzed in both an internal and external debriefing process. Internal debriefings were conducted throughout the site visit process during which team members reviewed site visit experiences and observations on-site and shared themes with other off-site team members in real-time, so they could be tested during concurrent and/or subsequent site visits. External debriefings were also conducted on the last day of each site visit with key transplant center staff interviewed, HRSA Division of Transplantation (DoT) staff, and HRSA guests from other transplant centers and organizations. The purpose of these debrief meetings was to review and validate the site visit findings and emerging best practices.

After the site visits were completed, data were compiled and analyzed to assemble a Change Package Document, consisting of a set of strategies/drivers and corresponding key change concepts and action items associated with high organ transplantation rates and expected or higher than expected patient and graft survival outcomes. The Change Package Document, including the strategies/drivers, key change concepts, and action items, was vetted with an expert panel that included representatives from transplant centers included in the study and HRSA guests from other transplant centers and organizations. Appendix A and Appendix B include a copy of the Change Package Document and a list of the expert panel members, respectively.

C. Study Limitations

The purpose of this study was not to isolate true cause-and-effect relationships between practices and performance measures, but to identify promising or likely best practices and to seek to validate them across sites. This was essentially a retrospective observational study intended to identify practices conducted to date that are likely to have contributed to recent historical (2000-2006) performance in terms of organs transplanted and patient and graft survival outcomes. This study is an initial phase of identifying and sharing “what works” across transplant centers to obtain higher numbers of organs accepted and transplanted.

For the purpose of identifying true best practices (i.e., that are known to be causally related to high performance in transplantation and outcomes), this study has several limitations, including the following.

• Small sample. Due to time and resource constraints, only a limited number of site visits could be conducted for this study. Based upon just this sample of 15 transplant centers, it is evident that organ acceptance and transplantation and patient care practices vary across the country. As a result, other best practices might not have been identified within the limited scope of this study. Also, certain practices considered to be “best” based on their appearance in some or all of this limited sample might not have been confirmed as such given a larger sample of observations. Similarly, some best practices among our sample of transplant centers may be artifacts or otherwise specific to those institutions, and therefore, not generalizable to other institutions.
• No control group. This study did not compare the practices of higher-performing transplant centers with the practices of lower-performing centers. Including such controls in this study would have enabled a more valid distinction between practices that simply co-exist with, but do not contribute to, higher performance and those practices that exist more often in higher-performing centers and less often in lower-performing centers.
• Limited perspectives. The practices that contribute to higher performance in organ transplantation and outcomes involve or affect many parties. Although information was collected about potential best practices from a wide range of transplant center staff who, as a group, are very likely to be aware of most potential existing best practices, it is possible that some best practices were overlooked by not involving other parties with perspectives not encompassed in this study. In particular, given the limited study scope, the study team did not have the opportunity to hear the perspectives of transplant recipients and families involved in the organ transplant process.
• Halo effect.³⁰ In this best practices study, transplant center staff were aware that they were participating in a study based on their higher numbers of organs transplanted and expected or higher than expected patient and graft survival outcomes and were being asked to identify (i.e., observe and report) what factors might contribute to this. The transplant centers may have considered that, ‘We must be doing something right,’ or, ‘Whatever we’re doing must be working,’ and categorized some practices not associated with more organs transplanted and expected or higher than expected patient and graft survival outcomes as “best” practices. To diminish the impact of the halo effect, to the extent possible, best practices were validated or c onfirmed across multiple sites.
• Hawthorne effect.³¹ Rather than being an interventional study, this was a retrospective observational study of best practices. Interviewees were aware of being observed; however, the observations were of potential causes of effects (e.g., number of organs transplanted that had been recorded previously). Although this was not a setting for the Hawthorne effect in its traditional form, there may be a Hawthorne effect at work affecting future performance. Many interviewees have noted that, as a result of being visited and interviewed and having the opportunity to reflect on their work, they identified what they had previously considered to be certain typical routine practices as being likely best practices. Aside from contributing to a potential halo effect as described above, this may enable sites to codify and track these practices internally and share them with others when they might not otherwise have done so. Further, some interviewees noted that the reflection prompted by the interview process, as well as the feedback provided to them about their center’s high performance, have led to performance improvements. While unintentional, this effect reportedly has motivated staff at the sites to rethink how they might improve existing policies, procedures, and practices and may have left the “observed” sites in a stronger position to think creatively and mobilize toward further improvements that might benefit themselves as well as others through the sharing of the study’s best practices.³²

³⁰The halo effect refers to a bias in observation or measurement that reflects an observer’s tendency to rate, perhaps unintentionally, a person or event or other phenomenon in a manner that is consistent with what the observer anticipated.
³¹In the Hawthorne effect, the act itself of observing people may prompt them to change their behavior. This might result, for instance, in subjects improving their performance due to their knowledge of being observed rather than due to an intervention such as training or use of some technology.
³²Similarly, in describing how the Hawthorne effect can contribute to improved performance, S.W. Draper notes, “This might be because attention made the workers feel better; or because it caused them to reflect on their work and [this] reflection caused performance improvements, or because the experimental situation provided them with performance feedback they didn’t otherwise have and this extra information allowed improvements.” (Draper SW, University of Glasgow, 2005 ).