The challenges of implementing a data use culture

The Tanzania Electronic Immunization Registry allows health workers to review immunization data in real-time. Photo: © PATH/Trevor Snapp

Vaccines have proven to be one of the best investments in improving people’s health around the globe. In 1974, the World Health Organization established the Expanded Programme on Immunization with the objective of vaccinating children worldwide. However, in many countries, immunization programs suffer inefficiencies and challenges stemming from poor data quality and use. Increasing or maintaining a high coverage rate is challenging when data quality, use, and accessibility are poor. The Tanzania Ministry of Health, Community Development, Gender, Elderly and Children identified the most critical data quality issues in routine immunizations, and a PATH team collaborated with a user advisory group in Arusha Region to develop a package of scaleable interventions across Tanzania to increase coverage and equity of immunization services. The interventions were implemented in 285 facilities over 2014-2017 in seven districts in the region over three phases, and adjustments were made to the implementation methodology between each phase in response to lessons learned.

Local setting

Arusha Region was chosen given its mix of urban, peri-urban, and rural settings; pastoral communities; and strong leadership at all levels, as well as its porous border with Kenya where families cross to receive services in Tanzania. Despite high immunization coverage rates in some districts of Arusha, several have less than 80 percent coverage.

Approach

The package of interventions covered three key areas: an electronic immunization registry, stock management at the facility level, and strengthening a data use culture, which included organizing peer networks through WhatsApp, producing micro-training videos, and developing data use guides. The PATH implementation team, comprising members with backgrounds in information and communication technology, operations, and change management, led the first two implementation phases with

1. user-centered design and testing of the interventions and
2. implementation of the interventions in the first five districts in Arusha.
   
   

The team then shifted into partnering with district data use mentors who led the third implementation phase in the two additional districts.

The first phase occurred from mid-2014 through early 2015 in Arusha City district. During this phase, the interventions were designed and tested in an iterative process with the user advisory group, which included community, facility, district, and regional representatives. This user-centered design process ensured the package of interventions would have the context to address identified challenges and allow for scale. Several testing sessions were completed in facilities and the district to frame the implementation process for on-site training.

In the second phase, the PATH team implemented in four more districts. This phase occurred from mid-2015 through late 2016; the prolonged time frame was due to changes made in the software platform for the electronic registry. On-site training was conducted to ensure nurses were trained on the interventions within the context of clinical care workflows and procedures.

Figure 1: Depicts the timeline of training at district and facility levels (which was also adapted for BID Initiative work in Zambia).

The third phase began in early 2017 in the remaining two districts. After a “learning and appraisal meeting” in September 2016, the district staff and PATH team reflected on lessons learned and agreed that going forward interventions would be deployed by engaged district government staff with support from a PATH team focal person. The phase three districts were implemented following this methodology by district team members known as “data use mentors.” Mentors were selected based on knowledge of the health sector, experience using data for decision-making, and basic computer skills. The mentors were inaugurated and given their roles in front of district administrative and political figures to increase accountability and commitment. The mentors were then trained by PATH staff to implement the package of interventions.

This methodology from phase three was then used as the model for Tanga and Kilimanjaro regions. The mentors introduced the tools and interventions over a three-month period to allow time for facilities to fully integrate the interventions into their service delivery, with mentors providing continuous monitoring and supportive supervision.

Women and children wait for vaccinations at a health facility in Tanzania. Photo: © PATH/Trevor Snapp

Adjusting of the method in order to creat better ownership

PATH staff planned five visits per facility in phase one; this was reduced to four in phase two due to budget constraints. In phase one, the PATH team modified the timing between touches (facility visits), specifically the timing between the first and second touches, which was decreased from five weeks to two weeks. The initial longer time was to allow the PATH team to enter data from the paper forms into the electronic registry. However, as data quality in the paper registers was incredibly poor, the back-entry process proved to be more costly than helpful. Therefore, the timing between touch one and touch two was reduced. In phase three, the nurses back-entered data into the tablet for the electronic registry directly from child health cards.

Training in phase two began with immunization department nurses, and later shifted to include all nurses available in each facility on the day of training, to address the challenge of knowledge lost due to high staff rotation between departments and facilities. This improved capacity-building across each facility, and it created a greater sense of ownership as the tools became the facility’s shared responsibility.

The most significant change between the second and third phases was the shift of leadership of the touches from the PATH implementation team to the mentors to establish a fully government-owned model. As unpublished data from the PATH costing work in 2015-2017 shows, the government-owned model reduced implementation costs, with a decrease between 10%-65% per district with the shift to the government-led strategy (Figure 2). At the health facility level, it is even clearer the difference in implementation costs, with significant reductions as seen in Figure 3 in the Arusha region districts.

Figure 2: Cost per district for each implementation strategy.

Figure 3: Cost per health facility in each district for each implementation strategy.

The shift in strategy also impacted the time needed to achieve regional scale. Where the PATH-led method took four to seven months to implement in the early implementation districts in Arusha Region, the government-led model allowed implementation in all of Tanga Region in two months, as each district’s mentors could be deployed simultaneously.

Lessons learned

Taking an iterative design and implementation approach, making adjustments based on lessons learned, and engaging stakeholders resulted in an efficient and sustainable model to take to scale. After developing the on-site, government-led training model over three phases in Arusha Region, the model was used in Tanga and Kilimanjaro regions.

Engaging a user advisory group with representation from different backgrounds and levels of the health system was critical in the design and development of interventions like the registry. Feedback from the advisory group supported an iterative design process for a package of interventions with country context to address the data-related challenges in Tanzania’s immunization program. These interventions are now scaling nationally with strong leadership from all levels of the health system.

Training of staff from multiple departments within facilities was a lesson from phase two that helped to address the challenges of staff rotation between departments and facilities, and it increased the level of ownership of the tools.

The PATH-led approach of the second phase was critical for solidifying the interventions but too expensive to scale and did not allow for full government ownership. Shifting to the mentor-led strategy reduced costs and allowed for faster rollout, making it possible for the government to plan for scale beyond the three regions.

Additionally, once the mentors led the process with facilities, nurses knew the government backed the initiative. This strengthened the adoption of interventions and sustainability of their use.

Summary of main lessons learned

- User-centered design of interventions contributes to government ownership; the designed interventions can include country context, and only the interventions found relevant during the testing phase are deployed for use.

- On-site training can be more effective than classroom-style training because more facility staff can participate and interventions are deployed within the context of clinical workflows and data use at the facility.

- Government-led models of implementation with data use mentors strengthen sustainability and efficiency of rollout and adoption of data quality and use interventions.