Using Data Quality Audit as a Strategy for Improved Sexual Reproductive Health and Rights Programming

Data Quality Audit is a critical process that entails constant assessment of a program’s data, identifies gaps, and informs correction for improved data quality. A majority of donor-funded programs rely on targets and reporting on achievements at the end of the implementation period to track progress. For this reason, it is critical for such institutions to report data that is accurate and complete, as this informs the next steps of the program in achieving the program’s aim. The quality of data generated from a program is a critical function of the program’s M&E systems and data verification processes. Tropical Institute of Community Health and Development (TICH) implemented a project for young people- Get Up Speak Out (GUSO) with an aim to achieve enjoyment of young people’s sexual and reproductive health and rights. The TICH-GUSO project adopted the USAID Guidelines to evaluate the M&E system, data verification processes and the data quality reported by program outcomes. The DQA was done twice at an interval of six months. Each DQA process entailed a two-stage process that entailed objective measurement of the M&E system and a data verification process to assess the data accuracy and completeness. The DQA process was done at the institution (data centre), where all the primary and secondary data are stored. The DQA process evidenced that audit and feedback facilitate learning and improvement. The second DQA recorded an improvement across all the sectors (M&E system, data verification process, and data quality). DQA processes are critical components of program implementation since they help identify weaknesses hence informing the type of correctional intervention needed to produce quality data, reports, and evidence for strengthening program implementation, future programming, policy recommendation and further research where needed. It is primary that programs and institutions at large adopt DQA processes for continuous improvement.


INTRODUCTION
Donor funded programs are often working towards ambitious targets hence the need for robust means of measuring success. Refining the running of these programs rely primarily on robust Monitoring and Evaluation (M&E) structures that lead to data of high quality, therefore, informing further program planning and execution (USAID, 2008).
The main objectives of conducting a Data Quality Audit (DQA) include: verifying the quality of reported data; and assessing the underlying data management and reporting systems hence an assessment of the entire M&E system (USAID, 2008). It is crucial that programs conduct Routine Data Quality Audits (RDQA), a simplified version of a DQA, to facilitate programs in assessing their data, thereby strengthening their systems with respect to data organisation and presentation (USAID, 2008).
Data quality precisely refers to the accuracy, completeness, reliability, timeliness, integrity, and confidentiality of the reported data across all the levels of an organisation's data flow system. The main factors that guide the assessment of data quality include the M&E organisation and abilities, Pointer descriptions and presentation procedures, Collection of data and presentation tools, and the processes of management of data, including the availability of standard operating procedures with reference to data management (USAID, 2008).
According to Kiwanuka et al. (2017), Health program planning should be based on evidence; however, precedence setting in Countries considered as Low-and Middle-Income (LMIC) is ad-hoc and rarely pegged on evidence. Evidence-Based Organisation is the practise of decision making in addressing an issue based on credible information with an aim of achieving the best results (PEPFAR, 2014). In order to rightfully implement the right strategies in terms of place and time, programs ought to have accurate, reliable, and timely data (quality data) so as to focus on populations most in need and tailor strategies with reference to evidence-based findings (PEPFAR, 2014). In addition, studies emphasise data completeness for high-quality data to support the high-quality implementation of programs due to the availability of reliable data for decision making (Xiao et al., 2017).
It is in line with this, that TICH-GUSO conducts Annual Data Quality Audits both internal and external (Courtesy of the SRHR Alliance GUSO M&E System strengthening). The aim of conducting DQA for the TICH-GUSO program was to assess: Outcome Performance across the four outcome areas of GUSO implementation (Meaningful youth participation; Young people accessing comprehensive SRHR education and information; Young people's access to quality SRHR services, and creating an enabling environment supporting young people's SRHR) targeting young people's enjoyment of their Sexual and Reproductive Health as shown in Table 1 below and assessing the M&E system for TICH-GUSO data management and reporting processes. The first DQA was conducted in October 2019 (Assessing the January-June 2019 reports), with a follow up conducted in May 2020 (Addressing the July-Dec program reports 2019). % of young people (under 25) representation in your organisation's structures and decisionmaking processes 2a2.
% of young adults (aged 25-30) representation in your organisation's structures and decisionmaking processes 2b.
Number of collaborations among young people from different alliance related organisations/ networks that represent the youth constituency OUTCOME AREA 3 Increased utilisation of comprehensive SRHR information and education by all people 3a.
Number of educators trained 3b1. Number of young people reached with (comprehensive) SRHR education 3b2. Number of young people reached with (comprehensive) SRHR information OUTCOME AREA 4 Increased utilisation of high-quality SRH services that respond to the needs and rights of all young people 4a.
Number of service providers who have been trained in YFS 4b.1 Number of direct SRH services provided to young people 4b.2 Number of indirect SRH services provided to young people 4b.3 Number of condoms provided directly to young people 4b.4 Number of condoms provided indirectly to young people OUTCOME AREA 5 The improved socio-cultural, political, and legal environment for young people's SRHR 5a.
A number of people are reached by campaigns and (social) media.

5b.
Number of people structurally involved in the implementation of the programme at the community level (for example, young people groups, CBOs, peer educators)

MATERIALS AND METHODS
The DQA process was guided by a DQA tool in two steps as follows: Checking of data management and presentation structures; and authentication of presented data for strategic pointers at designated sites. Based on the two objectives of DQA, the process was guided by two practises: System Assessment and Data Verification Protocols (USAID, 2008). Table 2 illustrates the M&E systems assessment objectively. Do you collect and present data using standard and systematically used forms? 6 Is there required precision in indicator measurement through recording of data? 7 Do you maintain your data based on national and global confidentiality measures? 8 Is the storage and availability of your source document in line with existing policies? Data Organization Practices 9 Is there an existing documentation with regard to aggregation, collection and manipulation of data? 10 Are defies in quality of data and tackling mechanisms identified and put in place? 11 Are discrepancies in information noted and addressed using clearly defined means? 12 Is verification of source data done under properly set out and adhered to procedures?
The second stage of the DQA process (Data authentication of presented data for primary pointers) cross-checks reported results against the primary data sources flagging out transcription errors across the program/project levels of the M&E unit. The data verification was conducted in two phases as follows: Detailed verification at points of delivery of service and supplement proofs at transitional collection levels at the task M&E department.

In all Cases
Cross-Checks Undertake "verifications" of the checked report matches with other sources of data (e.g., attendance lists, minutes, schedules etc.).

Verification Description Required
Spot-Checks Undertake "spot-checks" to check the exact transfer of services and/or wares to the marked populaces.

If feasible
Assessment of transitional collection level majorly focuses on the verification by document review and trace identification hence assessing the reported data, its accuracy, timeliness of the data, its completeness and its availability (USAID, 2008).
Controlled site strategy was used (one selected site) for the DQA. The DQA was conducted at TICH-GUSO Central Database Point, where data from all implementing sites, M&E Data, and Data submitted to the donor (SIMAVI) were available. The only data that was not included in the process was the primary data at the link health Facilities (Indirect services provided to the young people). The benefit of this approach is that the process maximised efforts on the site and with better control on application of the audit protocols and familiarity of the field-specific structures from which the outcomes resulted (USAID, 2008).
Studies show that this method is ultimate for evaluating variation in the quality of data which can be attributed to a data quality improvement application (e.g. training on processes of data organization). In this tactic, the audit on the quality of data was realized at TICH-GUSO Central Database Point; interventions were conducted towards the recommendations from the first DQA, followed by a second data quality audit on the same site. Studies have proven that variations in data quality is most likely attributable to the intervention (USAID, 2008).

RESULTS
The first data quality audit brought out the following findings on M&E systems: M&E Structures were weak in 2019 (1.83), and the identified areas of weakness included poor documentation of M&E roles of all staff in the GUSO program as well as training of the GUSO staff on their M&E roles; Data Collection tools scored averagely well (2) with weak points identified in the documentation of who and when data is collected; Data management scores (1.67) with the identified weak points including documentation of timelines for data submission across the levels, documentation of data flow on areas of reporting and data storage via a shared drive for easy accessibility by authorised persons and data security from damage; Indicator definition (3) performed well as all indicators were clearly outlined on what is to be reported and how with the appropriate segregations outlined as shown in Figure 1.

Figure 1: M&E System Assessment
Following this, TICH Implemented a process to integrate recommendations arising from the baseline DQA. Adoption and implementation of these recommendations were also integrated into the weekly and monthly M&E review meetings. A continuous review of activities, including capacity for staff in their M&E roles, was undertaken.
The second data quality audit conducted after six months of implementing baseline improvement actions was conducted in May 2020. The results indicated an improvement overall across the M&E System indicators from an overall score of 2.13 to 2.9 ( Figure 2) with major enhancements in these areas: Systems in M&E had improved to (2.58), where documentation and training of staff on their respective M&E roles had been conducted; Data Collection tools scores improved (3) with weak points identified in the documentation of who and when data is collected; Data management scores improved to (3) with improved documentation of data flow and submission deadlines, documentation of data flow with an electronic database system developed (Google Drive) for easy accessibility by authorised persons and data security from damage; Indicator definition (3) maintained the good performance as all indicators were clearly outlined on what is to be reported and how with the appropriate segregations outlined (Figure 3). The second phase of DQA entailed data verification by outcome areas (Figure 3). The data verification in 2019 had an average score of (1.8) (Figure 4) with weak performance across the outcome areas (OA1-2.3; OA3-1; OA4-2; OA5-2). The gaps identified across the M&E report included: high transcription errors across the reporting levels (maximum error noted at 8% of expected values) due to the long reporting chain. The reporting system included 5 levels (Primary documents-Field reports-Regional Team Leaders-Outcome Area Leads-M&E). Additionally, some Primary documents were not traceable as the custodians were unavailable. This called for the development and adoption of common cloud storage for all GUSO data for easy accessibility and retrieval.

Figure 3: Data Verification by GUSO Outcome Areas across M&E Unit Levels
The 2020 DQA indicated a significant improvement in average data verification performance (2.4; Figure 4). The Outcome areas as well improved on accuracy and completeness of data as per the following codes (OA1-3; OA3-2.3; OA4-2.6; OA5-3). The second verification of data assessment reported minimum transcription errors; this is because: the data chain was reduced to 4 levels (Primary documents-Field reports-Regional Team Leaders-M&E) hence minimising the transcription errors (maximum error noted at 3% of expected values). Additionally, the program data management system had adopted the cloud data storage system; hence the primary documents were accessible for reporting.

DISCUSSION
Audit and feedback contribute significantly to improved quality of program performance (Jamtvedt et al., 2019). In most instances, professionals have limited capacity to assess their performance accurately (Davis et al., 2006). For this reason, information about their performance is a critical motivating factor that leads to better performance. In line with this, the feedback from the first audit was relayed to the project staff, followed by capacity building sessions to address identified gaps. The second DQA recorded an overall improvement in all program outcome reporting in terms of accuracy and completeness of data as a function of the M&E system and data verification performance.
Data is currently a key resource in informed decision making. Data quality is often influenced by a myriad of factors including but not limited to: time, resources, organisational support, professional capacity, and pragmatic considerations, among others (Jamtvedt et al., 2019). Quality data is a function of the program's M&E system and data verification processes. Quality data is additionally important for continuous improvement of the program's outcome reporting performance in health programs and other spheres (Xiao et al., 2017). This is in line with DQA results in this study, where outcome performance was low in the first audit, and an improvement was observed upon feedback and capacity building among staff on M&E roles, data quality and data verification process hence a structured M&E system.

CONCLUSION
DQA processes are critical components of program implementation since they help identify weaknesses hence informing the type of correctional intervention needed to produce quality data, reports, and evidence for strengthening program implementation, future programming, policy, and further research where needed

RECOMMENDATION
Programs should consider training their M&E staff on DQA and rolling out routine DQAs for constant assessment and feedback hence improving data availability, accuracy, and completeness for its reference in program planning and implementation.

ACKNOWLEDGEMENT
I would like to thank everyone who contributed to this manuscript through project implementation, critique, and review of the document. Thank you for your dedication and commitment to seeing me through the entire process of completing this manuscript. Thank you all for your support and patience.