SECTION 4. M&E, LEARNING AND DOCUMENTATION OF STAKEHOLDER AND COMMUNITY ENGAGEMENT AND QUALITY IMPROVEMENT PARTNERSHIP
Some initiatives may have, for the first time, integrated stakeholder and community engagement more systematically in their programme, based on a stakeholder and community mapping and analysis and after going through a process whereby collaborative forms of engagement were sought. It is important to monitor, evaluate and document the process and impact of this engagement, to learn what worked and what didn’t, and to establish evidence of the impact of stakeholder and community engagement on the effectiveness of the QI initiative. Existing evidence is mixed and experiences of QI teams will greatly contribute to understanding the benefits of and barriers to engagement.
When new partnerships are formed in the context of a QI initiative or when a multi-stakeholder TWG has been established, it is useful to conduct a partnership assessment: this is a planning, M&E tool that defines the features of a successful partnership. It can provide feedback on the current status of the partnership, suggest areas that need further support and work, and help to evaluate how well the collaboration is going.
Monitoring and documentation
During implementation of the QI initiative, document positive, negative, or unexpected contributions of stakeholders in decision-making, planning and M&E of the QI initiative. Where possible, include other resources for validation (policy memos, meeting minutes, etc.). Insights can be continuously shared and enriched by organizing network meetings and learning events across initiatives and settings. (38)
Evaluation of stakeholder and community engagement
Before engaging in an evaluation exercise, reflect on the purposes and needs of the evaluation:
- What are the results you would like to obtain from the evaluation?
- What specifically will you learn? What will others learn from this if you do it?
- What do you really need to know?
- What data do you need to understand processes and outcomes? What data do you need to understand why your partnership does or does not function effectively?
Goals and benefits of engagement will differ per programme or intervention, and hence the scope of the evaluation will differ. In the spirit of partnership building, evaluation should be collaborative by eliciting stakeholders’ voices – including that of women and communities – in the design and implementation of the evaluation. There are a few generic questions and indicators QI teams can use to assess the process and impact of their engagement activities. They are presented in Table 5.
It is important to provide feedback to the stakeholders and communities on the process and results of both the QI initiatives and the stakeholder and community engagement. If this is not done, it can lead to individuals and groups being reluctant to be further engaged or to engage in the future.
The QI team should document how they are improving the QoC, what challenges they are facing, and the results of their improvement efforts. Such documentation can be shared with other facilities, stakeholders and community members at facility and district levels.
Documentation and sharing should lead to the formulation of concrete recommendations to improve and adapt processes of engagement in QI initiatives in the future. Recommendations can be formulated with the support of the recommendations outlined in Table 6. Box 7 lists some resources supporting M&E.
Box 7. Resources supporting M&E
Tools and references for M&E of stakeholder and community engagement
- VicHealth (2016). Evaluation of partnerships: partnership checklist (p. 5–7). The partnership analysis tool.
- Jansen et al. (2018). Checklist for stakeholder participation (p. 975). The TWG can use the checklist at the planning stage to identify important stakeholders and to review partnership processes. It supports the identification of strategies for improvement of stakeholder engagement.
- Save the Children (2008). Team effectiveness questionnaire: simple tool to discuss the functioning of multi- stakeholder QI teams (PDQ-Youth guide, p. 109).
- UNICEF Minimum quality standards and indicators for community engagement [in development] provides guidance on how to collect quantitative and qualitative measures of community engagement processes and outputs. The tool can be applied to situations of comprehensive community engagement, such as in a partnership approach described in this module, at the evaluation stage, and also at the community engagement planning stage.
- Network for improving quality of care for maternal newborn and child health (p. 14). Indicators to assess the accountability of a QI programme.
Implementation guide for improving quality of care for maternal newborn and child health.
Evaluating participation: a guide and toolkit for health and social care practitioners. Glasgow: Scottish Health Council; 2013 (http:// scottishhealthcouncil.org/publications/research/evaluation_toolkit.aspx#.XjLa6C17FQI).
UNICEF. Minimum quality standards and indicators for community engagement. Forthcoming, standards, pp. 44–52. https://www. unicef.org/mena/reports/community-engagement-standards