“WHAT DOESN’T KILL YOU MAKES YOU STRONGER.” …the messy business of generating and measuring impact

[Lindelwa Nxele]

A Learning in Action reflection by Richard. M. Angelo, Policy Forum – Tanzania

Policy Forum adopted social accountability monitoring (SAM) in 2008 as one of its strategic approaches to capacitate its members working at the sub-national level in Tanzania to engage in policy processes more meaningfully. Since then, I have seen SAM initiatives contribute to building teacher’s housing at remote schools, the operationalisation of primary health care facilities, and the empowerment of communities to lead participatory forest governance efforts among others. I also had to come to terms with our persistent failure to obtain a sustained shift in behaviour at the eco-system level to sustainably improve the way public resources are managed. Of course, the many factors and actors influencing a change at this level make this an arguably unrealistic objective for a single CSO network. The question was then how to develop a realistic and measurable strategy that keeps us focused on the desired result while providing an appropriate balance between flexibility and rigour.

A disappointing impact evaluation in 2016 motivated our decision to take part in a Learning Pilot Exercise,  both of which influenced our decision to rethink our strategic approach to SAM. Instead of merely providing formal training to a growing ecosystem of SAM actors and assuming that this would change behaviour at scale, we decided to be deliberate about which capacities to build, for whom, and why. We did this because we wanted to define for ourselves what our impact would be measured against through a fit-for-purpose MEL[1] strategy so that future evaluations contained no surprises. We then redesigned our SAM strategy to reflect our new strategic approach. Our most persistent problem was that of accountability for the implementation of SAM findings at the subnational level. We therefore made a strategic decision to focus on working with Councillors because of their legal power to enforce their recommendations. It therefore made sense for us to focus on improving their capacity to make informed recommendations and to enforce timely, appropriate and meaningful corrective action. We chose to work in four under-performing local government authorities (LGAs)[2] (according to CAG[3] reports) where at least one Policy Forum member was also working. In December 2016, we began to train Councillors on a version of SAM training that has been tailor-made for their oversight role. Since then, 106 Councillors have been trained.

The feedback from both councillors and the local government officials they oversee has been overwhelmingly positive on the impact of training on their ability to implement their role. In fact, we are now getting requests from councillors from other LGAs who have heard from their colleagues about how this capacity building has benefitted them. Earlier this year, our work with councillors was evaluated and the findings recommended that we consider mechanisms to scale up the intervention and ensure its sustainability.

Despite the success we have had, three main things still keep me up at night:

• Councillors are elected for five-year terms at the end of which they might not return to the role. Even those who have been trained require on-going support to ensure that relevant capacities are built and used. How do we ensure that the capacity to perform effective oversight is retained and improved within Councils even beyond the electoral term?
  • To address this Policy Forum is currently negotiating a partnership with the Local Government Training Institute, a government training institution for public officials at the subnational level, to collaborate to institutionalise the training so that it can become part of their institutionalised capacity building support to LGAs across the country.
• Capacity is not the only thing that drives the decisions and actions of politicians. How do we ensure that we take this into account in setting goals and developing realistic indicators? Better yet, how do we navigate the other political and non-political incentives in a way that motivates councillors to use their newly acquired capacities to effectively perform their oversight role accountably?
  • This is a difficult one, but we at Policy Forum have begun to invest in our learning more strategically beginning with our new PMEL[4] strategy. In implementing the strategy, we have also begun to set up an electronic management information system to ensure that what we learn individually is available to others within the Policy Forum secretariat and membership, both currently and in future.
• Our primary goal in training councillors is for them to ensure that LGAs become and remain accountable to their constituents and communities in an informed and inclusive way. How can Policy Forum strengthen the influence of citizens over their councillors in between elections?
  • Here is where the Policy Forum membership is expected to play a key role. Policy Forum also sponsors community radio programmes in the districts where it works. This provides a guaranteed platform for locally based CSOs, CBOs and community members to ensure that their most important issues receive enough publicity to motivate action on the part of their political representatives.

Our biggest lesson over the last five years has been that we can only fail if what we learn fails to change us, so we keep learning, and we keep changing.

[1] MEL = monitoring, evaluation and learning

[2] Kiteto, Kilwa, Mafia, and Mafinga

[3] Controller and Auditor General

[4] Participatory Monitoring, Evaluation and Learning

[Author]

Posts