At the very first national evaluation team meeting for Circuit, on April 10th 2013, we agreed a need for an evaluation framework that would clarify the scope, methodologies and timescale of the evaluation, in order to work out what needed to be done, by whom and by when. The framework needed to outline the methodologies that would provide all the evidence and information we need.
Whenever beginning work on any project I have developed a process, informed by Charities Evaluation Services processes amongst others, of mapping out the evidence that is needed, in relation to the aims and objectives of a project or programme. Such a process, and the use of a template based on an ‘outcome and output indicator form’ seemed to be a useful starting point for our evaluation framework.
The need for a framework demanded a common and shared understanding and clear articulation of the aims and objectives of Circuit. We therefore planned a ‘sharing session’ with staff from all partner galleries, in which everyone contributed to a set of aims, objectives and principles for Circuit.
These aims and objectives were then used to draft the first version of the evaluation framework. All members of the national evaluation team worked together to develop and refine the framework into a document that could be used by all gallery partners. All members of the team had key roles in finalising the document and making it more easily understood, for example, through careful formatting and colour coding the aims. The document was then shared with partners, young people, the Circuit steering group and board, who were consulted about the content and usability. The final version of the framework and a guide to using it can be downloaded here.
The framework outlines the evidence that needs to be gathered, relating to all programme aims and objectives. It also thereby outlines the dialogue that needs to take place, for meaningful reflection, amongst all involved, to nurture a shared and explicit understanding of what is happening in the programme. On a practical level it also shows the different roles involved in evidence gathering, monitoring and analysis.
The process of developing a framework is, in itself, a useful process, as it demands that everyone involved reflects on the aims they have for the work and how they are going to achieve those aims, as well as how they will gather relevant evidence in order to work out how (far) the aims have been met.
The framework is then an ongoing tool and resource as it enables a process of checking back to remind ourselves of the aims that relate to our objectives, or to put it another way, why we are doing what we are doing. Very often, in long term projects or processes it is easy to lose sight of our original aims and just keep on doing the things we planned to do; even if those things are not achieving the aims we originally had. The framework enables us to remind ourselves why we are doing the things we are doing and revisit and revise our objectives if they are not achieving our aims.
The framework has also been a useful resource for other partners, especially in the youth sector. Some youth sector partners have commented that it has helped them to refine and tailor the detail of what they are doing; for example, how they facilitate workshops; as it has informed the decisions they make (as reflective practitioners) in an ongoing way. This is because it is only when we understand why we are doing something (the aim of the work) that we are able to make informed decisions about how best to do what we do (the objective). In other words, it is only when we understand the aims of the work that our approach to fulfilling the objectives is meaningfully informed.