Evaluation for Meaningful (Ex)change

Evaluation is really important because it has a pivotal role in relation to equality. This is why it is important that evaluation activities are planned and carried out by all involved, not by a few members of staff, or by external ‘experts’ in isolation from those facilitating the work or those being engaged through the work.

This paper was written for and delivered at the Engage Conference, ‘A Different game: Young people, working with art and artists’ in Glasgow, November 2015

I’ve been involved in socially engaged arts practice since 1991. The difference that socially engaged arts practice makes for all involved has intrigued me since my very first experience of facilitating such work. My specific interest in evaluation grew from two concurrent observations that I made during the action research that I planned and conducted for my PhD.

The first of these observations was that the young people engaging (with) me, making images using photography and digital media, in the old indoor markets at the Bull Ring in Birmingham, were informing my own evaluative understanding and interpretation of the images they made. Their images were powerfully resonant, for me, as my understanding was informed through the meaningful dialogue that I had with them around their image production. That dialogue was not planned, forced or in any way pre-determined.

Young people led this dialogue; which was partly reliant on having situated myself, strategically, in a space where the usual power relations of adult to young person were significantly and usefully exploded. The dialogue did not (and I would suggest could not) take place in response to any questions or prompts, it was an organic exchange, within which they shared with me, glimpses into their everyday experiences alongside insights into what the creative processes they were engaged in meant, for them, often in terms of making sense of their own lives, the issues they encountered and the conflicts they were seeking to resolve, within their often very complicated and busy young lives.[1]

The second of these observations was that the evaluation processes I witnessed, at that time, being used within various settings; both in formal education and in what was then called ‘community arts’; seemed to be severely lacking. They were lacking in any qualitative, meaningful sense making of what had happened or what might be learnt from it. I witnessed organisations getting repeat funding for projects that had engaged a high number of people, without any attempt being made to get any sort of feedback from those people about the quality or meaning of that experience. I was also aware of how, in formal contexts, criteria were defined independently of those engaged. What I began to recognise was a connection between evaluation and power and control.

During the first day of presentations at this year’s ‘Engage’ conference, many of the points made chimed with my own thinking in significant ways. One of these was the quote that Barby Asante[2] shared with us, when she said ‘Who determines who is, objectively speaking, exceptional and talented enough to be admitted into higher art education?’ (Saner, P et al, 2015)

Barby Asante later showed me the website that this quote comes from, which I read with interest. The article articulates how less than transparent ‘yardsticks’ inhibit the ‘quest for “diversity”’ because ‘notions of who has “potential” and what potential looks like tend to be inherently classed and racialized.’ (Saner, P et al, 2015)

This article and the quote that Barby Asante shared with Engage conference delegates struck a chord with me, as it led me to think about how I could articulate the motivation I have for being interested in, and developing a role in, evaluation. For me evaluation is really important because it has a pivotal role in relation to equality. This is why it is always important that ‘criteria for success’ are determined through dialogue, with all involved, not by a few members of staff in isolation from those facilitating the work or those being engaged through the work.

Alongside my observations, as outlined above, I was also aware of how, as a photographer, as a creative practitioner myself, when engaged in any creative process, we reflect in an ongoing way, to make decisions about what to do next. As an action researcher I was aware of the similar function of reflective processes in the action research cycle.

I began to consider how an action research approach to evaluation might be a more meaningful way of thinking about, planning and conducting evaluation than any of the models I witnessed taking place around me at that time (1997-2000). I also began to consider how, by using such an approach, we might ensure a greater equality of input into evaluation processes than might be allowed by more formal or rigid processes, whereby all involved are understood as evaluators and as experts and whereby evaluation takes place as part of and throughout a process, not as something ‘in addition to’ the process, or bolted on at the end.

What is always important when nurturing such an evaluation process is to ensure and establish appropriate mechanisms for generating exchange and stimulating reflection for all involved. In order to ensure an equality of input it is essential to ensure that the mechanisms by which people can engage in evaluative exchange are not exclusive; in other words they do not simply invite responses from those who feel most confident about talking or writing and they do not simply invite responses from people who already have the confidence to believe that they have something worth contributing.

‘Creative’ evaluation activities that enable people to think through as well as to contribute their opinions therefore need to be devised from within a project and in response to the specific contexts and people involved. This needs to happen, not in isolation of a project, event or context, but once we are immersed in the work and can identify what will work in any one specific context. It is no good to just be armed with lots of techniques, it is far better to work with people to find out what criteria they have and what solutions they might have to enable a meaningful process of dialogue, exchange and evaluation to take place.

By working in this way, the important opportunity to reflect on what has happened and the difference it has made, for each individual, becomes a more accessible opportunity. This is important because often, without the chance to think about and ponder the significance of an experience, for ourselves, our own lives and futures, we may not be fully aware of the positive nature of that experience. For example it may be that we don’t recognise what we have achieved until we stop to think about it or to tell someone else about what we have done. This is most significant when working with young people for whom there may not be a support network willing or able to listen to them on a regular basis.

Without reflecting on what we have achieved, it is difficult, if not impossible, to identify the opportunities we may have to build on that experience in further positive ways. I therefore advocate for an evaluation process that is driven by the role and function the evaluation process can have in terms of making a positive difference for all involved. The approach is about seizing the power there is in such a process to nurture self-reflection and learning amongst all involved.

In Circuit one of the key strategies for evaluation has been the development of a group of young evaluators, from all the galleries involved in the programme, nationally. My role, as Circuit critical friend has been partly about supporting this group of people to develop evaluation methods and approaches that are relevant to their contexts and to the work they have developed, curated and programmed. Members of the group have considered who it is that they need to have exchange with and have therefore sought feedback from a wide range of people involved, including audiences, staff, artists and each other. Circulate members have also been supported to plan and facilitate their own ‘sense making’ sessions with the core groups at their galleries, to ascertain not just the value, but also the quality, of their engagement in the Circuit programme.[3]

Because methods and approaches have been devised in a wide range of contexts, a wide range of ideas and processes have been developed and used. For example, in Cambridge, the experiences of members of the core group, from Wysing Arts Centre and Kettle’s Yard, have been evaluated through a sense making session that included the use of ‘Jenga.’

Meanwhile, in Liverpool, Tate Collective Liverpool Circulate members have reflected on the evidence gathered by the national Circulate team about the ‘Blueprint’ festival, as well as on their experiences of programming and delivering the festival. The findings were used to develop a set of ‘top tips for organising peer led festivals.’   The findings also informed a paper written by Tate Collective Liverpool Circulate members, as a group, which they presented at last year’s iJade conference and have since published in iJade. (Cawley-Gelling, I et al, 2015.) This has led one member of the group, Steven Hyland, to write a solo paper, entitled The Trouble with Youth Voice, which he presented at a BERA seminar, in which he reflects on, but also critiques some of the ideas on which Circuit is based. (Hyland, S. 2015)

For further example, at the Whitworth gallery in Manchester, one member of Circulate, Charlotte Davies, has developed a project using photo-elicitation to evaluate the organisational change that is taking place because of Circuit.

By devising a wider range of creative evaluation activities, informed by the context and the experiences of those involved, it is possible to engage with a wide cohort of people, so that a broad set of perspectives can inform the project or programme detail and direction. The evidence and data thereby generated, through these activities and consequent exchange and dialogue, will be more representative of a wider sample than would otherwise be the case.

Furthermore, the difference the programme has made is considered, not just in terms of ways in which it may have brought about change for young people, but also in terms of the ways in which it may have brought about change for artists, facilitators, staff and the organisations involved. To return to the idea of articulating what it is that motivates us, for me, this is key, as ultimately my aim in working in this field is linked with transforming organisations, in order that everyone’s cultural entitlement is fulfilled; rather than focusing on increasing or diversifying audiences as an ultimate aim.

I would argue that the emphasis on the transformative potential of engagement in the arts has led to an unhelpful set of assumptions about who it is that is in need of transformation. This has become most apparent when engagement is talked about in terms of the ‘impact’ it has. For a few years now the increasing use of the term ‘impact’ has troubled me and I have asked people I work with if we can, instead, talk about the difference being made through working together, rather than assuming one set of people might ‘have impact on’ another set of people. I have also requested this in order to ensure we are not complacent about the use of such violent terminology, or ignorant of the power relation inherent in the use of the term. In a recent paper, François Matarasso has also talked about the term impact, clearly articulating some of the issues there are with this use of the word:

“The problem is that impact suggests something forcefully striking an object, like a die impressing itself on a blank. It implies an active agent and a passive recipient, a subject and an object.” (Matarasso, F 2015)

François Matarasso goes on to point out that:

“In this thinking, the social art project is conceived as an experience whose ‘impact’ changes those who take part. And in this context, ‘change’ means ‘improve’, in terms of the problem-solving mission identified, more ore less cooperatively, by the artist and the commissioner.” (Matarasso, F 2015)

François Matarasso thereby uncovers a set of problematic assumptions, which underpin this way of thinking, about some people being ‘in need of improvement.’

Another term that gets used regularly in our field and which I often challenge is ‘hard to reach.’ I would argue that people are never hard to reach, but galleries often are hard to reach for some people. The emphasis needs to be placed on the gallery to become easier to reach, not the other way around.

Furthermore, if ultimately our intention is to do with achieving greater equality, then I would argue that our ultimate aim should not simply be about diversifying or increasing audience numbers for the sake of it, rather, our aim would be better focused on how such an extension of engagement can then bring about significant positive social change. Ultimately I think work in the field needs to be about enriching the cultural experience of everyone, by ensuring that a diverse set of experiences and understandings inform galleries, so that galleries become more reflective of a multifarious range of cultural experiences and understandings, valuing, endorsing and validating them in ways that lead everyone to feel that all unfamiliar cultural expressions are of equal value to those that they are familiar with.

During the introduction to the Engage conference Alicia Miller talked about Stuart Hall’s ‘Divided City’ (Hall, S. 2004) and how Hall identified cultural sites of exchange as the ‘one glimmer of hope’ for overcoming the social divides wrought by this neoliberal era. Alicia Miller said:

‘The ‘fragile promise’ as he calls it, comes through a ‘genuine cultural syncretism’ that emerges from exchange between young people… which function as ‘critical zones of interchange’ that generate what Hall calls ‘globalisation from below.’’ (Miller, A 2015)

Ultimately I would argue that our aim should be about ensuring that everyone’s entitlement, to meaningful cultural engagement, is fulfilled. This does not happen through simply increasing and diversifying audiences, it is more complex than this and demands transformation, not of individuals, but within and across galleries. Through such transformation ‘critical zones of interchange’ can be established, nurturing a great equality of exchange and input and respect for different experiences and understandings.



[1] For a detailed account of this work see Hall, R. 2005

[2] Barby Asante is an artist and a co-director of Agency for Agency. @barbyasante @agencyforagency

[3] More information about this process and how notions of quality have informed Circulate’s evaluation process is available on the Circuit website: https://circuit.tate.org.uk/2015/06/quality-honesty-and-keeping-it-real/

The presentation of this paper can now be seen at: https://vimeo.com/149743927



Cawley-Gelling, I., Hyland, S., McCarron-Roberts, C., Mohamad Noor, S and Morrissey, R. (2015)

The Value of the Blueprint Festival

International Journal of Art & Design Education. Vol 34 Issue 3

See: http://onlinelibrary.wiley.com/doi/10.1111/jade.12091/abstract


Hall, R. (2005)

The Value of Visual Exploration: Understanding Cultural Activities with Young People

The Public: West Bromwich.


Hall, S. (2004)

Divided City: the Crisis of London. https://www.opendemocracy.net/arts-multiculturalism/article_2191.jsp


Hyland, S. (2015)

The Trouble with Youth Voice

Paper delivered at BERA seminar: Youth work, informal learning and the arts: exploring the research and practice agenda

18 April 2015

The paper is available at: https://circuit.tate.org.uk/2015/04/presenting-the-trouble-with-youth-voice/


Matarasso, F. (2015)

Music and Social Change: Intentions and Outcomes

Unpublished Symposium paper available at Academia.edu:



Miller, A. (2015)

Unpublished Programme Introduction

Engage Conference, ‘A Different game: Young people, working with art and artists.’ Glasgow, November 2015


Saner, P., Vessely, P. and Vögele, S (2015)

‘De-privileging Art School: Antisexist and Antiracist Methods’

Available at: http://blog.zhdk.ch/artschooldifferences/en/2015/03/12/de-privileging-art-school-antisexist-and-antiracist-methods/