READ – Reflect, Evaluate, Analyse, Document


A diverse and successful range of creative partnerships, new works, and community arts and cultural development outcomes were enabled via the Nuclear Futures program.

This section outlines the evaluation process undertaken, and summarises some key findings.

For a snapshot of the program’s impact and achievements, see recent posts.

 

About Nuclear Futures’ evaluation process

Program evaluation of Nuclear Futures aimed to generate evidence-based findings about the quality and impact of the program’s activities, outputs, and outcomes over its three-year operation. Alphaville sought a wide summative lens in order to learn about, and appraise, the program’s impact to: 

  • reflect the community arts ethos of the program 
  • cater for the interests of the culturally diverse stakeholders involved 
  • and comply with the program’s contractual obligations for evaluation rigour. 

During the program development, Alphaville negotiated with the Australia Council for the Arts and key stakeholders to design and test an experimental, integrated approach to community arts evaluation, that would be conducted internally at the Nuclear Futures sites of practice.

Based on the areas of evaluation interest identified during early stakeholder consultation, and including stated contractual obligations, the Nuclear Futures evaluation process aimed to:

  • Reveal and assess the stories of program impact upon partnering community participants, communities, organisations and personnel.
  • Measure program implementation against stated objectives and planned outcomes.
  • Provide a critical lens for the community arts practice in order to gain insight into specified community arts and cultural development (CACD) areas of interest.
  • Encourage skills development, capacity building and information sharing during the evaluation, to support a participatory approach.

The key evaluation questions for the program determined by partner needs and interests were:

  1. To what extent did the program meet its stated objectives?
  2. How effective was the program for generating opportunities for individuals and communities to actively participate in excellent artistic practices?
  3. What was the quality and depth of new partnerships formed and developed?
  4. In what ways did Nuclear Futures impact upon the delivery partners, participating groups and the wider community?
  5. How does the Nuclear Futures increase understanding of the contribution of the CACD sector, in the stated areas of interest?

 

The ‘READ’ model

The program evaluation was conceived as part of a broader, interpretative enquiry and based on a model for evaluative practice coined as ‘READ’ – to integrate Reflection, Evaluation, Analysis and Documentation. The READ approach was devised by Creative Producer Paul Brown and Program Manager Ellise Barkley, with the collective input from a mix of stakeholders. 

During the program, the READ model was developed, trialled, and assessed in conjunction with research undertaken at the Queensland University of Technology (QUT), to promote participatory learning and appraisal involving community participants, creative artists, and interested partner representatives in the community arts context. Distinct from program activities, the research was led by Ellise Barkley as part of doctoral research (http://eprints.qut.edu.au/97728/). 

The innovation of the READ model arising from the Nuclear Futures program lies on its systematic integration of Reflection, Evaluation, Analysis and Documentation (READ) – that creates a wide summative lens for recording, reviewing and representing community arts activities (Barkley, 2016).

Of high relevance, were the documentary art-making methods central to the collaboration, prominent across the majority of Nuclear Futures projects, that utilised creative research, photography, video, digital media and written publications. To achieve a robust evaluative framework, these creative documentation methods were coupled with conventional evaluation instruments and organisational learning processes, as well as critical analysis generated via academic research. In addition, the cohort considered whether the outputs generated could be created and shared in such a way so as to extend program impact and legacy outcomes– and how this may dovetail with the arts practice.

More broadly, for community arts and Creative Industries practitioners, the emergent READ model offers (Barkley, 2016):

  • an integrated evaluation method that can be done ‘in house’
  • a model of evaluation best practice that embraces the ethos of community arts
  • a flexible framework for generating robust data, linked with management and creative arts practice so that data can be fed back, continuously, into the project for real-time impact
  • original strategies for engagement and for developing stakeholder ownership of evaluation, particularly where trepidation or resistance to traditional evaluation means exists
  • possibilities for more general application in (research-led) creative management.

 

(Source: Barkley, 2016, p.173)

 

Evaluation findings

The summative evaluation for the Nuclear Futures Partnership Initiative revealed program impact in the domains of artistic vibrancy, community development, advancement of practice, and broader social and political influence. Findings were represented and communicated with partners via a number of means, including a comprehensive summative evaluation report and a series of program impact stories.

Some of the key findings from Alphaville’s evaluation reporting for Nuclear Futures are as follows:

  • Kana and Mariko at the youth leadership workshop in the Marshall Islands. Photo by Jessie Boylan.

    The program involved atomic survivor participants from Australia, Japan, Britain, the Marshall Islands, and Kazakhstan. Over 100 community arts and cultural development (CACD) events and workshops were delivered, involving 150 community of interest participants/artists, 35 professional artists/ artworkers, and over 40,000 attendees at events and exhibitions. The program received consistent, positive review from the culturally diverse communities of interest, and involved community-led initiatives, skills development, collaborative art-making, community and individual capacity building, international cultural exchange, professional development, audience development; and the performing, exhibiting, publishing, recording, touring, and permanent acquisition of artwork.

 

  • Alinta, Shilloh and Russell next to Tree of Life: Gift of Peace sculpture. Photo by Jessie Boylan

    The program unequivocally met its partner agreed objectives, performing beyond recorded pre-program stakeholder expectations. Outstanding results were documented in the domain of ‘making the story go far’ (the first agreed objective), and this linked to identified capacity building and personal transformative impact for participants. Educational engagement via school programs involved 2,000 students, with evaluation findings revealing most had no prior knowledge of Australia’s nuclear test history. Student reviews suggested artworks were impactful and innovative, and effective in staging atomic community stories and history.

 

  • Ngurini

    ‘Ngurini (Searching)’, immersive projection installation. Photo by Jessie Boylan.

    Feedback from the 3 major showcase seasons from audiences, partners, and participants (corroborated with assessment against Australia Council’s Artistic Vibrancy framework) verified the artworks are of high artistic merit. The feedback collected indicated participant and audience impact, educational outcomes, and technical innovation.

 

  • Across the 40 established partnerships, high quality, mutually beneficial relationships were demonstrated, particularly between core program partners, that delivered organisational and community benefits, personal transformative impact, and community access to new funding/ opportunities/schemes. Benefits beyond the life of the program (legacy impact) are evidenced including 5 spin off projects; sustained partnerships between communities, artists, and cultural and academic institutions; new artwork; long-term archival collection development; and research, publication, and dissemination in the domains of CACD and atomic art.

 

  • Nuclear Veteran Avon Hudson, used media interviews to make the story go far.

    Extensive international media coverage was achieved (millions reached via TV, radio, newspaper and online news stories) and this contributed to the wider social and political impact of the program, and in significantly extending program engagement beyond the South Australian place-based residencies in Yalata Aboriginal Community and Balaklava.

 

For more information and access to full evaluation findings, please contact Alphaville via this website.

Alphaville acknowledges the support of Yalata Community Incorporated, Maralinga Tjarutja Council, Australia’s nuclear veteran community, the Fallout descendant network, the British Nuclear Test Veterans Association, and the many community participants, creative professionals, and organisational partners contributing to the program and its evaluation. Also acknowledged is the financial assistance of the Australia Council for the Arts, Graham F Smith Peace Foundation, Mayors for Peace, Australian Embassy Tokyo, Country Arts SA, Arts South Australia, and the program’s philanthropic and private donors. 

****************

Barkley, E. (2016) An integrated approach to evaluation: A participatory model for reflection, evaluation, analysis and documentation (the ‘READ’ model) in community arts. Professional Doctorate thesis, Queensland University of Technology.  http://eprints.qut.edu.au/97728/