Why collect evidence?

Evaluating the impact of your public engagement project is essential for developing approaches and demonstrating accountability. The evaluation you undertake should be timely, effective and appropriate in scale and should be able to demonstrate you have achieved the impact(s) you set out to deliver. The body of evidence you will gather throughout your project will be a useful asset for:

  • Informing next steps
  • Making a case for future funding
  • Submitting your impact to the REF

Ultimately you want to be able to articulate what change, effect, benefit has happened as a result of your intervention, and ideally which of these changes would not have happened had the public engagement not taken place.
The reach (e.g. no people interacted with) and significance (e.g. depth of interaction/change affected) should be captured in any evidence-gathering.

Gain evaluation skills through training

Our Engaged Researcher training course includes modules on evaluating public engagement activity, we really recommend you attending these!

When to plan your approach

We strongly advise you consider evaluation at the earliest stages of developing your public engagement initiative. You may find that when you articulate the effects/changes/impacts you plan to deliver in your project and then capture in subsequent evaluation, that there are more effective approaches than those you were originally planning.

How to evaluate your public engagement

Consider who will carry out the tasks of designing feedback mechanisms, distributing these to audiences, analysing and reporting on feedback.

There are three options depending on the funding and resources you have available:

  1. Do it yourself
  2. Pay a professional to evaluate your project
  3. Get your partners/publics to help you collect and process data.

Each public engagement activity is unique and will require a specific approach to be adopted. If you would like to discuss the appropriate methods to evaluate your project, contact the public engagement team for advice and support.

Methods

Things to consider
Response cards, comment cards, comment books, graffiti walls Ensure you have the capacity to analyse comments and reflect on both positive and critical feedback items.
Questionnaires and surveys

Consider the project aims and objectives and ensure that the questions you ask will help you to evaluate whether these have been met. 

Questions need careful phrasing to reflect age, language and expertise of the target group and must not ‘lead’ the audience to give biased responses.

Ensure you have the capacity to analyse quantitative survey responses as well as qualitative data provided in response to open-ended questions

Testimony,  interviews, focus groups

 

Ensure that public/s understand the project aims so that their testimony can address the extent to which these aims have been met.

Consider attribution. Ask for informed consent to use quotes (e.g. in the next REF), or gather contact details so that permission can be sought later.
Observations

Consider creating a standardised document to record observations e.g. For observation of a family group around a hands-on exhibition record set information including; approximate age of children, number and gender of adults, length of time spent at each exhibit.

Remember to ask permissions where more in depth observation of an individual or group is required.
Video, photographs and artworks   

Get relevant permissions/release forms for re-use of images and digital footage. Be vigilant when collecting footage/photographs of children in particular.

Digital materials can be a valuable resource when revisiting public engagement activity to evidence audience demographics and interactions.

Additional evidence to collect

  • Public response to engagement activity: telephone calls, letters and emails
  • Reviews
  • Prizes and accolades
  • Audience, listener and viewer numbers,
  • Sales figures
  • Editorials, posts by ‘opinion leaders’
  • Website views: page hits, website viewers, average time spent on a site, average number of pages viewed, downloads, geographic location of users
  • Social media: online comments, hits, tweets, mention in blogs, including details of conversations/engagement with research, as well as statistics
  • Follow-on activities: rebroadcasts, translations, podcasts, invitations to subsequent public or professional events, media coverage, consultancy work
  • Invitations to give advice (e.g. to governments, professional bodies, community groups etc.)
  • Evidence of debate among practitioners, leading to developments in attitudes or behaviours
  • Evidence of citation in policy, regulatory, strategy, practice or other documents
  • Citation in a public discussion, consultation document or judgement
  • Economic impact: book sales, tickets sales, programmes sales, increase in tourism
  • Contributions (financial or in-kind) by third parties to enhance services or support for the public, or evidence of funds from third parties to enhance or extend the engagement activity