Skip to main content

Research and Projects

PenCHORD: Evaluation of the PenCHORD Health Service Modelling Associates programme (EmMA)

Who is involved?

Theme: Evidence for Policy and Practice
Status: Live

 

Background

This evaluation (EmMA) provides an independent and qualitative review of the first iteration of the PenCHORD Health Services Modelling Associates (HSMA) programme.

The HSMA programme aimed to increase the use of operational research (OR) methods by NHS organisations to support evidence based decision-making, and increase collaboration within and between NHS organisations and between research and NHS organisations.

The evaluation involved interviews with programme developers and HSMAs early-mid stage on in the programme to understand expectations and perceptions of how the programme is progressing. After the programme was completed, we conducted further interviews with the HSMAs, mentors and workplace supervisors. Our final analysis was also informed by a discussion with the modelling software supplier (Simul8), documents outlining the aims and recruitment to the programme, an evaluation questionnaire, logs and monthly reports recorded during the programme by mentors and associates.

 

Project findings

The evaluation suggests the programme has been very successful in developing skills in the modelling process and the free software supplied (Simul8). Associates and workplace supervisors both perceived a significant improvement in the quality of information and data analysis produced during the programme and a desire to sustain this approach and links with the PenCHORD team.  Four key themes have emerged as relevant to how the programme has succeeded through:

  1. the transfer of modelling knowledge;
  2. facilitating collaborations;
  3. technical training of modelling software;
  4. increasing organisational learning of modelling to solve problems in healthcare contexts.

 

Key outcomes were identified from the evaluation of the HSMA programme:

  • Increase in technical skills within organisations to complete simulation of healthcare systems;
  • Increased associate skills and value recognised by organisation through promotions and modified job descriptions,
  • Enhanced skills relevant to ensure engagement of all stakeholders;
  • The application of the modelling process to engage different levels of NHS organisations e.g. executive levels, commissioners and act as a tool to develop constructive conversations;
  • Production of evidence with high face validity to support business cases, secure or protect resources and obtain financial reward by supporting the achievement of performance targets;
  • Mentor and coaching supported the co-production of evidence, which associates suggested provided a greater level of confidence in the modelling outputs;
  • Greater visibility of modelling was achieved but varied depending upon the partnership between associate and workplace supervisor and their position or influence up the chain in the organisation.

 

The evaluation also highlights the aspects of the programme that went well and indicates how existing challenges might be addressed in the future. This includes allowing for flexibility in the timing of the programme, to reflect a more realistic timeframe for NHS decision-makers and providing specifications for each role within the programme. The CLAHRC support of the programme was clearly welcomed as training and expertise was at no cost to NHS organisations. The expertise offered by PenCHORD was highly valued and increased the confidence in the modelling outputs produced by associates. An ongoing collaboration between the associates and PenCHORD was suggested as desirable. Collaboration within and across NHS organisations was welcomed and seen as a direct impact of the HSMA programme. Additionally, an unexpected consequence of the programme was job enrichment for all participants: mentors, workplace supervisors and associates.

 

What lessons have been learned:

The evaluation has highlighted a number of challenges in ensuring the projects have been achievable within the timeframe. The protection of associates’ time and access to necessary data are such examples.

This evaluation study has also provided evidence on how and who the programme has worked for as well as the process of engaging modelling to enable NHS decision making and service change.  Theoretical insights into how the programme achieved its aims have been drawn through considering data in the context of the Integrated - Promoting Action on Research Implementation in Health Services (i-PARIHS) framework [1]. This has provided insight and greater understanding of the comments and factors which were suggested as influential to the HSMA programme during the evaluation. These factors relate to those participating in the programme (recipients), the style of facilitation by mentors and workplace supervisors, the design of the HSMA programme itself (innovation) and contextual factors that influenced the outcome and process of individual projects.

[1] Harvey G and Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implementation Science, 2016. 11:33. DOI 10.1186/s13012-016-0398-2

 

Outputs

  • A full internal report of evaluation method and findings
  • Poster presentation accepted for HSRUK conference 2017 (see below)
  • A findings paper for academic publication is in progress
  • Findings will inform the development of a HSDR grant

 

Relevant links

  • To find out more about the HSMA Programme, visit the project page
  • For an update on the programme's progress one year after it was launched, read our news story
  • You can also download a PDF poster which was used at a HSR UK event in July 2017