REF 2021

 Download the REF results presentation slides  (PPT).

Outputs

Download the REF 2021 outputs summary (Excel).

Impact cases

Unit of assessment (UoA) 1 (Clinical Medicine):

• Driving strategies to prevent neonatal Group B streptococcal infection (Heath, Le Doare, Ladhani, Jones)
• International recommendations for ECG interpretation in young athletes to identify those at risk of exercise related sudden cardiac death (Sharma, Papadakis, Malhotra)
• Novel molecular serotyping technology advances worldwide pneumococcal vaccine impact and development (Hinds, Gould, Witney, Butcher)
• Optimising antibiotic prescribing in children (Sharland, Bielicki, Hsia)
• Optimising malaria treatment with artemisinins (Krishna, Planche)
• Sudden unexplained death in the young: enabling diagnosis and promoting preventative interventions (Behr, Sheppard, Sharma, Papadakis)
• Transforming the treatment and prevention of HIV-associated cryptococcal meningitis (Harrison, Bicanic, Loyse, Molloy, Wake).

Unit of Assessment (UoA) 2 (Public Health, Health Services and Primary Care):

• Early introduction of allergenic foods and the prevention of food allergy in infancy (Perkin, Strachan)
• Outdoor air pollution and mortality: systematic reviews supporting the UK Government Air Quality Strategy (Atkinson, Butland, Huangfu, Anderson, Strachan).

Environment statements:

• UoA1 
• UoA2 
• Institutional statement.

The Research Excellence Framework (REF) is the UK’s system for assessing the quality of research in UK higher education institutions. It takes place every seven years and the next exercise will be conducted in 2021.

See Research Excellence Framework for more information. 

REF seminars at St George’s

Professor Jodi Lindsay, Director of Cross Cutting Themes and Academic Lead for REF, provided an overview of the REF 2021 and introduced St George’s strategy for REF to staff at roadshows that took place in May and June 2018. The presentation has been updated for 2020, REF 2021: impact and strategies for St George’s (Powerpoint). 

The next REF seminar takes place on Wednesday 4 March 2020, 11am – 12pm, in H2.8. The seminar is ideal for new academic staff, those that would like a refresher or an update, or those who are part of the mock REF of outputs and need assistance. Everyone is welcome. 

Regular emails sent to academic and research staff

Regular email communications sent to academic and research staff about St George’s preparation for the REF 2021 are sent from the Professor Jodi Lindsay, Director of Cross Cutting Themes. Emails are available for all staff to read:

• 15 August 2018: REF 2021 update for academics

• 29 January 2019: REF 2021 update 1: SGUL's Code of Practice - we need your feedback 

• 5 February 2019: REF 2021 update 2: What is an eligible output (paper)? And some interesting new guidance! 

• 7 May 2019: REF 2021 update 3: Latest version of our Code of Practice (CoP) and new Joint Institute membership call 

• 11 June 2019: REF2021 update 4: What is a 3* or a 4* star paper?

• 19 June 2019: REF2021 update 5: What makes a good Impact Case?

• 5 November 2019: REF2021 update: Staff eligibility for REF 2021

• 9 April 2020: REF 2021 update 6: Submission date is delayed

• 10 August 2020: REF 2021 update 7: update on the new REF2021 deadlines

We have made minor changes to our approved Code of Practice for REF 2021 (PDF) due to the impact of COVID-19 on our REF submission. This version of the Code was submitted to REF in October 2020. The changes meet REF’s criteria for minor changes: a) The spacing of milestones within the timetable have not changed; b) The revised timing will allow processes to be concluded (including appeals process) in advance of the submission deadline; and c) The changes will be communicated to all relevant staff with ample time before final submission decisions are made. 

The Code of Practice (PDF) applies to all staff involved in the preparation and submission of St George's REF 2021 return. It provides a framework within which recommendations and decisions of identifying staff with significant responsibilities for research, determining research independence and selection of outputs, are conducted in a fair and transparent way with the aim of promoting equality and diversity, complying with legislation and avoiding discrimination. St George's will adhere to the four main principles of REF 2021 to guide the Code of Practice: transparency, consistency, accountability and inclusivity.

St George’s submitted its Code of Practice to REF in June 2019. Following the assessment of its Code of Practice, REF’s Equality and Diversity Advisory Panel and Research England determined that St George’s Code of Practice was not considered to meet the requirements set by REF. It was amended and resubmitted in September 2019, when it received approval on 8 November 2019.

Staff agreement with the Code of Practice

The Code of Practice required staff consultation before the final version was agreed upon and submitted. Draft versions of St George’s Code of Practice for REF 2021: 

1. St George's draft Code of Practice (PDF) that was consulted on by staff.
2. St George's Code of Practice (PDF) submitted to REF by 7 June 2019. It was presented at REF Steering Executive, Research Committee, Executive Board and Senate between January to May 2019.
3. St George's Code of Practice with amendments (September 2019) (PDF). Amendments made to Code of Practice submitted to REF in June 2019 following its assessment by REF’s Equality and Diversity Advisory Panel and Research England.

St George’s Code of Practice sets out the criteria for eligible staff to be returned to REF 2021. Staff must fulfil the criteria:

1. have an employment contract of 0.2 FTE or greater with St George’s.
2. are on St George’s Payroll.
3. have a primary employment function to undertake either ‘research’ or ‘teaching and research’ as evidenced in the Higher Education Statistics Agency (HESA) ACEMPFUN field as 2 or 3. All staff returned as ‘research only’ or ‘teaching and research’ are members or joint members of a Research Institute (Infection and Immunity, Molecular and Clinical Sciences, or Population Health). All staff returned to HESA as ‘teaching only’ are members of the Institute of Medical and Biomedical Education.
4. work in an area of research aligned with the REF’s Units of Assessment that St George’s will submit to REF 2021 (staff who are members of the Institute of Infection and Immunity Research Institute work in an area of research aligned with Unit of Assessment 1: Clinical Medicine. Staff who are members of the Molecular & Clinical Sciences Research Institute work in an area of research aligned with Unit of Assessment 1: Clinical Medicine. Staff who are members of the Population Health Research Institute work in an area of research aligned with Unit of Assessment 2: Public Health, Health Services and Primary Care).

Are an independent researcher. At SGUL this is defined as being employed as a Professor, Reader, Senior Lecturer or Lecturer (including Clinical posts) OR holding an independently funded fellowship on the approved REF2021 ‘List of independent Research Fellowships’ (Appendix 17: List of Independent Research Fellowships) and have already been awarded their PhD or MD.

Appeals

Grounds for appeal

The grounds for appeal against a decision not to include an individual in the REF 2021 submission on the grounds of identifying staff with significant responsibility for research; criteria for eligible staff to be returned to REF 2021; research independence; and joint membership between Institutes will be that:

• individual staff circumstances have not been given appropriate consideration;
• the procedure outlined in this Code of Practice was applied incorrectly.

Appeals process

An individual who feels that he or she has been excluded from the submission to REF 2021 on grounds set out in the grounds for appeal can appeal against the decision.

1. The appellant writes to the appropriate Institute Director setting out the grounds for appeal.
2. The Institute Director shall meet with the appellant to discuss his or her appeal, and if possible shall, through discussion, come to an agreement over the appeal.
3. If the appeal is not resolved at the informal stage, the member of staff may write to one of the members of the Appeals Panel, the Deputy Principal (Institutional Affairs) or the Associate Dean for Culture, Development and Inclusion, formally stating that he or she wishes to appeal against the decision to include or not to include him or her in the REF submission, setting out the grounds for appeal.
4. The Appeals Panel will seek written comments on the appeal from the Research Institute Director.
5. The Appeals Panel will meet with the appellant and the Research Institute Director to hear the case. At such a meeting, the appellant may, if he or she so wishes, be accompanied by a Trade Union Representative and/or a colleague. The Appeals Panel will be assisted by a representative from HR, to act as clerk to the process and prepare a written report on the appeal.
6. The Appeals Panel will make a judgement and will make a report with a recommendation whether to uphold or dismiss the appeal.
7. The appellant will be informed of the decision in writing by the Appeals Panel.

Timetable for Appeals Procedure

Provisional timetable for decisions on staff selected for inclusion in REF 2021:

Research outputs (e.g. journal article/paper) are assessed on the basis of ‘originality’, ‘rigour’ and ‘significance’. Outputs count for 60% of the overall quality rating. Guidance on what is an eligible output and what is rated as 3* or 4* is available to read at:

• 5 February 2019: REF 2021 update 2: What is an eligible output (paper)? And some interesting new guidance! 
• 11 June 2019: REF2021 update 4: What is a 3* or a 4* star paper?

Ensure journal articles are compliant with the REF 2021 open access policy

Guidance to authors about complying with the process that will ensure journal articles are compliant with the REF 2021 open access policy is available on Library Open Access.  

 Using publication metrics responsibly

‘Using publication metrics responsibly’ event took place on 17^th October 2019. The event was aimed at staff who use publication metrics to hire staff, are involved in promotions, or assess grants or papers based on research. The event covered what types of metrics and metric providers are out there, and what other factors should be considered before using metrics such as journal impact factors, citations and h-indexes. Understanding publication metrics is important for academia, and a major topic for St George’s both for our REF 2021 submission and because we have recently signed the San Francisco Declaration on Research Assessment (DORA)

View the presentation slides for 'Using publication metrics responsibly'.

Contact Professor Jodi Lindsay, if you want to discuss the topic of responsible publication metrics.

All eligible academic staff for the REF 2021 are being asked to identify their top papers using the CRIS assessment tool to support the REF Steering Executive with the internal mock REF of outputs. Staff are being asked to select their top papers and provide supporting information about eligibility, authors, citations for your field, and originality, significance and rigour of the paper. Staff who are eligible will be notified by email by Professor Jodi Lindsay, Academic Lead for REF in late January 2020, when the assessment is live on the CRIS.

Guidance for academics about how to access and use the CRIS mock REF assessment, how to select top papers, and considerations for the supporting information is available here: Guidance for CRIS REF Assessment – output selection (PDF)

Guidance for internal reviewers of outputs (PDF).

Internal reviewers of outputs were contacted in mid-August 2020, and have been asked to complete reviews by 30 September 2020. 

To support reviewers, Professor Jodi Lindsay hosted a session on Friday 21^st August 2020 to go through the assessment criteria for the internal review of outputs (papers) for REF 2021, with time for questions. The recorded presentation is available at: (login required using St George’s login): https://web.microsoftstream.com/video/c22ca2d2-035e-4b69-addd-bcd1d2518b3a

Impact is at the heart of St George’s mission. Our world-leading research underpins a range of innovations which improve quality of life, mitigates harm and reduces mortality. We did well in this area for REF 2014 and based on the case studies available thus far, we seem to be in a good position to repeat an excellent performance in 2021.

St George’s will submit approximately nine impact case studies (ICS) for REF 2021 (7 ICS to Unit of Assessment 1 for impact in MCS and I&I, and 2 ICS to Unit of Assessment 2 for impact in PHRI). It and is a key component of our submission. Impact cases are assessed on the reach and significance of the impact. An impact case study summarises the underpinning research of the impact, and a narrative of the impact generated, supported by evidence including quantitative indicators.

If you have a potential impact case study that you would like to discuss, contact Professor Juan Carlos Kaski, Chair of the Impact Case Studies sub-committee.

Examples of previous impact cases submitted in REF 2014 are available at  impact.ref.ac.uk/casestudies/

Reach and significance of impact for REF 2021  

The main REF assessment panel welcomes case studies which describe impacts that have provided benefits to one or more areas of the economy, society, culture, public policy and services, health, production, environment, international development or quality of life, whether locally, regionally, nationally or internationally, beyond academia. Impact includes the reduction or prevention of harm, risk, cost or other negative effects. An impact case study may describe more than one type of impact arising from a single activity, for example, a new drug can generate both health and economic impact. We are expected to submit our strongest case studies, regardless of the type of impact they describe.

Impacts will be assessed in terms of their ‘reach and significance’ regardless of the geographic location in which they occurred, whether locally, regionally, nationally or internationally. The UK funding bodies expect that many impacts will contribute to the economy, society and culture within the UK, but equally value the international contribution of UK research.

Academic impacts on research or the advancement of academic knowledge (whether in the UK or internationally) are excluded. This contribution to academic research and knowledge is assessed within the ‘outputs’ and ‘environment’ elements of REF.

Dates of impact and underpinning research for REF 2021  

Each impact case study submission must describe specific impacts that have occurred during the assessment period 1 August 2013 to 31 July 2020 that were underpinned by excellent research undertaken at St George’s. The underpinning research must have been produced by the researcher at St George’s during the period 1 January 2000 to 31 December 2020.

New for REF 2021 – Guidance for standardising quantitative indicators of impact within REF case studies

Guidance for impact case study authors on provide evidence in an impact case study:

• Standardising quantitative indicators of impact makes the evidence to support the impact case study more discoverable. Stylistic standardisation includes: numbers, percentage and rates, measure of change, time periods, units, currency.
• Evidence should be gathered to support an impact case study in areas of: engagement, mentions in non-academic documents and the media, employment, financial figures, and emissions. This evidence should be uploaded to or linked from the Records of Impact module on CRIS.

Members of St George’s REF Impact Case Studies Sub-Committee

The Impact Case Studies Sub-Committee formed in the academic year 2018-19. The sub-committee will support the REF Steering Executive in impact. The REF 2021 Impact Case Studies sub-committee is comprised of:

• Professor Juan Carlos Kaski, Chair of ICS sub-committee
• Professor Jodi Lindsay, Cross Cutting Themes / REF Academic Lead
• Professor Deborah Bowman, Deputy Principal (Institutional Affairs)
• John Butler, external advisor, Bayer, Global External Innovations & Alliances
• Professor Mark Fisher, Professor of Molecular Biology
• Professor Jon Friedland, Deputy Principal (Research and Enterprise)
• Professor Peter Garrard, Clinical Academic Representative (Neuroscience)
• Professor Tom Harrison, Clinical Academic Representative (Infection)
• Dr Sven Hoffman, Head of Enterprise and Innovation
• Professor Peter Whincup, Director, Population Health Research Institute

The environment statement will make up 15% of St George’s overall quality profile rating and will be assessed against vitality and sustainability.

In the context of REF, vitality is understood as the extent to which the institution promotes and facilitates a culture of collaboration, and enables and actualises impact within research units, within a thriving and inclusive research culture. This should be based on a clearly articulated and overarching strategy for research and enabling its impact across the institution. Relevant factors in assessment are: interdisciplinary research, national and international collaboration, and engagement with appropriate networks both nationally and internationally.

In the context of REF, sustainability is understood as the extent to which the research environment ensures the future health, diversity, wellbeing and wider contribution of the institution and its research units, including investment in people and in infrastructure. Relevant factors for assessment are the extent of cross-disciplinary working and research partnerships, external partnerships more broadly and contributing to and achieving positive change within national research capacity.

The environment statement covers the following areas:

• Unit context and structure, research and impact strategy
• People
• Income, infrastructure and facilities
• Collaboration and contribution to the research base, economy and society

St George’s environment statement is being coordinated by Louise Phillips, Head of Research Funding, Joint Research and Enterprise Services. 

One of the key ways of supporting equality and diversity in Research Excellence Framework (REF) 2021 is to recognise the effect that equality-related circumstances can have on an individual researcher’s productivity. Both St George’s and the UK funding bodies are committed to supporting and promoting equality and diversity in research careers. As part of this commitment, St George’s has established a process in line with REF guidance through which staff can voluntarily declare their individual circumstances ahead of the REF submission (due by 31 March 2021). This enables us to take these into account in preparing our submission as well as to consider support for our staff.   

Types of individual staff circumstances

• Qualifying as an early career researcher (defined as starting their career as independent researcher on or after 1^st August 2016);
• Absence from work due to secondments or career breaks;
• Qualifying periods of family-related leave;
• Other circumstances (for example Junior Clinical Academic);
• Circumstances equivalent to absence, that require a judgement about the appropriate reduction in outputs, which are:
  • Disability
  • Ill health, injury, or mental health conditions
  • Constraints relating to pregnancy, maternity, paternity, adoption or childcare.
  • Other caring responsibilities (such as caring for an elderly or disabled family member)
  • Gender reassignment
  • Other circumstances relating to the protected or relating to activities protected by employment legislation.
• Part-time working.

• Circumstances affected by COVID-19.

Disclosing individual staff circumstances

Staff who feel that individual circumstances may apply to them and who wish to submit their circumstances form will be able to do so at any time in the lead up to the REF submission. This is a voluntary process open to staff who are eligible for the REF 2021. All eligible staff were notified of their eligibility by email in early November 2019 by Professor Jodi Lindsay, Research Director for Cross-Cutting University Themes and Academic Lead for REF. Information on how to submit this is below.

The outcome of the review of circumstances may lead to the removal of the requirement for the minimum of 1 output (eg journal article) for each member of staff, a unit level reduction for the whole output pool, or no reduction in the outputs. Following this assessment, additional support for individuals will be considered by Liz Grand and Jenny Winters in HR. 

Round one: Submit your staff circumstances disclosure form by 6 January 2020

All round one applications for staff disclosing personal circumstances must be submitted to Liz Grand, Diversity and Inclusion Adviser, HR, by 6 January 2020 to ensure due consideration by the time that St George’s makes its submission about staff circumstances to REF in March 2020.

Round two: Submit your staff circumstances disclosure form by 20 January 2021

All round two applications for staff disclosing personal circumstances must be submitted to Liz Grand, Diversity and Inclusion Adviser, HR, by 20 January 2021 to ensure due consideration by the time that St George’s makes its submission about staff circumstances to REF in March 2021.

Disclosure forms may be submitted and revised after this date up to the REF submission deadline.

Contact Us

If you have any questions about disclosing individual staff circumstances for REF, contact Liz Grand, Diversity and Inclusion Adviser, HR.

If you have any questions about St George’s submission to the REF, contact Professor Jodi Lindsay, Research Director for Cross-Cutting University Themes, or Yvonne Castle, Research Strategy and Development Manager, JRES.

Download the full guidance and form:

• REF 2021 Declaration of Individual Staff Circumstances guidance (Word)

• REF 2021 Declaration of Individual Staff Circumstances guidance (PDF)

• REF 2021 Declaration of Individual Staff Circumstances form (Word)

The REF Steering Executive was formed in January 2018 to guide the process for SGUL’s submission for REF 2021. The REF Steering Executive, which is a subset of the Research Committee, is chaired by Professor Jodi Lindsay, Director of Cross Cutting Themes (formerly the Academic Lead for REF). The REF Steering Executive delivers the Code of Practice and provides leadership to enable a successful REF 2021 submission.

Members of the REF Steering Executive

• Professor Jodi Lindsay, Chair, Director of Cross Cutting Themes (formerly the Academic Lead for REF). Academic Lead for REF 2021
• Professor Jon Friedland, Deputy Chair, Deputy Principal (Research and Enterprise)
• Dr Rachel Allen, Head of Graduate School (and deputy for Director, IMBE)
• Professor Dot Bennett, Research Institute Director, Molecular and Clinical Sciences
• Professor Deborah Bowman, Deputy Principal (Institutional Affairs)
• Mark Cranmer, Director of Joint Research & Enterprise Services
• Dr Dan Forton, Associate Medical Director (Research), St George’s University Hospitals NHS Foundation Trust  
• Professor Peter Garrard, Clinical Academic Representative (Neuroscience)
• Professor Priscilla Harries, Associate Dean for Research and Innovation, FHSCE
• (deputy) Professor Annette Boaz, Professor in Health Care Research, FHSCE
• Professor Tom Harrison, Clinical Academic Representative (Infection)
• Dr Vanessa Ho, Associate Dean for Culture, Development and Inclusion
• Professor Juan Carlos Kaski, Chair of REF Impact Case Studies sub-committee; and Clinical Academic Representative (Cardiology)
• Professor Julian Ma, Research Institute Director, Infection and Immunity
• Louise Phillips, Head of Research Funding, JRES
• Professor Alicja Rudnicka, Researcher Representative
• Professor Jane Saffell, Institute Director, Medical and Biomedical Education
• Dr Laura Southgate, Researcher Representative
• Professor Peter Whincup, Research Institute Director, Population Health
• Jenny Winters, Director of Human Resources and OD
• Yvonne Castle, Secretary, Research Strategy and Development Manager, JRES

Any questions about St George's submission to REF 2021, please direct them to Professor Jodi Lindsay, Director of Cross Cutting Themes (Academic Lead for REF 2021), or Yvonne Castle, Research Strategy and Development Manager, Joint Research and Enterprise Services.

Any questions regarding the impact case studies, please contact Professor Juan Carlos Kaski, Chair of the REF Impact Case Studies Subcommittee.

Any questions regarding the environment statement, please contact Louise Phillips, Head of Research Funding, Joint Research and Enterprise Services.