ERGP Project - online resources: Evaluation, quality assurance, enhancement

This section of the website contains links to articles and resources that look at ways in academic development activities may be evaluated. It also contains some resources of the role that quality assurance plays in the Higher Education context and the implications for academic development with an institution.

1. EVALUATING LEARNING ENVIRONMENTS

Cleveland, B. & Soccio, P. (2015) Evaluating the pedagogical effectiveness of learning spaces. Living and Learning: Research for a better Built Environment, pp. 507 – 516 

http://hdl.handle.net/11343/191728

___________________________________

Imms, W.,  Cleveland, B. & Fisher, K. (Eds.) (2016)  Evaluating Learning Environments: Snapshots of Emerging Issues, Methods and Knowledge. Rotterdam, Sense Publishers

https://www.sensepublishers.com/media/2871-evaluating-learning-environments.pdf

____________________________________

JISC (2018) UK Higher Education Learning Space Toolkit: case studies 

https://www.jisc.ac.uk/full-guide/learning-space-toolkit-case-studies

___________________________________

Standing Conference for Heads of Media Services (SCHOMS), Association of University Directors of Estates (AUDE) & Universities and Colleges Information Systems Association (UCISA) (n.d.) The UK Higher Education Learning Space Toolkit: a SCHOMS, AUDE and UCISA collaboration

https://www.ucisa.ac.uk/learningspace

___________________________________

Wood, P. , Warwick, P. & Cox, D. (2010) Evaluation of Experimental Learning Spaces. University of Leicester

https://www2.le.ac.uk/offices/lli/staff-development/docs/academic-prac/developing-learning-spaces-he.docx

 

2. EVALUATING PEDAGOGIC  & PROFESSIONAL PRACTICE

 

Guskey, T. (2002) Does It Make a Difference? Evaluating Professional Development Educational Leadership 59(6) pp. 45-51

http://www.ascd.org/publications/educational-leadership/mar02/vol59/num06/Does-It-Make-a-Difference%C2%A2-Evaluating-Professional-Development.aspx

In this article Guskey advocates the use of five critical levels of evaluation, to enable improvement of professional development programmes based on the enhancement of student learning. The five levels he identifies are as follows:

  1. Participants' Reactions
  2. Participants' Learning
  3. Organization Support & Change
  4. Participants' Use of New Knowledge and Skill
  5. Student Learning Outcomes

He suggests that evaluation can take place at different levels, but that by progressing through the 5 levels, you will better placed to demonstrate the direct impact of quality professional and academic development on the student learning experience.

 ___________________________________

Harvey J (1998): Evaluation Cookbook. Learning Technology Dissemination Initiative

http://www.icbl.hw.ac.uk/ltdi/cookbook/contents.html

A practical guide to evaluation methods for lecturers which provides a list of the methods with summaries, together with a page grouping evaluation approaches by their uses. It also provides step by step guides to the time, resources and process involved in different evaluation methods, with hints and tips relating to the stages of the process and links to related pages.

 ___________________________________

Kneale, P., Winter, J., Turner, R., Spowart, L., Hughes, J., McKenna, C. & Muneer, R. (2016a) Evaluating teaching development in higher education. Towards impact assessment: Literature review.  York , Higher Education Academy

https://www.heacademy.ac.uk/knowledge-hub/evaluating-teaching-development-higher-education

 ___________________________________

Kneale, P., Winter, J., Turner, R., Spowart, L., Hughes, J., McKenna, C. & Muneer, R. (2016b) Evaluating teaching development activities in higher education: A toolkit. York , Higher Education Academy

https://www.heacademy.ac.uk/system/files/evaluating_teaching_development_in_he_-_toolkit1.pdf

This literature review aims to update the Parsons et al. review commissioned in 2012 by the HEA in order to inform the development of a framework to evaluate the effectiveness of CPD. The review focuses on research into the impact of CPD in teaching and learning published between 2012 and 2015. This review broadens the scope and orientation of the original study to take into account a wider range of CPD activities and to include literature that critically engages with the impact discourse. It also considers prominent themes in the research on impact of CPD in relation to work on excellence in teaching in HE.

 The accompanying toolkit is developed as a resource for providers of teaching-related CPD in HE. It focuses on capturing the longer-term value and impact of CPD for teachers and learners, and moving away from immediate satisfaction measures. Informed by the literature review, the toolkit provides a series of templates that can be used to create bespoke CPD evaluations to capture impact before, during and after an activity or event.

 ___________________________________

Lewis J (2001): “Reflections on evaluation in practice”. In Evaluation Vol. 7 pp 387-394

 ___________________________________

MacDonald R (2006): “The use of evaluation to improve practice in learning and teaching”. In Innovations in Education and Teaching International Vol . 43: 1, pp 3-13

 ___________________________________

Moore I (2011): Guide to Practice: Evaluating your Teaching Innovation. National HE STEM Programme.

https://www.birmingham.ac.uk/Documents/college-eps/college/stem/evaluating-your-teaching-innovation.pdf

The purpose of this booklet is to offer some practical advice and guidelines on evaluating the impact of your teaching practice. Although it focuses on development projects in support of the STEM (Science, Technology, Engineering & Maths) programme, it will be relevant to any teaching practice. It explains the significance of evaluation in the context of the overall STEM programme, and particularly the Engineering Innovation scheme, as well as giving some general guidelines as to what you may wish to focus on when designing an evaluation strategy for your particular project or teaching practice.

 ___________________________________

University of Exeter (n.d.) Evaluating teaching: guidelines and good practice. Teaching Quality Assurance Manual Chapter 3

http://as.exeter.ac.uk/academic-policy-standards/tqa-manual/lts/evaluatingteaching/

Evaluation of teaching involves collecting evidence, from various stakeholders, for the purpose of improving the effectiveness of the teaching-learning process. A successful evaluation generates outcomes that are valid, reliable and indicate directions and action for development. These guidelines from the University of Exeter suggest five key questions to be addressed when considering the practical issues of evaluating teaching.

 ___________________________________

University of Michigan (n.d.) Guidelines for Evaluating Teaching

http://www.crlt.umich.edu/tstrategies/guidelines

These guidelines from the University of Michigan provide a sound overview of the types of methods and sources you might use to evaluate the quality of teaching.

 

3. QUALITY ASSURANCE & ENHANCEMENT

 

Matei, L. & Iwinska, J. (2016) Quality Assurance in Higher Education: A Practical Handbook. Budapest, Central European University

https://elkanacenter.ceu.edu/sites/elkanacenter.ceu.edu/files/attachment/basicpage/57/qahandbook.pdf

This Handbook is not meant as a manual for quality assurance specialists. Rather, it aims to provide a practical introduction to broader but crucial considerations and questions, and help facilitate a national debate on the topic of higher education quality and quality assurance in Myanmar. The Handbook introduces and discusses a number of key concepts, supplemented by examples of practices and methods from different higher education systems across the world. The examples presented, mainly from Europe, but also from Asia and North America, aim to illustrate the diversity that exists, across higher education systems, in the ways of addressing the issue of quality in higher education, and facilitate the policy planning and decision making in Myanmar, in particular by helping to identify which questions to address in priority and what measures to adopt and begin to introduce incrementally.

  ___________________________________

Elassy, N. (2015) The concepts of quality, quality assurance and quality enhancement, Quality Assurance in Education, Vol. 23(3), pp.250-261.

https://doi.org/10.1108/QAE-11-2012-0046

This paper aims to critically review and discuss different definitions of the concepts of quality, quality assurance (QA) and quality enhancement (QE) in higher education (HE) with presenting critical perspectives of the literature.

The paper looks at literature concerns with the meaning of quality, QA and QE, regarding HE context. It analysis and critically reviews the different definitions of these key concepts.

This paper suggests that the concepts of QA and QE should be dealt as part of a continuum and showed the need for both as an ongoing process in HE institutions.

The paper provides a unique analysis of the widely cited pieces of research regarding the concept of quality, QA and QE. It contributes to increase the understanding of those key concepts in HE sector, its origin and mean stream view. It outlines the importance of having a clear understanding of these terms and highlights the difficulties of having a unified definition.

  ___________________________________ 

Organisation for Economic Co-operation and Development (OECD) & Institutional Management in Higher Education (IMHE) (n.d.)  Learning our lesson: Review of quality teaching in Higher Education

https://www.oecd.org/education/imhe/44058352.pdf

In the context of the sustained growth and diversification of higher education systems, civil society is increasingly concerned about the quality of programmes offered to students. As a result, there is an increase in public assessments and international comparisons of higher education institutions, not only within the higher education sector but in the general media. However, evaluation methods tend to overemphasise research, and to use research performance as a yardstick of an institution‟s value. If these assessment processes fail to address the quality of teaching, it is in part because measuring teaching quality is complex and difficult..

The project examined the two main approaches to quality teaching: the top-down approach (those quality teaching initiatives taken by the institution collectively and determined by its leadership) and the bottom-up approach (those quality teaching initiatives taken by the teachers and which may nevertheless have an influence on the institutional policy on quality teaching). The focus of this review is mainly on the reasons for, and the effectiveness of, those initiatives. It is less concerned with the practical aspects and the concrete mechanisms used to put them into practice, which are heavily dependent on the circumstances of each institution.

  ___________________________________

Organisation for Economic Co-operation and Development (OECD) (2018) Enhancing Higher Education System Performance  - Report on benchmarking higher education system performance: conceptual framework and data

http://www.oecd.org/education/skills-beyond-school/Benchmarking%20Report.pdf

This document presents the report Benchmarking Higher Education System Performance:

Conceptual Framework and Data (the benchmarking report). The report presents an account of higher education today, the debate around the performance of higher education regarding its contribution to economic, social, cultural and environmental development, and the conceptual framework for the benchmarking higher education system performance project, including its data architecture, and the benchmarking data to be used.

  ___________________________________

QAA Scotland: Quality Enhancement Framework

https://www.qaa.ac.uk/scotland/quality-enhancement-framework

The Quality Enhancement Framework (QEF) is the enhancement-led approach to quality in Scottish higher education. Collaboration and partnership are at the heart of this innovative method. The QEF supports institutions, in managing the quality of the student learning experience. It also provides public confidence in academic standards and the quality of the student experience. It comprises a range of resources on the following themes: 

  • Enhancement-led Institutional Review
  • Enhancement Themes
  • Institution-led Reviews
  • Student Engagement

  ___________________________________

Ryan, Tricia. (2015). Quality assurance in higher education: A review of literature. Higher Learning Research Communications. 5. DOI: 10.18870/hlrc.v5i4.257.

https://www.researchgate.net/publication/288891841_Quality_assurance_in_higher_education_A_review_of_literature

The aim of this paper is to present a general view and a brief literature review of the main aspects related to quality assurance in global higher education. It provides an overview of accreditation as a mechanism to ensure quality in higher education, examines models of QA, and explores the concept of quality. In addition, this paper provides a review of research on the effectiveness of quality assurance practices, with a particular focus on student involvement with quality assurance. In reviewing the concept of quality assurance itself, the author noted there is a need for a common framework for a quality assurance model; however, there is no agreement as to a QA definition or a QA model. Furthermore, although quality is the utmost significant concern for accrediting bodies, accreditation structures are decentralized and complex at both the regional and international level. Another challenge identified revolves around the concerns of faculty members and other stakeholders, such as students, about the QA process. Given that students are at the center of higher education, and invest time and money in the system, the author concludes involving them could improve QA processes.

  ___________________________________

Seyfried, M. & Pohlenz, P. (2018) Assessing quality assurance in higher education: quality managers’ perceptions of effectiveness. European Journal of Higher Education 8(3) pp. 258-271

https://doi.org/10.1080/21568235.2018.1474777

The present article offers a mixed-method perspective on the investigation of determinants of effectiveness in quality assurance at higher education institutions. We collected survey data from German higher education institutions to analyse the degree to which quality managers perceive their approaches to quality assurance as effective. Based on this data, we develop an ordinary least squares regression model which explains perceived effectiveness through structural variables and certain quality assurance-related activities of quality managers. The results show that support by higher education institutions’ higher management and cooperation with other education institutions are relevant preconditions for larger perceived degrees of quality assurance effectiveness. Moreover, quality managers’ role as promoters of quality assurance exhibits significant correlations with perceived effectiveness. In contrast, sanctions and the perception of quality assurance as another administrative burden reveal negative correlations.

  ___________________________________

Williams, J. (2016) Quality assurance and quality enhancement: is there a relationship?

Quality in Higher Education 22 (2) pp. 97-102

https://www.tandfonline.com/doi/full/10.1080/13538322.2016.1227207

It is arguable that Quality Assurance and Quality Enhancement activity  are the two concerns which underpin all discussions of quality in higher education. Most of the existing definitions of the two terms show that they are clearly distinct activities but there have been remarkably few studies that have directly explored the relationship between them. This editorial reflects upon the relationship between quality assurance and quality enhancement with a view to stimulating further discussion within this journal.