HEC Ranking Criteria and its impact on quality assurance: implications for universities in Pakistan

Abstract

Globalization and market-based orientation of higher education institutions has increased the interest of students, parents, employers, universities, funding agencies, governments, and other relevant stakeholders in knowing the ranking of particular universities they have an interest in both at a national and a global level. This has led to the emergence of several systems for the ranking of universities at national and global levels. Aligned with international trends of ranking, the Higher Education of Pakistan (HEC) also initiated ranking of universities at the national level in Pakistan. Consequently, the HEC designed comprehensive criteria for the ranking of universities in Pakistan and has been implementing these since 2010. Furthermore, the HEC ranking criteria are aimed at pushing universities in Pakistan to streamline and adopt such quality assurance and enhancement mechanisms that are likely to improve the quality of their teaching, learning, research, and service. Consequently, most of the HEC ranking criteria indicators appear to be linked with the notion of accountability (or quality assurance) and improvement (or quality enhancement) in universities of Pakistan. In this context, the main purpose of this study was to examine the HEC ranking criteria and its impact on quality assurance in higher education of Pakistan. Moreover, this study also examined its implications for Pakistani universities. Specifically, this study initially examined the indicators of selected global and HEC ranking systems and their constituent indicators for their orientation as quality assurance and quality enhancement, as found in policies. This study also examined the perceived impact of the current indicators of the HEC ranking criteria on assuring and enhancing the quality of teaching, learning, and research in universities of Pakistan. Finally, this study suggests a potential framework for ranking universities in Pakistan that addresses needs of universities in relation to the notion of accountability and improvement. To achieve these objectives, a multi-phased mixed method research design was used. The study was conducted in three phases. The first phase of the study comprised document analysis of policies and practices related to ranking of universities at global and national levels, while the second phase comprised collection and analysis of both quantitative and qualitative data. The first two phases were conducted concurrently. Results of the first two phases of the study led to the third phase, which used explanatory sequential mixed method design. In Phase III, interviews were conducted with university teachers and administrators to seek their opinions about a potential framework for ranking of universities in Pakistan. The results of Phase I, Phase II, and Phase III were integrated to develop a potential framework for ranking of universities in Pakistan. This framework is likely to assure and enhance the quality of education and strengthen the current indicators of HEC ranking criteria. The main objective for Phase I of the study was to analyze the nature, focus, and weightage of the indicators of global and HEC ranking systems, as well as their constituent indicators, based on their orientation towards quality assurance and quality enhancement. In this phase, document analysis was used to conduct the examination. Data were collected through publicly available documents on the HEC website and websites of three other global ranking agencies. The three well-known global ranking agencies used in the analysis are ARWU (Shanghai Jiao Tong), Times Higher Education (THE), and Quacquarelli Symonds (QS). Furthermore, document analysis was used to examine in detail all indicators involved in the ranking procedures of ARWU, THE, QS, and HEC. Phase I of this research study found that the global university ranking systems focused on various academic areas. Some of these areas were found to be common to all the ranking systems, while others were slightly different. For example, the Times Higher Education ranking system places emphasis on teaching, research, and citations, while ARWU focuses more on quality of education and quality of faculty. Likewise, QS ranking places emphasis on academic reputation and HEC of Pakistan prioritizes quality of research and teaching. In summary, all three global ranking systems, including the HEC, have a high focus on quality of teaching and research. The main objective for Phase II of the study was to examine the nature and focus of current HEC ranking criteria and its constituent indicators and their impact on assuring and enhancing the quality of education in universities of Pakistan, as perceived by academics and administrators. Phase II of the study was mainly quantitative in nature, but also included a set of structured qualitative questions. This phase of the study was conducted in parallel with Phase I of the study, and both phases used a triangulation design. For collection of data in Phase II, a questionnaire comprising checklist items, five-point Likert scale items, and open-ended items was developed as a tool. This phase was further aimed at determining the extent to which the HEC ranking criteria and indicators address the needs of accountability (or quality assurance) and improvement (or quality enhancement) in universities of Pakistan. For Phase II of the study, all the Directors of Quality Enhancement Cells (QECs), Deans, Chairpersons/HODs, and the faculty members who were serving as members of Program Assessment Teams from all public sector universities of Pakistan served as a population. Using purposive sampling technique, the sample for Phase II comprised all Directors of QECs, Deans, Chairpersons/HODs, and the faculty members who were serving as members of Program Assessment Teams from public sector universities of Punjab and Federal Universities of Pakistan. Furthermore, the sample represented faculty members from ten departments of each university, and 517 academics and administrators were represented in the sample. The data were collected from the Directors of QEC, Deans, Chairpersons/HODs, and the faculty members of Program Assessment Teams. For the analysis of quantitative data, both descriptive and inferential statistics were used, with the help of SPSS software, and thematic analysis was used for analysis of open-ended quantitative items. The analysis of data in Phase II of the study found that respondents believe that current HEC ranking criteria indicators place a low (or moderately low) focus on both quality assurance and quality enhancement. It was further found that academics believe that the focus of the indicators of HEC ranking criteria is, however, more focused on enhancing the quality of education than on assuring quality. Respondents further believe that the current system of HEC ranking does not promote quality of teaching and quality of research in universities of Pakistan. Overall, respondents believe that the current system of HEC ranking of universities in Pakistan is not helpful in improving and enhancing the quality of teaching, learning, research, and service. They also confirmed that the current system of HEC ranking is neither serving the purpose of accountability, nor providing any strong system for accountability. The main purpose of Phase III of the study was to develop a potential framework for the ranking of universities in Pakistan (FRUP) that embeds the notion of accountability and improvement in universities of Pakistan. Phase I and Phase II of the study provided the basis for Phase III of study. Furthermore, this phase of study comprised both quantitative and qualitative methods. For the quantitative section, a questionnaire was used to seek opinions from academics and administrators about the potential indicators for inclusion in HEC ranking criteria in the future. Five-point Likert scale items and open-ended items were included. In this phase of the study structured interviews were also conducted with Directors of QECs, Deans, Chairpersons/HODs, and from faculty members of the Program Assessment Teams. The population for this phase was the same as that used for Phase II. Thematic analysis was employed to analyze the data. Finally, this phase of the study concluded that the potential framework for ranking of universities in Pakistan (FRUP) would be likely to address the demands of accountability and of improvement in universities of Pakistan. The framework (FRUP) comprises basic, complementary, and supportive components.