A psychometric study: The validation of a school quality assessment tool
https://doi.org/10.17853/1994-5639-2022-4-112-139
Abstract
Introduction. School evaluation is a complex process, and it is now a central priority for education systems with varied practices and multiple actors. It affects many aspects: teachers, institutions, training, management, educational policies, and design. Thus, it is necessary to think first about the regulation of the indicators that will serve as a system of valuation/sanction of the quality measured.
Aim. The present research aims to construct a grid for evaluating pedagogical and administrative quality of secondary school. Furthermore, producing a quality evaluation system based on indicators to allow quality to be witnessed remains a challenge. We have optimized and validated a coherent evaluation system of indicators (School Quality Assessment System SEQES). It is based on rigorous scientific research, evaluating school quality and testing with 196 stakeholders (school administrators, teachers, educational inspectors, and planning).
Research methodology and methods. The design and validation of this evaluation grid were carried out based on Churchill’s (1979) theory of measurement with a methodological process of numerous steps of analysis and emergent categorisation of items, scaling, refinement, and field testing.
Results and scientific novelty. The exploratory factor analysis (EFA) was administered to 196 stakeholders and initially yielded 153 items from 289. They articulated qualitative steps (interviews with members of audit cells, techniques: Focus Group, TGN, TRIAGE) and quantitative (Exploratory Factor Analysis). The results made it possible to identify a scale of seven dimensions and fields of 37 items, considering theoretical, empirical, and methodological considerations.
Practical significance. This measurement instrument is a toolbox that can be developed for decision-makers to establish a reference system for systematic external quality evaluation at the national level and a toolkit for inspectors and headteachers.
About the Authors
H. Berbar
Hassan II University
Morocco
Morocco
Hicham Berbar – Pedagogical Inspector of Education, PhD Student (Training in Educational Engineering and Science Didactics), Faculty of Sciences of Ben M’sik
Casablanca
S. Lotfi
Normal Superior School, Hassan II University
Morocco
Morocco
Said Lotfi – Dr. Sci. (Training in Educational Engineering and Research Methodology), Director of a Research Laboratory
Casablanca
M. Essaoudi
Inspectors Training Center for Teaching
Morocco
Morocco
Mohamed Essaoudi – Dr. Sci. (Engineering Science), Quality Audit Specialty
Rabat
M. Talbi
Hassan II University
Morocco
Morocco
Mohammed Talbi – Dr. Sci. (State in Sciences, Evaluating Analysis Processes and Educational Systems)
Casablanca
References
1. Bowe A. G. The development of education indicators for measuring quality in the English-speaking Caribbean: How far have we come? Evaluation and Program Planning. 2015; 48: 31–46.
2. Dayan C., Scelles R., Boutin A.-M., Ponsot G., Arnaud C., Storme M. Création et validation d’une échelle de qualité de vie et d’une grille d’observation des caractéristiques personnelles et environnementales pour les enfants polyhandicapés : Enseignements d’une étude. Motricité Cérébrale: Réadaptation, Neurologie du Développement. 2016; 37: 127–138. (In French)
3. Karatepe O. M., Yavas U., Babakus E. Measuring service quality of banks: Scale development and validation. Journal of Retailing and Consumer Services. 2005; 12: 373–383.
4. Roussiau N., Bailly N., Renard E. Premières étapes de construction et de validation d’une échelle de spiritualité explicite areligieuse. Pratiques Psychologiques. 2018; 24: 277–291. (In French)
5. Machado T., Desrumaux P., Van Droogenbroeck A. Indicateurs organisationnels et individuels du bien-être. Étude exploratoire auprès d’aides-soignants et d’infirmiers. Bulletin de Psychologie. 2016; 541: 19. (In French)
6. Talbott E., Maggin D. M., Van Acker E. Y., Kumm S. Quality indicators for reviews of research in special education. Exceptionality. 2018; 26: 245–265.
7. Karatepe O. M., Yavas U., Babakus E. Measuring service quality of banks: Scale development and validation. Journal of Retailing and Consumer Services. 2005; 12: 373–383.
8. Torres Fragoso J., Luna Espinoza I. Assessment of banking service quality perception using the SERVPERF model. Contaduría y Administración. 2017; 62: 1294–1316.
9. Dahler-Larsen P. Publication des données sur la qualité des établissements scolaires au Danemark : contingence et non-linéarité. Education et Sociétés. 2011; 28: 21. (In French)
10. Maghnouj S., Belanger J., Clarke M., Fordham E., Kitchen H., McGregor I. Examens de l’OCDE du cadre d’évaluation de l’éducation: Maroc: OECD Reviews of Evaluation Framework Education. Morocco: L’Organisation de cooperationet de developpement economiques (OCDE); 2018. 270 p. (In French)
11. Billing D., Thomas H. The international transferability of quality assessment systems for higher education: The Turkish experience. Quality in Higher Education. 2000; 6: 31–40.
12. van Damme D. Tendances et modèles de l’assurance internationale de la qualité de l’enseignement supérieur en relation avec le commerce des services d’éducation. Gest Enseign Supér. 2002; 14: 107–158.
13. Ramseook-Munhurrun P., Nundlall P. Service quality measurement for a secondary school setting. Quality Assurance in Education. 2013; 21: 387–401.
14. Riahi S., Riahi A. The Pedagogy of higher education: How to evaluate the quality of training in Morocco to improve it. International Journal of Engineering Pedagogy (iJEP). 2018; 8: 92.
15. Ghaicha A., Moroccan higher education system: Reality and prospects. Higher Education of Social Science. 2018; 14: 10–17.
16. Mohjbilou J. ةيؤرلا صن [Internet]. نيوكتلاو ةيبرتلا ةموظنم حالصإل ةيجيتارتسإلا ةيؤرلا يملعلا ثحبلاو - The strategic vision for the reform of the education system 2015-2030. [cited 2021 May 20]. Available from: https://vision.csefrs.ma/integral/ (In Arabic)
17. Badran A., Baydoun E., Hillman J. R. Major challenges facing higher education in the Arab world: Quality assurance and relevance [Internet]. Cham: Springer International Publishing; 2019 [cited 2021 May 20]. 381 p. Available from: http://link.springer.com/10.1007/978-3-030-03774-1
18. Hildesheim C., Sonntag K. The quality culture inventory: A comprehensive approach towards measuring quality culture in higher education. Studies in Higher Education. 2020; 45: 892–908.
19. Kohoutek J. Deconstructing institutionalisation of the European standards for quality assurance: From instrument mixes to quality cultures and implications for international research: Deconstructing institutionalisation of the European standards. Higher Education Quarterly. 2016; b70: 301–326.
20. Ntim S. Embedding quality culture in higher education in Ghana: Quality control and assessment in emerging private universities. Higher Education. 2014; 68: 837–849.
21. Sattler C., Sonntag K., Götzen K. The Quality Culture Inventory (QCI): An instrument assessing quality-related aspects of work. In: Deml B., Stock P., Bruder R., Schlick C. M. (Eds.). Advances in Ergonomic Design of Systems, Products and Processes [Internet]. Berlin, Heidelberg: Springer Berlin Heidelberg; 2016 [cited 2021 May 20]. p. 43–56. Available from: http://link.springer.com/10.1007/978-3-662-48661-0_3
22. Gunawan W., Creed P. A., Glendon A. I. Development and initial validation of a perceived future employability scale for young adults. Journal of Career Assessment. 2019; 27: 610–627.
23. Huson N. Oman. Quality culture in higher education a good-practice example. Zeitschrift Für Interkulturellen Fremdsprachenunterricht. 2015; 20: 101–115.
24. Shah A. A., Uqaili M. A., Qureshi A. S. Adoption of quality culture — A case study of Mehran University of Engineering & Technology. In: 2017 IEEE Global Humanitarian Technology Conference (GHTC) [Internet]; Jamshoro, Sindh, Pakistan. San Jose, CA: IEEE; 2017 [cited 2021 May 20]; p. 1–5. Available from: http://ieeexplore.ieee.org/document/8239254//
25. Karaca M., Inan S. A measure of possible sources of demotivation in L2 writing: A scale development and validation study. Assessing Writing. 2020; 43: 100438.
26. DeVellis R. F. Scale development: Theory and applications. Newbury Park, CA: Sage publications; 2016. 205 p.
27. Ozdemir Y., Kaya S. K., Turhan E. A scale to measure sustainable campus services in higher education: “Sustainable Service Quality.” Journal of Cleaner Production. 2020; 245: 118839.
28. Abdullah F. HEdPERF versus SERVPERF: The quest for ideal measuring instrument of service quality in higher education sector. Quality Assurance in Education. 2005; 13: 305–328.
29. Arribas Díaz J. A., Martínez-Mediano C. The impact of ISO quality management systems on primary and secondary schools in Spain. Quality Assurance in Education. 2018; 26: 2–24.
30. Detert J. R., Schroeder R. G., Cudeck R. The measurement of quality management culture in schools: Development and validation of the SQMCS. Journal of Operations Management. 2003; 21: 307–328.
31. Soria-García J., Martínez-Lorente Á. R. Development and validation of a measure of the quality management practices in education. Total Quality Management & Business Excellence. 2014; 25: 57–79.
32. Varouchas E., Sicilia M.-A., Sánchez-Alonso S. Towards an integrated learning analytics framework for quality perceptions in higher education: A 3-tier content, process, engagement model for key performance indicators. Behaviour & Information Technology. 2018; 37: 1129–1141.
33. Snijders I., Rikers R. M. J. P., Wijnia L., Loyens S. M. M. Relationship quality time: The validation of a relationship quality scale in higher education. Higher Education Research & Development. 2018; 37: 404–417.
34. Heo W., Park N., Park K. Classifying students using an expectation-perception survey about a hospitality laboratory class: Empirical research with the finite mixture model. Anatolia. 2020; 31: 50–61.
35. Ramseook-Munhurrun P., Naidoo P., Nundlall P. A proposed model for measuring service quality in secondary education. International Journal of Quality and Service Sciences. 2010; 2: 335–351.
36. Tovey P. Quality assurance in continuing professional education: An analysis [Internet]. 1st ed. London: Routledge; 2013 [cited 2021 May 20]. 224 p. Available from: https://www.taylorfrancis.com/books/9780203423684
37. Silva D. S., Moraes G. H. S. M., Makiya I. K., Cesar F. I. G. Measurement of perceived service quality in higher education institutions: A review of HEdPERF scale use. Quality Assurance in Education. 2017; 25: 415–439.
38. Sweis R., Diab H., Mahmoud Saleh F. I., Suifan T., Dahiyat S. E. Assessing service quality in secondary schools: The case of Jordan. Benchmarking: An International Journal. 2016; 23: 1207–1226.
39. Gronroos C. Service quality: The six criteria of good perceived service. Review of business. St. John’s University. 1988; 9: 10.
40. Lai Fong Cheng A., Keung Yau H. Principals’ and teachers’ perceptions of quality management in Hong Kong primary schools. Quality Assurance in Education. 2011; 19: 170–186.
41. Domínguez A. Q., Ruiz M. Á., Huertas J. A., Alonso-Tapia J. Development and validation of the School Climate Questionnaire for Secondary and High School Teachers (SCQSHST). Anales de Psicología. 2019; 36: 155–165.
42. Gaudreau N., Frenette É., Thibodeau S. Élaboration de l’Échelle du sentiment d’efficacité personnelle des enseignants en gestion de classe (ÉSEPGC). Mesure et évaluation en éducation. 2015; 38: 31. (In French)
43. Cheng Y.-Y., Chen L.-M., Liu K.-S., Chen Y.-L. Development and psychometric evaluation of the school bullying scales: A Rasch measurement approach. Educational and Psychological Measurement. 2011; 71: 200–216.
44. Thomas H. J., Scott J. G., Coates J. M., Connor J. P. Development and validation of the bullying and cyberbullying scale for adolescents: A multidimensional measurement model. British Journal of Educational Psychology. 2019; 89: 75–94.
45. Saitoti G. Education sector review: How far have we come since independence and what still needs to be done to meet the educational needs of all Kenyans. In: Report of the National Conference on Education and Training held at the Kenyatta International Conference Centre; 2003 Nov 27–29; Nairobi. p. 50–64.
46. Kasetwar R. Quality in higher education. University News. 2008; 46: 6–12.
47. Collignon E., Wissler M. Qualité et compétitivité des entreprises: du diagnostic aux actions de progrès. 2nd ed. Paris, France: FeniXX; 1988. 293 p. (In French)
48. Churchill G. A. A paradigm for developing better measures of marketing constructs. Journal of Marketing Research. 1979; 16: 64–73.
49. Helmer O. The use of the Delphi Technique in problems of educational innovations. Rand Paper Series. 1966. 22 p.
50. Nunnally J. C. Psychometric theory (revised). New York, NY: McGraw-Hill; 1978. p. 97–146.
51. Cronbach L. J. Coefficient alpha and the internal structure of tests. Psychometrika. 1951; 16: 297–334.
52. Cronbach L. J. Test validation. In: R. Thorndike (Ed.). Educational Measurement. 2nd ed. Washington DC: American Council on Education; 1971. p. 443–507.
53. Costello A. B., Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation. 2005; 10: 7.
54. Kieffer K. M. Orthogonal versus oblique factor rotation: A literature review regarding the pros and cons. In: Mid-South Educational Research Association Annual Meeting (MSERA); New Orleans, 1998 Nov 3–6. New Orleans, LA; 1998. p. 4–6.
55. Guttman L. A note on Sir Cyril Burt’s ‘factorial analysis of qualitative data. British Journal of Statistical Psychology. 1953; 6: 1–4.
56. Bourque J., Poulin N., Cleaver A. Évaluation de l’utilisation et de la présentation des résultats d’analyses factorielles et d’analyses en composantes principales en éducation. Revue des Sciences de L’éducation. 2006; 32: 325–344. (In French)
57. Elliot A. J., McGregor H. A. A 2×2 achievement goal framework. Journal of Personality and Social Psychology. 2001; 80: 501.
58. Hurley A. E., Scandura T. A., Schriesheim C. A., Brannick M. T., Seers A., Vandenberg R. J., et al. Exploratory and confirmatory factor analysis: Guidelines, issues, and alternatives. Journal of Organizational Behavior. 1997; 18: 667–683.
59. Neuville S., Frenay M. La persévérance des étudiants de 1er baccalauréat à la lumière du modèle expectancy-value. In: Michaut C., Romainville M. (Eds.). Réussite Échec Abandon Dans L’enseignement Supér. France: De Boeck; 2012. p. 157–75. (In French)
60. Gorsuch R. L. Factor analysis. Hillsdale, NJ: Lawrence Erlbaum Associates; 1983. 425 p.
61. Thode H. Testing for normality. New York; Marcel Dekker: 2002. p. 99–123.
62. Clark V. L. P., Ivankova N. V. Mixed methods research: A guide to the field. Vol. 3. Thousand Oaks, CA: Sage publications; 2015. 361 p.
63. Field A. Discovering statistics using SPSS. London: SAGE Publications; 2005. 816 p.
64. Van Maele D., Van Houtte M. The quality of school life: Teacher-student trust relationships and the organizational school context. Social Indicators Research. 2011; 100: 85–100.
65. Casanova M. A. Evaluación y calidad de centros educativos. Madrid: La Muralla; España; 2004. 276 p. (In Spanish)
66. Claude Ah-Teck J., Starr K. Principals’ perceptions of “quality” in Mauritian schools using the Baldrige framework. Journal of Educational Administration. 2013; 51: 680–704.
67. Olson L. M. An examination of quality management in support functions of elementary and secondary education using the Malcolm Baldrige National Quality Award’s criteria for performance excellence. 2009. 187 p.
68. Poston W. K. Factors impacting perceived quality in school organizations. Education Policy Analysis Archives. 1997; 5: 19..
69. Svensson M., Klefsjo B. Experiences from creating a quality culture for continuous improvements in the Swedish school sector by using self-assessments. Total Quality Management. 2000; 11: 800–807.
70. DeVellis R. F. Scale development: Theory and applications. London: Sage publications; 2016. 280 p.
71. Roussel P. Méthodes de développement d’échelles pour questionnaires d’enquête. In: Roussel P., Wacheux F. (Eds.). Management des ressources humaines. méthodes de recherche en Sciences Humaines et Sociales. France: De Boeck; 2005. p. 245–276. (In French)
72. Bertrand R., Blais J.-G. Modèles de mesure: l’apport de la théorie des réponses aux items. France: PUQ; 2004. 389 p. (In French)
73. Hair J. F., Black W. C., Babin B. J., Anderson R. E., Tatham R. Multivariate data analysis. NJ: Pearson Prentice Hall; 2006. 424 p.
74. Lagrosen S., Seyyed-Hashemi R., Leitner M. Examination of the dimensions of quality in higher education. Quality Assurance in Education. 2004; 12: 61–69.
75. Newton P. Validity and educational assessment. 1st ed. Thousand Oaks, CA: Sage Publications; 2013. 253 p.
76. Wu S., Lin C., Wu S., Chuang C.-L., Kuan H.-Y. Factors affecting quality of service in schools in Hualien, Taiwan. Procedia – Social and Behavioral Sciences. 2014; 116: 1160–1164.
77. Wilson A., Zeithaml V. A., Bitner M. J., Gremler D. D. Services marketing: Integrating customer focus across the firm. McGraw Hill; 2012. 529 p.
78. Shewhart W. A., Deming W. E. Statistical method from the viewpoint of quality control. Courier Corporation; 1986. 155 p.
79. Karatzias A., Power K. G, Swanson V. Quality of school life: Development and preliminary standardisation of an instrument based on performance indicators in Scottish secondary schools. School Effectiveness and School Improvement. 2001; 12: 3: 265–284.
80. Kells H. R., Mundial B. Performance indicators for higher education: A critical review with policy recommendations (No.PHREE/92/56). Washington, DC: Education and Employment Division, Population and Human Resources Department, The World Bank; 1992. 76 p.
Review
For citations:
Berbar H., Lotfi S., Essaoudi M., Talbi M. A psychometric study: The validation of a school quality assessment tool. The Education and science journal. 2022;24(4):112-139. https://doi.org/10.17853/1994-5639-2022-4-112-139
Views:
790
Email this article 