Exploring University ESP Teachers’ Beliefs about Course Evaluation
Abstract
The paper examines tertiary English for Specific Purposes (ESP) teachers’ experiences and perceptions of course evaluation within a contemporary theoretical framework. It aims to determine the evaluation impetus, identify gaps in the dissemination of evaluation results, provide insight into teachers’ beliefs about their evaluation competencies and needs, and address their concerns and suggestions for the evaluation process. The study was motivated by the expanding influence of ESP courses, criticism of current evaluation practices initiated by universities, and lack of research in the field. The purpose was to deepen understanding of the evaluation process among teachers and other relevant parties and to obtain teachers’ recommendations for enhancing the evaluation capacity of the above-mentioned groups, which ultimately contributes to educational excellence and student success. The sample comprised 43 ESP teachers from private and state institutions of higher education in Serbia, who completed an electronic questionnaire. The quantitative data were analyzed using descriptive statistics, while thematic content analysis was employed for the qualitative data. The results indicate that course evaluation is still not fully accepted as a regular procedure at the tertiary level in Serbia, since not all institutions initiate it and there is an obvious gap in communication of the results. Yet, teachers generally consider evaluation an essential component of an ESP course and require training opportunities for broadening their knowledge and developing the skills relevant for the effective administration of course evaluation. The pedagogical implications of the study underscore the necessity of teachers’ professional development, the alignment of evaluation procedures with pedagogical realities, and the enhancement of cooperation with institutions and other relevant stakeholders.
References
Abrami, P. C. (2001). Improving judgments about teaching effectiveness using teacher rating forms. New Directions for Institutional Research, 109, 59–87. https://doi.org/10.1002/ir.4
Anthony, L. (2018). Introducing English for Specific Purposes. London: Routledge.
Assessment and Evaluation Language Resource Center [AELRC]. (2024). References on Evaluation in Language Education Programs. https://aelrc.georgetown.edu/evaluation-bibliography
Beretta, A. (1986). Toward a methodology of ESL program evaluation. TESOL Quarterly, 20, 144–155. https://doi.org/10.2307/3586398
Birckbichler, D. W. (Ed.). (2006). Evaluating Foreign Language Programs: Content, Context, Change. Columbus, OH: Foreign Language Center, Ohio State University.
Borch, I. (2020). Lost in translation: From the university’s quality assurance system to student evaluation practice. Nordic Journal of Studies in Educational Policy, 6(3), 231–244. https://doi.org/10.1080/20020317.2020.1818447
Borch, I., Sandvoll, R., & Risør, T. (2021). Student course evaluation documents: Constituting evaluation practice. Assessment & Evaluation in Higher Education, 47(2), 169–182. https://doi.org/10.1080/02602938.2021.1899130
Burden, P. (2008). Does the end of semester evaluation forms represent teacher’s views of teaching in a tertiary education context in Japan? Teaching and Teacher Education, 24(6), 1463–1475. https://doi.org/10.1016/j.tate.2007.11.012
Cousins, J. B. (2003). Utilization effects of participatory evaluation. In: T. Kellaghan & D. L. Stufflebeam (Eds.), International Handbook of Educational Evaluation 9 (pp. 245–265). Dordrecht: Springer. https://doi.org/10.1007/978-94-010-0309-4_16
Dassier, J. P., & Powell, W. (2001). Formative foreign language program evaluation: Dare to find out how good you really are. In: C. M. Cherry (Ed.), The Odyssey Continues: Dimension 2001 (pp. 91–110). Valdosta, GA: SCOLT Publications.
Davis, J. M. (2018). Best practices for language program evaluation success. In: J. M. Davis & T. H. McKay (Eds.), A Guide to Useful Evaluation of Language Programs (pp. 8–14). Washington, DC: Georgetown University Press.
Davis, J. M., & McKay, T. H. (2018). Planning for useful evaluation. In: J. M. Davis & T. H. McKay (Eds.), A Guide to Useful Evaluation of Language Programs (pp. 15–26). Washington, DC: Georgetown University Press.
Davis, J. M., Sinicrope, C., & Watanabe, Y. (2009). College foreign language program education: Current practice, future directions. In: J. M. Norris, J. M. Davis, C. Sinicrope, & Y. Watanabe (Eds.), Toward Useful Program Evaluation in College Foreign Language Education (pp. 209–226). Honolulu, HI: National Foreign Language Resource Center.
Đurić, M. (1998). ESP textbooks: Who should produce them and how? In: R. de Beaugrande, M. Grosman, & B. Siedlhofer (Eds.), Language Policy and Language Education in Emerging Nations (pp. 219–229). New York, NY: Ablex Publishing Corporation.
European Commission/EACEA/Eurydice. (2017). Modernisation of Higher Education in Europe: Academic Staff—2017. Eurydice Report. Publications Office of the European Union. https://op.europa.eu/en/publication-detail/-/publication/8b5c98e7-c501-11e7-9b01-01aa75ed71a1/language-en
Flowerdew, L. (2013). Needs analysis and curriculum development in ESP. In: B. Paltridge & S. Starfield (Eds.), The Handbook of English for Specific Purposes (pp. 325–346). Boston, MA: Wiley-Blackwell.
Gorsuch, G. (2009). Investigating second language learner self‐efficacy and future expectancy of second language use for high‐stakes program evaluation. Foreign Language Annals, 42(3), 505–540. https://doi.org/10.1111/j.1944-9720.2009.01034.x
Gruba, P. (2024). Designs for Language Program Evaluation. Cham: Springer International Publishing.
Janković, A., & Jarić, I. (2009). Uslovi rada nastavnog osoblja na Univerzitetu u Beogradu: Evaluacija rada nastavnika. Filozofija i društvo, 20 (3), 3–22. https://doi.org/10.2298/FID0903003J
Kiely, R., & Rea-Dickins, P. (2005). Program Evaluation in Language Education. London: Palgrave Macmillan.
Kırkgöz, Y., & Dikilitaş, K. (2018). Recent developments in ESP/EAP/EMI contexts. In: Y. Kırkgöz & K. Dikilitaş (Eds.), Key Issues in English for Specific Purposes in Higher Education (pp. 1–10). Cham: Springer International Publishing.
Llosa, L., & Slayton, J. (2009). Using program evaluation to inform and improve the education of young English language learners in US schools. Language Teaching Research, 13(1), 35–54. https://doi.org/10.1177/1362168808095522
Loh, J., Renandya, W. A., Pang, E., & Aryadoust, V. (2023). Language programme evaluation. In: Z. Tajeddin & C. Griffiths (Eds.), Language Education Programs: Perspectives on Policies and Practices (pp. 37–52). Cham: Springer. https://doi.org/10.1007/978-3-031-38754-8_3
Lynch, B. K. (1996). Language Program Evaluation: Theory and Practice. Cambridge: Cambridge University Press.
Manić, D. Lj. (2016). Evaluacija projektnih zadataka u integrisanom učenju stranog jezika i sadržaja (neobjavljena doktorska disertacija). Filološki fakultet, Beograd.
Milutinović, J. (2023). Koncepcije nastave univerzitetskih nastavnika i njihovi pristupi nastavi. Nastava i vaspitanje, 72 (3), 307–323. https://doi.org/10.5937/nasvas2303397M
Morgan, P. (2008). The Course Improvement Flowchart: A description of a tool and process for the evaluation of university teaching. Journal of University Teaching & Learning Practice, 5(2), 1–14.
Norris, J. M. (2009). Understanding and improving language education through program evaluation: Introduction to the special issue. Language Teaching Research, 13(1), 7–13. https://doi.org/10.1177/1362168808095520
Norris, J. M. (2016). Language program evaluation. The Modern Language Journal, 100, 169–189. https://doi.org/10.1111/modl.12307
Norris, J. M., Davis, J. M., Sinicrope, C., & Watanabe, Y. (Eds.). (2009). Toward Useful Program Evaluation in College Foreign Language Education. Honolulu, HI: National Foreign Language Resource Center.
Patton, M. Q. (1997). Utilization-Focused Evaluation: The New Century Text (3rd ed.). Thousand Oaks, CA: Sage.
Patton, M. Q. (2008). Utilization-Focused Evaluation (4th ed.). Thousand Oaks, CA: Sage.
Patton, M. Q. (2013). Utilization-Focused Evaluation (U-FE) Checklist. Western Michigan University, Evaluation Center. https://wmich.edu/sites/default/files/attachments/u350/2014/UFE_checklist_2013.pdf
Patton, M. Q., & Campbell-Patton, C. E. (2021). Utilization-Focused Evaluation (5th ed.). Thousand Oaks: SAGE Publications.
Prieto, J., Guede-Cid, R., Cid-Cid, A. I., & Leguey, S. (2023). Major increases in teachers’ performance evaluations: Evidence from student evaluation of teaching surveys. Tuning Journal for Higher Education, 10(2), 105–125. https://doi.org/10.18543/tjhe.2299
Räisänen, Ch., & Fortanet-Gómez, I. (2008). The state of ESP teaching and learning in Western European higher education after Bologna. In: I. Fortanet-Gómez & Ch.
Räisänen (Eds.), ESP in European Higher Education: Integrating Language and Content (pp. 11–51). Philadelphia, PA: John Benjamins Publishing Company.
Rea-Dickins, P., & Germaine, K. (1992). Evaluation. Oxford: Oxford University Press.
Rea-Dickins, P., & Germaine, K. (1998). Managing Evaluation and Innovation in Language Teaching: Building Bridges (1st ed.). New York, NY: Routledge. https://doi.org/10.4324/9781315841014
Richards, J. (2001). Approaches to evaluation. In: J. Richards (Ed.), Curriculum Development in Language Teaching (pp. 286–310). Cambridge: Cambridge University Press.
Stojković, M. (2012). Evaluacija udžbenika u nastavi engleskog jezika za posebne svrhe na univerzitetskom nivou. U: B. Dimitrijević (ur.), Filologija i univerzitet: tematski zbornik radova (str. 810–826). Niš: Filozofski fakultet.
Stufflebeam, D. L., & Coryn, C. L. S. (2014). Evaluation Theory, Models, and Applications (2nd ed.). San Francisco, CA: Jossey-Bass.
Theall, M., & Franklin, J. (2001). Looking for bias in all the wrong places: A search for truth or a witch hunt in student ratings of instruction?. New Directions for Institutional Research, 109, 45–56. https://doi.org/10.1002/ir.3
Tsou, W., & Chen, F. (2014). ESP program evaluation framework: Description and application to a Taiwanese university ESP program. English for Specific Purposes, 33(1), 39–53. https://doi.org/10.1016/j.esp.2013.07.008
Vidaković, M., Jerković, J., & Rakić, D. (2022). ESP course evaluation purposes and methodology: Tertiary teachers’ perspective and policy implications. TESOL Journal, 13(2), e653. https://doi.org/10.1002/tesj.653
Watanabe, Y., Norris, J. M., & Gonzales-Lloret, M. (2009). Identifying and responding to evaluation needs in college foreign language programs. In: J. M. Norris, J. M. Davis, C. Sinicrope, & Y. Watanabe (Eds.), Toward Useful Program Evaluation in College Foreign Language Education (pp. 5–56). Honolulu, HI: National Foreign Language Resource Center.
Yang, W. (2009). Evaluation of teacher induction practices in a US university English language program: Towards useful evaluation. Language Teaching Research, 13(1), 77–98. https://doi.org/10.1177/1362168808095524
Youker, B. (2018). Internal evaluation. In: B. B. Frey (Ed.), The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation (pp. 857–859). Thousand Oaks, CA: SAGE.
Кулић, Д. и Костић Минић, А. (2022). Стручно усавршавање наставника енглеског језика: Анализа каталога програма стручног усавршавања запослених у просвети. Зборник радова Филозофског факултета у Приштини, 52 (4), 51–68. https://doi.org/10.5937/zrffp52-40662
Миленовић, Ж. (2015). Објективност студената у евалуацији рада факултета, његових делова и учесника у наставном процесу. У: С. Маринковић (ур.), Настава и учење: евалуација васпитнообразовног рада (стр. 99–108). Ужице: Учитељски факултет у Ужицу.
The details about the publication policy, including copyright and licensing, are available at: