TIMSS 2019 PAKISTAN: Where to next?

Introduction

Pakistan participated for the first time in TIMSS 2019, in fourth grade mathematics and science study. TIMSS, the Trends in International Mathematics and Science Study, is a flagship programme of the International Association for the Evaluation of Educational Achievement. Since 1995, the study measures international trends in student achievement in mathematics and science at grades four and eight. In December 2020, TIMSS results were announced. About 600,000 students participated from 58 countries and six regions. Only three countries represented the lower-middle income category: Pakistan, Morocco and Philippines.

TIMSS in Pakistan

TIMSS assessment focuses not just on content knowledge of mathematics but also on a range of problem solving, application and reasoning skills. These content domains are quite standard across primary school curricula internationally. These domains also align with Pakistan’s National Curriculum 2006 (Mathematics).

The TIMSS 2019 tests were held in 142 centers across Pakistan accommodating more than five thousand students of grade four. The National Education Assessment System (NEAS) carried out TIMSS under its supervision. The sample was drawn from the four provinces and the federally administered areas. Public and private schools with English, Sindhi or Urdu as their medium of instruction were also included in the sample. NEAS, however did not release the information about the cities included in the sample, or about the proportional participation of the schools from public or private sector.

The information about proportional participation of public and private sector schooling and profile about the medium of instruction is key as Pakistan is a linguistically diverse country with fifty-seven or more languages spoken throughout the country. Urdu is the national language and the lingua franca (common language), although 10% of the population qualify as native speakers. Urdu is the language of instruction in government schools and in low-fee paying private schools, while English is the medium of instruction in elite private schools and institutions of the federal government. In the province of Sindh, schools also have the option of keeping Sindhi as a medium of instruction.

TIMSS international benchmarks for its four-point achievement scale are as follows: a) Advanced international benchmark; b) High international benchmark; c) Intermediate international benchmark; and d) Low international benchmark.

Pakistan’s Performance in TIMSS 2019 and Possible Contributing Factors

Pakistan’s performance in TIMSS 2019 was dismal. Performance-wise, it stood second from the bottom. Only 27 percent of 4th graders in the country met the low international benchmark in mathematics, 8 percent met the intermediate international benchmark, and just one percent (1%) met the high international benchmark.

To sum up, learners performed well in simple content domain such as i‘Number’ but struggled in the more complex domain of ‘Measurement and Geometry and Data’. In short, it seems that a majority of the students were only able to cope with concepts such as solving simple problems and those relating to simple fractions and common geometric shapes. Items related to application and reasoning or requiring advanced mathematical knowledge had them floundering.

Language plays a crucial role in cognition and communication. In TIMMS 2019, there was relatively little variation in the average achievement of students who “sometimes”, “almost always” or “always” spoke the language of the test at home. However, the fourth graders who “never” spoke the language of the test at home had much lower average achievement in mathematics.

The Way Forward

While the dismal TIMMSS 2019 performance given Pakistan’s education crisis, is arguably is not unexpected, finding a way forward depends on looking at lessons learnt.

A key question to ask is what do we learn from these results, and, where do we go from here?

Large-scale, high-quality student-achievement data indubitably has value. It enables evidenced-based decision making about education systems and curricula. Pakistan’s participation in TIMSS significantly served to raise awareness about the issues with and around student achievement in mathematics and science. Several lessons can be learnt from this.

Firstly, all decisions arising from our TIMSS 2019 performance must factor in our 22M children and youth being out of school. Hence the non-formal education sector for out of school children and youth must heed lessons learnt about the quality of mathematics teaching, learning and achievement.

Secondly, reform in curriculum content requires concomitant change in curriculum delivery processes and curriculum evaluation. Otherwise, the intended curriculum goals would be difficult to achieve. Curriculum is delivered in the real world of classrooms. In effect, curricular change processes depend on teacher action. The quality and rigor of the academic challenge in classrooms are determined by teacher decisions about methodology, choice and use of materials and attending of student expectation. Lessons could be drawn from high performing TIMSS countries. The Japanese Lesson Study is one such strategy. It promotes teacher inquiry and can be incorporated within teacher education models.

Another alternative is carrying out a systematic comparative study of selected examination boards in Pakistan as students’ secondary and tertiary level academic career hinges on their performance in high-stakes examinations. Rind and Malik (2019) analysed examination items according to the six levels of the revised Bloom’s Taxonomy. Results demonstrated that mostly Examination Boards assessed lower-order thinking at level one and level two and with these items being recycled every year.

The third lesson relates to the issue of medium of instruction. As shared earlier, the TIMMS 2019 sample included students from all three mediums of instruction: English, Sindhi and Urdu. Yet, as per data, 36% of sample participants reportedly ‘never’ spoke the language of the test, English, at home. Not surprisingly, their performance, particularly in those areas, which require some language and literacy skills along with relevant mathematical knowledge, such as application, problem solving and reasoning, was the weakest. Developing students’ proficiency and achievement in mathematics is the valued ideal. Language instruction policies must facilitate achievement of this ideal alongside other goals such as achieving distributive justice (Fraser, 2001; Halai & Muzzafar, 2016) and enabling access to a global language.

Increasingly, students’ first or proximate languages are being recognised as a resource for teachers to employ for scaffolding mathematical learning in the target language. Using this strategy requires not only the introduction of technical change of pedagogy and techniques, but a challenging of deep- seated assumptions about language and pedagogic practices.

Fourth, another important step forward would be utilizing education research to inform policy and practice in the education sector. There is no dearth of studies looking at links between quality of school education in Pakistan and students’ dismal performance in TIMSS 2019. However, insights from these studies have yet to be used to inform policy.

Fifth, large in-country data sets need to be used effectively to inform decision-making about education. In Pakistan, data on education, including students’ achievement data, is collected by different agencies (e.g., NEAS, Secondary and Higher Secondary Examination Boards). However, these data sets are fragmented; do not necessarily speak to each other and often lack transparency about quality and governance of data. Availability of high-quality, reliable, comparable data on key education indicators across the four provinces and federally administered areas could be a significant resource for informing reform in the education system, curricula and teacher and school development. Researchers/ academics and research-students could also be given data access, to spur deliberation and further inquiry, ultimately informing decision making. Indeed, international bodies such as TIMSS have moved in this direction.

To conclude, should Pakistan participate in the next four-year cycle of TIMSS?

Participating in successive cycles of TIMSS could help develop a stronger baseline of data: enabling outlining of trends and evidence-based decision making, thereby increasing the assurance for efficacy. However, trends are not likely to change if targeted reform in education is not taken along the lines noted above. An alternate could be missing the next four-year cycle and entering the one after. This time could be used judiciously to address the gaps along the lines noted above. However, these gaps are fundamental and depend on policies for resource allocation, targeting interventions and policy implementation and thus political will pivotal.

References

1) Fraser, N. (2001). Social justice in the knowledge society. Invited keynote lecture at the conference on the “Knowledge Society,” Heinrich BöllStiftung, Berlin.

2) Halai, A., and I. Muzaffar. 2016. Language of instruction and learners’ participation in mathematics: dynamics of distributive justice in the classroom. In A. Halai and P. Clarkson (Eds), Teaching and Learning Mathematics in Multilingual classrooms: Issues for Policy and Practice (pp 52-72). Rotterdam: SENSE.

3) Rind, I., & Malik, A. (2019). The examination trends at the secondary and higher secondary level in Pakistan. Social Sciences & Humanities Open. doi:https://doi.org/10.1016/j.ssaho.2019.100002.

To share your thoughts and queries, you can comment here.