Trends in International Mathematics and Science Study (TIMSS) Essay Example for Free

Trends in International Mathematics and Science Study (TIMSS) Essay The Trends in International Mathematics and Science Study (TIMSS) is an international comparative assessment about mathematics and science education that is organized by the International Association for the Evaluation of Educational Achievement (IEA). The survey has been administered periodically in cycles of 4 years since 1995. Initially TIMSS was an acronym for the Third International Mathematics and Science Study, which identified its position as the third study following the First and Second International Mathematics and Science Studies (FIMS SIMS) in 1964 and 1982 respectively. A similar study to the 1995 TIMSS (using the same technical framework) was carried out in 1999, and was referred to as the Third International Mathematics and Science Study Repeat (TIMSS-R). The 2003 study was the third cycle of studies based on the 1995 assessment framework, and the acronym TIMSS was redefined to mean Trends in International Mathematics and Science Study (http://nces.ed.gov/timss). With funding from the U.S. National Science Foundation, the TIMSS assessment was to be offered more timely in intervals of 4 years. Table 1 gives an overview on the TIMSS assessment, target populations, and the number of countries that participated in each administration. Table 1. TIMSS assessments and participating countries at the 8th grade level Study Name Acronym Year Number of Countries Target population Grades tested Third International Mathematics and Science Study TIMSS 1994/5 42 3rd /4th, 7th/8th , 12th Third International Mathematics and Science Study Repeat. TIMSS-R 1998/9 40 8th Trends in International Mathematics and Science Study TIMSS 2002/3 46 4th 8th Trends in International Mathematics and Science Study TIMSS 2006/7 Over 60 4th, 8th, (12th rescheduled 2008) TIMSS is considered the largest, most comprehensive, rigorous, and extensive international comparative educational study ever conducted (Alejandro, 2000). It touches on almost every facet of the learning of science and mathematics. The 2003 TIMSS project was comprised of over 360 000 students, over 38 000 teachers, and over 12 000 school principals, and as many as 1 500 contextual variables were included in addition to variables on student achievement scores (Nelson, 2002; TIMSS‟ User guide 2003). TIMSS aims at providing policy makers and educational practitioners with information and indicators about their national educational systems from an international perspective. Alejandro argues that TIMSS serves a â€Å"mirror† function to participating countries to reflect comparatively on their education systems. Martin and Kelly (2004) suggest that TIMSS tests go beyond measuring achievement to including a thorough investigation of curriculum and how it is delivered in classrooms around the world. In a nutshell, the assessment is perceived to be valid and reliable as a measure of student achievement. However, the TIMSS assessments are not without critics. Among the critical voices is that of Wolf (2002) who questions the validity issues of the TIMSS studies. His contention is that TIMSS, being an international assessment, may have testing procedures that are not appropriate for some countries‟ usual testing practices. Related to that, Zuzovsky (2000) specifically questions the reliability of these tests. He argues that the heavily elaborated coding systems inherent in the TIMSS scoring procedures yield lower inter-rater consistency and also that translation of achievement tests into different languages brings the reliability of the assessments into question. The downside of cross-sectional assessments such as the TIMSS projects did not escape the attention of de Lange (2007) who questions the assumption behind international studies that a single test can give comparable measures of curriculum effects across countries. In spite of these criticisms, Alejandro, the chairman of IEA, sees the worth in countries participating in the TIMSS surveys. He argues that â€Å"More than just league tables, the TIMSS data place achievement in an international context where it can be considered from multiple perspectives† (Alejandro, 2000, p. 2). A similar statement was expressed more than 35 years ago by Bock (1970). He perceived the world as shrinking through technology. In support of international studies, he argued that through participation in international studies and by sharing educational reports, countries get a glimpse of each other‟s cultural practices. The TIMSS international reports give an overall impression of the impact that different education factors have on students‟ performance in different countries. It is by further engaging in secondary analysis of the TIMSS data and by conducting further studies related to the findings of TIMSS assessments that countries better understand their schooling system. In particular, it is important to identify areas of concern and address those systematically rather than attempt to replicate another country`s curriculum system simply because the schooling system there seems to be effective. Education indicators are numerous and vary in their effect from country to country. It is possible that each of the 1,500 contextual variables administered in the TIMSS surveys were included because of some supporting literature on their influence on students‟ learning. TIMSS as model of Educational Achievement The literature on models of educational indicators and their performance as a system together with research studies that model student learning achievement as a function of the characteristics of their schools and their family background is extensive (Oakes, 2006; Kaplan Elliott, 2007; Kaplan Kreisman, 2000; Koller, Baumert, Clausen, Hosenfeld, 2001). However, in spite of the extensive literature on the subject of modeling of educational data, no single model of educational performance has gained widespread acceptance (Oakes, 2006; Nelson, 2002). This lack of a global model was noted by Nelson in relation to the TIMSS projects, that, although rigorously executed, and with all its popularity (with more participants than any other IEA study), TIMSS has not attempted to come up with a prescriptive model that relates educational factors to student achievement that can be applicable across nations. Nelson is adamant that any attempt to provide a universal model would not be wise because countries differ in their educational policies and instructional practices so that a one-size fits- all model would not be realistic. That said, though models of educational performance seem diverse as suggested by the literature, in Haertel, Walberg, and Weinstein‟s (2003) view, they have more commonalities than differences. Haertel et al. conducted a meta-analysis of studies that modeled school performance data and found that the presented models had a common structure. Though the models differed in their specifications, their structures were comprised of three categories of pre-existing conditions (cognitive/affective attributes and resources), instructional processes (opportunity to learn, quality of instruction), and outcome measures (achievement, affective behaviors). These models presented student performance as a function of student, teacher, and/or school background variables. Studies of interest to my research are those that modeled students‟ performance as a function of their background variables; the model of interest that informed my variable selection is the Input-Process-Output (IPO) model by Oakes (2006), or Rand Model (Shavelson, McDonnell, Oakes, 2006). The IPO model has similar structural components as the models that were reviewed by Haertel et al. (2003) and models student achievement as a function of some resources. I selected this model for my conceptual framework because of its scope of coverage of educational indicators. The model presents a holistic conception of student learning in a classroom setting and it appears frequently in literature that analyzes large scale data (e.g. Kaplan and Kreisman, 2000; Koller, Baumert, Clausen, Hosenfeld, 2001 analyzing TIMSS data). Additionally, it has been used extensively to guide education researchers in the selection, specification, and analysis of educational variables that correlate with student learning outcomes (Kaplan Elliott, 2007; Kaplan Kreisman, 2000; Koller, Baumert, Clausen, Hosenfeld, 2001). It is taken as one of the influential models in shaping public opinion and policy on how to foster school improvement. TIMSS Input-Process-Output Model The input-process-output (IPO) model by TIMSS is one of the improved versions of the traditional input-output (IO) models of school organizational data (Glasman, Biniaminov, 2001). Glasman and Biniaminov reported that the input-output traditional model employed research strategies that measured changes in the systems‟ outputs brought about by changes in the systems‟ inputs. The IO model was criticized for not taking the academic environment into account and for oversimplifying the schooling process by portraying it as linear. According to Glasman and Biniaminov, â€Å"the input-output analyses [did] not deal with characteristics of the dynamic and ongoing interrelationships between students and teachers or those among students themselves† (p. 509). To overcome the problem, Oakes‟ model added a third component (processes) that mediated the input variables into the output variables and that also provided an educational context for the model. The process component focuses on classroom characteristics such as instructional quality issues (explained in the next section under measures for the model). Oakes‟ model is therefore comprised of three components of an educational system: inputs, processes, and outputs (IPO) (Figure1). Figure 1. A comprehensive Model of an Educational System INPUTS PROCESSES OUTPUTS Fiscal and Curriculum Achievement other quality resources School quality Instructional Participation Quality Dropouts Teacher Characteristics Teaching quality Student Attitudes Background Aspirations Note. The arrows indicate the direction of effect Briefly stated, according to Shavelson, McDonnell, and Oakes, (2001) â€Å"the TIMSS model‟s inputs are the human and financial resources available to education† (p. 13): This includes teacher quality (e.g. certification and experience), student background (e.g. parents‟ education and home possessions), and school quality (e.g. school climate). â€Å"Its processes are what is taught and how it is taught† (p. 13): This includes classroom characteristics such as curriculum quality (e.g. pace and coverage of materials), teaching quality (e.g. integration between teacher, pupil, and materials), and instructional quality (e.g. instructional tasks, teaching methods, and classroom climate) and â€Å"its outputs are the consequences of schooling for students from different backgrounds† (p. 13) such as academic achievement, participation (what percentage graduate versus drop out), and attitudes (e.g. any desires to continue studying math or career goals that are math related). The TIMSS model is complex and provides insight into how the various components of the education system relate to one another. As noted earlier, one of the shortcomings of the traditional input output model was its structure in which nested data was treated as though linear, and the TIMSS model addressed that problem. This was noted by Kaplan and Elliott that â€Å"the TIMSS‟ model, is one instantiation of the organizational structure of schooling that captures its hierarchical nature† (Kaplan, Elliott, 2007; p. 221). The two suggested that the model was multilevel in form and was testable through statistical methodologies that take the multilevel nature of educational data into consideration. Kaplan and Elliot used the model for their framework to propose a model-based approach for validating educational indicators that explicitly took into account the organizational features of schooling. The two contended from their model that it was not necessary for every indicator that has ever been suggested for collection to actually appear in the model. To them, it is the research questions and the goals for the investigations that should determine which indicators to include in a statistical model. Nevertheless, Oakes (2006) advised that a single indicator of each component of the educational system was inadequate. What was needed, in his view, was for each component to have indicators of all its most critical dimensions, and, â€Å"without a series of indicators that assesses all important facets of the schooling processes (the 3 components of the model), we can neither understand the system`s overall health nor determine the conditions under which a particular goal is met† (Oakes, 2006, p. 8). Oakes further observed that each of the three components appeared to be necessary but insufficient by itself to convey full information about school effects. That is, although a system of indicators measures distinct components of the education model, it also provides information about how the individual components work together to produce the overall effect. What can be deduced from Oake`s remarks is that studies that model only one component of the educational system may not be doing an adequate job of conveying the necessary information about the school effects. Conclusions What I considered informative from TIMSS and through the deliberations by Kaplan and Elliott (2007) about TIMSS`s model in general, and its use in particular, was how to utilize its structure to reach the different components of an education system holistically. It was of interest that the TIMSS model has flexible attributes in those variables could be rearranged to reflect the hierarchical nature of classroom data as evidenced in the current study in subsequent chapters. In one study, Kaplan and Kreisman (2000) used the TIMSS` model to validate indicators of mathematics education using its data. Rather than group their variables into the three distinct categories of Input-Process-Output as outlined in the model structure, they contended that since TIMSS` model was inherently multilevel, a subset of the inputs and processes occurred at higher levels of the education system. As a result, they grouped their indicators into three organizational levels: student, teacher, and school. That is to say, although TIMMS` model (Figure 1) groups school resources, teacher quality, and student background as one category of input (or prerequisite) variables, these indicators occur at different hierarchical levels of the school organization. Some of the variables included in the Kaplan and Kreisman model were: †¢Ã¯â‚¬  At student level: mathematics achievement, attitude toward mathematics, utility of mathematics, parents’ education, and mother’s expectations. †¢Ã¯â‚¬  At teacher level: method of instruction, teacher collaboration with colleagues, teacher’s level of education, and teaching experience. †¢Ã¯â‚¬  At school level: opportunities for continuing professional development, good facilities, school climate, level of discipline, and outstanding teacher recognition. Needless to say, Kaplan and Kreisman`s (2000) variables were representative of the three components of the IPO model even though the variables were grouped differently. Some of the variables they used were composite indicators. These were variables such as attitudes and methods of instruction. The two authors ran factor analyses to help group related items into the composite indicators and they had mathematics achievement as their outcome measure. In summary, TIMMS` model is more of a conceptual framework than a prescriptive one. That is, it does not prescribe what variables one should include in a statistical model for testing educational performance, but offers guidance about the components from which to draw the variables. Directions for Future Research TIMSS data have opened extensive avenues for further research work. In conclusion, this paper has evidenced the complexities involved in TIMSS data. It is hoped that this study and other studies that continue to model TIMSS data and pilot TIMSS instruments will help to illuminate the factors that explain student achievement in the us and in many other countries and to direct policy interventions.