Critical Review of the Journal Article Sample
Critical Review of the Journal Article Sample Educators are researchers and decision makers. The rapid advancement of information technologies and the changing demands of the postmodern reality require educators to develop new methods of learning and knowledge delivery in the classroom. Every day educators face a myriad of challenges. They must constantly expand their knowledge and improve their skills to make valid decisions regarding the best instructional methods and practices. Research is a foundational ingredient of all learning practices; it provides direct and reliable information about education effectiveness and contributes to the speed and efficiency of professional decision making in education.
Evaluating the Research Article: A Brief Insight
The goal of this paper is to review a research article in the field of education. The paper is focused on the analysis of the study performed by Mullens, Murnane and Willett in 1996 and titled The Contribution of Training and Subject Matter Knowledge to Teaching Effectiveness: A Multilevel Analysis of Longitudinal Evidence from Belize. The purpose of the study was to explore whether pedagogical training, educational attainments and subject matter competence predicted the effectiveness of primary school teachers in Belize and student achievements in learning mathematics (Mullens et al. 1996, p. #). The need for the study was justified by the complexity of student experiences in classroom settings and the difficulties faced by researchers in their striving to evaluate teacher’s effectiveness. The study aimed to close the methodological and informational gap in literature, but failed to produce usable results. Mullens et al. (1996, p. #) found that none of the variables was significantly related to teacher effectiveness in elementary mathematics. However, of all variables included in the study, subject matter competence was found to produce the greatest impacts on student outcomes in learning advanced mathematics (Mullens et al. 1996, p. #).
Theoretical Frameworks
Theoretical frameworks often become the first step in designing professional research projects. Present day educational researchers are moving away from a rigid distinction among various theoretical frameworks, but theories continue to serve the basis for developing relevant research designs. Mullens et al. (1996, p. #) do not discuss any theoretical frameworks, nor do they use any of them to guide their research. However, it is possible to assume that at the heart of the discussed research is the scientific realism philosophy, which supports quantitative research approaches and is used to produce numerical research results (Lodico, Spaulding & Voegtle 2010, p. #).
Scientific realism is rooted in the positivist philosophy of research, which reflects the realism of the surrounding reality and motivates researchers to capture the objective reality in which they exist (Lodico et al. 2010, p. #). Scientific realism implies that all phenomena and the world itself are measurable and can be accurately described/ measured. This is also what Mullens et al. (1996, p. #) try to prove in their article: the researchers try to demonstrate that the relationship between teacher training, teacher effectiveness, and student outcomes is objective and can be easily quantified. Like scientific realists, Mullens et al. (1996, p. #) break the complex world of learning and the process of teaching into smaller, comprehensible parts. The researchers devise multi-level models of teaching effectiveness that incorporate the major elements of the teaching process and facilitate assessment of various factors and their impacts on student outcomes. However, Mullens et al. (1996, p. #) forget that the scientific realism framework is not without limitations. Scientific realism is built on the assumption that all individuals perceive the world in the same way (Giere 2005, p. #). In reality, humans are extremely different in their ability to perceive the reality in which they exist (Giere 2005, p. #).
Therefore, it is slightly incorrect to view the scientific realism framework as the ultimate source of truth about the world. All ideas, suggestions and results produced in educational research are just a tiny share of the sophisticated, multifaceted learning realities. Scientific realism lacks an interpretive angle; therefore, the numerical results produced during the study will need additional qualitative interpretation to ensure greater validity and reliability of the findings.
Of particular interest is the use of concepts in the study. In Mullens et al. (1996, p. #), teacher effectiveness is one of the central concepts.
Unfortunately, the researchers do not provide any explicit definition of teacher effectiveness; nor do they try to derive such definition from the earlier studies. In the discussed study, the concept of teacher effectiveness is interpreted in terms of three variables: (a) years spent in the pedagogical training program; (b) whether or not the teacher has a high school degree; and (c) the level of teacher’s competence in mathematics (Mullens et al. 1996, p. #). The concept of teacher effectiveness is extremely vague, and the variables used to define it lack empirical support. The way the researchers define teacher effectiveness and the variables they include in the definition meet the purpose of the study, but can hardly be used in a different research project. As a result, the concept of teacher effectiveness proposed by Mullens et al. (1996, p. #) and the results to which it leads should be treated with caution.
Data Collection and Data Analysis
Mullens et al. (1996, p. #) used pretest and posttest procedures to evaluate student outcomes and their relation to teacher effectiveness in learning math. According to Mullens et al. (1996, p. #), a 69-item curriculum-based instrument used in the study was developed by the Ministry of Education and administered with the help of Peace Corps teacher-trainers. In other words, the researchers used a standard questionnaire that was distributed among third-grade math students during the school year of 1990-91 (Mullens et al. 1996, p. #). 1,043 students were included in the sample (Mullens et al. 1996, p. #). In terms of data analysis, Mullens et al. (1996, p. #) developed two multilevel statistical models that described the relationship between student learning and selected student characteristics and the relationship between student learning and external influences, namely, teacher characteristics and effectiveness (Mullens et al. 1996, p. #). The data collection and analysis techniques used by Mullens et al. (1996, p. #) raise the question of their appropriateness and applicability in the context of the discussed research.
To begin with, questionnaires exemplify a unique and extremely convenient instrument of collecting information from a large number of respondents. Questionnaires are believed to be a structured means of collecting information, since the researcher determines the range of possible answers that can be produced (Gillman 2000, p. #). Although respondents have a choice, the range of answers is limited and defined by the researcher. This is the main reason why questionnaires facilitate and speed up the process of collecting primary data. In Mullens et al. (1996, p. #), the choice of pretest and posttest questionnaires is justified by the presence of a large sample and the relatively low costs of collecting the data. Questionnaires allow getting information from respondents quickly.
Respondents, in turn, are free to complete their questionnaires when they deem it necessary and possible (Gillman 2000, p. #). Questionnaires also reduce researchers’ bias and allow for the fast standardization of questions and answers (Gillman 2000, p. #). Unfortunately, the use of questionnaires in educational research is associated with a number of limitations, such as poor quality of the data, respondents’ misunderstanding of the questions and possible answers, as well as problems with literacy among respondents (Gillman 2000, p. #). Mullens et al. (1996, p. #) do not report any problems with their pretest and posttest procedures and do not mention any limitations concerning the use of questionnaires as a data collection technique.
Whether or not questionnaires are appropriate in educational research depends on a number of factors. First, the type of the information required is the principal criterion of questionnaire effectiveness in educational research (Chandra & Sharma 2004, p. #). Respondents should not spend more than 20 minutes answering questions or, in case of Mullens et al. (1996, p. #), solving mathematical problems (Chandra & Sharma 2004, p. #).
Mullens et al. (1996) do not provide any details of the data collection process; for this reason, defining the appropriateness of the data collection method is extremely problematic. Second, the appropriateness of questionnaires depends on the type of respondents reached (Chandra & Sharma 2004, p. #). The goal of Mullens et al. (1996) study was to measure the relationship between teacher training, teacher effectiveness, and student outcomes in mathematics, and the use of pretest and posttest questionnaires seems to be feasible. Again, since Mullens et al. (1996, p. #) do not describe the items included in the pretest and posttest questionnaires and do not mention any difficulties faced by the respondents while completing the questionnaire form, there is no possibility to assess the quality and effectiveness of the data collection process.
Mullens et al. (1996, p. #) used numerical data analysis methods, which made the interpretation and comprehension of the results rather problematic. The fact is that the researchers did not use traditional data analysis methods, but, instead, created multilevel models of data analysis that incorporated numerous of parameters and statistical codes.
Nonetheless, the multilevel models of statistical analysis fit in the nature and purpose of the learning process in the classroom: the process of delivering knowledge in the classroom is extremely complex and encompasses a variety of factors, processes, events, contexts, and techniques. Quantitative data analysis by itself is a powerful source of unique information that renders most studies in the educational research field more systematic and organized (Cohen et al. 2007, p. #). Many education researchers object to the use of quantitative methods, because the latter promote unreasonable mathematization of the nature and purpose of education (Cohen et al. 2007, p. #). Yet, in Mullens et al. (1996), the use of statistical data analysis techniques is justified by the need to systematize the knowledge of the complex learning process, the main factors affecting it, and the results to which it leads. Despite the complexity of the numerical data inferences offered by Mullens et al. (1996, p. #), the proposed statistical models allow measuring and estimating the statistical significance of the relationships between teacher effectiveness and its indicators.
Literature Review and Inferences Made from the Findings
No research is possible without reviewing earlier research questions and findings. More often than not, previous studies and findings help researchers to identify the existing gaps in scientific knowledge and develop their projects in ways that close these gaps. Mullens et al. (1996, p. #) performed a brief review of literature, and the goal of the review was to assess and critique the earlier methods of research in the field of teaching effectiveness and its implications for student outcomes. That Mullens et al. (1996, p. #) focused on the methods aspect of research is one of the major benefits of their article: the researchers were able to identify the existing methodological gaps in literature and use them as the guidance in the development of more effective research frameworks. In many instances, the design of Mullens et al. (1996, p. #) study is a response to the methodological weaknesses found in the earlier studies. However, the results of the literature review have little value for the development of teaching effectiveness indicators, since the researchers failed to define the best measure of teaching effectiveness (Mullens et al. 1996, p. #).
Mullens et al. (1996, p. #) provide reasonable and grounded inferences from the statistical data obtained in the process of collecting and analyzing student responses. The researchers found no significant relationship between teacher effectiveness and student learning of basic concepts, but concluded that teacher knowledge of mathematics was critical for student learning in advanced mathematics (Mullens et al. 1996, p. #). It is noteworthy that Mullens et al. (1996, p. #) did not forget about the importance of confounding factors (control variables); in other words, the researchers also controlled the influences that could mediate the relationship between the research variables. For Mullens et al. (1996, p. #), those confounding factors included the location of the school and its proximity to urban areas, the number of days between the pretest and posttest procedures (for students), years of teaching experience and teacher gender, as well as student gender and the presence or absence of a mathematics textbook in student homes.
Unfortunately, the researchers did not mention the basic limitations of their study, including the fact that the study took place in one geographical location and targeted only third-grade math students. Also Mullens et al. (1996) did not discuss the main ethical issues. However, in educational research or elsewhere, ethical concerns are common at all stages of the research project, from defining and conceptualizing the problem to interpreting and reporting the findings (Sharma 2002, p. #; Simons & Usher 2000, p. #). Directly related to the discussed research is the problem of using children (namely, third-grade students) as research participants, which raises the questions of anonymity and parent informed consent. Given the importance of teacher effectiveness and its implications for the quality of education and learning, the researchers should design and implement a new research project that will address these weaknesses and shed new light on the relationship between teacher effectiveness and student learning outcomes.
Conclusion
Academic research is a field of activity that encompasses numerous elements, actions and decisions. The study conducted by Mullens et al. (1996) exposes the main advantages and deficiencies inherent in most research projects. The absence of explicit theoretical frameworks, as well as failure to discuss research limitations and ethical issues, has the potential to reduce the validity and reliability of the study findings. Of particular concern is the participation of children in the discussed research project. However, the value of studying and critiquing earlier studies cannot be overstated: the results of this analysis create a foundation for the development of future research projects.
