Critiquing Qualitative Research Articles
by Mark Firth
Review of literature
Is the review comprehensive and up to date? The literature that is cited is recent but tends to lack depth. Much of the review consists of discussion of the tool that will be used for data analysis, which results in lengthy explanation of its elements. While this can be seen as being necessary, little else is discussed. Maybe little research exists within the specific area of evaluating listening through a cognitive framework, however more relevant information concerning cognitive processes e.g. other possible models could have deepened the discussion. By the nature of the study the researcher knows what she will be looking for; through the use of an ‘evaluative’ study, the narrative accounts (in this case students’ own reflections) are intended to be explained and judged as described in Meriam (1998). More description of research that suggests this approach could have been advantageous.
Is there an emphasis on primary resources? Put simply, we think so. Most of the citations appear to be based on empirical studies but few details are given. It must also be mentioned however that this study (as stated) was carried out in contribution to a larger body of research. Cohen et al. (2000:p.183) mentions that case studies are often utilized to provide finer details to complement large-scale investigations. While this body of research could not be termed as being a case study per se, by studying the listening difficulties a small group of learners face, what is deemed common for this case may well also be considered common for others as described by Punch (ibid:pp152-3). Little else resembles a case study-like investigation except to say that perhaps the elements that make up a psychological case study may also be relevant. Discussion on psychological research methodology is beyond the abilities of this author and so except for the above-mentioned similarities, the article by Goh is generally considered to be ethnographical research. Discussion of which is described later.
This may also explain the reason as to why the article doesn’t contain many details on how the research question was operationalized supported by the use of a large number of primary resources. As it reads by itself the article tends to lack rich description in the literature review.
Is there a critical review or a review of findings? Whilst the review argues strongly for the strategic (methodological) approaches it uses, no disadvantages or limitations are described. For example, Anderson’s cognitive framework of language comprehension is lauded and backed up by other researchers’ testimonies but no weaknesses or criticisms for conceptualizing an invisible process in this way are presented.
Is the review well organized? As mentioned above the introduction should have been kept separate. The layout does however clearly demonstrate the way in which the researcher intends to go about the study – the disadvantage of this being the somewhat predictability of results and certainly the approach to be taken for data analysis (though this could be also seen as a strength of the study).
Does the review clearly relate previous studies to the current research problem? Looking at research that has gone into listener difficulties in general, the review goes in to great detail about the stages of listening comprehension according to Anderson’s model. It seems unfortunate that no other investigations are referred to regarding listening processes within the field of psychology or language acquisition. In fact a large body of research exists: Dirven & Oakeshott-Taylor (1984), (1985) have both researched and reviewed the psychological processes involved in various levels of listening; and Nagle & Sanders (1986) present an information processing model of listening comprehension of their own. If the author could have referred to such models and discussions and pointed out where Anderson’s construct is more advantageous, then a clearer link with previous research could have been established.
Does the review help establish the importance of the research? Whilst the conceptual framework for the study is clear, the literature review does not strongly argue for the need for such a study. This is except to say the author notes the extensive preceding research that has focussed on other types of factors affecting listener difficulties. If a lack of research in an area does exist, this does not necessarily automatically validate the need for research and so critical discussion about related material could be seen to be necessary.
The author does refer to her own previous research, which we assume has led her to this investigation. Other than this, the significance of the research is left up to the reader to decide. When a research article refers to other documentation of the same author this is generally a sign of weakness as it suggests the researcher has a good idea of what she intends to find in order to support some kind of argument, McMillan & Wergin (1998:p146). Again to be fair however, we must bear in mind the study is part of Goh’s own larger research.
To sum up, the introduction and review of literature thoroughly introduce the approach to be taken and elements comprising the design (use of learners’ self reports and analysis via Anderson’s model), yet we still don’t know how it is the author came to the decision for this particular design.
To page 3 of 6
To the article index