Learner identification

This information about Beatriz is taken from an informal interview between her and I as well as from a learner profile sheet that she completed herself. (Appendix A)

General background

Beatriz is an eighteen-year-old first year university student from Madrid, Spain. Her major course of study is computer engineering. According to Beatriz, English is not only important for her because of the reading involved in her current studies, but also for future job prospects as well. She is an attentive and sociable student but because of her demanding university studies, and being her first year as well, she often comes to class tired and mentions lack of sleep as the reason. She is the youngest of two daughters and her sister also studies English.

Language learning background

Beatriz began learning English at the age of three while attending a bilingual Spanish-English school for young learners. She attended the bilingual school until the age of twelve at which point she began attending regular Spanish state schools. She has been a student at International House, Madrid for three years and passed the Cambridge First Certificate exam two years ago. During her high school years, she spent three months in Canada as an exchange student and last year went back to study English for the summer. She is currently attending class to prepare for the Cambridge Certificate in Advanced English and she has approximately ninety-percent attendance.

Learning style

Nunan (2000:168) describes 'learning style' as:

Any individual's preferred ways of going about learning. It is generally considered that one's learning style will result from personality, including psychological and cognitive make-up, socio-cultural background, and educational experience.

Observing Beatriz over the past seven months, she appears to be quite self-contained and although she works well in pairs and groups, she is just as happy to listen to the teacher, and take notes. In order to better determine her learning style I asked her to complete both a learner profile and needs analysis (Appendix A & B), and two additional questionnaires (Appendix C & D), which were designed to determine favored learning styles.

From the sources mentioned above, Beatriz stated that she preferred working in pairs to other options, learned better by 'being in physical contact with things', liked learning through problem solving, listening, reading, and taking notes. She prefers to be corrected 'later, in private' to 'immediately, in front of everyone', and feels somewhat uncomfortable having others see and be asked to comment on or correct her written work.

She was shown to be well balanced in the primary senses; Visual, Auditory or Kinesthetic which people use principally in learning.

Beatriz was also shown to be a 'concrete' learner by scoring highest for this type at 76% in a questionnaire based on learner types taken from findings reported by Willing (1988) found in Nunan (2000:170). She scored similarly in the 'authority oriented' type at 73%. This was followed by 66% in the communicative learner type. Her lowest scoring was in the 'analytical' learner type, scoring 56%.

This information is in line with my observations of Beatriz in class and I believe is also somewhat representative of her age. Being a teenager, she has still to fully develop her preferred learning styles and learners of this age group often feel self-conscious about being 'criticized' or corrected in front of others. Having said that, I find Beatriz to be an attentive, well-balanced learner.

Motivation

Williams (1999) cites the work of Gardner (1985) and defines motivation as consisting of effort, plus desire to achieve the goal of learning, plus favorable attitudes towards learning the language.

A distinction is made between integrative (or intrinsic) orientation, which occurs when the learner wishes to identify with the culture of the target language, and instrumental (extrinsic) orientation, which occurs when motivation arises from external goals, such as passing exams, financial rewards, or furthering a career. LittleJohn (2001) adds a third category: success in the task, which is a combination of satisfaction and reward.

Beatriz is motivated to a large extent by the exam at the end of June as well as continuous encouragement from her father. This points principally to instrumental or extrinsic motivation. However, Beatriz also cites being able to communicate in English as a source of accomplishment and hopes to travel again to Canada or to the U.S. and work for a year when she finishes her university studies. This suggests integrative or intrinsic motivational factors as well. In class she is visibly motivated when she does well on a particular exercise and vice-versa.

Principles of assessment

Diagnostic test-validity and reliability

A diagnostic test is defined by Huges, (1990:13) as a test used to identify students' strengths and weaknesses and is intended primarily to ascertain what further teaching is necessary.

In Language Test Construction and Evaluation (1995:171) the authors divide validity into two areas, internal and external. Internal validity relates to studies of the perceived content of the test and its perceived effect while external validity relates to studies comparing students' scores, which measures their ability gleaned from outside the test.

Internal validity is broken down into:

- Face validity: the test's surface credibility or public acceptability.
- Content validity: the representativeness or sampling adequacy of the content.
- Response validity: how individuals respond to test items.

External validity is explained through the terms:

- Concurrent validity: the comparison of the test scores with some other measure taken at roughly the same time as the test such as candidates self-assessment, or teacher or other specialist assessment.

- Predictive Validity: a concept of validity meant to predict how well someone will perform in the future, beyond the test, and which is common with many proficiency tests.

The CAE demonstrates both face and content validity. It is recognized by the majority of British Universities for English language entrance requirements. Pre-testing plays a central role in the making of the test, as it helps to ensure all versions conform to the test requirements in terms of content and level of difficulty. The CAE falls within Level four of the ALTE framework, being proof of the necessary language level needed to work at a managerial or professional level or study at university level, an example of its concurrent and predictive validity. (CAE handbook: 4-6)

'Backwash'
Prodromou (1995) refers to the 'backwash effect' as the direct or indirect effect of examinations on teaching methods. Positive backwash may occur when classroom activities clearly relate to the test and the student. Negative backwash can occur if reference to the test is abused and there is an over reliance on its extrinsically motivational effect at the expense of sound classroom procedures (which are intrinsically motivating to the student).

Reliability of a test is a test of its consistency. The circumstances in which the test is taken, the way is which it is marked and the uniformity of the assessment it makes are important elements in judging test reliability. In Communicative language teaching, Weir (1990:32) warns that:

Validation might prove to be a sterile endeavour unless care has also been taken over test reliability.

According to the CAE handbook (2000:4-8), the exam is constructed, administered and marked in such a way to ensure uniformity and reliability.

Rationale for devising Beatriz's test

I have chosen sections of CAE practice tests (book 3). I believe it adheres to the criteria mentioned above for a test to be both valid and reliable. (See appendix E2 & 5 for bits of the test)
The CAE handbook (2000) assures:

The work of the UCLES-EFL team is involved in a programme of ongoing validation, and specialist staff work and test analysis and evaluation.

Another obvious reason is that because Beatriz is taking the official exam in June, 2002, doing a mock exam now will serve as a benchmark as well as a motivational factor as she continues to study.

Content and administration of the test

The exam consists of five sections: reading, writing, listening, speaking and use of English. Each section carries equal weight in grading the final outcome. The overall grade is based on the total score gained in all five papers. A pass grade is given for a minimum of sixty-percent over all five sections

The CAE handbook (2000) breaks down the different sections and gives guidelines of expectations based on successful completion of the exam. For a full breakdown, see appendix E.

The test was administered over two one and a half-hour classes. Beatriz was allowed to do the writing over the weekend and was encouraged to write within the time limit of one hour. The speaking section was done the following week in her free time, with a classmate.

Analysis of test data

In this section I will analyze the data from Beatriz's test to find her strengths and weaknesses in terms of her knowledge and use of the language systems and skills tested.

Analysis of letter writing task (Appendix E2)

Writing skills will be analyzed using the following criteria:

- Task achievement

- Communicative competence -coherence and intelligibility of the letter

- Appropriacy - of style, organization and presentation

- Range - of lexis, grammar, and general language used

- Accuracy - of lexis, grammar, expressions, spelling, punctuation and use of cohesive devices.

Task achievement, Communicative competence and Appropriacy

Strengths

- Beatriz has achieved the task by competently writing a letter to the newspaper using the conflicting information in the texts provided, pointing out the inaccurate information and asking for a reprint. She includes a short note to Malcom but forgot to include mention of enclosing a copy of the letter. This omission would likely cost her a one-band reduction. I do think that despite the errors and inaccuracies (including the use of contractions), the letters would have a positive effect of the reader, in that they would achieve what they had set out to do.

- The letters are intelligible and organized in clear paragraphs making them easy to follow despite their flaws. The information is presented in a logical pattern and she uses both grammatical and lexical references, which hold the piece together.

- The letters are presented in letter format and she has used appropriate opening and closing expressions, and opened her first paragraph stating her reason for writing in letter one.

Weaknesses

- Beatriz has inappropriately used contractions, which are not used in formal letters. In the second letter, however, they are used well. This would keep her in a band 3.

- She has also gone over the word limit by some 60 words in the first letter, although in my opinion this does not have an overtly negative effect on the target reader.

- In the final paragraph she uses rather strong language threatening legal action. Other, more subtle wording might have been more appropriate.

Range and accuracy

Strengths

- Beatriz uses some appropriate expressions - I'm writing with regard to the incident (3), In orden to clearify the facts…, I would like to explain…, (6), You are a wonderful man (4-#2), Don't worry (2-#2), I'm sure (2-#2), Yours faithfully (23), Best wishes (6-#2), in who we all trust(5-#2)

- Although marred by poor spelling and other inaccuracies, Beatriz uses a range of structures such as: Present continuous (3), present simple (4), relative clauses (4,5), past simple for main events (7,8,9,10), past continuous for background information and incomplete actions (8), future perfect (11), present perfect (15,18), Use of modals (6,9,11,18,21), use of conditional 'if/will' structure (21) (letter 1)

- A range of cohesive devices are used such as, the fatal day-that night, the Rex Cinema-after the show, such unforgivable mistakes-referring to discrepancies between article and reality, Not only this.. but…, so…also…I am the American tourist who…In regard to the incident which…handbag-handbag, thief-thief, cut-cut, facts-facts, Malcom-Malcom, article-article,

- Beatriz also makes an attempt at using appropriate vocabulary items and some collocations which, although often inaccurately spelled or in the wrong form, shows that she is attempting more complex ways to express herself and indicates development in her language system. Examples are: accompained, fatal day, to clearify the facts, ran bravely, heroic action, badly cut, tends to misunderstand readers (mislead), aggresor, unforgivable mistakes, clearifying article, disafortunate incident, legal actions, horrendous lack of information.

Weaknesses

- Beatriz has showed poor control of spelling and word formation, which is also evident in the other sections of the diagnostic test. Writting, clearify, disafortunate, legal actions, misunderstands readers (misleads), apologise for apology, lyes for lies, inepts (a L1 transfer from 'ineptos'), have ever belive those lyes.(slip-forgetting to use past participle)

- Also problematic are omissions of the definite article in 'outside Rex Cinema' (3), ignoring the limit of subjects in 'I would like to explain what it really happened' (6,7), omission of main verb in 'In (performing) this heroic action...(10) and ignorance of transitivity in 'what really happened and what your article described had nothing to do with (each other, the facts)', and omission of subject, twice, in ' (I) am American.

- An attempt at using collocations was made, which I think is a sign of risk taking and although she does not always get them right, making mistakes is a sign of learning. Examples are: fatal day (all right in form but not appropriate in this context), clearifying article (marred by spelling) disafortunate incident (wrong prefix-unfortunate) misunderstands readers (wrong root word-lead).

- Misuse of 'regard', line 19. Either wrong word (demand, would like to see) or incomplete clause, (so I regard a…..as necessary, to be the correct thing to do)

Overall the letters have more positive elements than negative ones. I have seen Beatriz improve over the months in her writing and although there are definite areas that she needs to work on, I think that she shows positive steps towards becoming a more competent writer in English.

An analysis of the Speaking test (Appendix E5)

To analyze the speaking test the following criteria suggested in the CAE handbook (2000) will be used:

- Grammar and vocabulary
- Discourse management
- Pronunciation
- Interactive communication

Strengths and weakness based on the Diagnostic Speaking test (examples are taken from the transcript) The line number where each phrase can be found in the transcript is noted in parenthesis at the end of each entry. See appendix E5.

Grammar (accuracy and appropriacy)

Problems

- She's right now working (word order) (1)
- I'll be in the beach (wrong preposition) (5)
- It seems as (if) he has just climbed it (omission of 'If' in structure) (9-10)
- Just in the ice (wrong preposition) (18)
- I like the one…where's a little girl with her mom's shoes (omission of there) (25-26)
- Well, I didn't think about the bears, I was thinking about the one…(confusion of past simple for past continuous) (56)
- I don't think it can gets the people because it isn't cute (wrong modal verb-should be would, third person 's' following modal verb) (47)

Good usage

- She likes travelling and learning languages and she has a very good accent. (2-3)
- …Yes, since I was three (4)
- …I hope I'll be with some of my friends having fun. (5-6)
- …the man who is resting from his work because he seems exhausted and depressed. (21-22)
- They want to learn and it shows how they are growing up. (41-42)
- I would probably choose kids around ten years old because they are old enough to understand what you are telling them. (62-63)
- I would rather just have pictures about little kids. (38)

Beatriz is still forming her hypothesis of how grammar and syntax work, as she is as often accurate as inaccurate in this area. In my informal classroom observation of her, I find that she has very good days and very bad ones. This may be due to the strain of University studies and projects. This may also be a consequence of not reading sufficiently on her own outside of class.

Vocabulary (accuracy and appropriacy)

Problems

- He must be just glad of himself (wrong word-should be 'proud' inappropriate use of just)) (11)
- Shows the innocencey of the childhood (consistent with her problems with morphology) (26-27)
- The clouds and the ceiling (wrong word-should be sky) (13)
- Probably he's one of those person that …(confusion of plural/singular-should be people) (20)
- What would you do to make them know it's kid's day? (confusion with make and let) (68)
- Get friends with each other (wrong word (collocation)-should be make friends) (72)
- I would make them discuss about the different points of view (use of make when 'have' would have been more appropriate-common Spanish L1 interference following discuss with adverb about. In Spanish: Discutir sobre) (70-71)…

Good usage

- She's been living abroad (1)
- He seems exhausted and depressed (22)
- He's completely confused (23)
- I like the idea of the discussion (70)
- The man who is resting from his work (21)

Beatriz has a somewhat limited and often inaccurate lexical resource for an advanced student. She has mentioned that she feels this is her weak area in writing but because her communicative competence is good, I do not think she has realized that she also lacks in this area when speaking. This may be due to her limited opportunities to use English outside of class.

Discourse Management (range and coherence)

Poor coherence

- "So he must be just glad of himself as he could get through this because the mountain is quite difficult to climb. And he can only see the clouds and the ceiling and he might feel so free but not like the other one, he seems flying while in this one he feels glad of himself, proud and very happy."(10-15)

It was difficult to follow the monologue at this point. I think that this is mainly due to the misuse of modals, the lack of clarity in which picture she was comparing to which, and for lack of pausing.

Good range and coherence

- Well, I think it would be a good idea if it really does...do that…does something. Because if it's just the day and it doesn't get to the kids because just older people know it, I don't think it works. (95-97)

- Well, I would probably choose kids around ten years old because they are old enough to understand what you are telling them but also they are young enough to change their look at it, their view, because older one I don't think they would pay much attention. Don't you think so?

Although there are some (non-impeding) errors her meaning is clear. She gives her opinion and invites the other speaker to make a comment in the second example. In the other example, she responds coherently and uses a range of structures to get her message across.

Pronunciation (intonation and individual sounds)

Intonation

Generally Beatriz has good control over suprasegmental features such as intonation, connected speech, and stress, mostly due to her time spent in Canada, I would imagine. She does, however, show a tendency to use an inappropriately high pitch range on the prominent words of an utterance when she is uncertain. This is common in many North American teenagers' speech.
(rise) (rise) (rise)
e.g. Well, she's right now working, she's been living abroad quite, I think, many
(rise) (rise)
years I'm not sure about it.

This does occur now and again in her speech when she is not interacting with the other student but although somewhat distracting, it does not interfere with coherence or intelligibility.

Individual sounds

Problem sounds

- She's working- /s/'s working (sound like 'See's') (1)*
- It -/i:/t (sounds like 'eat') (2)*
- The beach-/d/ (sounds like dee) (5)*
- Could feel-/l/ (she is pronouncing the silent /l/ sound.) (8)
- Climb-/i/ (She confuses the short /i/ sound for the longer /ai/ sound) (10)*
- Sports-/es/ (She places a vowel sound in front of the /s/ sound. An L1 interference error. (20)*
- Doubt-/b/ (she pronounces the silent letter 'b') (21)*
- Depressed -/t/ (She fails to release the final /t/ sound. Sounds like depress) (22)
- Confused-/d/ (She fails to release the final /d/ sound. Sounds like confuse) (23)
- Shoes -/z/ (She pronounces /s/ instead of /z/) (26)*
- Book-/u:/ (She pronounces the long /u:/ sound instead of the short /u/ sound (40)*

*These are all individual sounds, which often prove difficult for learners whose first language is Spanish. However, they rarely impede understanding.

Interactive communication (turn-taking and responding)

Good example of responding (grammatically correct)

- (Claudia) -they are start just looking at pictures and learning the first words. (Beatriz) -Yes. And you can see how they are trying to see the book with the pictures so they are…they want to learn and it shows how they are growing up) (58-60)

Basic example of responding

- (Claudia)-They are children about four years old do you think? (Beatriz)-Yeah, probably. because I don't think they can read.(55-57)

Asking for an opinion

- Well, what do you think? (52)
- But what do you think about this one? They are animals! (47)

Beatriz communicated and interacted quite well showing an awareness of polite turn taking and handled the development of the discussion, including frequent topic shifts, with relative ease. Although we have seen that she can be quite inaccurate and use relatively simple vocabulary, her strength is being able to handle discourse management and communicate competently. The structural and lexical errors I see as being largely developmental, as she is able at times to correct herself and realize she has made a slip. Her pronunciation problems, however, will need to be addressed as I do not think they will disappear on their own.

Listening

Beatriz did quite well on this section of the exam, scoring 74 percent. The most difficult was section A , which is based on note taking and sentence completion from a monologue of about 2 minutes. She later commented that it was difficult to follow because the subject matter (a specialist in cookery giving a talk about bread) was quite foreign to her. I expected Beatriz to do well in this part of the exam as she has a very good ear and is very communicative in both speaking and listening.

Reading

Beatriz had some problems with the reading section, especially in part two, which tests understanding of how texts are structured and the ability to predict text development. She later commented that she found that part very challenging. The poor showing on the reading section is perhaps due, in part to the fact that she reads relatively little in English outside of the classroom. I have since convinced her to begin reading more, for pleasure, as that will certainly help her in this part of the exam. Her score for this section was 59 percent.

Use of English

Poor usage

Prepositions

- He gradually discovered their individual effects towards his food and well being….
- Should be: on/upon (part 2 no. 24)

Verb forms

- Because the horrid woman of the pictures she thinks I took them from books.
- Should be: likes/liked/admires/admired (part 4 no. 53)

Lexis

- Property crimes develop on the easy opportunity.
- Should be: thrive (part 1 no. 4)

Word building

- This practical, straight forward and unreliable guide,…. (part 4 no.57)

- …especially designed for those who are not knowledged about computers… (part 4 no.58)

- …an invaluable publishment. (Part 4 no. 63)

- The first to express their dissaprovement… (Part 4 no. 69)

Good usage

Reported speech (no backshift)

- She says it is rather disorganized. (Part 4 no.49)

Auxilaries

- But anything that has been…will never be…(Part 2 no. 28)

Conditionals

- It would be a very different world without them. (Part 2 no. 18)

Prepositions

- Based on the trial and error of early man (Part 2 no. 22)
- Our present day search for all things natural (Part 2 no.30)

Language learning objectives

Areas to focus on in and out of the classroom- Activities and resources
These activities and resources are cross-referenced with and matched to the appropriate learner types found on page one and detailed in appendix D. The number in parenthesis matches the learner type.

Conclusion

From the test results and my observations in class, Beatriz appears to be a fair advanced student whose main areas to work on are vocabulary development, control of register, spelling, and awareness of text structure and cohesive devices. Her pronunciation needs to improve although her accent does not impede understanding. If she continues to improve in the areas mentioned, she should be able to achieve a pass grade in the exam at the end of her course.

Bibliography

Nunan, D. (2000) Language Teaching Methodology. Pearson Education Limited, Edinburgh.

Willing, K. (1988) Learning Strategies in Adult Migrant Education. Adelaide: NCRC. Found in Nunan, D. 2000. Language Teaching Methodology. Pearson Ed. Ltd., Edinburh.

Williams, M. (1999) Motivation in Language Learning. English Teaching Professional issue 13, October1999

Gardner, R.C. (1985) Social Psychology and Language Learning: the role of attitudes and motivation. Edward Arnold 19885. Found in Williams, M. Motivation in Language Learning. English Teaching Professional issue 13, October 1999.

Littlejohn, A. (2001) Motivation: Where does it come from? Where does it go?. English Teaching Professional issue 19 April 2001.

Huges, A. (1990) Testing for Language Teachers. CUP, Great Britian.

Alderson, C.J., Clapham, C. & Wall, D. (1995) Language Test Construction and Evaluation. CUP, Great Britian

Prodromou, L. (1995) The backwash effect: from testing to teaching. ELT Journal Volume 49/1 Junuary 1995. OUP

Weir, C.J. (1990) Communicative Language Testing. Prentice Hall International (UK) Ltd.

Baxter, A. (1997) Evaluating your Students. Richmond Publishing, London.

Swan, M. & Smith, B. (1987) Learner English: A teacher's guide to interference and other problems. CUP, Cambridge

Brazil, D. (1995) A Grammar of Speech. OUP, Oxford

Hedge, T. (1988) Writing. OUP, Oxford

Wright, J. (1998) Dictionaries. OUP, Oxford

Lewis, M.(1993) The Lexical Approach: The state of ELT and a way forward. LTP, Hove.

References

Certificate in Advanced English Handbook (2000) UCLES, Cambridge

CAE examination report December 1997/ June 1998. UCLES, Cambridge

Biodata

Appendix A
Appendix B
Appendix C
Appendix D
Appendix E
Appendix E2
Appendix E5

Appendix A

Appendix B

Appendix C
(see J. Wingate's 'learning preferences' in LTP (10/2000) issue 17)

Appendix D

Appendix E

Appendix E2

Appendix E5

To the original article

To the article index