‘Me Tarzan, you Jane’. Is this me?
By citing the cases of Alberto and Wes in my last post, I may have given the impression that my Spanish is irremediably bad: that it has either pidginized to a similar degree as had Alberto’s English, or is only minimally grammatically inflected, like Wes’s.
It’s probably not as bad as that. Before I report two objective measures of my Spanish, though, here is how I rate it myself, using criteria borrowed from the literature on cognitive psycholinguistics (e.g Skehan 1998), plus one or two others that I think are important:
- Accuracy (i.e. the extent to which my Spanish is target-like): 4/10: it’s only really accurate under conditions of zero time pressure, e.g. writing, or in non-vocalized retrospection immediately after the event, of the type ‘I said hizo. I should’ve said hice‘. Specific problems include verb inflections, gender, pronouns, prepositions; pronunciation is heavily L1 accented; common words are frequently confused;
- Fluency (i.e. the capacity to cope with production and reception in real time): 5/10 (although this varies widely depending on context and interlocutors); word-searching often halts speech; correct stress placement in polysyllabic words is also a problem in terms of fluid articulation; communicative effectiveness is often impeded by problems of comprehension that result from poor phonemic coding ability (also known as ‘a bad ear’).
- Complexity (i.e. ‘the size, elaborateness, richness, and diversity of the learner’s linguistic L2 system’ ( Housen & Kuiken 2009: 464)): 4/10: vocabulary range is serviceable for day-to-day production needs, and quite good for reading relatively difficult texts; however, key areas of grammar, such as pronoun choice, reflexive verbs, past tenses, ser and estar, por and para, etc., still operate in free variation – i.e. the choice of one form over the other is random and unsystematic. There is insufficient grammatical knowledge to express complex ideas involving modality and hypothesis, or to flag the logical relationship between utterances, or to vary and embellish a simple narrative.
- Idiomaticity: (command of formulaic language, collocation, etc) 3/10: poor command of formulaic language means I would never sound native-like, even ignoring other variables; collocations etc are still influenced by L1;
- Appropriacy: (also known as sociolinguistic competence and includes the capacity to function in different registers): 5/10 – limited to mostly informal and one-to-one situations, due in part to difficulties choosing between tu and usted, and in using second person plural forms comfortably.
- Strategic competence: (capacity to compensate for a limited linguistic competence, e.g. to paraphrase, to repair breakdowns, etc): 4/10: I tend either to avoid situations where this might be put to the test, or to operate on a ‘wait and see’ principle, hoping that communication breakdowns will be resolved further down the track.
For a more objective measure of my spoken Spanish, I turned to Pearson’s ‘Versant Test’, which is conducted over the phone and scored both automatically and instantly. (I was intrigued by a demonstration of the English version of this test when I saw it at a conference a few years ago). In their own words:
The Versant™ Test of Spanish, created by leading language testing experts using advanced speech processing technology, evaluates a non-native Spanish speaker’s ability to understand and to communicate appropriately in Spanish on everyday topics. Versant automated tests allow language teachers, program coordinators, and administrators to quickly, objectively, and accurately assess students’ levels of spoken Spanish skills.
And they add:
Making use of advanced speech recognition techniques, the Versant Test includes several different sections: Reading, Repeat, Opposites, Short Answer Questions, Sentence Builds, Story Retellings, and Open Questions.
The test costs $25, takes about as many minutes, and is easy to do. Moreover, it’s an interesting experience, in that, in some of its sections at least, it requires a degree of spontaneity, combined with the pressure of working against the clock, that cleverly simulates the demands of real-time speech production.
How did I do?
Here are the results (80 is the top score in each band):
Part 1: Click to enlarge
Part 2: Click to enlarge
What’s curious is how polarized my accuracy and fluency results are – over-generous in terms of grammatical accuracy, but a little mean, I thought, about my response to the story-telling and open questions tasks. It also rates me more highly on the use of formulaic language than I do. (How does it know, you may well be asking!) Anyway, I plan to take the test again in a few months’ time, to see if it can detect any improvement. For those of an empirical bent, this will constitute the pre- and post-test elements of the study.
Finally, having enrolled in a language school (of which more in the next post), I was level-tested using a customized online instrument that comprised a self-adjusting multiple-choice grammar test, followed by four short texts in C-test format. (A C-test is a kind of cloze test where the second half of every second word is blanked out and requires completion).
Here you can see a screencast of me doing the grammar section (forgive the typically Spanish background ‘jaleo’!) Those of you who speak Spanish might like to make a preliminary assessment of my level.
I was disappointed, though, that there was no test of my oral Spanish – surely easy to set up nowadays, using Skype. When I queried this, however, I was assured that, on Day 1, my speaking would be evaluated by means of an interview.
How did I do? On the basis of the online test, I was provisionally placed in an advanced level (C1 on the Common European Framework), the actual placement being contingent on the speaking test.
It seems, then, that both external assessments rated my Spanish more highly than I do, especially with regard to grammatical accuracy. What will this mean in terms of how I respond to formal instruction, I wonder? Specifically, will I be over- or under-challenged? Will the classroom experience penetrate the areas of my Spanish that seem to have stabilized? In what areas (accuracy, fluency etc), will I show improvement – if any?
¡Continuará!
References:
Housen, A., & Kuiken, F. (2009) ‘Complexity, accuracy, and fluency in second language acquisition’, Applied Linguistics, 30/4.
Skehan, P. (1998) A Cognitive Approach to Language Learning, Oxford: Oxford University Press.
The (De-)Fossilization Diaries 
September 8th, 2013 at 8:43 am
Hello Scott,
I saw in your first post that you said you were going to use a wide range of means to achieve your goal (but comments weren’t possible on that one).
I would like to recommend Duolingo. Von Ahn invented CAPTCHA and RE-CAPTCHA and the story behind duolingo is fascinating. It is the type of learning programme that you would not normally recommend as there’s no context and it’s a bit dogmatic in its insistence on some answers being correct and others definitely wrong. However, I think it’s a fascinating project and knowing that I’m helping Von Ahn translate the internet increases my motivation. On every disputed point, there’s also a forum discussion which is refining the system on a daily basis. There’s already a million users. I would love to see if you think it can help you.
Best wishes,
Stuart
September 8th, 2013 at 12:58 pm
Hi Stuart… thanks for mentioning Duolingo. I do have it bookmarked, but I’d sort of forgotten about it. (Here’s the link for those of you who don’t know it: http://www.duolingo.com/ It’s worth watching the little promo video).
I’m definitely going to incorporate this into my non-class-based strategies – could be a good way to use down time when I’m on the road. Thanks for the reminder!
September 8th, 2013 at 10:26 am
Another valuable post, Scott. I most appreciate the question you ask yourself at the end, in the last paragraph: What will this mean in terms of how I respond to formal instruction? My spontaneous answer would be: as the results of all various tests indicate, your Spanish grammar (e.g. sentence making) is not bad, traditional formal instruction is not exactly what you need, and I suppose you will be under-challenged. This happened to me many times when learning German; I felt I could barely express myself but the teachers were surprised how good my grammatical competence was. And as the courses always focused mainly on grammar, I felt I wasn’t challenged enough. I would have appreciated an intensive, challenging speaking course with the focus on formulaic language.
What I also like about this post is that you’ve just demonstrated that if you analyze your performance the way you did, the chances that you will find a tailor-made language course will increase. However, I would also be cautious; the results are only mathematical calculations but languages behave like living things – they change, grow, and deteriorate. So any kind of categorizing may get us into trouble. And as you write, the way you feel about your performance does not always correspond with what the tests show…
Hana
September 8th, 2013 at 1:03 pm
Hi Hana: thanks for your comment. ‘Tu has dado en el blanco’ (an expression I learned in class!) – you’ve hit the nail on the head! Viz: ” I would have appreciated an intensive, challenging speaking course with the focus on formulaic language”. This sums up in one neat sentence what I yearn for.
Let’s see if this is what I get!!
September 8th, 2013 at 11:41 am
I’ll be following this blog with great interest, probably to vicariously avoid doing anything with my Polish, which after 19 years in Poland is what I half-jokingly/half-shamefully place at a level of ‘rudimentary but fluent gibberish’. I have excused myself by thinking of it all as a grand experiment to discover what can be attained sheerly through osmosis and being ‘pushed’ to output here and there, and thereby gaining an insight into the value of teaching a language in the classroom by ‘noticing the gap’ between naturalistic learning and formal instruction.
Similarly to what you have described, I am now running out of excuses to overhaul the system since the results of the experiment have long ago reached its findings: I’ve reached a level where I have very adept usage of probably a 1000 chunks and patterns and probably 3000 words. The chunks I pronounce so well that often a listener can be initially fooled into thinking my Polish is at a much higher level that it really is. Being a sharp-eared chunk hunter, I am quick to notice the surrender value of a particular chunk in conversation and make a mental note of the context and effects on listeners – I especially like the short chunk that seems to pack a lot of communicative power into mercifully few syllable, and am wary of any chunks that defy quick uptake in pronunciation. I wonder if you have done the same.
Polish is notorious for its Latin-like case system for nouns, and one of my excuses for not taking up Polish lessons was that courses for foreigners (from all reports) were often grammar-heavy from the outset and, to my mind, overemphasized endings relentlessly, course books typically replete with tables and rules (though recent publications are more ‘communicative’ and ease up on this). Is this the case in Spain with conjugations etc. ?
September 8th, 2013 at 3:51 pm
Hi Declan, nice to have you back again. I’m intrigued by your chunk-oriented acquisition, as you seem to have acquired a larger ‘chunk lexicon’ in Polish than I did in Spanish. Is this an aptitude, do you think? Ann Peters, one of the first to identify formulaic learning in FLA (see The Units of Language Acquisition, CUP, 1984) claimed that children learning their first language divide into two groups: the holistic learners, who initially assemble a vocabulary of chunks that are only later broken down to ‘release’ their grammar, and the more analytic ones, who seem to build utterances piece by piece, mapping them on to structures that are (somehow?) already in place. Whether this distinction persists into second language learning is a moot point, but maybe you belong to the former type.
Either way, as as Hana suggested above, a pedagogical bias towards grammatical morphology (all those endings) is not what I think I need. Nor do I need to (just) memorize 1000 formulae. I need some way of harmonizing these two strands of language use so as to achieve maximum communicative effectiveness, and in real time.
September 8th, 2013 at 4:22 pm
I think the division between holistic and analytic is reflected in my native speaker circles; a mate of mine, let’s call him Walt, has a real focus on endings and getting things right (he is definitely attempting to acculturate as well – not sure to what extent that’s connected). He is now so good that he even corrects Poles (to their chagrin); however, he has lamented more than once that he lacks some of the instant and succinct ripostes I seem to have ready-made when it comes to conversation ( I have quite a few years on him in terms of period of gathering these). I have another friend, Morten, (not trying to acculturate) who was really attentive to endings and was able to correct me on a thing or two, but also was really lacking in phrases, ones that I had gathered and thus came easily to me.
As to your musing on a possible ‘chunk aptitude’, I think it is less this and more of a strategic skill – I have found that overhearing/eavesdropping on conversations and exchanges have been my primary source of these, and I seem to have a stamp collector’s instincts for vacuuming these up – because I realise how high value and effort-saving they are – and thus am highly motivated to gather them at every opportunity. With chunks, a little goes a long way.
In any case, (pun now intended) my uptake of chunks is so good that, because the chunk contains (of course) the correct ending, it makes me sound accurate 50-80% of the time – but it is sadly (?) not rule-based, rather purely associative. Other times when accurate with endings, it seems to be based on some implicit probabilistic process, that again, I seem to have no control over. Such is my plight 🙂
September 8th, 2013 at 12:41 pm
Having now read previous posts and comments, I see some of the issues of motivation, embarrassment and traditional teaching methods as factors have already been covered (though it would interesting to know what sort of chunks you have appropriated into your mental lexicon).
So to add something fresh, I offer another (perhaps odd) factor that inhibited me wanting to learn the L1 of the country very well: the observation that, among (some) colleagues who became very fluent in Polish and very immersed in Polish in social circles, that they seemed to be starting to suffer from language attrition – more specfically, they would have momentary lapses of not remembering certain words in English during conversation; TEFL teachers’ English is, we know, susceptible to such ‘downsizing’ due to constant language grading in the classroom and talking in teacherese with students. But this was something more; it was, I felt, language loss through sheer lack of use or recent activation; these teachers would laugh it off but I took fright at this a little and vowed never to get so lost in another language as to lose my own (but, of course, perhaps this was yet another convenient excuse to do nothing with my Polish).
September 8th, 2013 at 3:56 pm
Hi Declan (again). Interesting comment on attrition, which prompts me to make a mental note to look into the literature on this subject, since attrition (i.e. language loss) may share some psycholinguistic territory with fossilization. I can’t say that I, personally, resisted becoming a native-like Spanish speaker through fear of attrition in English. It’s more: I could never be as good in Spanish as I am in English, so why bother?
September 8th, 2013 at 4:06 pm
Yes, the latter is also a factor. But maybe this is ‘the perfect becoming the enemy of the good’ – perhaps it’s a matter of setting the bar a bit lower in the interests of more realistic goal-setting, another plank in the motivation platform.
September 8th, 2013 at 3:14 pm
Para mí, fue interesante ver en el screencast cómo respondías en la parte de gramática de la prueba. Ese proceso consciente de hacer la selección de la alternativa apropiada (falsos amigos, singular-plural, subjuntivo). En mi opinión, no todos los aprendices hacen esto y menos si inician el aprendizaje de una lengua extranjera por primera vez – que no es tu caso. Lo que, tal vez, ayude a explicar la ubicación en el nivel avanzado en esta prueba. Por otra parte, parece ser común en algunos aprendices el hecho de evaluar su competencia en el idioma muy por debajo de lo que puede indicar una prueba e inclusive de la misma evaluación de nativos.
September 8th, 2013 at 4:07 pm
Gracias, Mike, por tu comentario, que me hace preguntar si yo hubiera hecho las mismas eleciones en la prueba si no las hubiera vocalizado. Seguro que Vygotsky hubiera dicho que el acto de vocalización ayudará el acto de pensamiento. ¿Quizás deberíamos animar a nuestros alumnos a hacer algo parecido?
September 9th, 2013 at 7:46 pm
Well done Scott
A mi me hubiera gustado tener tu paciencia para estudiar tan profundamente Saludos Que te vaya todo muy bien!
September 12th, 2013 at 7:44 pm
Hi Scott,
Thanks for this. A great story developing and an opportunity to (re)visit the theory and match it to real practical issues.
I was wondering if you’ve done a needs analysis on yourself. You’ve said you’re not happy with your current command but what will your new Spanish look like? What will you be doing better and more of? What outcomes do you expect from the learning strategies you’ve chosen? What are your linguistic objectives and how will you know you’ve achieved them?
Thanks.
September 12th, 2013 at 9:37 pm
That’s a very good point, Richard. And this relates in part to my motivation, that is to say, what is the ‘vision’ I have of myself as a Spanish-speaker that drives me … or that his motivated me to take this step? This is such a key issue that I plan to address it very soon in a separate post. Can you wait? 😉
September 14th, 2013 at 1:57 am
This is really fascinating, Scott. You report with sincerity and your usual critical acumen on this splendid experiment. What particularly strikes a chord in this latest episode is the difference between your own assessment of your competencies and those of your assessors.
First, is this not very common- I mean that learners often have a poorer opinion of their level than their teachers? Who’s right? Are the teachers/assessors just trying to drum up trade or is their some real attempt to genuinely assess? Are you typical in thinking you’re hopeless when you’re not? Obviously, you’re better informed than most learners who take these tests: how does that affect the process?
Second, the assessment process itself. The tests you did are, IMHO, not very convincing. And this, of course, leads us into a discussion of testing in general and diagnostic testing in particular. In the case of writing, just for example, I’m sure you’re aware of all the relatively new stuff on dynamic rather than static assessment. In their extensive literature review of research on dynamic assessment (DA), Grigorenko & Sternberg (1998, p. 76) list three areas in which DA differs from static assessment:
1. the focus on psychological processes rather than on the evidence of already developed skills;
2. the integration of testing and instruction rather than the separation of the two;
3. and the centrality of invidualised interaction between tester and testee rather than a largely standardised interaction between the two.
The first point might be reexpressed as a concern of what higher level constructs should be addressed by assessment. DA sees the subject of its analysis not as the finished products, or past achievements, of a testee’s experience but as the process that a testee uses during the assessment. As Grigorenko & Sternberg state, “what is tested is not just previously acquired knowledge but the capacity to master, apply, and reapply knowledge taught in the dynamic testing situation” (1998, p. 76).
Gingorenko and Sternberg’s second point is potentially the most radical in that it goes against the traditionally accepted idea that assessment and instruction should remain separate with every effort made to stop the possibility of testees being exposed to anything during the test that might result in learning. Whereas both summative and traditional formative SA are largely concerned with assessing the amount of learning that has occurred as a result of previous study, DA is concerned with asessing the amount of development that occurs during a collaborative study activiy. That is, in DA it is assumed that the assessment process will result in a change in the testee’s development state. DA assumes this is not an incidental or unwanted side-effect of the assessment process but that assessing the extent that a testee’s current state can be modified is the only way to see the “complete picture of the individual’s capabilities” (Poehner & Lantolf, 2005, p. 234).
Grigorenko and Sternberg’s third point may require a rethinking not only of the administration of tests but also of the construct being tested. The change of construct might be necessary since it could be argued that what is being tested, in this study for example, is not the ability to write but the ability to develop and adapt and alter ones writing (Poehner, 2008).
There are obvious implications for areas other than writing.
Grigorenko, E. L. & Sternberg, R. J., 1998. Dynamic Testing. Psychological Testing, 124(1), pp. 75-111.
Poehner, M. E., 2008. Dynamic assessment. A Vygotskian approach to understanding and promoting L2 development. New York: Springer.
Poehner, M. E. & Lantolf, J. P., 2005. Dynamic assessment in the language classroom. Language Teaching Research, 9(3), pp. 233-265.
September 14th, 2013 at 2:07 am
Sorry, first for all the typos (“their” where “there” was meant, for example) and second for not making it clear that I’ve used an unpublished paper by Niel MeGregor for some of this.
September 14th, 2013 at 9:30 am
Thanks, Geoff. I agree that the test instruments I was subject to were wanting in many ways – the institution’s level test, in particular. Not only was it targeting static knowledge, but also inert knowledge, in the sense that – in the absence of any production task – it tested only grammaticality judgements and (in the C-test) a rather random set of textual features, including vocabulary recognition, and some knowledge of collocations, morphology, and syntax. No attempt was made to assess my ability to marshall these resources for a communicative purpose. (The subsequent spoken interview seemed to be more a way of rubber-stamping the results of the online test).
The Versant test, on the other hand, did seem to place a wider range of demands on my Spanish, and there was even a dynamic element, in the sense that as I did some of the tasks (such as the spontaneous speaking) I adjusted my strategies in subsequent iterations of the task (e.g. speaking faster, and eliminating extraneous detail in order to finish within the time limit). Obviously, the fact that the test was scored using purely computational means, without any human intervention, raises all sorts of questions about its validity. But, having been a user of voice recognition software for some time now, I am less sceptical now than I might once have been, as to what can be achieved through technology alone (the advantage of the technology being its practicability and relative low cost). Your comment makes me wonder if the same tools could be enlisted within a more dynamic testing framework – where, for example, the principle of reviewing and repeating a task was a key component of the test design.