Practical measures of proficiency

[freknu]

Ah, standard deviations, bell-curve, all that. Why would you assume that language learning is in a normal distribution? (It might be, but I'm not sure that's an obvious statistical assumption to make.)

Not necessarily language learning, but perhaps taking a sample of native speakers and statistically comparing your "proficiency" to that.

It would possibly require some sort of standardised test to query specific linguistical features and understanding of your language. It wouldn't surprise me if that was close to a bell curve.

I don't really know if comparing proficiency to some superset of all native features is useful. It's how you compare to a native speaker.

[Daniel]

Native speakers, yes, I'd expect a normal distribution.
Learners, I'm not sure. Mostly I bet it wouldn't be symmetrical: you'd get a relatively steep increase until the midpoint, then a wide/shallow extension off toward infinity beyond it. The midpoint would be determined by when people on average stop studying/learning, and I don't know where that would necessarily be. Maybe on average at something like the two-years-of-class level?

[Corybobory]

The Japanaese Language Proficiency Test is a standard test, that in conversation people can say to give a rough idea of fluency.  The problem is, like the problems discussed above, it doesn't give a resolution into whether you might be a strong or weak reader, or better with informal language or academic language.

That's a good point about the bell curve and language learners - there is always going to be more early level learners than intermediates or advanceds. It would be a changing 'language-scape' too, because of trends in langauge learning and language popularity I imagine.

I wonder if there can be a universal proficiency index used for all languages, or if they have to be language individual as they otherwise won't work well.

I was imagining something like the Myers Briggs personality test with the 4 letters that could indicate proficiency broken down into reading writing speaking listening, though that leaves out register.  Hmm...

[IronMike]

I'd like to add to this discussion as one (hopefully not the only one) who has attended their nation's military language school (for U.S.: Defense Language Institute, or DLI), who was required to take language proficiency exams (Defense Language Proficiency Test) annually from 1986 to just a couple years ago. A couple of notes, none of which are based on science:

1) I like to talk about passive and active proficiency/skills. Under passive I put reading and listening (hold on, I'll explain) and under active, speaking and writing. Now I say passive purely based on my unscientific experience on how I study the language, take proficiency tests in the language, or work in the language. Reading and listening, I'm taking L2 input into my brain, which translates it into my L1 for me to understand. Then, if I want to respond to that input, I switch to active:  My brain takes my L1 response, and has to actively translate that into the L2 so that I can respond to my interlocutor.

2) This was the way I explained it to the military linguists I worked with when I would catch them always studying vocabulary one way:  L2 to L1.  I would tell them to flip their flashcards over and study L1 to L2.  Much harder.  But those who made a concerted effort at studying that way would do better on the speaking proficiency test.

3) Now I know the above can be language specific. In my experience, which was Slavic-heavy, most of my colleagues were very good at reading and listening, but only so-so at speaking. This was mostly due to the job not requiring speaking (and their study habits, as I describe above). But I had friends who were Chinese linguists, who would always do really well on the speaking and listening test and not so well on the reading. But I contend that if they knew the character, they'd be able to understand it passively, but the action of having to study so many characters and learning/memorizing them is an active skill.

4) With regards to practical measures of proficiency, personally I use the term "Berlitz level." If I can order food (beer!), get a hotel room, follow and ask for directions, buy a ticket to something, I say that I'm at "Berlitz level in X-language."  In ACTFL terms, I'd say that's about a 1 to a 1+.  And for most languages, I'm happy with that.

[sieledorothy]

I think it's all in the ability to converse, to sustain a conversation on and on. If it's with a native speaker of that language, the better.

[Guijarro]

As I have never thought seriously about the ways to measure proficiency, let me try a lateral thought on it.

Could PROFICIENCY be related to RELEVANCE in some way or other?

I mean,

(1) When I am with my Italian friends having a wonderful time out in, say, Ferrara, I do think that I am proficient in Italian. But when I am trying to read Dante Alligheri in that language and make out something about his ideas, I confess that I am less so.

(2) Similarly, when I write letters to my German friends, I tend to be fluent. But i don't consider myself fluent enough to write a paper on Linguistics in that language.

So, as relevance, fluency could probably be considered a function of the effects I try to achieve against the effort it would cost me in each case.

Does that make sense?

[Daniel]

I think that's exactly it. "Fluent" of course is a label we apply when we find someone to be in general, across various domains fluent (fluid) in speaking a language. Being fluent in saying "hello" is not enough (or I'd be fluent in maybe 20 languages).

However, the arbitrary tasks you list may not be very reliable across learners or languages. Is the difference between ordering a cup of coffee and writing a linguistics article always constant?

But certainly, I would agree that proficiency relates to ability in accomplishing certain tasks, or just more generally ability to speak the language functionally in certain circumstances.

At the same time, there may be other measures of proficiency such as pronunciation metrics, which may not necessarily impede communication but still mark someone as non-native.

[freknu]

If the documentary I recently posted bears any weight then I would dare say that gossip would be a true test of fluency

PS. It's towards the end, at 43:50 minutes.