Author Topic: Practical measures of proficiency  (Read 4538 times)

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Practical measures of proficiency
« on: December 18, 2013, 03:48:16 AM »
What are your favorite ways to measure proficiency that actually feel effective?
I often wonder whether, for example, I'm "good at" Spanish, and I don't feel like I have a good way to answer that. As a linguist I'm probably harder on myself than most, but I'm also realistic: I'm certainly no native speaker of Spanish and I do have trouble understanding sometimes (perhaps often).

Personally I don't see the value in labels like "intermediate" or "advanced" or even "fluent". (Maybe "beginner" is an exception.)

And official scales like "B2" don't seem to mean much unless you can compare yourself to others on exactly the same scale.


So what's a good solution?

Personally I like thinking about practical aspects:
--Can you give/get directions to the airport? [Also known as the "train station test"]
--Can you read a newspaper?

Of course different measures like that might have very different results (maybe you can read a newspaper but not ask for help, or the opposite), but that seems natural to me.

Based on these tests, I'd say:
I am comfortable living in Spanish.
I can read academic research in most (Germanic, Romance) European languages, at least with a dictionary.


Does that seem reasonable? Any better suggestions?
What's your favorite way of measuring proficiency?
Welcome to Linguist Forum! If you have any questions, please ask.

Offline Corybobory

  • Global Moderator
  • Linguist
  • *****
  • Posts: 138
  • Country: gb
    • English
    • Coryographies: Handmade Creations by Cory
Re: Practical measures of proficiency
« Reply #1 on: December 18, 2013, 04:10:01 AM »
I think those are good measurements - for myself, I gauge in terms of being 'conversational' or not, if I can carry on a conversation with a stranger, and  if I can read academic texts with a dictionary or not.

It's hard to define, since language learning can improve in different areas at different rates - as in I'm conversational in Japanese and French (and Spanish at a push), but I struggle to read Japanese, and reading French or Spanish for academic works is fine (with a dictionary!).  I don't think 10 Japanese dictionaries could get me through an academic paper sadly...
BA Linguistics, MSt Palaeoanthropology and Palaeolithic Archaeology, current PhD student (Archaeology, 1st year)

Blog: http://www.palaeolinguist.blogspot.com
My handmade book jewellery: http://www.coryographies.etsy.com

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Re: Practical measures of proficiency
« Reply #2 on: December 18, 2013, 10:58:52 AM »
I agree about "conversational"-- I'd describe myself as strongly conversational in Spanish. But sometimes I wonder if others might expect more of my Spanish than is realistic, because "conversational" is almost a technical term like "fluent" with no real definition.

Quote
It's hard to define, since language learning can improve in different areas at different rates - as in I'm conversational in Japanese and French (and Spanish at a push), but I struggle to read Japanese, and reading French or Spanish for academic works is fine (with a dictionary!).
Absolutely. But it might be a good thing that a proficiency test/description would take that into account.

Quote
I don't think 10 Japanese dictionaries could get me through an academic paper sadly...
I know what you mean. There's also a degree to which being "advanced" in Japanese and "advanced" in Spanish mean very different things. Somehow we want to almost separate the difficulty from the experience. It's hard feeling like we'll always be at the "beginner" level in some languages (like my Arabic, for example), even though that may be true (unless we find the time to go to the country for a while and really put in a lot of specific effort).
Welcome to Linguist Forum! If you have any questions, please ask.

Offline freknu

  • Forum Regulars
  • Serious Linguist
  • *
  • Posts: 397
  • Country: fi
    • Ostrobothnian (Norse)
Re: Practical measures of proficiency
« Reply #3 on: December 19, 2013, 01:00:11 AM »
This is not something I have thought about before, but being able to read a newspaper is something I would consider a very good indicator of proficiency.

Of course, this does not mean that you have an extensive knowledge slang, colloquialism, idioms, proverbs, academic or technical terminology, and so on. However, newspapers do seem to be a pretty good indicator of the standard language.

It's also important to remember that proficiency is not a "scalar":

  • reading proficiency
  • writing proficiency
  • speaking proficiency
  • listening proficiency
  • general understanding
  • morphology

... and so on.

Of course, it all depends on the newspaper you are reading. An economic (or otherwise academic or technical newspaper) is going to be more difficult.

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Re: Practical measures of proficiency
« Reply #4 on: December 19, 2013, 01:30:44 AM »
Quote
Of course, it all depends on the newspaper you are reading. An economic (or otherwise academic or technical newspaper) is going to be more difficult.
Not necessarily! I do relatively well reading Linguistics articles in languages I'd have no ability in otherwise (like Swedish, if I have some patience and a dictionary). [It's possible some of that is a cognate effect, I admit, but not all; context also helps and my specialization in the field and learning that specific jargon.]

As you said: it's not scalar at all. Definitely not. (But on the whole it does feel that way... perhaps we could somehow measure a "core" proficiency, in things like syntactic structures and phoneme contrasts??)
Welcome to Linguist Forum! If you have any questions, please ask.

Offline freknu

  • Forum Regulars
  • Serious Linguist
  • *
  • Posts: 397
  • Country: fi
    • Ostrobothnian (Norse)
Re: Practical measures of proficiency
« Reply #5 on: December 19, 2013, 01:39:39 AM »
Quote
Of course, it all depends on the newspaper you are reading. An economic (or otherwise academic or technical newspaper) is going to be more difficult.
Not necessarily! I do relatively well reading Linguistics articles in languages I'd have no ability in otherwise (like Swedish, if I have some patience and a dictionary). [It's possible some of that is a cognate effect, I admit, but not all; context also helps and my specialization in the field and learning that specific jargon.]

I was talking more in general, not taking into account any specialisation or previous knowledge, the average general population.

As you said: it's not scalar at all. Definitely not. (But on the whole it does feel that way... perhaps we could somehow measure a "core" proficiency, in things like syntactic structures and phoneme contrasts??)

I would are say reading and writing would then be secondary, as language is firstly an oral skill, so perhaps:

  • pronounciation
  • syntomorphology (basic syntax and inflection)
  • vocabulary

Those would probably be three core features that I would concentrate on if I absolutely had to create a constant of proficiency. I'm unsure about weighting, though. It's very difficult to say which one is more important.
« Last Edit: December 19, 2013, 01:55:00 AM by freknu »

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Re: Practical measures of proficiency
« Reply #6 on: December 19, 2013, 01:51:08 AM »
Right. Weighting is the hard part.
I was thinking about some kind of "necessary core", though. I mean, you can't speak/read/do anything with English if you don't know things like SVO order, "of", the contrast between "z" and "s", etc. Beyond modality, beyond genre, a few core things are there. Although perhaps knowing all of the core doesn't really take too much; I'm not sure.
Perhaps a definition: core = generalizable knowledge, rather than specialized details?
(It's not a sharp distinction, of course, but perhaps practical.)



As for specializations, I think it applies to almost everyone: lots of "average" people really like sports and can probably understand a whole lot more of that than an average economics article in a newspaper :)
Welcome to Linguist Forum! If you have any questions, please ask.

Offline freknu

  • Forum Regulars
  • Serious Linguist
  • *
  • Posts: 397
  • Country: fi
    • Ostrobothnian (Norse)
Re: Practical measures of proficiency
« Reply #7 on: December 19, 2013, 01:56:04 AM »
As a side note, is it always "weigh-t-ing" when speaking of statistical factors?

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Re: Practical measures of proficiency
« Reply #8 on: December 19, 2013, 01:57:37 AM »
Yes. weighing = measuring weight; weighting = assigning weight. (interesting/odd contrast now that I think about it)

[However there's the non-technical usage "weigh X down" which I suppose is an exception to the rule above. "weighting" is always about sets/lists and relative weights, I think.]
Welcome to Linguist Forum! If you have any questions, please ask.

Offline freknu

  • Forum Regulars
  • Serious Linguist
  • *
  • Posts: 397
  • Country: fi
    • Ostrobothnian (Norse)
Re: Practical measures of proficiency
« Reply #9 on: December 19, 2013, 02:01:41 AM »
I was thinking about some kind of "necessary core", though. I mean, you can't speak/read/do anything with English if you don't know things like SVO order, "of", the contrast between "z" and "s", etc.

Funnily enough, I'm not sure I would consider z/s that important ;) When it comes to phonology I would be more interested in strict phonemicity, maybe even dropping some minor phonemes.

So [s~z], [θ~ð], [ð~d], and so on, is not something I would necessarily require for basic proficiency.

I guess I'm more inline with being able to understand and make yourself understood being more important that strict correctness. It could certainly be a part of further proficiency, but for basic proficiency I'm somewhat uncertain.

I guess it depends a lot on the definition of proficiency as well.

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Re: Practical measures of proficiency
« Reply #10 on: December 19, 2013, 02:06:57 AM »
Sure. But I meant that it's not modality/genre dependent. If you know s/z then you know English better, simple as that. It's not required for basic proficiency, you're right.

Perhaps for basic proficiency (but still no specialization) consider t/d.
[Or a less practical but still very literal example, "e" vs. "t" ;) ]
Welcome to Linguist Forum! If you have any questions, please ask.

Offline freknu

  • Forum Regulars
  • Serious Linguist
  • *
  • Posts: 397
  • Country: fi
    • Ostrobothnian (Norse)
Re: Practical measures of proficiency
« Reply #11 on: December 19, 2013, 02:18:56 AM »
For something that would seem quite central to language acquisition, has there not been any linguistic study into proficiency and maybe things like sigma-based proficiency scales?

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Re: Practical measures of proficiency
« Reply #12 on: December 19, 2013, 02:34:33 AM »
'Sigma-based'?

There has been a lot of study. I'm wondering at the moment mostly about intuitive measures. Not for the sake of competition, but just linguistic research, is there some intuitive way to say that you speak German better than I do? (Certainly we'd need to make some assumptions, but ignoring the details...)
Welcome to Linguist Forum! If you have any questions, please ask.

Offline freknu

  • Forum Regulars
  • Serious Linguist
  • *
  • Posts: 397
  • Country: fi
    • Ostrobothnian (Norse)
Re: Practical measures of proficiency
« Reply #13 on: December 19, 2013, 02:36:48 AM »
'Sigma-based'?



;)

There has been a lot of study. I'm wondering at the moment mostly about intuitive measures. Not for the sake of competition, but just linguistic research, is there some intuitive way to say that you speak German better than I do? (Certainly we'd need to make some assumptions, but ignoring the details...)

I see. Kind of difficult to say.

Offline Daniel

  • Administrator
  • Experienced Linguist
  • *****
  • Posts: 1539
  • Country: us
    • English
Re: Practical measures of proficiency
« Reply #14 on: December 19, 2013, 02:39:56 AM »
Ah, standard deviations, bell-curve, all that. Why would you assume that language learning is in a normal distribution? (It might be, but I'm not sure that's an obvious statistical assumption to make.)
Welcome to Linguist Forum! If you have any questions, please ask.