Why does linguistic structure exist?

[jkpate]

I've mentioned on this board before that there is a tradeoff between incrementality and efficiency. I just posted on my blog about just why this is, along with some speculations about how this view allows us to express grammaticalization as an optimization procedure: as speakers, we want some tradeoff between immediately understanding or producing each part of a message (incrementality) and quickly understanding or producing the message as a whole. As I see it, this is essentially a different view of usage-based approaches to grammaticalization, as advocated by e.g. Joan Bybee.

The most prominent alternative explanation for the existence of linguistic structure is the Generativist one, in which language is fundamentally about thought rather than communication, and the hierarchical structure of language reflects the inherently recursive nature of thought. Are there other explanations for the existence of linguistic structure beyond these two?

[Daniel]

Why linguistic form? Historical accidents, and communicative need.
Why linguistic structure? Modification (human language has a fundamental property of one concept modifying another-- by another name, Chomsky calls this "Merge"), and communicative need.

In short, I think it's important to separate the two.

...but my dissertation is still in progress.




--
Edit: reading your blog entry--

First, why do you treat language as information/code? I thought it was established, at least in pragmatic theory, that language is not a code in a technical sense. I've always wondered about that. In some sense, of course it must convey information. But what exactly is it conveying? Are there 1s and 0s? Or are there just hints for the hearer to associated particular concepts that are somewhat loosely related to similar concepts in the mind of the speaker? If we're going to talk about information, then what exactly is information? And if there is information, why not just say it directly? Why do languages vary so much, if it's just "information"? Or is there more going on?

For example, if the phonological form of a word is much longer than its Shannon information, we could improve our efficiency by making that word shorter. Conversely, if the phonological form of a word is much shorter than its Shannon information, we could improve our efficiency by lengthening that word, thereby freeing up a short phonological form to be used by a different, more frequent word.

I suppose this makes sense, but I'm not sure it works exactly that way. For one thing, not all short word forms are utilized in languages. And some languages have about a dozen phonemes while others have toward 150. Language is more complex than just information: it's also form. Adding more contrasts to the signal would also be more efficient (encode more information), but then it would run the risk of being less efficient for the hearer. Probably in some way all of these factors balance out.
In the end I do wonder if the arbitrariness of language (and I mean that to extend to syntactic form, for reasons I'm working out in my dissertation) is part of how it works: we refer to culturally defined shortcuts ("lexical items") rather than some objectively 'best' code.

So in short, I'm still lost about what "information theory" has to do with language-- what information does language encode anyway? If we can answer that and get to a formalization, this will all start making sense, but there are still some complicated pieces. One is, as I said, that we refer to shared lexical entries, not to objective information. (We can quantify those references to the lexical entries as information, of course, but this gets complicated, because some of the work is done by the lexicon then.)

In a block code, we encode sequences of message characters instead of individual message characters.

Ooh, now I'm starting to wonder if this has to do with syntax, periphrasis and 'structure'

First, observe that a substantial portion of linguistic meaning is compositional

If you're referring to processing (which is what gets us historical change and then current usage), this is complicated. I believe there are shortcuts so that everything isn't strictly compositional. Much is, but not everything. In terms of quantifying any information in the signal then, it becomes tricky. One imbalance might be idioms where syntactic form is still relatively transparent despite lexicalization. And if non-compositionality (at least for processing, think "chunking") plays a part, then this also gets tricky. There's a balance between competence-compositionality and processing-chunking. Why does the signal evolve faster? Because we need to have some way to reconstruct some of the shortcuts in our lexicon, something like an index to search a database. But eventually some of those shortcuts leave the competence-compositionality behind ("grammaticalization" I suppose).

In other words, because natural language is compositional, it assumes that different components of a message are statistically independent! However, different parts of linguistic messages are highly dependent. Thus, natural language implicitly relies on a family of probability distributions that is far too constrained to capture the true distribution over messages for optimal source coding.

Much of parsing, and probably a substantial amount of speaking, involves just knowing what's coming next. This is one area where native speakers far surpass learners-- try speaking in a noisy environment (pun intended?) and just see how much you can still understand despite the weak signal.

This view suggests that language is far from optimal in the sense of information theory. It does not, however, mean that there is not any pressure in the direction of optimal behavior ...

While I'm not a favor of unfounded "everything balances out" arguments in general, this reminds me of the "simplification" problem in historical linguistics-- "simplification" can be used to explain many changes, and it seems natural. Yet the "complexity" (whatever that may refer to) must come from somewhere. So languages are not, on average, becoming simpler, or in the end they'd just stop changing and be, well, simple. So I imagine that this is the same here-- information theoretic concerns are a force acting on language, but just one of a number of such forces.

We could, for example, use the same block coding trick as before: instead of encoding grammatical elements as separate message components, encode ensembles of grammatical elements simultaneously. Now, however, we are dealing with a much larger space of message components, such as all the words in the language, so we'd want to get a lot of "bang for our buck:" the increase in fit of our low-dimensional approximation must be very large relative to the additional parameter introduced by the larger block.

This (like the going to / gonna example) is fundamental to 'Construction Grammar'. And I think you have a point. I like that it goes deeper than just observations of meaning, though. And I'm wishing I understood the math a bit better (it's me, not your explanations-- I haven't looked at equations like that for a long time, and you're reminding me I probably should). This is interesting (and see my comment above).

...the frequency effects are assumed to be given, as some kind of basic feature of how brains work.

This I can help you with. There's a very interesting (thesis and now) book:
Zeldes, A. (2012). Productivity in argument selection from morphology to syntax. Berlin: De Gruyter Mouton.

This framing suggests that the role of frequency effects in grammaticalization itself may reflect an adaptation towards some balance between incrementality and brevity.

I think this is right. I'm not sure whether it's blatantly obvious or especially insightful. I haven't seen it phrased that way before, anyway, so let's call it insightful. I'm a bit inspired by the post at least.


So while I'm trying to write the formal proposal for my dissertation, these thoughts are distracting me about what's next-- I'll be very interested to follow up on some of this after I have something more concrete from my research. I'm curious if it would tie in. I'm looking at the (cross-linguistic) data-driven side of things, but ending up in the same way intersecting with grammaticalization, construction grammar vs. generativism, and so forth.

[MalFet]

Great read; thanks for posting.

You might be interested to know that the entire field of phonology seems to be moving in similar directions. Many are finding the generative architectures behind Optimality more cumbersome than descriptively helpful, and while the alternatives are still duking it out most of the major contenders describe something very much like what you're describing: structure as an interplay between information theoretic pressures and (in phonology's case) the material properties of articulation and psychoacoustics.

If you're taking questions from the peanut gallery, I've got two:
1) If information theoretic optimization is a real but limited pressure, what's pushing the other way? It seems to be something related to message compositionality<->segmentability, but is that itself something that can be "optimal"?
2) How useful do you think this approach is to explaining apparent typological gaps? I'm thinking specifically of something like non-conservative determiners, here.

[Daniel]

(jkpate should address these questions, so I'll keep my replies short)

1) If information theoretic optimization is a real but limited pressure, what's pushing the other way? It seems to be something related to message compositionality<->segmentability, but is that itself something that can be "optimal"?

I did hint at something like "compositionality/segmentability" in my post; that sounds reasonable. And another pressure is just convention-- not everyone wants to speak Esperanto even if it's wonderful. (That is, I assume the average language user is not easily able to judge whether Esperanto is legitimately more efficient/etc., so that seems a simple test case.) Over time we might expect it to converge on optimal, but convention is still a force over time.

Plus some would argue for linguistic relativity and that all languages are (essentially) optimal for the culture's communicative needs. The balance between information content being long/short and the form being long/short may be close to perfect for languages based on frequency effects. But it's a cycle too. This seems to lead to the question of why some languages have categories like evidentiality and others don't-- the only attempted answer that I'm aware of is in linguistic relativity.

2) How useful do you think this approach is to explaining apparent typological gaps? I'm thinking specifically of something like non-conservative determiners, here.

Universals (or in this case negative universals) are far from convincing in my opinion. I think the simpler answer (whether it's statistical or absolute) is functional: are non-conservative determiners really useful? (And, moreover, how would they evolve diachronically?) So I'm not sure how deep we'd need to go to explain such things. But for those cases, if they exist, that aren't explained by other factors, I'd wonder about the same question.

[jkpate]

First, why do you treat language as information/code? I thought it was established, at least in pragmatic theory, that language is not a code in a technical sense. I've always wondered about that. In some sense, of course it must convey information. But what exactly is it conveying? Are there 1s and 0s? Or are there just hints for the hearer to associated particular concepts that are somewhat loosely related to similar concepts in the mind of the speaker? If we're going to talk about information, then what exactly is information? And if there is information, why not just say it directly? Why do languages vary so much, if it's just "information"? Or is there more going on?

From the perspective of information theory, information is just ruling out alternatives. I've heard people talk about pragmatics showing that language is not a code, but I haven't found these arguments convincing. People argue, for example, that what the listener needs to infer is the talkers' intention, not a logical form divorced from context. However, this is just means that the "messages" of information theory can be intentions as well as more traditional logical forms, and talker behavior provides information by ruling out particular intentions. It's not fundamentally a problem for information-theoretic accounts.

For example, if the phonological form of a word is much longer than its Shannon information, we could improve our efficiency by making that word shorter. Conversely, if the phonological form of a word is much shorter than its Shannon information, we could improve our efficiency by lengthening that word, thereby freeing up a short phonological form to be used by a different, more frequent word.

I suppose this makes sense, but I'm not sure it works exactly that way. For one thing, not all short word forms are utilized in languages. And some languages have about a dozen phonemes while others have toward 150. Language is more complex than just information: it's also form.

You are of course right that language "skips" some short word forms. But this just another way that language does not approach information-theoretic efficiency limits.

Adding more contrasts to the signal would also be more efficient (encode more information), but then it would run the risk of being less efficient for the hearer. Probably in some way all of these factors balance out.

I know this contrast between efficiency "for the talker" and efficiency "for the hearer" has been influential in linguistics, but I really think that is the wrong way to talk about things. Providing too little contrast (easy "for the talker") is troublesome for both, because if the listener fails the talker will have to repeat himself. Providing too much contrast (easy "for the listener") is also troublesome for both, because they both have to wait a long time for the utterance to finish. I would prefer to call these "short-term ease" for the talker and listener rather than "efficiency."

So in short, I'm still lost about what "information theory" has to do with language-- what information does language encode anyway? If we can answer that and get to a formalization, this will all start making sense, but there are still some complicated pieces. One is, as I said, that we refer to shared lexical entries, not to objective information. (We can quantify those references to the lexical entries as information, of course, but this gets complicated, because some of the work is done by the lexicon then.)

Hmm, what do you mean by "objective information"? Information is just ruling out alternatives, whether those alternatives are words, trees, logical forms, sociolinguistic categories, or something else.

In a block code, we encode sequences of message characters instead of individual message characters.

Ooh, now I'm starting to wonder if this has to do with syntax, periphrasis and 'structure'

Me too!

First, observe that a substantial portion of linguistic meaning is compositional

If you're referring to processing (which is what gets us historical change and then current usage), this is complicated. I believe there are shortcuts so that everything isn't strictly compositional. Much is, but not everything. In terms of quantifying any information in the signal then, it becomes tricky. One imbalance might be idioms where syntactic form is still relatively transparent despite lexicalization. And if non-compositionality (at least for processing, think "chunking") plays a part, then this also gets tricky.

I'm not relying on a specific processing account. However humans perform the mapping, at the end of the day, most message components will be encoded with largely the same code, regardless of what else is going on. For this to really sink in, I think it's useful to look at arithmetic coding. In arithmetic coding, each successive message character is encoded by picking a more constrained range of numbers between 0 and 1. The final code for the message is then just a representation of some number in the final, most constrained range, and there is not any portion of the final signal that can be specifically attributed to any one message character.

And I'm wishing I understood the math a bit better (it's me, not your explanations-- I haven't looked at equations like that for a long time, and you're reminding me I probably should). This is interesting (and see my comment above).

The first eleven chapters of David MacKay's book is an excellent and friendly introduction to all of this.

...the frequency effects are assumed to be given, as some kind of basic feature of how brains work.

This I can help you with. There's a very interesting (thesis and now) book:
Zeldes, A. (2012). Productivity in argument selection from morphology to syntax. Berlin: De Gruyter Mouton.

Thanks!

If you're taking questions from the peanut gallery, I've got two:
1) If information theoretic optimization is a real but limited pressure, what's pushing the other way? It seems to be something related to message compositionality<->segmentability, but is that itself something that can be "optimal"?

I think this is a hard question. We can of course devise different ways of operationalizing degree of incrementality, and then trade it off with efficiency, but is there any reason to prefer one operationalization or trade-off to another? Pure mathematical simplicity would argue for just a weighted interpolation of how long it takes to decode against the average amount of redundancy (= amount of signal over and above what is necessary), but... maybe there's some other way to adjudicate this tension.

2) How useful do you think this approach is to explaining apparent typological gaps? I'm thinking specifically of something like non-conservative determiners, here.

Hmm, I hadn't thought about this perspective. I'm always a bit wary of relying on gaps. There seems to be a "fat-tail" of possible linguistic phenomena, and so it's not too surprising to fail to see some phenomenon or combination in the few thousand languages we have. I think this account would have to be made much more specific to address non-conservative determiners; so far, I've tried to be agnostic about what exactly the message is, since there are many different ways to analyze language behavior into form/meaning pairs.

[MalFet]

I've heard people talk about pragmatics showing that language is not a code, but I haven't found these arguments convincing.

I hear this claim frequently too, and to be honest I can't even really decipher it. The alternative to language as code seems to be...I have no idea...the kind of phonosemantics we hear about occasionally on this forum? Some other unmediated domain of pure experience? The objection seems to be that language does not exhaustively represent all potentially consequential circumstances of the speech context, but that is a flatly impossible expectation for any kind of code.

I think this is a hard question. We can of course devise different ways of operationalizing degree of incrementality, and then trade it off with efficiency, but is there any reason to prefer one operationalization or trade-off to another? Pure mathematical simplicity would argue for just a weighted interpolation of how long it takes to decode against the average amount of redundancy (= amount of signal over and above what is necessary), but... maybe there's some other way to adjudicate this tension.

I guess I'm not asking for a particular kind of incremental processing so much as a way of thinking about competing forces: what kind of advantages are you accounting for with incremental processing? Is it chiefly about memory load? Opening the door for listener back-channeling? Allowing for comprehension as a matter of degree? Something more directly aligned to human cognitive organization (and its apparent gestalt preferences)?

Hmm, I hadn't thought about this perspective. I'm always a bit wary of relying on gaps. There seems to be a "fat-tail" of possible linguistic phenomena, and so it's not too surprising to fail to see some phenomenon or combination in the few thousand languages we have. I think this account would have to be made much more specific to address non-conservative determiners; so far, I've tried to be agnostic about what exactly the message is, since there are many different ways to analyze language behavior into form/meaning pairs.

I'm wary too, but to the extent that this is an account of learning (among other things, perhaps), there's a potentially very powerful explanation here about why some organizations are extremely common and some are extremely rare, no? Within an framework that treats language as an ongoing dynamic of informationally-driven grammaticalization, wouldn't we expect to see relatively more and relatively less stable forms?

[jkpate]

I think this is a hard question. We can of course devise different ways of operationalizing degree of incrementality, and then trade it off with efficiency, but is there any reason to prefer one operationalization or trade-off to another? Pure mathematical simplicity would argue for just a weighted interpolation of how long it takes to decode against the average amount of redundancy (= amount of signal over and above what is necessary), but... maybe there's some other way to adjudicate this tension.

I guess I'm not asking for a particular kind of incremental processing so much as a way of thinking about competing forces: what kind of advantages are you accounting for with incremental processing? Is it chiefly about memory load? Opening the door for listener back-channeling? Allowing for comprehension as a matter of degree? Something more directly aligned to human cognitive organization (and its apparent gestalt preferences)?

Now that you mention it, backchanneling could be really useful. In the post, I did not talk about avoiding errors. Briefly, noise is modeled as random changes to the signal between the sender and the receiver, so the the signal the receiver gets might be different from what was sent. The sender can guard against this noise by re-encoding the signal in a way that the intended signal can be reconstructed, but the re-encoding process will have to make the signal longer. For example, you might just repeat each signal character three times, so the receiver can tolerate one random change by taking a majority vote. Because this makes the signal longer, we might be worried that the short signals we get from source coding will become unmanageably long. The central result of the Noisy Channel theorem is that you can bring the error rate arbitrarily close to zero with a finite amount of extra re-encoding, if you know that noise is like.

However, I can imagine that in many situations talkers will not know what the noise is like, and a backchannel cue at the precise moment of error could help be immensely helpful for learning a good noise model. In fact, work by Jennifer Roche and colleagues suggests that this is exactly what's happening, with talkers revising their referring expressions after communication breakdowns.

Hmm, I hadn't thought about this perspective. I'm always a bit wary of relying on gaps. There seems to be a "fat-tail" of possible linguistic phenomena, and so it's not too surprising to fail to see some phenomenon or combination in the few thousand languages we have. I think this account would have to be made much more specific to address non-conservative determiners; so far, I've tried to be agnostic about what exactly the message is, since there are many different ways to analyze language behavior into form/meaning pairs.

I'm wary too, but to the extent that this is an account of learning (among other things, perhaps), there's a potentially very powerful explanation here about why some organizations are extremely common and some are extremely rare, no? Within an framework that treats language as an ongoing dynamic of informationally-driven grammaticalization, wouldn't we expect to see relatively more and relatively less stable forms?

I do think it's likely that any kind of dynamic system with feedback is going to have some sparse areas and some attractors. I'm just not convinced that we have specified our system in enough detail to make quantitative predictions about where those areas should be, or to distinguish between chance gaps and systematic gaps. But it's definitely something to work toward!

[Daniel]

From the perspective of information theory, information is just ruling out alternatives. ... It's not fundamentally a problem for information-theoretic accounts.

What is the "alternatives space" then? That is what I meant about pragmatics. I'm not sure I believe it either. But I think part of the argument is that we point toward meanings rather than encoding them. The intention isn't transferring information but rather approximating it. I'm happy to use a more code-based definition, but I still want to know what domain we're talking about. Is it sounds? Morphemes? Lambda expressions?

I know this contrast between efficiency "for the talker" and efficiency "for the hearer" has been influential in linguistics, but I really think that is the wrong way to talk about things. Providing too little contrast (easy "for the talker") is troublesome for both, because if the listener fails the talker will have to repeat himself. Providing too much contrast (easy "for the listener") is also troublesome for both, because they both have to wait a long time for the utterance to finish. I would prefer to call these "short-term ease" for the talker and listener rather than "efficiency."

With none of these terms defined, we run into problems like that. Your point is valid. Still, I do think speakers tend to try to abbreviate even when they go too far-- I've many times spent more time explaining what I meant than the amount of time I saved when I've tried to say things quickly.

Hmm, what do you mean by "objective information"? Information is just ruling out alternatives, whether those alternatives are words, trees, logical forms, sociolinguistic categories, or something else.

Mhm. So which one(s)? (I'm really not sure what I'd start "counting" if I wanted to count things in an utterance.)

I'm not relying on a specific processing account. However humans perform the mapping, at the end of the day, most message components will be encoded with largely the same code, regardless of what else is going on. For this to really sink in, I think it's useful to look at arithmetic coding. In arithmetic coding, each successive message character is encoded by picking a more constrained range of numbers between 0 and 1. The final code for the message is then just a representation of some number in the final, most constrained range, and there is not any portion of the final signal that can be specifically attributed to any one message character.

Ok, but convince me that language works that way. It might. What is complicated about linguistics is all the layers. Our jobs would be much easier if, say, we were just determining phonemes and stopping at that. The trick is that all of these components overlap and while the final message is of course going to be the sum of information in that message (how could it not be?) the explanation for how we use/know language must involve something about the different strategies available. Think of this as a compression algorithm-- to understand a compression algorithm we need to know about the different ways the signal is encoded, not just the string of 0s and 1s that somehow can be transformed into the full original data.

The first eleven chapters of David MacKay's book is an excellent and friendly introduction to all of this.

Thanks. I've saved that. I'm not sure when I'll end up with time to get into it in any detail, but it's worth knowing about the resource

--

I hear this claim frequently too, and to be honest I can't even really decipher it. The alternative to language as code seems to be...I have no idea...the kind of phonosemantics we hear about occasionally on this forum? Some other unmediated domain of pure experience? The objection seems to be that language does not exhaustively represent all potentially consequential circumstances of the speech context, but that is a flatly impossible expectation for any kind of code.

As I understand it, the point being made is that the "message" in the speaker's head is not "encoded" in the signal and "decoded" by the listener into that same "message" in their head. Instead, the signal is a tool that hints at the intended message, and shared pragmatic strategies allow the speakers to (appear to) "communicate". In other words, the reason it "isn't a code" is because of the domain being considered-- the full intent of the utterance (basically what you said).
And this is why I'm wondering... what exactly is the signal composed of? It must encode something (even if that's phonosemantics, right?)...

Within an framework that treats language as an ongoing dynamic of informationally-driven grammaticalization, wouldn't we expect to see relatively more and relatively less stable forms?

But to be clear, that doesn't give us typological gaps. It just explains frequencies. And I'd assume you're right about this.

--

...or to distinguish between chance gaps and systematic gaps.

I'm not sure there is such a distinction. It's all chance. But the probabilities are guided by certain parameters. Again, "gap" is the wrong term in the sense that there's nothing that would rule out any particular configuration (at least in what you/we have said so far), just that some configurations are much more likely to develop given certain constraints in the system. This is where my (unrelated) thoughts about the nature of linguistic form have gone too: there are patterns and expectations, but there are no strict rules (universals). Anything is possible, but many things are highly improbable. Finding the system that properly calculates these odds would basically explain language (at least change-- it would probably have a prerequisite of a synchronic theory first, at least an outline).

[MalFet]

Now that you mention it, backchanneling could be really useful. In the post, I did not talk about avoiding errors. Briefly, noise is modeled as random changes to the signal between the sender and the receiver, so the the signal the receiver gets might be different from what was sent. The sender can guard against this noise by re-encoding the signal in a way that the intended signal can be reconstructed, but the re-encoding process will have to make the signal longer. For example, you might just repeat each signal character three times, so the receiver can tolerate one random change by taking a majority vote. Because this makes the signal longer, we might be worried that the short signals we get from source coding will become unmanageably long. The central result of the Noisy Channel theorem is that you can bring the error rate arbitrarily close to zero with a finite amount of extra re-encoding, if you know that noise is like.

However, I can imagine that in many situations talkers will not know what the noise is like, and a backchannel cue at the precise moment of error could help be immensely helpful for learning a good noise model. In fact, work by Jennifer Roche and colleagues suggests that this is exactly what's happening, with talkers revising their referring expressions after communication breakdowns.

There's also just a question of sufficiency of reference. This morning I went to that store...next to the bank...with the name that sounds really German...that sells those funny hats...where we went last weekend...and (stop me when you get what I'm talking about).

At a discursive level, backchanneling not only allows a listener to indicate whether a communication event has been successful, but moreover it lets a speaker organize a message in such a way as to unfold incrementally and only to the extent necessary. That's tremendously useful (from an efficiency perspective and otherwise) and it's made possible by the kind of sub-utterance parsability that you're describing.

I do think it's likely that any kind of dynamic system with feedback is going to have some sparse areas and some attractors. I'm just not convinced that we have specified our system in enough detail to make quantitative predictions about where those areas should be, or to distinguish between chance gaps and systematic gaps. But it's definitely something to work toward!

Predicting them precisely is indeed a tall order, but even just to be able to say qualitatively that "this thing we don't see often/ever is something our hypothesis would predict to be relatively/extremely rare" is a powerful start. That's why non-conservative determiners popped into my head. The mainline explanation here is that they are unlearnable, a notion that both illustrates and depends on a very particular narrative about how the mind does language. But, your model seems to suggest something else. Modeling information formally is well above my pay-grade, but the thing that has never sat right for me in this claim about non-conservative determiners is that they make for some very odd constituencies. "Some dogs" and "all dogs" references various scopes of dogs, but with "[non-conservative determiner] dogs" we're actually scoping something as yet inexplicit. If your hypothesis suggests that words go together in order to allow for incremental processing, we'd *expect* n-c ds to be extremely rare. You've got a pretty powerful model for explaining typological frequencies on your hands, even if only qualitatively.

[MalFet]

As I understand it, the point being made is that the "message" in the speaker's head is not "encoded" in the signal and "decoded" by the listener into that same "message" in their head. Instead, the signal is a tool that hints at the intended message, and shared pragmatic strategies allow the speakers to (appear to) "communicate". In other words, the reason it "isn't a code" is because of the domain being considered-- the full intent of the utterance (basically what you said).
And this is why I'm wondering... what exactly is the signal composed of? It must encode something (even if that's phonosemantics, right?)...

You're conflating "message" with "intent". Message is much simpler. The signal of a message  (in spoken language) is a perceivable sound-stream that is parsable into nested hierarchies of categorical units. The thing being encoded to comprise the message are those categorical units. What you are observing is that messages are interpreted in ways determined by context, but there's nothing particularly unusual about that. A truly context-independent code system is (perhaps by definition) inconceivable.

But to be clear, that doesn't give us typological gaps. It just explains frequencies.

Unless you've got a non-finite corpus squirreled away somewhere, those two things are empirically indistinguishable.

[Daniel]

we're actually scoping something as yet inexplicit.

Hmm... head-ordering varies between languages, so I don't think the issue is precisely precedence. Rather, it is something about proximity--

If your hypothesis suggests that words go together in order to allow for incremental processing, we'd *expect* n-c ds to be extremely rare.

This seems very similar to me to a claim that discontinuous constituents should be rare. And indeed they are, but not impossible.
Essentially, like things are grouped, usually. And that's a major property of language.
There are a few different ways to get there (a universal tree-structure skeleton, information-theoretic grouping, near-isomorphism to conceptual structure, etc.).

You're conflating "message" with "intent". Message is much simpler.

Terminology. And otherwise I agree. If my intent is to have you close the window, though, it seems that something in my message must relate to closing the window, so when I say "It's cold in here", it doesn't seem unreasonable to me, pre-theoretically, to consider something about that message to relate to the window. And as I understand it, that's basically the pragmatic argument there-- the message is incomplete.
But absolutely: we can redefine these terms and come up with something a lot more workable. Language is a code for messages and a system of implicatures for intents. That seems right to me. And back to the question: what information makes up messages? (That's the 'big question', of course.)

The signal of a message (in spoken language) is a perceivable sound-stream that is parsable into nested hierarchies of categorical units. The thing being encoded to comprise the message are those categorical units.

So is the signal made up of sound segments? Or is it made up of nested hierarchies? Or both? I suppose both, and in terms of information theory we'd optimize over both, perhaps independently? Maybe that's one of the tradeoffs.

What you are observing is that messages are interpreted in ways determined by context, but there's nothing particularly unusual about that. A truly context-independent code system is (perhaps by definition) inconceivable.

Right. I don't disagree. I'm just discussing the perspective I've heard that language isn't a code. I think it should be, at least at some level.

Unless you've got a non-finite corpus squirreled away somewhere, those two things are empirically indistinguishable.

There's a categorical distinction at the theoretical level: "gaps" implies something important for nativist universal grammar. Frequencies just describes distributions. And more than that, the apparent gaps are just one kind of frequency, whereas discussing frequency in general can explain more about the distribution of linguistic features, if indeed information theory can explain it.

But if you just meant lacking data in our current knowledge, then that's fine. (I was taking note of the phrasing because phrasing like that can be used, not by you in particular, to argue for a nativist perspective. Frequencies don't allow for that, so I prefer that term.)

[pljames]

My problem is I do not understand verbs nouns and such. Just words alone. I hear words in my mind, but do not understand then until I see them down, like on this screen. I have a fanaticisium about being understood, without question. Mabe I should underline or or () the word instead of usig Html? I love to write and just want to be understood. Thoughts? Paul

[Daniel]

Paul, as we've said numerous times in other places, it doesn't sound like you are interested in Linguistics. It sounds like you're interested in writing and perhaps web design. "Being understood" is a communicative goal independent of the linguistic signal. It involves, for example, being in the right context and interacting with other people. It is related to pragmatics, but pragmatics as a field within linguistics is built on top of linguistic analysis including things like identifying nouns and verbs. If you don't/won't do that, this is probably not the forum for you.

[pljames]

Understood, I will unsubscribe. Thank you. Paul