So, first: I don't understand the meaning or importance of "motivating a linguistic analysis" (more generally, I don't understand "motivating"). What motivates me to get out of bed is the knowledge that I can tend my garden or write a book, depending on my mood. I assume that you're asking about justification, according to intellectual / epistemological criteria, specifically, "What justifies analysis X of language Q, as opposed to analysis Y of language Q".
But: you then observe that "Somebody has to collect data to analyze. The type of data a person analyzes is very much motivated by that person's theoretical framework". This suggests that you are not asking about evidence / warrant / justification and other such epistemological constructs, you are actually interested in the personality question "what motivates me to get out of bed?".
Not at all, and please don't take anything I say here personally. What I was referring to was more in the sense of a Kuhnian paradigm, where the data under analysis shifts depending on one's theoretical assumptions. So pre-generative descriptivists shunned linguistic intuitions and pretended that their corpora were somehow acquired in an objective fashion. So the data one collects depends very much on what one's theory dictates. Let me digress with an anecdote to illustrate my point.
My first field methods course was with Mary Haas in 1970, who did things strictly by the descriptivist playbook. We had an Aymara informant, Mr. Yapita, whose language Professor Haas did not know. As her ideological principles demanded, we were required first to work out the phonetics. Only then could we proceed to phonemic analysis, but not morphology. After the phonemic inventory was established, we could proceed to address morphology. As you can imagine, that didn't sit well with young generative linguists, so we accosted Mr. Yapita outside of the classroom to ask questions about the more interesting aspects of his language and query his intuitions. When we tried to swear him to secrecy, he laughed and told us that Professor Haas was also meeting him outside the classroom and asking roughly the same questions.
As a competent linguist, she knew the difference between theory and practice, but she was letting theory dictate the practice in the classroom. She didn't want to have to explain why to her students.
I don't understand what you mean by "theoretical framework". In terms of names, I'm a substance-non-abusing Formal Phonologist. I also do field work (more of the latter than the former). My theoretical framework absolutely does not determine of motivate the kind of data that I collect and therefore analyze. I won't deny that some people who are primarily theoreticians but who also dabble with first-hand language description only ask a limited set of questions about long-distance extractions out of islands, but that's a reflection on poor understanding of best practices for field work. As to why I tend to gather / analyze more of one kind of data than another, that is because the day is finite and I believe in greater depth rather than greater breadth.
Perhaps I should have said "theoretical paradigm" rather than "theoretical framework". Luckily, one has great tools these days when analyzing the speech of informants. We used to have to make sound spectrographs on a rotating drum on paper that gave off a poisonous gas.
Nowadays, you can produce acoustic analyses instantly and at will on a portable computer. Still, the aspects of the data that you pay attention to will very much depend on what research goals are directing your research. It is well-known that hypotheses act as a filter on data collection, which is not a random activity.
There are a number of ways to determine what counts as valid linguistic forms and relations in a language. Actual speaker behavior is the main one. If I ask a speaker "Can you say [tuʔəɬəd čəd kʷi sʔuladxʷ]?" and they say "Hell no" (and are consistent, and assuming I've done due dilligence with the context, like saying that I'm talking about a dream), then I am entitled to exclude it from the set. Or, they may say "Hell yes" (and I would of course do due dilligence on that as well). In the latter case, they might even volunteer that sentence. In any event, I may be relying on the speaker's ability to introspect, but what I'm accounting for is the behavior -- not just passive behavior, but active and reflective behavior.
If I were analyzing morphonology and not phonology (as I define them, of course), then I might ask such a question. Indeed, when I studied Breton as a grad assistant, we were sometimes looking at the behavior of Celtic mutations under conditions of language death. So the lead researcher would try to get a younger and older pair of informants together to ask such questions about intuitions. He then compared the answers to see which mutations were decaying fastest. But, when we studied the phonemic inventory, we never paid attention to how speakers thought they would answer such questions. We listened to recordings and voted on what we thought was being pronounced. Nowadays, we would just look at the display of a spectrogram. The problem is that naive speakers have a lot of false intuitions about what they can and can't say.
As I mentioned before (and every generative linguist has mentioned before), a grammar is an aspect of the mind, which is an entity of individual psychology. So a grammr is a psychological entity. That does not make generative linguists psychologists -- it makes them mentalists.
So here is where I push back a little. Just what is it that you think "an entity of the mind" is? I do not really understand what you think the difference is between a psychologist and a "mentalist", but I do think that psychologists would consider the subject matter of "mentalism" to be solidly within their purview of study. I don't think that you can do "mentalism" without at least relating it to behavior that is mentally-driven.
The "mere" description of patterns relates to the causal mental mechanisms in it that provides something that a hypothesized mental mechanism actually does -- it is a logical precursor to theory-construction. If for instance there were a hypothesized mental mechanism Fred that had no known effect, Occam's Razor would eventually slice Fred out of existence. Sometimes, the evidence (the merely-described patterns) suggest but do not prove the existence of a causal mechanism. So being agnostic about the mental reality of a particular theoretical construct is well justified.
I suggest that Occam's Razor applies rather sooner than later to cut Fred out of the picture. If Fred has no known effect, then he cannot be studied. Period. Science never actually "proves" anything in an absolute sense, but empiricism demands at least a modicum of plausibility. If you hypothesize a "Fred", then you need to be able to say something about what Fred does. Otherwise, there is no point in keeping him around.
And I urge you to reflect on the claim that "nobody can be agnostic about the validity of the data they are analyzing". There are three attitudes that you can have about data: "I know that X is part of language Y", "I know that X is not part of language Y" and "I don't know if X is part of language Y". If you don't have sufficient evidence that X is part of Y, you shouldn't conclude that X is part of Y. So not only can people be agnostic about supposed data, they should be.
What we are discussing here is what it means to even say whether "X is part of language Y". How do you know that "X" can be "part of language Y"? That is not a trivial question. It requires you to take a stand on just what it is that you are trying to prove. The same question arose in the last century, when linguists were told to shut up and just confine themselves to studying a "corpus" that was gathered by means that we won't bother to discuss seriously. That actually led to a lot of confusion and debate that, apparently, has still not been resolved.
Given that I know that a grammar is a thing of individual psychology, I know that knowledge of the nature of a grammar is "psychologically relevant". It does not follow that I know that every conjectured specific mechanism of grammar is in fact a mechanism of grammar, and if I don't know whether e.g. rule iteration is a property of grammar, then I don't know if it is psychologically valid.
I certainly agree with you that "grammar" can be construed as psychologically relevant. I question that you can get away with remaining agnostic on its psychological status. That's the kind of thing that really comes back to bite you in the end.
We do know that linguistic structure can be recursively and iteratively defined, but that just means that our theory must account for its iterative and recursive properties. We also know that people are not random sentence generators. That, too, must have a theoretical explanation. Let's make a distinction between structures of infinite length (not linguistic) and structures of indeterminate length (linguistic). Now we can talk about the factors that limit iteration and recursion. That's where the interesting questions come in about the relationship between structure and communicative function. You can't really have one without the other.
