I do actually disagree with the claim that "allophone" is a significant linguistic unit, but here I am assuming it and defending it (somewhat).
I have no objection to that. In fact, I think it's very interesting how these traditional ideas break down under pressure from a closer look at unusual empirical data.
However, it does seem useful for the sort of toy grammars taught in classes for analytical purposes. (And that's why the "right" answer can only be defined within such a class...)
In English, s and z are distinct phonemes
Yes. Because they are
sometimes contrastive. If you can find a minimal pair, then there is a phonemic contrast. That's the core rule.
whereas k and kʰ are allophones of one phoneme, usually thought to be /k/
Only for notational purposes. There's no reason to write it one way or the other. You could also write a random symbol if you wished, because phonemes are only meaningful in their contrasts. Of course the most natural symbol is the one that takes the fewest steps to actually derive the surface pronunciations. But theories or theoretical approaches do differ in whether the actual "value" of a phoneme is meaningful or not.
the word "allophone" does not mean "sound that another sound can have a derivational relation to" or "has some variation in realization", it is a special kind of derivational relation of the kind "all instances of x derive from y"
OK, I see what you mean now. But I disagree. Allophone means "alternative phone" as in "allomorph" meaning "alternative word-part". An allomorph of the plural "-s" is "-es", and that "same" allomorph is also found for the third-person singular present tense. That coincidence (or maybe not?) shouldn't be enough to decide that the form "-es" is no longer an allomorph of one affix or the other, or both. Uniqueness is not a particularly coherent defining property for allophone either.
the sounds are in complementary distribution, and that is the mantra that millions have memorized ("phoneme, allophone, complementary distribution")
OK, here we go! Yes, good point. But I disagree. Phonemes are unique because they are contrastive. Complementary distribution is not negative evidence for phonemes: they can be neutralized. Only contrastive distribution is evidence. Therefore, complementarity is secondary in a theory. Only contrastiveness has any basic theoretical weight. Yes, allophones would tend to be complementary in an ideal and simple case, but that idea breaks down (or is perhaps incoherent to begin with) in any real language when two phonemes might just happen to have the same pronunciation in some circumstances. If any other phoneme in English, other than /k/ is found to have the alternative/contextual pronunciation [kʰ], then does that suddenly not become an allophone of /k/? It seems to me that almost no allophones could ever exist given the natural variation in pronunciation, especially in fact speech. Examples of where you might encounter [kʰ] not from /k/ would be devoicing of /g/ in some cases, as in /gt/, or maybe the sequence /ŋt/ in certain cases. The problem is that allophones cannot be defined at any particular level of phonetic variation/derivation, because phonetics doesn't have specific levels like that. Otherwise we could have IPA notations for /broad/ {medium} [narrow]. There is no "medium" level (that isn't even a coherent idea as I understand it), so the fact that you could
ever get [kʰ] as the output of any other phoneme would invalidate the uniqueness idea. Complementarity is not a uniqueness relationship, but a failure to falsify contrastiveness. Therefore, either the idea of allophones is incoherent (using your restrictive definition), or we must consider allophones in the more flexible and contextual sense of phonetic variants of underlying phonemes. The idea of required complementarity only works in simplified data sets for textbooks, and relies on an artificial/imaginary "medium" level of transcription.
Another possible meaning for "allophone" would be "sound that is not a phoneme", which could be interpreted differently than either of use has suggested. That would be any segment for which we could never identify a contrastive relationship, such that it must be a contextual variant of some underlying phoneme, but perhaps multiple phonemes. (It also might be the case that this is not a coherent definition because it privileges one phonetic output of phonemes over others that are "just allophones", which doesn't necessarily make sense in all theories and regardless relies on already understanding the system and the underlying values for phonemes, e.g., after a full analysis, not as an observational property of "that's just an allophone" to begin with.)
Personally I prefer the simple "one of several phonetic realizations of a phoneme" because it's coherent and doesn't break down or imagine levels of representation that don't really exist.
"Allophone" can remain a (somewhat) useful concept of just saying 'phonemes vary in pronunciation' and talking about one element of those bundles of pronunciation.
Of course allophones per se are not really a coherent concept at all, in any sense, because we cannot meaningfully define explicit sets of phonetic outputs because they are scalar/variable beyond a simple list of "segments". Only phonemes exist as discrete units*, and allophones are just our "medium" symbols referenced above, although we call that "narrow" and allow various levels of detail.
[*Phonemes may also break down for various reasons, such as archiphonemes, a topic that theoretically interests me, but is beyond the scope of this conversation about definitions of allophone which presupposes phonemes.]