I don't know of anything particularly convincing.
There was a lot of talk following the paper a couple years ago in Science that discussed how languages have smaller phoneme inventories as they move away from Africa, with Khoisan representing an earlier state. I'm sure you can google that to find the reference. Searching for something like "phoneme inventory size and population size" brings up relevant information.
I saw a talk at the LSA conference a couple years ago that was about how environmental factors can affect perception of sounds, such that dense jungle environments are very different than high snowy mountains or wide grassy plains. The correlation was slight but apparently supported statistically.
I'm pretty skeptical of all of this, though.
As far as I'm concerned, this seems like just one more aspect of the arbitrariness of language. (Note that sounds aren't even really fundamental units, considering sign languages!)
As for Hawaiian, that's true. And it may be for just part of a system, such as the size of the vowel inventory. Arabic has a fairly complicated phoneme inventory, but the vowels are relatively straightforward (3 long, 3 short). Still, I never was able to get the vowels right, because they seemed to move around so much phonologically (or in free variation?). So I was intelligible (everything else being equal) but far from native sounding in my vowels.
A really interesting comparison is with Pirahã as described by Everett. The 7 (female) or 8 (male) phoneme inventory is (one of?) the smallest in the world, at least segmentally. There are also tones. The language can be spoken, yelled, hummed, and even whistled. The tones help a lot, but the main point is that the phoneme inventory is tiny and allows huge variation while maintaining intelligibility.
You can hear some discussion from Everett in this video:
https://www.youtube.com/watch?v=sYpWp7g7XWUThe video is interesting in itself (monolingual elicitation methods for fieldwork!) but you can skip to about 45 or 50 minutes to hear the discussion. I can't remember exactly when that question came up, but he discusses how completely inconsistent the Pirahã were in producing the same word on different occasions. Fascinating.
At a theoretical level, I wonder also what's going on with actually counting phonemes. There are some major (if overlooked) theoretical decisions about what exactly counts as a phoneme or segment:
--Are prosodic features phonemes? Or do they multiply out the number of phonemes? If Chinese has 6 vowels and 4 tones, then does it have 24 phonemes? 10? 6?
--What about complex articulations? Are affricates one phoneme or two?
--What about long vowels? Is there a length phoneme or are there twice as many vowels? Same with nasalization.
--What about systems with lots of dimensions of contrasts like voiceless-aspirated / voiceless-unaspirated / voiced-unaspirated / voiced-aspirated (like Hindi)? Are those just linearly more phonemes or does the predictability matter? Khoisan languages with something like 150 phonemes tend to do so in very predictable ways with features like nasalization on the clicks.