Linguist Forum
Specializations => Phonetics and Phonology => Topic started by: zaba on July 14, 2014, 09:49:57 PM
-
In layman's terms (to the extent possible), how does Praat calculate pitch? It occurs to me that I have a wonderful time calculating intonation, but without an idea of what's going on behind the scenes, so to speak, I'm putting an awful lot of faith into this program... curious what's going on ! Thanks.
-
As far as I know it tries to locate F0, which should be within a particular range. I assume it uses some heuristics then finds an average across an interval. I'm not sure exactly what the algorithm is (but you can certainly look that up). Does that help at all?
Essentially, if you find the large band of voicing near the bottom of the spectrogram, that's pitch. Praat does the same thing mathematically.
-
So pitch is f0 and/or voicing?
-
It's really a misnomer for Praat to call it "pitch." What Praat calls "pitch" is f0. These are not the same; pitch is a psychoacoustic phenomenon, f0 is a property of periodic signals. Creaky voice is often perceived as low in pitch, but it has no periodicity at all.
Praat's f0 estimates are based on autocorrelation (http://en.wikipedia.org/wiki/Autocorrelation). At each time point in a signal, an autocorrelation looks back in time over a range of lags. The lag that achieves the highest correlation is the wavelength of the fundamental frequency. You have to specify how far back you are willing to look; these provide hard bounds on the highest and the lowest fundamental frequency you can find. This is why Praat requires you to provide the pitch range.
Note that this is not the same as looking for a voicing band in the spectrogram and finding its peak. That technique will fail if the voicing band does not exist. Telephone speech does not include energy below about 400 hz; there may be a lot of noise at low frequencies; and so on. Autocorrelation can still recover the fundamental frequency by relying on the harmonics. Even if the fundamental is gone, the signal will contain the harmonics, and they will all hit their peaks when the fundamental would have hit its peak (in the time domain) if it were there.
-
Thanks for clearing that up. I had the suspicion just based on my analysis of the spectrogram that pitch was really fundamental freq.
Here's a crazy-ish question: Is there a script that automatically recognizes e.g. rising/falling f0/pitch?
-
There certainly are scripts for doing things with f0, but you'll have to define your problem more precisely. Do you want to find every place the f0 increases? Do you want to find the largest increase in an utterance? Do you want to decide whether the overall f0 contour went up or down in a given syllable? All of these require different approaches and different amounts of annotation. You probably won't find a script for exactly your needs that runs out of the box. However, you can look at the scripts by Pauline Welby (http://aune.lpl-aix.fr/~welby/pages/praat.html) and Christian DiCanio (http://www.haskins.yale.edu/staff/dicanio/scripts.html) for good examples and ideas.
-
Excellent tips for the scripts. Thanks a lot for that.
Humour me for a second (you're used to that by now) -- if I wanted to make a script that recognized e.g. polar interrogative formation pitches (e.g. rising pitch), would that be possible?
Would it be possible to do that in real-time? I know, crazy... but just curious.
-
This sounds like it could be a pretty major undertaking. Pauline Welby has a pipeline of praat and R scripts for detecting F0 elbows, but that doesn't necessarily get you all the way to recognizing polar interrogatives. How clean is your audio? Do you need to distinguish a polar interrogative boundary tones from L+H* pitch accents?
-
Well, I think/hope/believe that it's only a major undertaking if you assume I need 100% accuracy and no false positives / true negatives. Let's see I'm tolerant of both and I don't care about distinguishing polar interr boundary tones from L+H* pitch accents. In fact, all I care about is finding rising f0 phrase-finally. Plausible, no?
-
Is the system going to be told where the ends of the phrases are? Finding the ends of phrases automatically is not easy.
-
no. basically in its simplest iteration it will not. it'll simply be a "rising f0 indicator", and then get tweaked from there.
-
no. basically in its simplest iteration it will not. it'll simply be a "rising f0 indicator", and then get tweaked from there.
Given the meager coding accuracy that this will produce, I have to say that I'm very skeptical this kind of automation is going to make things any easier than just pounding through all the data by hand.
-
I agree with MalFet. F0 can increase for many reasons, including pitch accent (even if you aren't interested in finding pitch accents, they're a confound), the inherent noise in the autocorrelation algorithm, boundary tones for list intonation, cues to voicing contrasts in neighboring segments (http://web.ics.purdue.edu/~francisa/Articles/Francis-etal_JASA06b.pdf), and so on. Simply tracking f0 is not likely to provide a strong signal about intonation, especially if you don't have strong constraints on what the talkers can say or advance knowledge about where the phrase boundaries are.
-
Right, I agree with your misgivings. Fair enough. But I think if I ask speakers to say just one sentence pair in my uni lab setting, e.g. "The dog wears a hat" and "The dog wears a hat?", the algorithm will select which one is the polar question. Obviously in natural speech it won't work so well because of interruptions, etc.
-
Can you write an algorithm that will identify which of two audio clips has the largest positive change in pitch near the end? Sure. If you've got that much control over your data, I'm not sure why automation is necessary/helpful, but there's nothing to stop you from doing it.
You'll still get foiled by things like phrase final devoicing, but under very ideal conditions what you're describing should work some of the time at least.
-
It's really a misnomer for Praat to call it "pitch." What Praat calls "pitch" is f0. These are not the same; pitch is a psychoacoustic phenomenon, f0 is a property of periodic signals.
Very nice observation! Thank you. That really clears things up.
Praat's f0 estimates are based on autocorrelation. At each time point in a signal, an autocorrelation looks back in time over a range of lags. The lag that achieves the highest correlation is the wavelength of the fundamental frequency. You have to specify how far back you are willing to look; these provide hard bounds on the highest and the lowest fundamental frequency you can find. This is why Praat requires you to provide the pitch range.
This description looks very interesting - but I'm not sure what an "autocorrelation" is and what it means for f0 to estimates to be based on them. If you have a moment, I'd be so grateful if you could elaborate on this. I looked online and through several other linguistic resources, but to no avail. Perhaps its a technical term unfamiliar to my old dictionaries.
-
Praat's f0 estimates are based on autocorrelation. At each time point in a signal, an autocorrelation looks back in time over a range of lags. The lag that achieves the highest correlation is the wavelength of the fundamental frequency. You have to specify how far back you are willing to look; these provide hard bounds on the highest and the lowest fundamental frequency you can find. This is why Praat requires you to provide the pitch range.
This description looks very interesting - but I'm not sure what an "autocorrelation" is and what it means for f0 to estimates to be based on them. If you have a moment, I'd be so grateful if you could elaborate on this. I looked online and through several other linguistic resources, but to no avail. Perhaps its a technical term unfamiliar to my old dictionaries.
The place to look for more information on this is not linguistics but digital signal processing, or, for a more cognitive/artificial intelligence perspective, computational audition. To understand autocorrelation, we first need to define a few terms, the most important of which are discrete-time signal and cross-correlation. A discrete-time signal is a sequence of numbers, called samples, that encodes a signal. For speech, each number expresses the air pressure at a particular time, and the discrete-time signal is a discrete approximation to the continuous speech signal. The sampling rate is the number of samples per second. The sampling rate is usually set fairly high to provide a good discrete approximation to the continuous signal. So, ultimately, a discrete-time signal is just a very long sequence of numbers.
The cross-correlation of a signal [latex]x[/latex] with a signal [latex]y[/latex] at a lag of [latex]\tau[/latex] is defined as:
[latex](x\star y)( \tau ) = \sum_{t=0}^{\infty} x(t) y(t-\tau) [/latex]
Remember that [latex]x[/latex] and [latex]y[/latex] are just sequences of numbers, and [latex]x(t)[/latex] is just the [latex]t^{\mbox{\scriptsize th}}[/latex] number in the sequence of numbers called [latex]x[/latex]. Because we are multiplying the signals together, this sum will be large when [latex]x[/latex] and [latex]y[/latex] are similar at a particular time lag. (if you are comfortable with linear algebra, note the similarity between this definition and the definition of a dot product.)
An autocorrelation is a cross-correlation where [latex]x[/latex] and [latex]y[/latex] are both the same signal, call it [latex]h[/latex]. An autocorrelation asks how similar a signal is to itself, at a particular time lag. If [latex]h[/latex] is perfectly periodic, with fundamental frequency [latex]f[/latex], then we have:
[latex]h(t) = h\left(t + \frac{1}{f} \right) = h\left(t - \frac{1}{f} \right) [/latex]
for all [latex]t[/latex]. (remember that frequency is the inverse of period). So a periodic signal is maximally similar, maximally correlated, to itself at a lag corresponding to the wavelength of the signal's fundamental frequency. Real speech is not perfectly periodic, because it changes over time, and the vocal folds don't maintain perfectly constant tension, but the same basic principle applies. We can get a pretty reliable estimate of the f0 by searching for the lag that produces the highest correlation over a restricted range of wavelengths corresponding to feasible f0.
I said Praat's approach was based on autocorrelation because Praat does more work to get more reliable estimates. The method just described assumes the signal is periodic (i.e. it never says that there is no f0), and is not specialized to the characteristics of human speech. For the first point, Praat incorporates a system of heuristics to decide when the correlation is too weak to have an f0. For the second point, Praat uses the autocorrelation to return multiple f0 candidates at each time, and then finds a smooth sequence of estimates to avoid implausibly large and jagged jumps in the f0 trace.
So the overall approach is something like this:
- Decide to estimate the f0 at time [latex]t[/latex].
- Extract a window of the signal around time [latex]t[/latex], producing windowed signal [latex]h[/latex].
- Find a series of f0 candidates [latex]\tau[/latex] that are local maxima in [latex](h \star h)(\tau).[/latex].
- Decide whether there is any f0 at time [latex]t[/latex], and compare the f0 candidates to the f0 candidates of nearby time points.
For the complete algorithm, have a look at Boersma (1993) (http://www.fon.hum.uva.nl/paul/papers/Proceedings_1993.pdf). It's a pretty old paper (by technology standards) but it doesn't look like much has changed, based on the options in the To Pitch (ac) window.
-
Wow, thanks a lot for the very thorough and detailed answer. Much obliged!
-
By the way, how does one best "eyeball" pitch in a spectrogram? Is there a way to do this, just to get a rough impression
-
The height of the dark band of voicing near the bottom. But eyeballing it probably won't distinguish anything but the most extreme contrasts.
-
So the overall approach is something like this:
Decide to estimate the f0 at time .
Extract a window of the signal around time , producing windowed signal .
Find a series of f0 candidates that are local maxima in .
Decide whether there is any f0 at time , and compare the f0 candidates to the f0 candidates of nearby time points.
@jkpate: Do you think I can do this with "least squares method"?
http://en.wikipedia.org/wiki/Least_squares
-
Do which part? Least squares is just an approach to regression. Which variable would you be regressing against which other variable(s)?
-
lesastsq function in python minimises the sum of squared components of the array returned by function foo([k,m]). It does this by altering values of parameters k and m (of the straight line equation k*x + m). Then, foo returns a difference between each f0 value and a straight line by f = k * t + m for that time, t. Thoughts?
-
Right, but I don't quite see how you want to use a linear regression. Are you wanting to use this to replace the step that compares f0 candidates to their neighbors to produce a smooth f0 trace? Or are you wanting to characterize the final output of Praat's approach by using the parameters of the best-fit line? (i.e. k and m in your example)
-
I want to be clear; I'm genuinely asking what you have in mind, I'm not suggesting or endorsing either idea. The first one is a Bad Idea. For example, in "easy" cases, the top f0 candidate will be the true f0, and the second f0 candidate will be the first harmonic. In such a case, the linear regression will split the difference, and decide that the f0 is 50% higher than it actually is. More generally, the f0 candidates are not random samples of some distribution centered on the true f0, but a linear regression would assume they are. Praat's approach of finding a best path that penalizes sharp jumps is a much better strategy. The second one is more reasonable, but you might need to go beyond a simple linear regression (again, depending on exactly what your goal is). A linear regression will not distinguish between an f0 trace that drops and then rises and an f0 trace that rises and then drops, for example. Coming up with the right amount of freedom in the functional form is not straightforward
-
Before I elaborate, let me ask a question which may belie a surprising degree of ignorance:
How is the best candidate for f0 (AKA pitch) in praat calculated? I know that there is some kind of "goodness measurement" (what a strange term!) but am fuzzy on how that is determined. Is this something you know about?
-
How is the best candidate for f0 (AKA pitch) in praat calculated?
That's exactly what the autocorrelation function jkpate described in Reply #16 does.
-
I think zaba is asking about the Praat implementation around autocorrelation, which uses the autocorrelation to get a set of candidates at each time point, and then, in step 4 above, compares those candidates to each other to get a smooth f0 trace. Praat uses a lowest cost path algorithm, described on pages 105 and 106 of the paper I linked to.
At each time point t, there will be n candidates, each with a correlation score from the autocorrelation. We can draw a line from each candidate at time t to each candidate at time t+1. This creates a lattice, and any path along these lines (that always moves forward in time) corresponds to one f0 trace by choosing one candidate at each time. We say that traveling along a line from a candidate at time t to a candidate at time t+1 is a transition. We can furthermore associate each transition with a cost, so that a transition from a very low f0 to a very high f0 at the next time point has a high cost. Paths with lower cost will then be smoother than paths with higher cost. Finding the lowest cost path in this kind of lattice can be solved easily with a well-known algorithm called the Viterbi algorithm (http://en.wikipedia.org/wiki/Viterbi_algorithm) (The wikipedia article implies that the Viterbi algorithm requires the costs to be probabilities, which is not true. The probabilistic setting is just the most common use case for Viterbi). Praat also includes "no voicing" as one of the candidates at each time step, and assigns a high cost to transitions between one of the f0 states to one of the unvoiced states. This voicing/devoicing cost prevents Praat from interspersing very short f0 stretches with very short voicing stretches.
-
Thanks. Must think before replying. You've given me a lot to chew on.