------------
Pitch Interpolation Bug
------------
Yesterday I intended to ship out the data to my advisor for review. Before doing so, I carefully looked over the data. I wanted to make sure that there were no issues so I hand checked a small portion of the data. I noticed that the timing was off. For the pitch interpolation, I interpolate not only the pitch but also the time. Although pitch points for one audio file are matched with the corresponding pitch points in another audio file, based on when they occur in the labeled region, the times are not exactly the same, so the interpolation also shifts them.
The final result of this dual interpolation should result in a pitch contour that looks the same as in the target utterance, but in a compressed or expanded space.
Unfortunately, I was doing this interpolation using absolute, rather than relative time. The interpolation equation is:
Pitch Interpolation Bug
------------
Yesterday I intended to ship out the data to my advisor for review. Before doing so, I carefully looked over the data. I wanted to make sure that there were no issues so I hand checked a small portion of the data. I noticed that the timing was off. For the pitch interpolation, I interpolate not only the pitch but also the time. Although pitch points for one audio file are matched with the corresponding pitch points in another audio file, based on when they occur in the labeled region, the times are not exactly the same, so the interpolation also shifts them.
The final result of this dual interpolation should result in a pitch contour that looks the same as in the target utterance, but in a compressed or expanded space.
Unfortunately, I was doing this interpolation using absolute, rather than relative time. The interpolation equation is:
newTime = startTime + ((i)*(targetTime-sourceTime)/float(numSteps-1))
So you can imagine that (targetTime-startTime) could be quite different if one of the audio files had a significant pause at the beginning of the file (time does not start at the labeled region but at the beginning of the audio file). The solution was to make time relative to the region. One could bin it between 0 or 1 (for the start and end of the labeled region) or subtract off the time of the start of the labeled region. I chose to use relative time.
--------------
Duration Interpolation Bug
--------------
I also thought that there was a problem with the duration normalization. The length of the wavefile produced by the final stage of the interpolation should be the same length as the target wavefile--or so I thought. It actually should be the same, minus any silences, which are skipped for the sake of simplicity. Small pauses might appear in one audio file but not another. And they might be important for cuing prominence.
Thinking outloud now: if we were going to add it, it would need to be a post-process. It cannot be a part of psola, in the event that a pause is present in the target file but not the source (psola can't, from what I understand, stretch nothing into something). Also, currently, the script requires a one-to-one correspondence of labeled regions in the source and target files. It would be possible to have the user also indiciate
In sum, I think we should skip this for now, even if it is important. I can't think of an elegant way to incorporate this at the moment.
-------------
Next
-------------
I need to verify the duration interpolation by focusing on just regions. This shouldn't take long. And then I need to make sure the audio quality is ok. From what I have listened to so far, some audio gets interpolated better than others. I will need to listen to it more carefully before passing the data off to my advisor for approval.
No comments:
Post a Comment