Friday, August 23, 2013

Text that behaves as a checkbox in HTML

I'm working on building a custom web-survey interface.  It is a bit ad-hoc.  I am not familiar with the many web technologies out there, so I just scrap together whatever bits of stuff I can find.

The latest feature request were words that act as checkboxes--users can click on words in a sentence and the words would react with toggle behavior and change visually to reflect being checked or unchecked.

The end result is small and pretty simple but it took me a while to nail down the correct syntax exactly.  For the event handling I used jQuery.  It could have easily have been done without it but the code is a lot cleaner this way at least.

STEP 1:
Make the word technically a checkbox (but without any sort of visual response to clicks).

First the text and checkbox.

<body>
           <label for="1">
                <input type="checkbox" name="some name" id="1" value="whatever"/>
                today
           </label>
</body>

The <label> tag ties together the checkbox and the text.  Note that "for" and "id" need to have the same value.  Now if someone clicks "today" the associated checkbox will be marked.

Next, you need to set the checkbox to invisible and underneath/on top of the text:

<style type="text/css">
           label input {
                position: absolute;
                visibility: hidden;
           }
</style>

That's it!

STEP 2:
Now we want to make the checkbox respond visually when they are clicked.  There is a way to do this in regular javascript but I couldn't figure out a clean way to do this for multiple checkboxes.  Instead, I'm using jQuery

<script>
$(document).ready(function(){
  $('input[type=checkbox]').click(function(){
    $(this).closest("label").css({ color: this.checked ? "red":"black"});
  });
});
</script>

From what I understand, this code is listening for any changes to a state in the checkboxes.  It then looks for the nearest label (each checkbox is encapsulated in a label) and toggles that labels text color between red and black.  You could imagine other, more sophisticated changes but that is the basic idea.

And that is it.  The actual code has more complicated behavior than this but this is the basic idea.

Back from vacation

I meant to post a month ago an update on what I had been working on before I left (and of course, now I have no idea exactly what I was doing).

I had too many things going on before I left though, and ended up not having the time to post.  Anyways, I had an awesome time in Asia over the last month--documented here:
(http://timmahrt-travel.blogspot.com/)

Will hopefully be posting regular updates now.

Saturday, July 6, 2013

Stimuli First-Pass Complete

Yesterday, Friday July 5th I signed off on the stimuli.

----------
Verifying the Data
----------

I've handchecked both the duration and pitch resynthesis output.  The pitch resynthesis is working perfectly.  The first and last iterations correspond to the input and output and the intermediate interpolations are correctly being generated.

For the duration, the timing appears to be off by a small amount (~.001 second) for the whole file, observing the last iteration.  I tried to fix this or find the source of this difference but was unable to.  The difference is small enough that I am willing to overlook it for now.  I do not believe that it will cause any major problem in our experiment.

----------
Data Quality
----------

Both the duration- and pitch- resynthesized data did not sound very good.  The default pitch settings were 75 to 600.  I felt that this must be suitable because that is a pretty generous range.  For a few of the output files it was not bad but for others, the results were unusable.

For my voice, I tweaked these to 50 to 300.  The drop from 75 to 50 was significant.  I tried many values around 300 but did not find a significant difference so I left it at 300.  This greatly improved the data for the duration resynthesis.  I looked at the output for when the resynthesis was word-level and again at phone-level.  I expected the phone-level resynthesis would provide better results but the results were surprisingly indistinguishable in many situations although even more surprisingly it was even worse in some.  (I hand checked the duration tiers to double check that I wasn't just running the same code in both cases.)

So, finally, I am going with word-level resynthesis.  Even still, some of the audio files did not turn out well, so I may need to gather new recordings of similar sentences and then p

As for the pitch resynthesis, the output sounds better but is still tinny.  Paul Boersma commented that this can happen if one tries to flatten the pitch but shouldn't happen if changing the kind of pitch contour.  I played around with the window size but nothing sounded better than the default.  There are no other parameters to the pitch resynthesis, so I don't think there is anything more we can do.  The artefacts are noticeable but I think they're not too distracting.

Paul Boersma's discussion of this issue with pitch resynthesis:
http://uk.dir.groups.yahoo.com/group/praat-users/message/5745

----------
Up Next
----------

I need to figure out exactly how we are going to use these stimuli.  The idea is to do something like AB or AXB analysis.  Subjects try to determine which is a better, closer, or more correct usage of one item or which is more appropriate to use in a given context.  Should be able to report on this soon.

Friday, July 5, 2013

Yesterday's Work: Real final bugsquashing

------------
Pitch Interpolation Bug
------------

Yesterday I intended to ship out the data to my advisor for review.  Before doing so, I carefully looked over the data.  I wanted to make sure that there were no issues so I hand checked a small portion of the data.  I noticed that the timing was off.  For the pitch interpolation, I interpolate not only the pitch but also the time.  Although pitch points for one audio file are matched with the corresponding pitch points in another audio file, based on when they occur in the labeled region, the times are not exactly the same, so the interpolation also shifts them.

The final result of this dual interpolation should result in a pitch contour that looks the same as in the target utterance, but in a compressed or expanded space.

Unfortunately, I was doing this interpolation using absolute, rather than relative time.  The interpolation equation is:


newTime = startTime + ((i)*(targetTime-sourceTime)/float(numSteps-1))

So you can imagine that (targetTime-startTime) could be quite different if one of the audio files had a significant pause at the beginning of the file (time does not start at the labeled region but at the beginning of the audio file).  The solution was to make time relative to the region.  One could bin it between 0 or 1 (for the start and end of the labeled region) or subtract off the time of the start of the labeled region.  I  chose to use relative time.

--------------
Duration Interpolation Bug
--------------

I also thought that there was a problem with the duration normalization.  The length of the wavefile produced by the final stage of the interpolation should be the same length as the target wavefile--or so I thought.  It actually should be the same, minus any silences, which are skipped for the sake of simplicity.  Small pauses might appear in one audio file but not another.  And they might be important for cuing prominence.

Thinking outloud now: if we were going to add it, it would need to be a post-process.  It cannot be a part of psola, in the event that a pause is present in the target file but not the source (psola can't, from what I understand, stretch nothing into something).  Also, currently, the script requires a one-to-one correspondence of labeled regions in the source and target files.  It would be possible to have the user also indiciate 

In sum, I think we should skip this for now, even if it is important.  I can't think of an elegant way to incorporate this at the moment.

-------------
Next
-------------

I need to verify the duration interpolation by focusing on just regions.  This shouldn't take long.  And then I need to make sure the audio quality is ok.  From what I have listened to so far, some audio gets interpolated better than others.  I will need to listen to it more carefully before passing the data off to my advisor for approval.

Wednesday, July 3, 2013

Today's Work: Final Bugsquashing

Today I reviewed the results of the pitch interpolation.  The results seem ok from one iteration to the next but I stumbled on a few issues when it comes to edge cases.  The code worked fine in the general situation but would crash if only a single sample point was used (which occurs when the pitch tracker does not detect much pitch within a word or for very short words ('a' or 'the').  This issue has been patched.

I discovered this issue when debugging a related issue.  I found that all of the alignment between a source and target file were off by one in each labeled textgrid region.  Let me illustrate with an example. Suppose the word 'car' in the source utterance had 5 pitch samples and the word 'car' in the target utterance had 3 pitch samples, the code would output the source utterance with 3 pitch samples and the target utterance with 2.  It was always the case that the smaller of the two would lose one value (which is incorrect) while the the larger of the two would be scaled down to the size of the smaller (which is correct).

I couldn't figure out why this was, so I rewrote the fairly small function.  It is working now.

I was able to generate all of the interpolated pitch data.  I should review the results to make sure that there are no more errors.  Then I can ensure that the results are suitable for the experiment and then begin working on setting up the experiment.

Thursday, June 27th - Tuesday, July 2nd


------------
Thursday, June 27th
------------

I selected the audio files that I would use out of all of the stimuli that I collected.  I then began writing phone and word-level textgrid annotations for them.  The process was too tedious and taking too much time, so I used the penn forced aligner to generate the textgrids.  Then all I needed to do was to fine-tune the textgrids.  I finished most but not all of the textgrids by the end of the day.

------------
Friday, June 28th
------------

Not a very productive morning.  Got an email from the IRB representative in the early afternoon that my ethics training was out-of-date.  I did the ethics training which took a full 5 hours. After that I had headache and I called it a day.

------------
Sat - Sun
------------

Finished one or two outstanding modules in the ethics training.

I intended to work on the textgrid annotations but I ended up not having a productive weekend.  Next weekend I need to go to the office.

------------
Monday, July 1st
------------

Finally finished up the textgrids.  I went to generate the stimuli.  No problem with the duration resynthesis but the pitch resynthesis crashed.  The bug was nestled pretty deep but I had actually previously written a comment in the section where the bug originated.

The problem was that I was taking a constant number of pitch samples from every labeled region in the textgrid.  This works fine as long as there are enough samples.  However, the rest of the code assumes that there are a constant number of samples for each region.  If there are more samples in the target audio than the source audio, no crash occurs.  However, if there are more samples in the source audio than the target, then the code tries to map those last few samples to samples in the target audio that don't exist and the script crashes.

In either case, samples are mapped to non-corresponding samples, which is not what we want.

As a result, I changed it so that the pitch samples, and their time stamps, extracted from each region are stored in isolation until it is time to resynthesize, at which time they are pooled together (unlike the original code, where all samples were never separated out).

Additionally, in each region, for the source and target audio, I find the number of samples in both and pick whichever is smaller.  I take this many samples from both.  For the one with a larger number of samples, I sample at evenly spaced intervals such that the endpoints are preserved.

Before the pitch samples in the source and target audio are lined up, pitch values of 0 are removed.

After finishing with the code I finished for the day.  However, the code still needs to be tested and run on the rest of the data set.  I will pick a few random samples from both the duration and the pitch data and ensure that the interpolation is occurring as expected.  Auditorily, it seems to be doing the right thing, but this needs to be verified sample-by-sample.

------------
Tuesday, July 2nd
------------

On account of rain I ended up staying at home and having a not-very-productive day.  I need to go to the office every day, even if that means taking the bus on rainy days.


Thursday, June 27, 2013

Processing Audio

---------
Audio Files
---------

Yesterday I began looking at the audio files that I had collected a few weeks ago.

In my original audio recordings, I had two sessions.  While post processing these I originally normalized the volume on an utterance-by-utterance basis.  Of course, for my purposes, this is incorrect as it throws out all of the relativistic intensity values that we might utilize.  So I had to go back to the original two recordings, sound-normalize those and then chunk them into individual wave files again.

-----------
Resynthesis Code
-----------

I also worked with the code a bit.  The code for duration and pitch resythensis is complete but I am still trying to figure out what the output should be.  By that I mean, we started with utterance vs word level resythensis.  Today I tried phone level resynthensis.  It was brought on by a particular issue with 'bobby'.  In one broad production it came out something like [bavi] however, the [v] was not particularly salient--until I synthesized it into the contrastive focus context.

I thought that this was an artefact of the resynthesis code but it is not.  This became apparent when I tried phone-level resynthesis.  The phone-level resynthesis is only working for duration.  Because in pitch resynthesis we sample a constant number of points per sample region (something that should be changed anyways) phone-level resynthesis may not work for pitch.  We should instead to change from a set number of samples to time-based sampling.

(As an aside, it is interesting how one particular variant of a consonant ([v] for /b/) is perceived in a focused vs broad condition.  Also, it is interesting how there are pops and clicks in our speech that we don't even notice.  In all of my productions of "Bobby" there is a click half-way through that I cannot hear when I produce it (unless I am simply unable to reproduce it now?).  I'm not sure where it comes from.)

I don't think this issue is worth dealing with now, so for this first experiment we won't worry about phone-level resynthesis.  The question remains as to what to do for duration.

Next, my job is to select the audio stimuli that I will use in the experiment, generate word and phone-level textgrids, and create the hypothetical stimuli for use in this experiment.