Due to our adventurous history with Kotlin Native, we immediately knew that we wanted to implement our whistle transcription app with Kotlin Multiplatform. IntelliJ generated the base project for us, so we could throw Claude at it a few minutes later.
Claude was fast and ruthless: we prompted him to detect notes from whistling, and he happily reported that he was done.
Now that we had created our award-winning application, we were ready for shipping (with maybe a bit of testing before).
If it were that easy, our researched apps might not have flopped so hard! It’s probably obvious to anyone familiar with music theory, but pitch detection is not an entirely “objective” thing. In music, there are generally specific notes that you are aiming for. On the other hand, average people (= us in this case) don’t whistle perfectly in tune. We might hit somewhere between these predefined notes, or start whistling in one tone and slowly slip downward during the song. In these cases, the algorithm has to decide where to lean, where to correct the melody.
Another challenge is deciding how long a note is. Did you just accidentally slip a tiny bit, or are you going all-in on vibrato to sound fancy? Was it noise or was it a whistle? We have to optimize for detecting short sounds without cutting up long notes. If we get it wrong in either direction, we either end up with nothing transcribed or with the “jazzy” melodies that we got from other apps.
As we are quite diligent people, fully committed to optimization and no friends of overfitting, we recorded several samples to find the best parameter values.
Two.
“We might need a few more”, we concluded.
So we recorded a third one.
Okay, getting a nice bag filled with whistle samples and their expected outcomes might be quite time-consuming after all. In the next few days, we will have to come up with a strategy for a faster feedback loop. Let us know if you have any ideas!
And as usual, follow subscribe click the little bell!
