By the way, this is written by the sixth-place winner of last year's Kaggle Competition for detecting Whales. [1]
I think another very interesting point brought up was when that even such a well-ranked model on Kaggle did a poor job when applied to an different dataset and had to be retained, which is a nice example of over-fitting.
This second challenge actually featured a different dataset with different hydrophones used etc. But even without retraining (which was rather trivial to do at that point; the hard work of finding the right hyper parameters had already been done), I would have still scored well above 90%. And I think Nick Kridler reported the same.
So overfitting yes, but not too much considering there was a different sensor.
Overfit may be too strong of a criticism. If you spend your life only seeing 2's and one day I show you a 3 and you can't tell it's different (you think it's just a poorly written 2), are you overfitted to 2's or did you just not have enough varied experience?
I guess it is technically overfitting, but overfitting sounds wrong when you didn't have access to the extra data in the first place (or even realize you were being given pre-cleaned-up data).
Solution: the first de-noising done with actual noise?
The work is great though. More of this and less bitcoin, godaddy announcements, and SV gossip politics on the front page, please.
I have no idea if it's a common technique or not, but a couple years ago I met some guys similarly using image analysis of spectrograms. They were trying to diagnose sleep apnea based on of heart rate data. They had a company developing a device and claimed to have patents on the technique, but I forget the name. I just remember that it seemed like a convoluted algorithm to me, and they agreed, but they claimed it worked better than any traditional approach they tried.
The nice thing about these convolutional neural nets is that they're not convoluted at all. ;-) It's basically feed the raw data, in this case spectrograms. Traditional approaches in this field are usually much more convoluted, because they involve a complex feature extraction part. Which tends to be specific to a certain species.
From the perspective of other source data, I wonder if that limits you to five features (X,Y and RGB) or whether you could extend to fictional/non-human-visible colours as extra features and just be unable to view them in the weight maps.
I think another very interesting point brought up was when that even such a well-ranked model on Kaggle did a poor job when applied to an different dataset and had to be retained, which is a nice example of over-fitting.
Excellent article and nice details, thanks!
[1] http://www.kaggle.com/c/whale-detection-challenge/leaderboar...