"The AI accurately predicted 70% of earthquakes a week in advance, with 14 forecasts coming true… However, it issued eight false warnings and missed one earthquake." Beyond the fact that n is very small, I don’t think 14/23 is 70%…
"A week in advance" is overstating it. It forecast the week of the earthquake, counting a hit in a radius of 200 miles. That is probably exciting for the researchers, but of no current practical value.
"Neither the USGS nor any other scientists have ever predicted a major earthquake. We do not know how, and we do not expect to know how any time in the foreseeable future. USGS scientists can only calculate the probability that a significant earthquake will occur in a specific area within a certain number of years."
It's a simple matter of cascading your rounding operations. 0.636 rounds up to 0.64 of course. That four is almost a 5 so round that up to 0.65. Everybody knows that it's fair to round up from a 5 so this is how they 0.7, but these amateurs forgot to subsequently round 0.7 up to 1. Shameful, really.
The article is so information-lite that a possibility I wouldn't rule out was that of 20 earthquakes in the trial, it made predictions for 14 earthquakes that were close enough to count as a success criterion, it made predictions for 5 earthquakes that were substantially off in location and/or magnitude but not entirely misses, 1 earthquake that it missed entirely, and 8 more earthquakes that it predicted that didn't occur.
Sounds good. In that case, I just made a 100% accurate model by always predicting that every location on Earth is experiencing an earthquake all the time.
> The AI accurately predicted 70% of earthquakes a week in advance, with 14 forecasts coming true within 200 miles of their estimated locations and matching their anticipated magnitudes. However, it issued eight false warnings and missed one earthquake.
The precision was 14/(14+8) (64%) and recall was 14/(14+1) (93%) which means the F1 score was .756. The accuracy was 14/(14+8+1) (61%). I'm not sure where they got the 70% from, perhaps a different F metric. In any event, it's clear the author is confused about the terminology.
There isn't even a number. It's been generally regarded as just flatly impossible using current technologies. Even a partial step towards reliable earthquake prediction would be a massive advancement.
> The AI accurately predicted 70% of earthquakes a week in advance, with 14 forecasts coming true within 200 miles of their estimated locations and matching their anticipated magnitudes. However, it issued eight false warnings and missed one earthquake.
How does this work out to 70%? It made 23 predictions, 14 were right and 9 were wrong. 14 of 23 is 60.8%
I also wonder what counts as a success. If it predicts an earthquake and one occurs, but too weak to hurt anybody, does that count as success or a false warning? What do you do with earthquake warnings anyway, preposition yourself under a doorframe? I guess humanitarian organizations could use this to pre-stage emergency supplies, but if false warnings are common and true warnings don't seem actionable most people will probably ignore it.
I had whole comment questioning the usefulness of week before prediction. But prediction like that would save thousands of lives in my area. It is enough to evacuate the unreinforced masonry buildings. It is enough to evacuate the areas of coast swept by tsunami. It would require plans to deal with evacuees.
I still wonder about the accuracy, is it for specific time? It would also need to be reliable that worth evacuating thousands of people. Finally, how well does it work on different kinds of earthquakes?
If it technically predicts earthquakes but can't tell you which of the earthquakes will be strong enough to warrant evacuation, then will people heed the warning for the big one after they've had a few evacuation notices for imperceptible earthquakes? It's a "boy who cried wolf" problem.
The vast majority of earthquakes are very weak, so predicting the time and area of earthquakes seems worthless if they can't accurately predict the magnitude as well, with very few false positives.
> The proposed algorithm is trained using the available data from 2016 to 2020 and evaluated using real‐time data during 2021. As a result, the testing accuracy reaches 70%, whereas the precision, recall, and F1‐score are 63.63%, 93.33%, and 75.66%, respectively.
> The researchers said that their method had succeeded by following a relatively simple machine learning approach. The AI was given a set of statistical features based on the team’s knowledge of earthquake physics, then told to train itself on a five-year database of seismic recordings.
One common mistake with timeseries forecasting is leaking the evaluation set. It would be relatively easy to accidentally record the future into a DNN.
