The reason I say "if" and hedge my words is because the conventional wisdom is that autism is not a single disorder with a single underlying condition. It is a cluster of symptoms of varying intensity and it is (likely) caused by a host of underlying conditions.
Two children diagnosed with the disorder can have no overlapping symptoms.
If that is right, then the first step is to break it down into different kinds of conditions. (No eye contact is caused by X. Speech issues are caused by Y).
However, if this test can effectively pick apart one of the underlying conditions and its symptoms that would be a huge step forward. We could definitively say whether a child has this particular version of "autism".
After we tease apart a few more versions, the original condition will disappear and be subsumed by these other versions.
I suspect the reason autism has such a constellation of symptoms isnot because the cause is complex, but because the environment (human brain and body) is complex. Imagine there was "cardboard box spectrum disorder", caused by keeping a child in a 4x4-ft box for their first 5 years. They'd develop a host of comorbid symptoms, in varying intensities, based on how the child developed coping mechanisms and workarounds to it's confinement, even though the actual cause is quite simple.
For autism, the "cardboard box" is something to due with the pattern of connections in the developing brain, and pattern has wide ranging impacts on many brain functions, which then influence physical functions , which cause feedback loops back to and inside the brain.
Case in point: girls and boys are diagnosed with autism at wildly different rates. But how much of this is due to actual differences in the incidence of autism versus the differences in the way girls and boys are socialized resulting in different appearances of symptoms?
This is true of many behavioral and cognitive conditions. The symptoms and severity of things like schizophrenia vary significantly from culture to culture (in some cultures schizophrenics hear benign or even kind voices versus here in the western world where the voices tend to be harsh and violent).
Good point. However, this example could go either way.
Suppose everything you said is true, but in addition to all that, there is a set of symptoms caused by -- making this up -- endocrine disruptors in our drinking water. When pregnant women drink too much, they get some subset of symptoms from a set that overlaps 80% with the cardboard box symptoms.
Now we have two different causes for different symptoms. In addition, you can get the strange case where:
patient A, symptoms X, condition G
patient B, symptoms X, condition H
patient C, symptoms Y, condition G
patient D, symptoms Y, condition H
Although most of the time, condition G will have similar symptoms and condition H will have similar symptoms.
> Two children diagnosed with the disorder can have no overlapping symptoms.
Is there a precedent for this with other disorders? Seems to me that if there are no overlapping symptoms, it should be a separate disorder. Even if it's just arbitrary naming like "Type X" as in diabetes.
This might not be the most authoratative source, and I'm no medical expert, but this overview from Encyclopedia Britannica[1] nicely describes the diversity in how diseases are classified (I know we aren't using the term "disease" here but the concepts are presumably related in practice - I found this summary informative but welcome any informed corrections):
> The most widely used classifications of disease are (1) topographic, by bodily region or system, (2) anatomic, by organ or tissue, (3) physiological, by function or effect, (4) pathological, by the nature of the disease process, (5) etiologic (causal), (6) juristic, by speed of advent of death, (7) epidemiological, and (8) statistical. Any single disease may fall within several of these classifications.
> Seems to me that if there are no overlapping symptoms,
The full name is "Autistic Spectrum Disorder", so the name hints that there's a range of stuff happening.
To be autistic someone has to have problems with social communication, problems making or maintaining friendships, and fixed and repetitive interests.
Some people also have other stuff on top. These things are common with autism, but are not needed for the dx.
Alexithymia (the ability to recognise emotion in yourself and others) is one example. It's far more common in autistic people, but you don't need it to be autistic. Between 50% and 55% of autistic people have alexythimia. Sensory sensitivities are another. There are a range of these things that are more common in autistic people, but aren't needed for the dx.
And there's a lot of co-morbidity too. People with autism are more likely to have depression or anxiety or OCD. These aren't part of autism, but it's complicated to untangle what's going on. Is someone social isolated because they're depressed, or autistic, or is it a bit of both?
When you start looking at these other things it makes sense that autism might be an umbrella diagnosis.
In the DSM 5, "Pervasive Developmental Disorder - Not Otherwise Sepcified" (PDD-NOS) and others (Asperger Syndrome) got refactored into a single unified Autism definition.
It happens because there is so much overlap in most patients, and symptoms tend to occur in clusters.
General delays in motor skills and language skills are quite prevalent in many conditions. If you google videos of kids with autism, you will see varied abilities.
I am hesitant to go into more detail because this is an area that causes a great deal of anxiety for parents and discussions are often sensitive. One surefire way to annoy a parent and get into an argument is to claim their kid does or does not have a condition that differs from their own opinion. (And I can see why it would bother them).
> Is there a precedent for this with other disorders? Seems to me that if there are no overlapping symptoms, it should be a separate disorder.
Yes, this is a pretty normal way of doing diagnoses.
Specifically, many disorders are diagnosed according to the template "the disorder is present if the patient presents any X out of this list of Y symptoms". As long as X is less than half of Y, it's possible for two patients to "have" the disorder without sharing any symptoms.
"Have" is in scare quotes because, obviously, this state of affairs is an artifact of the diagnostic criterion. However, it's also possible for e.g. two people to host infections of HIV without sharing any symptoms -- one may have AIDS while the other is asymptomatic.
Most believe that there is a common source for varying diagnoses like ADD, ADHD and high functioning Autism. What that source is, is not yet known but people have hypothesized about certain genes causing it, or changes in the prenatal environment.
The disorder is much more prevalent in some families than others and even in some ethnic groups which strongly suggests that there are genes involved. One theory suggests that the genetic material came from when Homo Sapiens interbred with Neanderthals.
Autism is in some ways very similar to homosexuality. There is no one true test for it, but if you "look for the signs" you can "diagnose" even very young children. It implies that autism, like homosexuality, can't be cured. People with the disorder have to live with it and those around them have to adapt because they can't change themselves.
Even if diagnosis works perfectly, a brain scan tells you almost nothing about how to respond. The one benefit is to focus early intervention efforts on parents who will likely need ASD parenting skills in their child's next few years.
>The algorithms predicted a clinical diagnosis of ASD with high specificity, sensitivity and positive predictive value, exceeding 95 percent at some ages.
More about the metrics you care about[1]
Edit: Many people in this thread are talking about bayesian stats that it appears they don't full appreciate or understand. They're saying that 95% statistical accuracy is commendable. 95% sensitivity and 95% specificity aren't good enough to use in broad tests. Why? Autism has a 1/68 likely hood[2]. Meaning if you had a sample of 100 general-population people, tested them with this test, the likely hood of someone who tests positive for the test is actually positive (positive predictive value) is a measly ~20% (that is Probability that you have the condition given you test positive). Play around with these more at the following app:
https://kennis-research.shinyapps.io/Bayes-App/
Thanks for writing this summary. Plenty of people get the wrong idea when it comes to test accuracy for topics like health diagnosis.
In the case of this particular topic it does seem like the outlined test could be another tool that doctors could utilize. If for instance a child has shown a change in developmental milestones then that observation comes with it's own (somewhat doctor specific) sensitivity and specificity. That information could be combined with the EEG test to improve the overall doctor+test accuracy. Nothing's going to be perfect, but the outlook is a bit more positive than presented in your example.
>Nothing's going to be perfect, but the outlook is a bit more positive than presented in your example.
Ideally what would happen is that the doctor would use their judgement to narrow down the candidates who the test is applied to who have strong priors of Autism. That would substantively increase PPV. You'd need a ~50% prevalence before you get to 95% PPV
Thank you for this comment; it has been an eye-opener for me. I always eyeballed those statistic and assumed 0.95 is good; now I've realized that it's not specificity, but (100%-specificity) that matters.
Given that diagnosing Autism is hard and that all previous attempts at diagnosing it in a non-psychological way have failed this would be an extremely surprising result.
Call me skeptical until it's reproduced independently.
I was taking this with a hefty grain of salt, though at least on first pass they seem to have a reasonable cross-validation/testing procedure (which is usually one of my major complaints with comp-neurosci papers). As per 3 month infants, you can see that their classifier does have difficulties at that age bracket, though performance seems to start to saturate around 6 months (table 5).
It would be interesting to give the paper another pass to see how more operational data collection could impact the quality of the data and thus classification results. EEG can be really hit-or-miss with different equipment. More-so with simple features such as the band power ones used within this paper.
>"at least on first pass they seem to have a reasonable cross-validation/testing procedure (which is usually one of my major complaints with comp-neurosci papers)."
It looks like the usual overfitting the cv to me... They had 1000 features, 200 datapoints, tried out "several different algorithms".
That doesn't really diminish the results in my book. If you're trying to publish something it's basically assumed that you're going to try out several methods and show the one with better performance even if the performance difference is not statistically different.
As per the number of features and data points: In this field you generally don't have a ton of subjects to sample from and the high dimensional features are a natural result of the array based recordings. It should be possible to perform dimensional reduction on the data, however the ML methods are already implicitly doing that step so it's not necessarily that important.
My normal gripe is when the tested subjects have some data in the training fold and some data in the testing fold (even if the data points are separate). In those cases then the ML method can fit the statistics of a particular subject rather than the true target class (e.g. target movement of a cursor in a BCI). In this paper they explicitly are testing on a subject which was never trained. So, even though the data + particular supervised layer is going to budge the results around some, it should not be a night and day difference from what's expected in reality.
You have to account for the fact that you used so many algorithms, though. Using ten different algorithm makes it ten times more likely you'll fit your dataset well just by chance.
I acknowledge that the reported accuracy of a system will be higher if you take the max accuracy of 10 methods which have the same 'true' accuracy +- some noise. The results presented in table 5 of the paper are very unlikely in my opinion (as someone currently in the ML field and who has worked in the field of computational neuroscience) to be solely due to randomly trying different ML techniques without the underlying data providing a noteworthy difference between the target classes.
If these results are replicated independently with a different dataset then the magnitude of the overselling of the method will be seen. I just don't think that it makes sense to doubt the results (i.e. with a grid of EEG sensors and bandpower features it is possible to identify a portion of autism cases) based upon this factor alone.
5 % is quite a lot of misdiagnosed babies if this is implemented as a mass screening activity, 2-3 times higher than than what the internet seems to think is the actual ASD incidence, and I imagine that that number includes a lot of "highly functional" ASD cases
What should a parent do when this happens? It will be only perhaps 20-30% risk that the baby actually do have ASD and not just a false positive.
I imagine that ASD "prevention" is mostly behavioural training [I have no idea at all actually] - but how much time and effort would that take? What are the consequences for healthy babies? I imagine that most people would spend a lot of effort on anything that could help in cases like this.
It's a bit problematic since it's not possible to know until after a couple of years if it's was a false positive or not. It might turn out that a lot (or most) of all successful recoveries was in fact "false positives".
What? 95% accuracy is outstanding. I see that you're saying if you said "No" for all babies, that'd be 95%, so that's a legitimate point.
However, it sounds like it's better than that "We were also able to predict ASD severity, as indicated by the ADOS Calibrated Severity Score, with quite high reliability, also by 9 months of age."
> 5 % is quite a lot of misdiagnosed babies if this is implemented as a mass screening activity
No, because “screening” and “diagnosis” aren't the same thing.
It might be a lot of children identified for diagnostic follow-up that isn't strictly necessary, though, but that may not outweigh the early support and assistance for those who end up being correctly diagnosed earlier because of the screening.
What do you mean a false positive won't be detectable until a few years? I think by age 1 a parent will notice? Lack of eye contact, altered walking mechanism (foot dragging, tiptoes), non-responsiveness to voices, etc. So maybe a positive test means the kid gets more attention for a few months, that never hurt, right?
There has been recent research showing that part of the issue with asd is related to the glutamate system, specifically NMDA. Some sort of genetic transcription problem. Sorry I can't quote the exact paper off hand.
There's some really early eye movement based detection too. Good to see this. As others point out it isn't a binary but a hugely variant spectrum, but there are some commonalities that at the very least suggest that a child receive further screening or prepare the parent to watch for other symptoms. Regardless of the type and spectrum position, a kid with an early diagnosis/warning seems much more likely to have a good outcome than one that gets one while in pre-school or kindergarten.
I don't understand why there's such a massive focus on autism research, both on HN and in the news in general. It doesn't seem like a pareto optimal use of attention resources.
Because with early development, people on the spectrum could function better in the neurotypical environment, which means more people with ASD can work and need less support later.
Yes! The most effective treatment is behavioral therapy and the results are better if started early. At young age the brain is very plastic and for light cases you can teach all the things that are innate to neurotypical kids.
Agreed. That was my first thought, too. Part of the reason that these silly vaccine/autism conspiracy theories are hard to shut down is the fact that Autism is harder to detect pre-vaccination, so there's confirmation bias here among parents of Autistic children.
If this study provides nice evidence to the Anti-Vax crowd that Autism can be measured and detected well before vaccination age, this might help take some of the wind out of the sails of the movement.
Of course, for many, science won't help, much like usable retroreflectors will only break down the fantasy for a subset of moon landing conspiracy theorists, but if it gets brought up even once in a Whole Foods somewhere, they've done a good thing.
In the other comment thread that got shadow-banned, someone pointed out that there are several vaccinations recommended by 2 months of age [1]. I remember my child getting the first one before leaving the hospital (HepB [2]).
Well, first, you have to believe that the EEG's predictor model is accurate and trustworthy.
You have to trust those prescribing it, which, if you don't trust the advice of a doctor with a needle, would you trust them with a brain scan that labels your child defective, before they can even talk?
Some people (those who might consider skipping vaccinations) might avoid such an exam entirely, and choose to wait until their child is age 10 or 15, to decide whether they have some sort of problem with their social skills, or worse.
People adjust for perceived risk. For example, seatbelts and airbags make people less careful when driving.
Vaccination makes people less careful about disease. There is no vaccine for enterovirus-68, which sometimes paralyses people. There is no vaccine for adenovirus-36, which is a cause of obesity. I could go on for a long time I think; lots of "harmless" viruses are turning out to cause serious problems. They can damage your heart, set off dementia, or give you cancer.
The only effective answer is avoidance. This requires learning and behavior modification, so it isn't too popular, but nothing else is as effective.
I'm having a [Poe's law](https://en.wikipedia.org/wiki/Poe%27s_law) moment here. You've just applied Abstinence Only Sex Education 'logic' to infectious disease. This is either brilliant satire of the odd arguments against harm reduction in general, or... you're serious, and I agree, it's a crying shame nobody taught all those poor smallpox victims how to modify their behavior.
The reason I say "if" and hedge my words is because the conventional wisdom is that autism is not a single disorder with a single underlying condition. It is a cluster of symptoms of varying intensity and it is (likely) caused by a host of underlying conditions.
Two children diagnosed with the disorder can have no overlapping symptoms.
If that is right, then the first step is to break it down into different kinds of conditions. (No eye contact is caused by X. Speech issues are caused by Y).
However, if this test can effectively pick apart one of the underlying conditions and its symptoms that would be a huge step forward. We could definitively say whether a child has this particular version of "autism".
After we tease apart a few more versions, the original condition will disappear and be subsumed by these other versions.