Hacker News new | past | comments | ask | show | jobs | submit login
When they warn of rare disorders, these prenatal tests are usually wrong (nytimes.com)
146 points by phsource on Jan 2, 2022 | hide | past | favorite | 163 comments




My wife and I went through this a couple of years ago, with a 10 week NIPT calling a rare trisomy (chr 9), which is always fatal within a few weeks of birth.

It was absolute hell. The key problem here is the waiting and uncertainty. You have the NIPT at 10w, but you can’t have the amniocentesis until several weeks later. When that came back fine, there were questions about whether it was a “mosaic” meaning only a small proportion of cells are effected. We were only really in the clear after the 20 week ultrasound.

That’s a lot of weeks to be consumed by wondering about whether to terminate the pregnancy, or wait it out for more information. I have a masters in bioinformatics (in genomics!) and my knowledge of stats and the science was next to useless in the face of these decisions.

I know of couples who simply couldn’t deal with this uncertainty and chose to terminate on the basis of this test alone.

Fortunately for us our child was fine and is a perfectly healthy 18 month old now, but I wouldn’t do the rare trisomy test again.


Having gone through two twin pregnancies (where the odds of these tests being correct are especially low) we declined all of them. Anecdotally, I know of several parents who had a positive test for genetic disorder, went ahead with the pregnancy anyway and children were perfectly healthy. Until these tests are close to 100% reliable I don’t see the point.


The point is that it is a screening test. A positive test will be followed by a more invasive test that has a lower false positive rate.


"The good news is that the invasive test proved the screener was incorrect. The bad news is that it looks like you've now lost your baby. It was fine though!".

The risks associated with this extended testing are just not worth it, perhaps aside from Down's (from a numbers point of view). Even then, there are many completely gorgeous children and people with Down's .. chances are you'll have a curveball in life one way or another at any rate.

I have two children, one diagnosed with ADHD/ASD, the other likely not too different but too early to tell. Apple doesn't fall too far from the tree. Wouldn't change a thing, other than to avoid the ABA services companies like plague - they prey on your insecurities and you might face financial ruin for possibly no real benefit to the child if you go along with their spiels.

So, to see medical companies exploiting vulnerable new parents who will do anything for their children? I am shocked. (/S...)


You are free not to take the test, but I think we would have taken the 0.3% risk to see if it is really Down's. While children with Down's can be gorgeous, I am not up to the task.


We did get the test for it for my two; can't recall if we did any extra ones. Believe we skipped them, at least on the 2nd.

I've come around to maybe change my mind since then, however I'd need to be in a very good position to be able to be up to the task.

I truly wouldn't have been, the ASD diagnosis was hard enough - and made magnitudes harder due to the manipulations of the "autism industry". Do this, do that, or else - you only have one chance for an early intervention, so better throw your own life away or you'll be a bad, bad parent.


Tests (the consequences) are not harmless e.g., https://empowertotalhealth.com.au/new-study-on-screening-mam...


"it worked out fine for me" is not particularly reassuring when the alternative is a lifetime of medical bills and possibly a permanent dependent.


The alternative is loving a human being and playing the cards you’re dealt


No. I think this is a cruel and ignorant thing to say.

Wanting to have a child who is not special needs is not an evil thing. Choosing to not bring a a child with special needs to term is not evil.

Having a special needs child can dominate your finances, your life, and the lives of your family members and already existing children. People have a right to choose what what they want out of life, especially in the context of before a child is born.


“ Wanting to have a child who is not special needs is not an evil thing. ”

No its not.

“Choosing to not bring a a child with special needs to term is not evil.”

A huge evil. Just consider your next paragraph explaining why its not evil; it centers around how having a disabled child affects you and others. I do apologize, but I find your last paragraph frighteningly narcissistic .


I can see from your comment history you are catholic. I don't see any point playing the "yes" "no" game with someone. I doubt you are going to convince me of your ethics, I doubt I will convince you of mine.


Why stop there? It can be a challenge to have a kid who is ADHD, or is susceptible to depression or mental illness, or a different sexual orientation, etc. It’s not wrong to abort those children either, right?


> Why stop there? It can be a challenge to have a kid who is ADHD, or is susceptible to depression or mental illness, or a different sexual orientation, etc. It’s not wrong to abort those children either, right?

But we don't stop there. The bar is considerably lower than "the kid might have problems".

The mother, exclusively, can decide whether or not she wants to proceed with the pregnancy through to birth, and she doesn't need a reason other than "I don't feel like it". All those reasons you gave are still better than "I don't fell like it", and you somehow think that those are the wrong reasons?


To be fair, the pregnant woman isn't a mother yet - unless she has other children. She's deciding whether or not she wants to (or can!) cope with pregnancy and the hardships involved. She's deciding whether or not to become a mother or bring another child into the world.

And yeah, it can look like "I don't feel like it" to a casual observer, but then again, it really isn't the casual observer's business. They won't know if it was planned or the result of failed birth control, if the father was/is abusive, and a myriad of other things.

And to be absolutely clear: People would try to tell me "I didn't feel like it" - nevermind that I've never wanted children. I'd have been sterilized years ago if it were readily available and cheap in the US.


> And yeah, it can look like "I don't feel like it" to a casual observer,

I think you misread my argument: if the bar is currently at "I don't feel like it" for abortions, then judging someone as morally or ethically "wrong" (or evil) for a higher bar is silly.

[EDIT: I'm not making any judgement on a pro-choice position, nor am I attempting to trivialise the complex and often traumatising decision-making involved in abortions. I'm simply saying that if we accept "I don't need to give any reasons" then we shouldn't judge the people who give medical reasons]


> The mother, exclusively, can decide whether or not she wants to proceed with the pregnancy through to birth, and she doesn't need a reason other than "I don't feel like it".

The morality of that is quite vigorously contested. And even in places that have elective abortion, it’s often only available in the first trimester, while many of these tests are performed after that.

But even if we accept the notion of elective abortions, it doesn’t follow that any reason is morally acceptable. What if the parents don’t want girls, like often is the case in India and China. It’s okay to abort them? What if we had an in vitro test for sexual orientation? Would it be okay to abort fetuses in that basis?


Moral and legal are two different classifications.

I think it should be LEGAL to have an abortion no matter your reasoning. The reason for this is we need to preserve the bodily autonomy of people who can become pregnant.

Weather or not it is MORAL is another question.

It is LEGAL to protest outside of a veteran's funeral. However, I would argue it is not a MORAL thing to do.


There are plenty of ways we don't protect people's bodily autonomy though. For instance it was illegal for anyone to take Paxlovid until FDA approval.

If we can ban someone from taking a potentially life saving drug it doesn't seem crazy to ban someone from taking what half the population believes is a life ending drug.


We’re talking about morality in the context of this article, since abortions based on this testing are legal in the United States and most other developed countries.


This is not true in every state. Abortion is in the process of being outlawed in United States. Texas outlawed it. Supreme court is hostile to it for religious reasons.


Abortion isn’t primarily a religious debate in the US. There is a broad consensus view that elective abortion should be illegal after the first trimester: https://apnews.com/article/only-on-ap-us-supreme-court-abort.... Even highly secular countries like France draw the line at the end of the first trimester, because after that point the fetus looks a lot like a baby (has a face, fingers, toes, etc): https://www.babycenter.com/pregnancy/week-by-week/13-weeks-p...

Roe continues to be controversial because it prohibits states from adopting abortion laws that reflect the consensus view. Roe mandates elective abortions until viability, which is toward the end of the second trimester. Roe is thus opposed by many people who don’t believe that life begins at conception for religious reasons. Those people oppose Roe for whatever moral reasons that drive highly secular countries like France to prohibit abortions in the second trimester.


It is being outlawed in some states of the Union, not in the United States as a whole. Even if it repeals Roe/Casey, I don’t think that Supreme Court is at all likely to hold that abortion is in fact unconstitutional. As a result, some states will restrict abortion, but others will not.


Roe effectively does not exists. Protections that have been there are not anymore. And the court has very clear politics on the matter.


For one, we don't have a good enough understanding of those things to be able to test adults definitively, so it's a moot point.

That being said, if there was a test for ADHD and I knew what the odds were that my child would have ADHD, as someone with ADHD I would seriously consider whether or not I wanted to have kids or subject another human to that.

So, I don't think it's necessarily wrong, in some universe, but there's a line between adversities for which you can still overcome and thrive (the ones you listed) versus the sorts of things prenatal tests check for where, if they do have these conditions they are either dead on arrival, dead soon after, will wish they were dead for most of their lives, or at the very least will never be a contributing member of society.

I don't think the comparison you draw is fair.


As someone with ASD/ADHD (professionally diagnosed), and with two children (one diagnosed the same so far), let me just say I'm somewhat taken aback by this.

I hear you say you have ADHD yourself - each case is different and such, but you're posting on HN and I'd say that puts you far above the rest of society. Sure there are challenges but it can also be a kick-arse superpower; learn to embrace it, and most of all be grateful for the things you have achieved.

If anything it has opened my mind about the beauty of a diverse society. There is more to life than a strict adherence to some notion of what it means to be successful.

I've done pretty well for myself even though I've also had some rather hard times - no diagnosis or help existed in the 70s/80s. Then again, people will face hardships in the most unexpected ways so if anything it's made me more resilient.

With the extremely low confidence factor in some of the conditions, this genetic testing is dangerous and unethical on so many levels.


I don’t think we understand these subtle mental disorders well enough for this kind of absolute decision-making. What is ADHD, really?

“Your child has an extra chromosome and will probably never live independently” feels deeply different than “your child has a gene variant that subtly affects their brain in ways that make it harder to focus in certain circumstances, and also probably in dozens of other ways that we don’t understand, and also many very successful people have this variant and it’s possible it even contributed to their success”. A society that routinely aborts children in the second case is starting to tilt into wild dystopia.

Not that we’ve identified such a gene, as far as I know.


Wait, are you objecting to the abortion specifically? Or does it just bother you that people could have a conscious influence on the kind of children they do/don't get?

If this is just a coded abortion debate... Count me out

But if it's the latter... Parents have been "selecting" children of the "right" race, eye color, height, etc for millenia... It's called "being attracted to certain physical characteristics over others" and it's really quite pedestrian.

ADHD, depression, and schizophrenia all appear to have a strong genetic component. I think most people would definitely weigh the knowledge of their partner's family's mental health history when they decide who to have children with.

I mean, don't you think people should have the moral right to select their own mates, free from coercion? I think most people would consider that to be an absolute human right, and anything less would be morally condoning rape.


They haven't been consciously selecting for traits, have they? Not with the certainty of a 1% false positive test.

This is eugenism, with its adherents lacking the moral clarity of the of early 20th century eugenists - at leasts they honest about what they were doing!


Well, solely with respect to skin color, which is arguably THE single biggest physical characteristic of concern... Yeah, I'd say >95% of Americans consciously select their mates with skin color in mind. Google the rates of interracial marriage if you don't believe me.

And yes, the skin/eye/hair color of the parents is probably about 99% determinant in the perceived race of the children... There are always exceptions, but two random people who are "white passing" are overwhelmingly likely to have children who are also "white passing".

You can call this behavior "eugenism" or even just racism, and I'm not saying your terminology is wrong... But I dare you to start going around telling people that to their faces.


As someone who has all of those in my family, none involve having a lifetime dependent.

Most people don't want to die and leave behind someone who can't care for themselves.


The alternative is to take the same energy you'd spend on that one unlucky human being and spend it raising 3 healthy children who will go on to live full lives.

Parenting is an enormous time investment and families that take on the burden of raising disabled children almost universally reduce their family size. This is not a decision without cost, to those who lose their chance at life.


It's fine for you to live in whatever fantasy world that allows you to never have to make any hard life choices, but you shouldn't go out of your way to try to make others feel guilty for having the unfortunate fate of having to live in the real world.


You literally know nothing about me past my 24 karma point earning comments.


Many of these conditions would in the past have been fatal in infancy, and it is only through modern medicine that children can survive them at all. It's fair to expect medicine to provide the solution to the problem it has created. There's nothing natural about children having to live their lives in and out of a hospital.


So… let’s have HIV positive people die since they can’t naturally live without modern medicine?

Why do adults get to be rescued by modern medicine and not children ?


with one of those cards being a lifetime of medical debt


Thats a political and social problem, not an ethical consideration.


So what?


Depending on what state/country you live in, care for a disabled citizen may be partly or wholly paid for by the state. The parents may also be able to become qualified caregivers, in which case they can be paid to take care of the child.


It's still a full time job that precludes you from doing a great many normal family activities for the child's entire lifetime. There is also the danger that your state may elect budget hawks who decide to cut funding for the programs halfway through your child's life, leaving you on the hook for ruinous medical expenses. It can also be very unfair to your other children who are going to basically lose out on activities because their disabled sibling requires too much care.


I agree completely about all of the challenges, hardships, lost opportunities, and unfairness. It is an awful situation to be in for everyone involved and no amount of money in the world could ever make up for that.


Our experience was kicked off by a troublesome ultrasound and then confirmed by amniocentesis.

The tragedy of receiving news like this is probably fathomable, but I think it may be hard to grasp the emotional and intellectual agony of deciding whether to terminate a pregnancy based on a set of probabilities.

It breaks my heart to think that parents face this decision with erroneous data.


I hope you are doing ok; truly sad to read. Agree 100%.


Why was there doubt surrounding an entire chromosome trisomy? My understanding was that it is easy to have high confidence about that since allele frequencies in the sequence reads are skewed across the entire chromosome.


Thank you for sharing this.


So glad to hear that things turned out well for you and your family.


Did you repeat NIPT test?


IMHO, some of those criticizing the article for failing to understand statistics are missing the point.

The point is that people who get a "positive" result on these tests are often put through terrifying levels of anxiety when there is no actual problem; this anxiety is often exacerbated because they aren't informed of the false positive rate. This clearly has a harmful emotional effect on people, and explaining the false positives in Bayesian terms, or reframing it in terms of sensitivity and specificity, doesn't undo that damage.

That potential harm needs to be explained to patients, and it needs to be weighed carefully against the potential benefits of the test (as is done for PSA tests for prostate cancer, which also have a high false positive rate). Given that potential for harm, it's not unreasonable to ask that these tests be more tightly regulated.

To quote the OP:

> In interviews, 14 patients who got false positives said the experience was agonizing. They recalled frantically researching conditions they’d never heard of, followed by sleepless nights and days hiding their bulging bellies from friends. Eight said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive.

(Edit: clarified)


If you get a positive for a horrid cancer with a 90 percent false positives you should be afraid. Its lunacy for tests to be regulated beyond requiring rough false positive false negative rates, and if anything smacks of "I dont understand statistics and therefore have to protect my children from understanding statistics." The article is most likely written by some anti abortion idiot.


> if you get a positive for a horrid cancer with a 90 percent false positives you should be afraid.

No you should not panic, anymore than you should celebrate and buy a yatch if you think you have 10% chance of winning a lottery.

The doctor should phrase it as "the test indicates you have a 10% possibility of cancer. The majority of people who test positive do not have cancer. Further testing is required to confirm or rule it out."

Going with the expected value of 10% of a horrid thing is still bad, or 10% of $100m is still $10m, is not applicable to a single non repeated event.


The GP wasn’t saying go out and kill yourself because you are definitely condemned. They were highlighting that you still have an extremely elevated chance of having cancer. If you got into your car and found out you had a 10% chance of being in a fatal accident, would you not worry? Sure, there is a 90% probability you’ll be fine, but the probability for most other people is 99.999%. Pretending like there isn’t a potential issue is dangerous and willful ignorance.


> Its lunacy for tests to be regulated beyond requiring rough false positive false negative rates

There is a good reason tests are described in terms of sensitivity and specificity ("if the answer in reality is yes, how often will the test say no?") rather than in terms of false positive or false negative rates ("if the test says no, how often is the true answer yes?"). The sensitivity and specificity are facts about the test which can conceptually stay constant[1] as you apply the test to different people. False positive and false negative rates do not have that property; they are facts about the group you're performing tests on just as much as they're facts about the test.

[1] This is not to say that the sensitivity and specificity of a test do stay constant as you apply the test to different populations. Often they won't. But it is a theoretical possibility, and even that isn't true for false negative rates.


For a test manufacturer you are right, and they aren't responsible for accounting for general population frequencies, or conditional frequencies based on things like some sibling has it etc. But there is no need to regulate that. But the serivice in question isnt a test manufacturer, its diagnostics, and they are responsible for this. I think a company selling a diagnostics service should be required to prominently display reasonably accurate FP,TP,FN,TN rates for gen pop, as the next example will show.

This is is not a legal problem, its a customer experience one, and as such its very much what the free market does well. A clever company could say that the tests service we provide works in a two stage process where a preliminary test is used to identify which conditions need to be investigated further, the results of which are presented in a meeting with one of our doctors were the results are explained and additional tests are performed to resolve any ambiguity. This combines the benefits of sensitivity, and specificity in a resource efficient way, and does not provide the user with any scary results before a certain answer is available. The FP,FN,TP,TN probabilities for gen pop for such a company would be much better than those of companies trying to do so in a single pass. But for these figures to mean anything regulating that they are available and reasonably accurate is key to ensure an efficient market.

Another problem is of course that the tests themselves are shit, or the rates based on poor data. The requirement on reasonably accurately reported gen pop rates help with this however, and I see no way around legally forcing diagnostic services providers to prove this. And until someone does, my patent pending algorithm(){return false} tests for all rare genetic disorders with excellent accuracy^^


I'm not sure GP was intending to make that distinction (rates globally for the test vs. the sample population being tested by.. what a given doctor/hospital?) - I haven't come across that before.

If the population is the same then your changed-order definitions are just inverses, and they're just different terms for the same thing.


> I'm not sure GP was intending to make that distinction (rates globally for the test vs. the sample population being tested by.. what a given doctor/hospital?) - I haven't come across that before.

> If the population is the same then your changed-order definitions are just inverses

No, you just haven't understood the concept.

Let's assume some condition has a prevalence of 20%, and a test for it will correctly identify presence of the condition 95% of the time, while correctly identifying absence of the condition 90% of the time. We can immediately answer the first question: when the answer is "yes", the test will say "no" 5% of the time.

You have proposed that when the test says "no", the answer is "yes" a share of the time that might be the inverse of 5%, or perhaps 5% itself. I have no idea what you meant -- and I suspect you didn't either -- but the correct rate of false negatives is not 5%, 95%, nor 2,000%.

In a model population of 10,000 people, we will see this:

                  |  condition present  |  absent  |
    test positive |               1900  |     800  |
         negative |                100  |    7200  |
From this table we can see that the false negative rate is 100/7300 or 1.4%. The false negative rate looks much better than the sensitivity and specificity figures because the condition is rare. The corollary to that is a horrific false positive rate of 800/2700 = 30%.


I don't think I have, it sounds like you think I'm saying false negatives are the inverse of false positives? Not at all, that's obviously not true.

I was surprised at 'sensitivity and specificity' (jointly) being considered different from 'false negatives and false positives' (jointly).

The given reasoning was about population differences, which.. fair enough, I understand that makes a difference, I just wasn't aware that was a standard difference in definition (if it is) and suggested the up thread commenter wasn't (or wasn't meaning to use it) either.

> correctly identify presence of the condition 95% of the time, while correctly identifying absence of the condition 90% of the time. We can immediately answer the first question: when the answer is "yes", the test will say "no" 5% of the time. You have proposed that when the test says "no", the answer is "yes" a share of the time that might be the inverse of 5%, or perhaps 5% itself. I have no idea what you meant -- and I suspect you didn't either

10%. 'inverse', as I called it, of 90%, not 95%.

(That's why I think you think I think (..!) that false negatives/positives rates are derivable from one another. Sorry if not and I'm just still not getting it...)

I don't think I am misunderstanding though - Wikipedia calls them 'true pos/neg rate', and gives formulae for false pos/neg rates as 1-true: https://en.m.wikipedia.org/wiki/Sensitivity_and_specificity


> it sounds like you think I'm saying false negatives are the inverse of false positives? Not at all, that's obviously not true.

No, I think you're saying that the false negative rate as I defined it in my comment is the inverse of sensitivity. You've corrected me to say that you think the rate I defined is the inverse of specificity, which makes even less sense. And you did that despite the fact that I included a full calculation demonstrating that that isn't true.

Wikipedia's definition of the false negative rate differs from mine. Wikipedia indeed defines the false negative rate as (1 - sensitivity), though not, as you seem to believe, (1 - specificity). But you get no credit for this, because I explicitly defined what I meant by the false negative rate, and you echoed that definition in your response to my comment:

>>> your changed-order definitions

So: you think you haven't misunderstood what's happening. I ask you this: in my table above, I believe that prevalence is 20%, sensitivity is 95%, and specificity is 90%. Please verify that.

I have said that the conditional probability P(condition present | test negative) is 1.4%. You responded saying that that probability is actually 10%:

>> You have proposed that when the test says "no", the answer is "yes" a share of the time that might be the inverse of 5%, or perhaps 5% itself.

> 10%

Where are you getting that figure from? Show me in the table.


Ok, fine, 'as you defined it'. It's hard to read your table on mobile. My only suggestion was that your definition might not be what the original commenter meant, because it's not my layman's understanding that they're different, but I don't know, maybe they also use your definition, no point arguing about it.

(Though it does seem a little odd to me to object to the comment on the basis of a definition you introduce yourself that even if some sort of standard is something that varies enough that Wikipedia uses a crucially different one.)


> Though it does seem a little odd to me to object to the comment on the basis of a definition you introduce yourself

Notice that I always talk in terms of the question being asked. And your response to me was phrased in the same terms. You didn't say 10% was "the false negative rate", you said it was "the share of the time for which, when the test says 'no', the answer is 'yes'". You didn't say "the false negative rate is the inverse of the true positive rate"; you said "your changed-order definitions are just inverses [of each other]". Those claims are wildly false regardless of what you think "the false negative rate" means, for the simple reason that they make no reference to a "false negative rate".

> It's hard to read your table on mobile.

The table has four cells, eight if you count the labels. I have every confidence that you can do it.


> > Though it does seem a little odd to me to object to the comment on the basis of a definition you introduce yourself

> Notice that I always talk in terms of the question being asked.

You were more specific, you replied to a commenter using 'false positive rate' as an undefined term; you introduced a definition. It's not the definition I'm familiar with; it's not the definition used on Wikipedia; I suggested it might (I don't know either!) be the definition intended by the commenter who used it.


I've worked clinically and taught this in professional training programs. It's a classical clinical decision making problem.

The solution is fairly straightforward I think: require the test companies to report the posterior probability of the illness given the test result. Probably with some kind of uncertainty interval.


The posterior depends on the prior. So not very feasible in practice.


Generally speaking in clinical settings there are rough estimates of base rates of diseases available, either through biological theory (eg, mutation rates) or empirical studies or both. Not always to be sure, but often enough to mitigate against some of the problems discussed.


My 2nd daughter was flagged during our 20 week for something having to do with the way her skull was forming and they wanted to do a series of genetic test. They charged us through the wazoo and everything came back negative. She arrived 3.5 weeks early and contracted bacterial meningitis shortly after birth. We found her code blue in the crib. She ended up having a bilateral craniotomy to relieve the empyema that had formed. CP, CVI, global TBI - every day is hell on earth. This was 2019, so the nightmare of the last few years started early for our family. We've had a number of medical professionals drop hints at the fact there might be something wrong from a rare disorder perspective but we're in a league of our own and that is hindsight - the damage and trauma are non-stop. Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.


I am sorry for your horrible situation.

I don’t really understand what your implication is. Are you wishing you had not done the genetic tests? Suggesting you should have aborted the 20-week fetus based on unusual skull appearance? Suggesting that the meningitis was a result of some malpractice? Mad at medical professionals who are now “dropping hints” without saying something more substantive?

The problem with these tests highlighted (confusingly) by the NY Times article is that they test for extremely rare conditions but have a high enough false positive rate (by my napkin math, on order 0.05% = 1/2000 false positive rate) that they end up dramatically over-diagnosing these conditions.

That doesn’t necessarily mean the people designing the tests were lacking in “sound science” or were behaving maliciously. Their tests just need to be almost perfect to not overdiagnose rare abnormalities, and perfection in medical tests is a very high bar. Arguably we shouldn’t do tests for such rare conditions unless the false positive rate can be dropped by another order of magnitude or more, and if we do such tests communication to those who test positive should be much clearer. On the other hand, since genetic abnormalities are so life altering, even a 10% or 20% chance of an accurate positive might be grounds to abort a very early fetus.

It sounds like your specific problem was that the genetic tests did not return a positive result, even though it seems plausible that your child had some genetic abnormality. Not picking up every possible genetic condition is a different problem from the false-positive problem highlighted by the article.


I've always wondered-- is the error between tests correlated? Why does it not suffice to gather all the positive outcomes and test them again, then again and again until you have sufficient power depending on the rarity of the disease in the population? I know that for covid tests for example the test outcome errors aren't independent, which is why even taking several tests you can't be confident you're negative even if you tested negative. But I figured that was a special case and it was because their error comes from variance in viral load or something. But would that apply to these genetic marker tests?


I think of it like this:

Unlike a false negative where there isnt enough to trigger the test, false positive implies that something triggered a positive.

You have something, just not what the test thinks you have. For example, men with testicular cancer, apparently, show up as pregnant in pregnancy tests. Clearly thats a false positive for pregnancy.


[flagged]


you should probably step back and take a breather.

youre emotionally involved in a matter with your kid(s) - as one should be - but thats not the way to have a public discourse.


The reply is pretty tasteless. Especially:

> Suggesting you should have aborted the 20-week fetus based on unusual skull appearance? Suggesting that the meningitis was a result of some malpractice?

Don't ask people sarcastic questions on a subject like this. You wouldn't do it in real life would you? It's a real person who just shared a harrowing experience with a child, not talking about the latest "Rust framework for DeFi Widget Spinning".

Common sense and basic social skills would tell anyone this was someone talking about a related experience with thier child, not a calculated rebuttal meant to talk to some exactly overlapping point presented in the article.

Honestly the exchange captures the core of HN: You can be as nasty as you want as long as you're not direct about it.


I am not asking sarcastic questions, and my comment is not intended as any kind of attack. Please read my tone as 100% earnest, just confused. And I hope the top poster will accept my apology that I didn’t phrase my comment to make that clear enough.

The tone of voice in the top-level comment reads to me as extremely aggressive/aggrieved, like they had been personally wronged by someone (not just a victim of a tragic accident). But I don’t understand where that anger is being directed toward, or what precisely they are trying to say, so I am asking for clarification. Are they upset at doctors? the hospital? genetic test providers? themselves? Or perhaps I was grossly misreading the intended tone.

I am not judging or criticizing them: having a chronically sick kid must be a horrible traumatic experience, with no easy answers. I am sure I would be incredibly upset too, under the circumstances. I am just trying to figure out what they are trying to say.

* * *

Separately, I’m sorry for the “It sounds like your specific problem” part, which is phrased insensitively; clearly they have much bigger problems than this one. I didn’t mean it that way. I’m just trying to point out that taking a genetic test with zero false positives wouldn’t have made any difference to the top-level poster.


The entire comment you've written seems hellbent on attributing this person's anger to some specific entity... did you take 5 seconds to ask yourself what the goal of that is? Is the goal to find out what they're angry at so you can invalidate their pain over (what you seem to understand is) a terrible situation?

"Oh you're angry at the doctors? Well doctors can't read the future."

"Oh you're angry at the tests? Well the tests are perfect."

I mean, I don't know why I'm asking when it's exactly what your first comment tried to do. And for obvious reasons they didn't appreciate it... after all what does it add?

-

You think this person doesn't know tests are imperfect?

You think they don't know what false positives and false negatives are?

Or you think they know and it's still a terrible situation, and any article about the inaccuracy prenatal tests would have reminded them of their traumatic experience that at least somewhat involved the inaccuracy prenatal tests, even if the inaccuracies involved are not exactly the same

-

And I mean, let's consider hypothetically they don't know for a second! Or let's say your goal was for the benefit of another reader. You're going to be the one to inform them of these things and then... ???

I mean seriously, someone reads an article that causes them to recall their child's birth which they described as a living hell. I think it's safe to say:

a) the point was not asking HN strangers to tell them why they're upset with the wrong thing

b) they likely understood their situation was not 1-1 with the article's situation

c) they commented anyways because they were venting. And not an off-topic venting anyways, they didn't say "I hate prenatal tests that are false negative" they showed hatred for inaccurate prenatal tests which doesn't prescribe a direction and is pretty fair???

-

And yes I get HN's love this place to be as sterile as an OR, but it's not every day we get an article that has this level of personal implication, so what exactly does anyone lose just letting this person vent without trying to police their emotions or "correct" their anger?

If this was a website about prenatal care it'd be one thing, but posts about prenatal care make up: 0.0024% of all posts on this website. 47 mentions out of over 1 million stories.

Honestly I'm shocked this even needs to be said though. This is all covered under basic social skills, but then again maybe I'm being inconsiderate of the fact that certain personal situations can affect the ability to pick up on non-verbal cues and implicit social expectations

To that end, in the future: Never, ever, ask someone if they wish they had aborted a child they had.

(And before you rush to say you didn't, don't even ask it implicitly.)

If they feel the need to share an opinion on that they will. It's certainly not for you to ask, and even less so for you to ask an internet stranger.


I didn’t understand what they were trying to say (e.g. who is supposed to be "burning in hell") and asked for clarification. Asking people to clarify when their statements are confusing/ambiguous is a pretty normal part of human conversations, and is not as a general rule intended as an attack.

Nobody has a secret agenda here. Nobody (except you?) is attacking anyone, correcting them, telling them they are wrong, policing them, threatening them, insulting them, putting words in their mouth, invalidating their feelings, ....

You seem to be really dramatically upset about this. I don’t really understand why (I assume you are grossly misunderstanding my intention – e.g. “hellbent”?!), but I am sorry to have deeply offended you. I sincerely hope you feel better soon.


The passive-aggression will only get you so far... here's the comment you seem to have already forgotten replying to:

> Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.

Where's the ambiguity?! How much more clear could this be?

> Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science

-

Or is it that you don't understand why someone would be angered by people pedaling prenatal diagnosis based on unsound science for personal gain? And doubly so someone who experienced so much pain after these types of tests failed them?

> You seem to be really dramatically upset about this

So you can pick up an emotional cue! (Well... actually you still seem unsure despite me being very clearly upset, so still have some ways to go.)

Yes, I'm very upset I had to tell what I assume is a grown adult that casually asking people if they wish they had aborted their child is not ok!

And as I write that I have to say: if you actually do have some medical situation that affects your non-verbal skills, then I do apologize for being harsh.

-

Otherwise if you really want me to feel better soon, stop with the flimsy runabout and be better.


You are repeatedly making very strong, incorrect (quite aggressively insulting) claims about my intentions based on your own invented assumptions, despite having been told directly that you were incorrect.

You now claim that actually I am lying about what my intentions were. You insist that I was previously intending to be insulting and cruel and that by (a) apologizing for being insensitive and (b) denying that was my original intention I am being “passive aggressive”.

(To what end do you think I would do this, and who do you think my “aggression” is aimed at? You think I am just trolling you for my own amusement?)

I don’t see any way of convincing you otherwise here; continuing this conversation seems futile. Have a good day.

* * *

Edit: for the record, my top comment upthread is currently at net +25 votes. Apparently a whole lot of other people also found the top-level comment very confusing/ambiguous.


I love this.

I ask a very simple question:

> Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.

Where's the ambiguity?

And there is none so you're forced to divert the conversation to... literally anything but that.

"I don't see any way of convincing you that incredibly direct statement was ambiguous" would have been shorter, no?

-

Continuing this conversation was futile the moment you decided to lie instead of admitting there was no ambiguity and you had just asked a stranger if they wish they had aborted their child for no good reason.

And who knows. Maybe despite appearances some basic understanding of why you don't make statements like the ones you did so casually... has permeated into your subconscious after being schooled on the concept for this many words?

I guess the world will never know


Woah, that wasn't my reading at all, I also didn't understand top-level commenter's implication, as in what went wrong and should have happened, or what's regretted; no judgement or disbelief or whatever - I just didn't understand (not denying there was one) the failing that meant the test manufacturers should join the commenter in hell.


So you read:

> Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.

And didn't understand the failing? Really?

> Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science

I mean how much more obvious could they have made it? Do you just not understand why pedaling unsound prenatal tests is a failing?


Maybe it's because I'm not American and am pretty unfamiliar with that system, but yeah.

To be clear I'm not disagreeing with OP or saying nobody else is to blame, I just didn't fully understand. I was glad someone else had asked and hoped to scroll down and understand, not find an argument, I think everyone had good intentions here.


It's not an American concept. Pre-natal diagnoses based on unsound science is a bad thing regardless of if money is involved.

And bilking a medical system has nothing to do with if it's free or privatized.

Even with free healthcare the suppliers are still getting paid somehow... so them providing an unsound test is still making unjust profit.


From what was described in the NYT article, the tests are not “unsound”. Nor are those making the tests inherently corrupt or malicious (some medical testing companies are, e.g. Theranos, but the same could be said of any other occupation or industry). My impression is that people who design these tests go into the field to help give parents the best possible information, and do their best to make tests which are as accurate as possible. (Disclaimer: I don’t know anyone in medical technology, pharmaceutical, etc. industries.)

The tests are extremely good, but they are not perfect, with a specificity in the 99.9%–99.99% range, which is low enough that there are more false positives than true positives for extremely rare conditions. Making tests with specificity in the 99.999% range or beyond is an inherently extremely hard problem.

The current quality would be fine if the posterior probabilities were properly explained to patients and if physicians had enough basic intro-level statistics background to interpret the test results. Unfortunately patients often aren’t given the results directly or told accurately how to interpret them and apparently many physicians lack basic statistical competence. It isn’t too promising that this NYT article expressly aimed at clearing up laypeople’s misconceptions does such a poor job explaining the subject, and spends its first half on misleading clickbait about how “wrong” the tests are.


Disagree. I don't get your reading from it and frankly I read the above comment and also did not understand what they had intended to communicate due to seeming contradictions. I had the same questions.


My reading was being charitable, but I was corrected by the follow up.

I wanted to believe this person understood the top level comment was venting and asked anyways (implying sarcasm) because that seems like a pretty basic inference from most people's social experiences.

But instead they clarified they (and I guess you as well) truly do not realize that the top level comment is upset because prenatal tests were part of a traumatic event.

You apparently did not realize the top level comment is not conflating a false positive and a false negative, or solely blaming prenatal tests, but rather they're saying all forms of inaccuracy in a prenatal test have great potential to cause heartbreak... which is 100% true.

tl;dr: Their comment is venting about something unimaginably painful while staying on topic and closing with a relevant statement, so I REALLY don't get what the point of drilling into the more personal part of it buys you?

_

I also assumed you and the other reply did not un-sarcastically try to ask a stranger if they regret not aborting their child so casually. But again, I stand corrected.


How did this article, written by someone who clearly lacks an understanding of basic statistics, make it into the Upshot? They try to make it seem like the test is wrong 85% of the time, but that's not necessarily the case. All we know from the article is that 85 / 100 positive results are false positives, which means the test could actually be quite accurate. If the test correctly identifies 100% of real cases, then that sounds like an excellent test. Just as an example, if 1/4000 people have the disease, and the test identifies 100% of these cases, then around 0.14% of test takers will get a false positive.


Their infographics convince me that they understand the statistics. But one of the key issues here is that the statistics are radically counterintuitive in a way that most people don't understand - the patients, the testing companies, and even some medical staff all incorrectly believe that a positive test for a rare condition means you probably have the condition.


Their graphics say the tests are “84% wrong.” Do you really feel that’s an accurate description? That doesn’t feel like an accurate description to me, and their usage of “wrong” in this context highlights that they don’t understand the distinction and importance of true positives, false positives, true negatives, and false negatives when measuring accuracy.


Going through something like this is very VERY stressful. When you get a negative you immediately forget about it. When you get a positive you die inside. Speaking from experience here.

84% wrong sounds, to me, as an accurate description. Experiencing this from the inside out, only the false/true positive ratio matters. (Given sufficiently low false negative rates, of course)

84% of people whose world is turned upside down are actually getting a wrong diagnosis.


You’re talking about precision (true positive / true positive + false negative) but that’s only one part of the story.

There is a real human cost to having a child born with a rare genetic disease (and I would argue is immensely more stressful). You can easily adjust the sensitivity to the test but at the cost of detecting actual true positive cases. The correct response to receiving a positive is to do another test to ensure it’s not a false positive.

To say 84% wrong is clickbait and used to elicit a legislative response (FDA regulation), which will help the reporters career.

The actual ratio to tell if something is “wrong” is accuracy (True positive + true negative) / (true positive + true negative + false positive + false negative)


No, precision is true positive / (true positive + false positive).

Your first equation is sensitivity.


If you get a negative result and then your child is born with the condition, you won’t forget quickly either.


I really feel it's an accurate description. If you get a positive result on the test, there's a 16% chance your fetus has a 1p36 deletion and an 84% chance they don't.


As you said “if you get a positive result”. It’s true, if you ignore the 99.9% of the time the test is correct (true negative result), then you can say the test is 84% wrong.


84% of people who got a positive test result will end up telling their family "it's OK, the first test was wrong, my baby doesn't have a 1p36 deletion after all". The 99.9% of other people who got true negatives are important from a test design perspective, because specificity is closer to the actual levers you can pull on, but it's not super relevant to the decisionmaking process of someone who gets a positive result.


Ignoring all the true and false negatives which themselves are markers of how accurate the test is.

16% precision is the correct statement, saying the test is wrong 84% of the time implies that those getting negative results might actually have positive results.


He framed his statement correctly, limiting his observation to the condition that the test returned a positive result. Saying that 84% of positive results are false is correct if only 16% are true. You'd need to know false negative rates and base occurrence rates (modified by whatever other factors are unique to your situation) to inform the nature of information you get by performing the test.


I disagree. It is clear from the title, “When They Warn of Rare Disorders, These Prenatal Tests Are Usually Wrong”, and the lead that they’re focusing on false positives.


It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity


No, the article isn't talking about sensitivity. We don't actually know what the sensitivity is from the data the article gives us. We are told that lots of people were screened and a small number had a positive result, of which a proportion were actually positive. You can't calculate sensitivity from that because you don't know how many actually positive cases were missed.

This article is talking about precision, which is the proportion of positive results that are true. And it's okay for precision to be awful, especially when the condition is so rare. But it's only okay if the result is communicated alongside a statement saying what the precision is, which it seems these were not.


Yes you are correct.


While the author may not be well versed or focusing on the stats side, you're missing the human side here I think.

> the tests are inaccurate, when in reality the tests are accurate

If the test make someone consider terminating a pregnancy or even considering it, that's a lot of pain. So for that human, the test is failing its purpose potentially, depending on the value calculation of terminating a viable pregnancy vs the severity of the issue if it comes to term.

For a human, accuracy as you defined it means little to nothing. Usefulness and helpfulness are far better metrics, and such a high false positive rate is clearly causing issues in respect to those, which is what the article is highlighting.


Usefulness and helpfulness are far better metrics, and such a high false positive rate is clearly causing issues in respect to those

How exactly do you plan on codifying usefulness and helpfulness?

A high false positive rate is not necessarily a bad thing and may instead be the catalyst for additional tests to confirm the first one. The tests accuracy may actually be 100%, which is great because it avoids a child being born with a fatal genetic disease. Would you prefer a high false negative rate that misses these diseases instead?


Or maybe you’re missing the human side of having a child born with a serious genetic defect?


Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do? At what point is it not better to terminate 100% of pregnancies?


> Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do?

Yes, it’s absolutely better to do that. Of course, the actual ratio is much better than that because we do follow-up tests after the screen.


Your first question and answer is against medical ethics. If you’re a physician you know that terminating pregnancies based on a non-specific but sensitive screening test is inappropriate (do no harm).

However, you are correct in your second statement - there are much more specific follow up tests which improve diagnostic accuracy.

Medical utilitarianism should never be so cruel and gruesome as to eliminate 85 healthy and wanted pregnancies in order to find and eliminate 15 genetic aberrations.


> At what point is it not better to terminate 100% of pregnancies?

Everyone should decide for themselves. Having seen the long term consequences I would rather err on the side of caution, even if it were difficult to become pregnant.

Such diseases are often incurable and significantly degrade the quality of life of not only the person to be born but the whole immediate family. At least in the US the there isn't enough social safety net or support too offset the crushing costs.


Did they use the word accurate? You used the word accurate and then you yourself are going on a tirade about how that’s not correct?

It’s clear the article is talking about why sensitivity is important in layman’s terms and while it could use better writing it’s a real problem in diagnostics. This is why you don’t ask men to take a pregnancy test to check for prostrate cancer. It is accurate but not sensitive.


They used the word “wrong”. Whether or not they used wrong to mean inaccurate, or wrong to mean not sensitive is up to the reader.


The issue is that the tests portray themselves as being accurate (in the sense of low false positive rates), and portray the result as “your baby has XYZ rare syndrome” instead of “your baby has a 15% change of having XYZ rare syndrome”. If the test providers stated the false positive rate for their results more clearly, parents would be in a better position to make informed decisions.


The larger issue as I see it is that the medical system around these screenings are not well versed in the statistics and able to communicate that to patients. "Eight [patients] said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive." It's hard to know what happened in the room when the doctor spoke with them or what was on those particular patients tests, and that's (one hopes) the worst medical news those people will receive for a long time so listening comprehension is understandably impaired, but there needs to someone available who can help them interpret, even days or weeks later, and these people were let down by the entire system, not just the test manufacturers.


I don't think that it's useful for articles like this to try to educate readers on the way that a precision-recall curve works (and how that differs from the statistical definition of accuracy). Honestly, that would just confuse the vast majority of readers when it's simpler to point out that the tests produce more false positives than they might otherwise expect. Also note that even if we want to be incredibly pedantic, the article never calls the tests "inaccurate" and instead uses a layperson term without a hidden definition ("wrong").


Would a test that reported 100% positive similarly be "quite accurate"? It would catch all true positives, right?


This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail.

It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exaggerated confidence thereof), arbitrary-seeming decision thresholds (but their hyperventilating over '85% wrong' notwithstanding, many are probably too conservative, if anything, given how devastating many of these problems are, there should be more false positives to trigger additional testing, not less), costs of testing (sure why not but little is presented), tests which they claim just bad and uninformative (developed based on far too little _n_, certainly possible), implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... Sometimes in the same paragraph.

Plenty of valid stuff could be written about each issue, but they'd have to be at least 4 different articles of equivalent length to shed more light than heat.


> implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)...

This is true in so many areas of journalism but lately seems especially egregious in the NYT. And I don't really blame them, as the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter.

I basically can only use these types of articles as a jumping off point for my own research, as I usually find the moralizing conclusion the article comes to as unsupported.


"the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter"

Yep, this is the framing I came here looking for.

Investigative journalists live in the same asymmetrically-incentivized world as social science researchers. If the reporter had looked into the phenomenon and concluded "yeah, boring technical logic pretty much works as expected here" then there's no story.


I wonder if the NYT editorial staff receives zillions of these pitches from their reporters purporting to reveal nefarious phenomena, and most of them turn out benign. It's fun to imagine that the NYT editorship is actually exceedingly good at detecting these outrage-false-positives before publication, but the base rate of outrage-true-positives is just so low that you have to expect some to make it to publication.

I'd like to imagine there's an investigative-journalism-editor-news somewhere, and they're discussing this discussion saying "bah, these clowns are making sweeping generalizations about editorial standards based on only a false positive; this is totally specious with no mention of the prior distribution or sensitivity vs specificity trade-offs"


They even missed "base rate", which is the way I usually see this explained to ordinary people without stats backgrounds. Really disappointing.


They don't use that specific term, but the Down syndrome infographic does a pretty solid job at explaining the base rate issue.


> implicit calls for the FDA to Do Something and ban the tests

Not that you're necessarily wrong, but how did you get that from the article? It didn't seem to me like they wanted a ban.


So you are saying the testing companies in the article aren't fraudulently claiming much more effective tests than they are providing?


Specificity and sensitivity are two dimensions that you can measure tests in. You can claim your test is 99% accurate if you mean that "if the test says you don't have the disease, there is a 99% chance that you don't have the disease". That same test can still be 85% wrong if it says you DO have the disease, though.

I doubt that hyping one side of this equation is fraud. Pushing the error in this direction seems like a good idea, anyway. If you have some weird illness, and the test comes back as a false positive, at least you'll continue to explore that possibility for a while. If it comes back as a false negative, then you'll spend a ton of time exploring alternatives which will be true negatives. Probably infuriating.

https://en.wikipedia.org/wiki/Sensitivity_and_specificity


Here's the fliers mentioned in the article:

https://www.harmonytest.com/content/dam/RMS/harmonytest/glob...

https://web.archive.org/web/20211116203541/https://myriadwom...

https://images.health.questdiagnostics.com/Web/QuestDiagnost...

I'd appreciate it if you could point out where any of them walk the potential customer through sensitivity, specificity, and the fact that if they test positive, there is an 80-90% chance that their will not be affected. I can't seem to find any of that.


When I got my test results, they were clear that the odds of having a disorder were (for example) 1/144, even with a ‘positive’ result. This was through Natera. The problem is that this information is sent directly to the provider in most cases, so parents are left interpreting someone else’s interpretation of statistics. My midwife specifically told me that the test isn’t often wrong, even though the actual odds were there in the fine print.


> “The chance of breast cancer is so low, so why are you doing it? I think it’s purely a marketing thing.”

This mindset is ingrained in every doctor I speak to, but I think it's just so wrong.

Take DiGeorge syndrome. You have a 1/4000 chance of having it, and the test carries an 81% chance of a false positive. The above doctor calls this "marketing"? Foolishness. That's an incredibly useful test. The downside is small, and the upside is asymmetrically large.

We need far, far better screening for all sorts of things. Adult cancer and heart screens once a year, prenatal screening, and on. We do a good job with breast and prostate screens, but for rarer conditions our current approach of waiting for the disease to be symptomatic makes no sense. Part of that will be driving the cost down. There is so much market need for a legitimate version of Theranos and I'm glad there are some companies working on these things.


> We do a good job with breast and prostate screens

Do we? Unless I'm missing something, breast cancer is a huge killer and PSA tests are deeply imperfect. I am very much not expert in these areas.


This article seems a bit deceptive. We are going through NIPT soon and our doctor went over false positive and false negative rates for the common screens. Our doctor has pointed out some of the screens (esp for rare conditions) are not that accurate. The only procedure with high accuracy, amniocentesis, has a slight risk of miscarriage (our provider quoted 0.3% ) so its still statistically better to take NIPT and then only consider amniocentesis with a positive result since there is no risk from NIPT.

You are supposed to treat a positive on NIPT as “there’s a chance your baby has this, need a more accurate procedure to confirm”.

It sounds like their ob gyn wasn’t able to explain results to them or they didn’t understand the probabilities. To be fair our provider didn’t even suggest tests for the disorders in the article, probably because of the false positive rates and rarity. Sounds like these extra screens shouldn’t be offered.


"These extra screens shouldn't be offered" seems like exactly the point the article is trying to make.


Interesting.

We have been undergoing IVF with my wife since 2019. (Covid made a huge mess of those plans...) One of our embryos tested as a possible positive (but only slightly) for aneuploidy of one chromosome.

The doctor, a veteran of IVF, looked at the results and said "my experience is that this is either a very small mosaic error, which tends to be utterly invisible in real life, or a computer artifact. I have never seen embryos with those borderline results develop any serious problems later. Things would be different if the aneuploidy signals were clear, but definitely do not discard this embryo".


Good luck, keep up your hope. I hope things work out for you.


I am not a parent, but the criticism of the article appears to be around a misunderstanding of statistics, or at least how to apply them. While I agree that criticism is completely correct, it overlooks the human nature of the people receiving the tests. At an already-stressful point in someone's life, it seems almost like bad bedside manner for the medical community, even if in an automated fashion, to tell people that there might be a complication looming.

This _does_, however, seem like a framing issue, more than a utility issue. If the tests are 100% accurate at detecting true positives, they're a great aid. But rather than framing the tests as a be-all, end-all source for information, why not frame them as "a test that suggests whether or not you should get other tests"? That simple wording change would save a great deal of added stress on someone starting or growing a family.


I totally agree with this. Managing perceptions and expectations is super important here.

Having been on the receiving end of a false positive, I'd still do the test again for a hypothetical future pregnancy. Even though it was hell for a couple of days.


I recall a period in the early 2000s when unindicated whole-body CAT-scans were being advertised on television.

That got knocked down pretty quickly but wow a lot of folks picked up a big chunk of their lifetime radiation allowance because of that.

These tests seem to operate under a similar model, disregard the risks of unnecessary testing in return for information of limited utility that may cause material harm.


I think you're conflating "these tests cause harm" e.g. radiation and "the information gleaned from these tests could cause the patient to make poor decisions". Having a regulatory body make this value judgement for people has quite a bit of disadvantages. See "DON’T TRY THIS AT HOME: THE FDA’S RESTRICTIVE REGULATION OF HOME-TESTING DEVICES" https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article....


This isn't really a fair criticism. I could be wrong, but I believe your comment reflects a bit of naivete about the current state of evidence-based medicine.

To evaluate the value of performing a diagnostic test as an intervention, you DO have to look at final actual patient outcomes at an appropriate end target which includes sending people unnecessarily down different treatment paths, including additional testing with additional risks. And most importantly is that, in fact, mere knowledge of diagnostic results has been PROVEN to cause harm in many scenarios.

Now... if a patient WANTS that test, I think it should be available. But whether or not it should be performed routinely without prompting is an appropriate question for regulatory bodies.



A tweet about this very article caught my eye yesterday, and I'm glad HN's taken notice too.

https://twitter.com/JohnFPfaff/status/1477382805583716353?t=...

'For a disease w a 1-in-20,000 risk, a test w a false positive rate of 1% and a false negative rate of 0%—an insanely accurate test—would identify 1 correct case and 200 false positives every time. Or would be wrong 99.5% of the time.

This isn’t “bad tests.” This is… baserates.'


Medical professionals are often shockingly bad at statistics. My wife and I were talking about birth control to an RN* after our first child was born. The RN mentioned cooper IUDs were 95% effective. He asked what timeframe that was measured over and she couldn’t answer. Not only did she not know, but she couldn’t even understand why we were asking the question.

*) My wife insists that it was a doctor, not an RN, but my brain won’t let me process that possibility.


Before my daughter was born I sometimes felt like it was the doctors job to scare us with every worse case scenario possible. It was quite stressful and upsetting.


I'm not certain that perk of parenthood ends at birth.


When my then-pregnant wife called our fertility clinic with a worry the nurse calmed her down but also basically told her “welcome to the rest of your life.”


The point of the profession is to find and address bad outcomes before they happen.


I had rather my doctor be upfront about all possible scenarios than to be try and nice about them and save possible information.


I've heard that in the early days of HIV, the tests were (e.g.) 95% accurate, and when patients saw their positive results and the supposed 5% chance it's wrong they'd sometimes kill themselves.

They revised the tests so the first test would say Inconclusive rather than Positive, and ask them to repeat it. This saved some lives.

Maybe this a UX failure? Shouldn't the test designers present the results like this, even to doctors?


Absolutely a UX failure here, one that it seems some doctors translate for patients while others are left in the dark on. From the way people are responding on here about the use of statistics in the article, it's clear that a big portion of the techo community I think is undervaluing that often UX is far more important than it is treated.


I had a friend tell me their daughter tested positive for something and they didn’t do any follow-up testing because someone else they knew also had a positive result and their baby was fine, so “the test is worthless.” Luckily their daughter seems to be healthy herself, but it clearly wasn’t explained well to them by their doctor.


This seems to miss the point entirely. Even for their worst example the odds of the fetus having it go from 0.005% to 7%. That's valuable information even if it's not perfect or somewhat hard to understand.


This would be valuable for running some extra tests (possibly more expensive, but more accurate), but not for, say, decision to abort the kid, which is what usually "hangs in the air" after such a test result.


NIPT is not supposed to be used for termination decisions. A positive is meant to be “your baby might have this, test further with amniocentesis”.


The article in NYT nevertheless states:

"A 2014 study found that 6 percent of patients who screened positive obtained an abortion without getting another test to confirm the result."

Maybe people aren't informed enough. It is my experience that some doctors tend to cut conversations short and some people are shy/insecure enough not to pry answers out of them.

In this case, that would be a tragedy, given that statistically 5 of those 6 aborted fetuses were healthy.

Edit: I found the following comment in the comment section of this article, which appears to address the same issue:

I am a physician with a PhD in Biomedical Informatics. Most patients who receive these tests do not see a maternal fetal medicine doctor or genetic counselor, and no one actually explains that the tests they are receiving are “screening” or “diagnostic.” Your opinion that this article does a disservice to patients reflects your unrealistic assumption that most of the doctors ordering these tests are actually communicating effectively with patients (or frankly, even understand the tests themselves). In my experience, they usually aren’t/don’t. Articles like this “fill the gap” on patient education when doctors are unable to explain math and risk (i.e., most of the time).


That’s a tragedy. Maybe there needs to be regulation requiring results are delivered by genetic counselors rather than physicians. Or maybe this is willful patient error.


That depends on the person though doesn't it? I'm not sure what I'd do in that situation. But 7% seems awfully bad odds for painful and debilitating life.


I guess that depends on the exact scenario. There are likely people with a variety of conditions who enjoy their lives vs having not been born. It brings up a seemly logical contradiction that we terminate fetuses (potentially viable in some cases) on the assumption that they don't want that life yet we don't allow people who want to kill themselves to do so.


There's a lot of sibling comments going on about whether the value they're looking at is the right one. What the Times is showing as their headline number is Positive Predictive Value (True positive/(TP+FP)), which depends on the prevalence in the population. The "methods section" here is a little vague, but given the low prevalence I'm willing to accept on face value that it's basically accurate (i.e. that it's not assuming that the families getting these tests are not orders of magnitude more likely to be positive for these diseases). If the test result truly said one patient's 'daughter had a “greater than 99/100” probability of being born with Patau syndrome', then that's concerning, but given the fairly narrow quotes around the number, I'd suspect that what is actually on the test result is not inconsistent with the fairly low PPV on these screens.


We were told our son had a high chance of being born with down syndrome. It was quite stressful to hear this as we weren't going to do anything about it regardless (he was born with no issues whatsoever and is now a thriving young adult).


I was in this exact situation. I received a phone call from my midwives, saying that my son had tested positive for one of these disorders, and that these tests aren’t usually wrong. Fortunately I had done my research and knew that the false positive rate is high. But the entire system is set up to provide a terrible experience.

Your results are sent directly to your provider, so you can’t read the fine print yourself. And if you do get access to the results, the wording implies that a null result (not enough DNA collected) actually means you’re likely to have some disorder. In fact the wording here actually got worse in the three years between my two (healthy) births.

Ideally these companies should require genetic counseling before you take the test. Parents should understand that these tests are for screening purposes only, and that a definitive diagnosis can’t be gotten until 16-20 weeks. Unfortunately these companies have found a niche- parents wanting to know the sex and health of their children as soon as possible- and have no real reason to improve their practices.


Isn’t that often true with screens in general? The threshold often allows a good number of false positives in order to minimize false negatives. The goal is to know when to seek further diagnostics. Communicating that to patients can be a challenge but it doesn’t mean the screens were designed incorrectly.


Edit: They kind of do this farther down in the article.

Considering this as a UX challenge - imagine a grid of 10,000 dots (100x100).

Draw one box around the base rate - the rate at which you expect to find the problem in the population. If the base rate is 1%, then the box is 10x10 = 100 dots.

Then color in the dots for the test positive rate (not false positive, just all positive tests) False positives would be the colored dots outside the box.

Next to that, put strikes through the dots corresponding to your expected false negative rate.


This is an example of a problem that is so hard to explain. The vast majority of folks getting these tests will get a true negative. Such that for most people, this is not an issue. So I get that it takes effort to make people care.

That said, I do feel that pulling in abortions to the debate is specifically to trigger a set of readers. But to what aim? They have not established that the tests could be better. Just that when they say yes, they are still not perfect.


The state mandated tests in California are far worse. At least with NIPT tests, if you get a negative, it's fairly certainly a negative. The state tests have all kinds of unnecessary false positives, and if you don't have the NIPT to negate them, you are in for a lot of worry.


Watch this for a great explanation about the statistics of testing for rare diseases https://www.youtube.com/watch?v=R13BD8qKeTg


Wow, what an embarrassing mess. Front page feature of bad statistics and bad medicine.


https://xkcd.com/1132/ seems appropriate. Or perhaps https://xkcd.com/2545/




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: