I found the giant line graph problematic because of the hand waving they did to normalize when the breaks occurred during the day. It also disturbs me to see such large, high frequency variation in a line that is averaging a large number of events without explanation.
The dataset certainly begs for a better analysis. For instance, from the supporting material, is there really a judge (the 'X' marker) the never granted parole to the last three people before his morning snack, but granted 70% for the first three of the day on average? All year, for a significant number of cases? Did no one notice that the people before snack time were not getting due process?
No, that graph is incredible. Literally unbelievable. 70% accepted in the morning to down below 10% before snack? Either this data is BS or there's some hidden variable here.
The comments on the original article do suggest another variable. The schedule may not be randomly arranged. The judges or the court clerks could actively arrange for easy cases earlier and more diffficult or complicated cases later. IOW, the variables may not be independent.
Also it's worth pointing out that this was a study on Israeli judges. American legal precedents and opinions and scheduling norms may not apply.
"It’s not possible that someone ordered the cases in a special way. The judges know nothing about upcoming cases ahead of time, so they can’t decide to take a break in the knowledge that an easy positive case is coming up. They also have control over when they set their breaks, so prison staff cannot predictably schedule the hearings in order of ease."
They also have control over when they set their breaks
So... maybe they routinely take a break after a particularly emotionally trying or complicated case? Then we only need demonstrate the messy cases are more likely to produce an unfavorable decision.
That wouldn't be enough to eliminate the effects of ordering. Sure you'd get some judges that take breaks earlier or later than others, but they'll still have three sets of cases of about the same length at about the same time.
This is true and obvious to some extent, but there's some real downsides to the legal realism movement. Legal realism starts off with a correct premise - that judges are flawed and human like the rest of us - but some people go further and come to the conclusion that, thus, no justice is possible and we should just game the system as much as possible.
It's an absolute fact that the American legal code has become more much more bloated, and in my opinion, more haphazard, disorganized, and prone to severe contradiction following legal realism becoming a popular philosophy.
It's immediately obviously true in its premise, but some of the conclusions following from legal realism seem to make the justice system worse.
"Belief in the importance of interdisciplinary approaches to law. Many of the realists were interested in sociological and anthropological approaches to the study of law. Karl Llewellyn's book The Cheyenne Way is a famous example of this tendency. ... Belief in legal instrumentalism, the view that the law should be used as a tool to achieve social purposes and to balance competing societal interests."
This contributes to the current culture, I think, of having 1000 page laws that nobody except lobbyists have actually read. Legal realism says, "There's a problem here, justice isn't perfect" - but then proposes a cure much worse than the disease.
Yes, legal codes do tend to get bloated. But I think legal realism has been a foe, not a friend, of a straightforward and consistent legal code.
That's before even getting into things like fad psychology getting introduced to the courtroom, where psychiatric evaluators use things like the Rorschach test to evaluate people on trial. It's frigging scary that fad social science gets introduced without vetting into the court system, and it turns the common law system from an asset into a liability. But anyways, this is a long conversation - I just wanted to point out that despite legal realism's premises being correct, that doesn't necessarily make all of the conclusions lead to good places.
You're way overloading the idea of legal realism to make your argument, and ignoring things like the influence of critical theory (Marxist/Hegelian dialectic, which is not the same thing as Marxism), public choice theory, the pragmatic movement and so on. Yes, this stuff is very much inside baseball, but I don't think you should rely on Wikipedia to form your opinions about this complex subject.
FWIW I'm inclined towards liberal pragmatism, and biased with an admiration for Richard Posner, probably because I already shared his admiration for JS Mill. Posner's book 'overcoming law' is a good survey of judicial philosophies from the most conservative kinds of formalism/originalism to the most left-wing kinds of identity politics.
Fad social science? Sure, that sucks, but even "forensic science" evidence is often questionable (http://www.innocenceproject.org/understand/Unreliable-Limite...). Even worse, even if the test really does work (e.g. DNA testing), we get abuse of statistics like http://en.wikipedia.org/wiki/Prosecutor%27s_fallacy - "the probability of a random guy having this weight of evidence against him is 1/1 000 000, so this guy must be guilty!" (of course, there are 300 Americans you could convict of any one crime under this standard...)
Let's just say that modern justice systems are not without their faults.
To be clear, I wasn't really suggesting that one should game the system, hence the scare quotes. But a competent lawyer should, in my view, respect the fact that judges are human and respond just like other humans. That means paying attention to little details like dress, demeanor, font, grammar, and all the other elements of presentation that every marketer knows but few lawyers understand.
On a deeper level there are tactics like angling your argument to the judge's world view. (This makes judicial panels all the more important.)
Some vertical error bars would have been helpful on that plot, but having read the original article (from behind the pay wall) I'm pretty satisfied that they thought about all the obvious problems being discussed in these comments. I am a practicing experimental scientist but with minimal knowledge of statistics, so factor that into whatever I say here.
They point out that a negative ruling ("no" or "almost - come back next time") is much easier to deliver. There is less thought involved, nobody will raise eyebrows and the paperwork is much lighter. So the trend is more about mental energy than grumpiness. They note the significance of the breaks, but do not attempt to disentangle the role of mood improvement / a blood sugar boost / rest.
They explicitly discuss the possibility of hidden variables:
"A key aspect for interpreting the association ... is whether an unobserved factor determines case order in such a way that yields the pattern of results we obtain"
The dip and spike patterns in the rulings are quite pronounced, and similar graphs in the article for parameters such as the gravity of the offense, presence of a rehabilitation plan and the number of previous incarcerations do not show any such trends. This addresses questions about case scheduling by the clerks. Besides which - even if the clerks did organize easier cases for later in the day, this would manifest as one long trend over the entire day, not the spike/decay sequence shown.
So I didn't see any gaping holes, and more importantly it seems quite a plausible conclusion to me. Surely you must have had a similar experience going Christmas shopping - you start out thinking very carefully about getting the perfect gift for every person, but after 3 hours trudging around in a mall you'll just buy any old shit so you can go home. As you get tired you just start taking the easiest options.
My theory is that it's more about the judge's blood sugar than about the break. Here's a Science Daily post about one documented case of blood sugar affecting decision-making:
To combat these kind of issues, maybe the law should move to some kind of a majority of three decisions, each hearing being random in all possible senses.
Or we could use tele-judge technology so all defendants can get a hearing with an alert, well-fed judge from another timezone. (I'm only half joking.) This could also be a solution for load-balancing courts among different regions (though the laws may be regional differences).
If you didn't like the judge assigned to your case, and could pull together this kind data on their past decisions, would that be enough to argue for a different judge?
Or perhaps mandate that they eat a snack bar every so often.
Nice article. I would love to see similar data for orchestral auditions. It's common wisdom that it is best to play when the committee is fresh from a break but I'll bet the data would show that that is dramatically the case, since as with parol hearings, rejecting an auditionee would be the default choice.
The fact that the judges can pick their breaks makes me suspicious that there might be a correlation/causation conflation going on here. A possible explanation could be that the workload is trimodal, and judges tend to take breaks after particularly depressing (parole denied) cases.
If the breaks are pretty regular, wouldn't you want to hear more simplified (clear-cut?) cases immediately after a break and more complex ones when more alert?
It’s not possible that someone ordered the cases in a special way. The judges know nothing about upcoming cases ahead of time, so they can’t decide to take a break in the knowledge that an easy positive case is coming up. They also have control over when they set their breaks, so prison staff cannot predictably schedule the hearings in order of ease. And Danziger showed that the judges weren’t any more likely to take a break after particularly difficult cases or severe crimes.
I'm not inclined to purchase the paper, but I would like to see a better explanation of the mechanics of Israeli prison parole hearings before jumping to conclusions.
The big flaw in this study is that while the judges may not know anything about the cases coming up, their clerks do and the clerks are the ones who actually determine the schedule.
Afternoon cases are almost always easier cases because afternoons are largely reserved for actual work: trial/pre-trial hearings, legal research, or paperwork generated by the morning workload. Thus, the clerks always schedule the easiest cases for the afternoon to minimize interference with the important stuff.
- The judges don't know the order of upcoming cases, but do pick when they're going to get food breaks. When they take these breaks shows considerable variation.
- Scheduling usually happens based on the ordering the attorneys arrive, and the attorneys don't express any conscious choice to try to game their arrival time.
- The paper also looks at what is likely to make a prisoner get paroled (e.g. type of crime, recidivism, etc.), and the cases that, without considering the effects of the breaks, are likely to result in parole do not seem to be clustered throughout the day.
The relevant passage:
A key aspect for interpreting the association between the ordinal
position of a case and parole decisions is whether an unobserved
factor determines case order in such a way that yields the
pattern of results we obtain. For instance, if prisoners without
a rehabilitation program or recidivists were somehow more likely
to appear before a food break, we would naturally find a greater
proportion of rejections occurring before the food break as well.
A number of procedural factors preclude this possibility.
First and most critically, the judge both determines when the
break will occur during the course of the day’s proceedings and is
unaware of the details of the upcoming cases. Thus, the judge
cannot decide when to take a break based on information related
to the nature of the upcoming cases. So, in the example above,
a judge cannot decide to take a break because he or she knows that
prisoners after the break will have no previous incarceration record.
Relatedly, the type of case (e.g., severity of the crime) that
the judge had just ruled on exerted no significant effect on the
likelihood of taking a break (Table S15). Furthermore, the large
variability in break start times and durations attests to the fact that
their occurrence would be nearly impossible to predict by any of
the prison staff involved in the parole proceedings.
Second, the ordinal position of cases is, with rare exception,
determined by the arrival time of the prisoner’s attorney. The
attorneys are sequestered in a room where they are unable to view
the proceedings of the board and, therefore, are unaware of any of
the rulings of the judge, how many prisoners preceded their client’s
case, or when and whether the food break occurred (after
the board’s deliberations, attorneys exit through a different door).
Thus, by design they cannot learn about the advantage of
appearing after a break. Indeed, a survey administered to a sample
of these attorneys after the primary data collection period
indicated that they were unaware of the effect of ordinal position
on rulings (see SI Materials and Methods, S2 for details). A similar
survey administered to parole board members (judges, criminologists,
and social workers) revealed the same results (see SI
Materials and Methods, S3 for details).
Because of the factors discussed above, we did not expect significant
correlations between ordinal position within either the day
or the session and the control variables in our data (SI Materials
and Methods, S4 and Table S16). Consistent with our expectations,
there does not appear to be a deliberate ordering based on the
characteristics of the prisoners (Fig. 3 A–D and SI Materials and
Methods, S4); certainly there appears to be no effect of a food
break on the type of prisoner appearing before the judge. Note that
although there was a slight but significant correlation between
recidivism and ordinal position in the day, this correlation was not
significant within a decision session, i.e., between breaks. Thus, it
cannot explain the spikes in favorable decisions after breaks.
The work itself is behind a paywall, but you can read some of the supporting material at http://www.pnas.org/content/suppl/2011/03/30/1018033108.DCSu... to get an idea.
I found the giant line graph problematic because of the hand waving they did to normalize when the breaks occurred during the day. It also disturbs me to see such large, high frequency variation in a line that is averaging a large number of events without explanation.
The dataset certainly begs for a better analysis. For instance, from the supporting material, is there really a judge (the 'X' marker) the never granted parole to the last three people before his morning snack, but granted 70% for the first three of the day on average? All year, for a significant number of cases? Did no one notice that the people before snack time were not getting due process?