The issue appears to be, that lab testing does not emulate real-world driving conditions. So in the field, emissions are higher than in the lab.
These four car makers involved in the report differ from VW: they do not have a defeat device in their design. Their results are solely because of the lab-vs-road issue.
Its not clear this is any kind of problem. Reducing emissions in lab tests may very well result in car designs that have reduced road emissions. That was the hope when lab standards were introduced. Auto pollution in our cities has indeed gone down as (lab) emissions standards have been tightened over the years.
Its not clear this is any kind of problem. Reducing emissions in lab tests may very well result in car designs that have reduced road emissions.
This is the logic that European politicians have used in the past.
If there were a direct relationship between lab test emissions and real-world emissions, that approach may be acceptable. Unfortunately, that is not the case.
In the case of NOx, implementing the Euro standards should have resulted in an 80% reduction in emissions over the past 15 years. As the vehicle fleet is renewed, that should have translated into significant reductions in kerbside NOx levels. But this has not happened, and London continues to suffer some of the worst NOx levels in the world.
It turns out that an 80% reduction in lab test emissions has not translated into anything like an 80% reduction in real-world emissions - not even close!
The current system is inadequate because manufacturers simply "design for the test". There is huge real-world variability between vehicles that may record very similar lab test scores. A vehicle may claim similar test score results as another, but in fact have hugely differing levels of emissions in the real world.
Test environments need to reflect real-world conditions!
With the new testing procedure in place, you can still cheat like Volkswagen. But it will benefit the customers more, when the OEMs try to game the system by optimizing solely for the cycle.
What they need to do is more like a clinical trial for a drug.
Step 0) measure on a dyno
Step 1) measure on small closed course with a handful of drivers
who are unknown to the car companies
Step 2) measure on a larger group of cars in the real world
If these "costs" are unacceptable, then auto manufacturers shouldn't be putting cars on the road. I always thought trusting the ECU to tell the truth was ridiculous, mostly because over end user tampering not the original code. But the effect is the same.
It really is bullshit that so much time has been spent overfitting to the test when real universe exists right here. No model required.
It makes technically no sense if you can harmonize the regular driving condition into one single realistic cycle. Then the results are reproducible and comparable.
OEMs all have their in-house driving cycle to better reflect the reality[1] than the boring standard cycles.
The emission/consumption performance for some given customers is easily predictable.
To prevent cheating, you'll just have to open source the software inside.
My assertion is that you can't do that with a single realistic cycle precisely because there is a single one. Overfitting will occur, an ensemble of actors can't be gamed.
Sort of and sort of not right? So VW came right out and said "The difference is caused by defeat device code programmed into the computer." Great simple answer. None of these people have offered up a theory of why things are different. It could be defeat devices, or it could be something else, they just don't know.
So step two of this process is figure out all the ways in which actual driving differs from lab testing. And when you've eliminated all other explanations, start looking for defeat devices or their equivalents.
I'm guessing one or more of these companies will end up admitting, after looking into it, that their car control software runs differently if it detects the car is being tested.
At least in my corner of the industry (commercial) , we've been asking for better test cycles because we don't want to wind up in a situation where we are chasing fuel economy on the test cycle at the expense of fuel economy in the real world. The regulators want to keep the same test cycles so they can measure how emissions and fuel economy are changing over time.
Internally, we have much better test cycles correlated to real world routes that we use to validate fuel economy simulations.
I don't understand why regulators don't just do additional, more representative tests so they can track long-term trends and also have a better test? How much time/resources do the tests actually take?
A more representative test (like running a car on a treadmill) requires a dedicated testing facility that nobody wants to pay for or wait in line to use.
exactly, beside, a disconnect between test regime and real world results is expected.
what VW did was not just optimize for the test, but cheat the test, which is: those cars in the article are able to pass the tested restriction in the test regime, while disabling the cheating code wv engines were found not to be able to match test restriction even under test regime.
no one here was busted for real world emission. that's kind of understood already by regulators which built the test assuming the result would trickle down in real world usage even if not 1:1
It's not as though there haven't been issues with gaming these tests to a degree for a long time. Almost certainly this has been an issue of walking backwards into a big mistake little by little, which is the typical process of making big mistakes. You start by figuring out how to game the system right at "the line" as much as you can, then the day comes where you decide to be more explicit about it and build it into the software. You probably decide that you just want to play around at the edges, with incremental differences between "cheat mode" and real driving in the same realm as the bit by bit gaming they've been doing already. But once the system exists the natural process of optimization would push it to the limit of what was capable, and it turned out that was a lot. But by then you're committed, and it would take a lot of fortitude to walk back from such a mistake.
I'm sure the same sorts of dynamics have played out in many other auto makers, though it's notable that some, with a more developed sense of ethics seemingly, have not gone down that road.
Maybe we have a different idea about what a 'defeat device' is. Its a physical piece of hardware. Not some code. So to find out if these other cars have it, you stick your head under that car and look.
The term "defeat device" refers to any mechanism, physical or otherwise, designed to defeat emmissions controls. It's not literally a standalone gadget. I assume the term is a holdover from the 1970s.
A defeat device is an AECD “that reduces the effectiveness of the emission
control system under conditions which may reasonably be expected to be
encountered in normal vehicle operation and use. Motor vehicles equipped
with defeat devices cannot be certified.”
And the AECD is defined as
any element of design which senses temperature, vehicle speed,
engine RPM, transmission gear, manifold vacuum or any other parameter for
the purpose of activating, modulating, delaying or deactivating the
operation of any part of the emission control system.
"Its not clear this [lab-vs-road discrepancy] is any kind of problem."
I agree with what you said, except this.
Clearly these NOx emissions are a problem, because they cause smog (like ozone), which is a health hazard. The city where I live has too much ozone for many days of the year, for example.
Unless this problem can be controlled, vehicles like these should not be sold here, and people will have to find other options, or technologists will have to develop better cars.
My point is that this discrepancy has, in the real world, proved to be a huge public health problem. And we are not talking about mileage. We're talking about NOx that produces O3.
> These four car makers involved in the report differ from VW: they do not have a defeat device in their design. Their results are solely because of the lab-vs-road issue.
At the end of the day though, all the automakers did whatever was required to make them pass the lab testing, with no regard whatsoever for what that actually means on the road.
Maybe it's good on the road, maybe it's not, but in all honesty none of them care.
VW went about it different than the rest of them (that we know about), but the end result is the same - cars that pass in the lab and do who-knows-what on the road.
That's not necessarily unreasonable. They complied with the regulation, which is determined by lab test. Perhaps the test is defective, but what else were they supposed to do?
VW acted in bad faith and clearly exhibited the mentality you describe, but it doesn't seem to me that we know enough to determine how these few acted.
It does seem apparent the test is fairly useless. If we want to pursue this type of emissions control, everything I've read suggests that it's very possible to hit the marks in real-world conditions at the expense of power and fuel economy.
I agree. It's wildly accepted the winner in most sports contents is the one the pushes the rules the hardest - i.e. push the line as hard as possible, without "breaking" the rules.
I've not read the EPA rules, but I'm sure it says something like x amount of y gas under whatever conditions.
They definitely didn't. EPA didn't just make up the term "defeat device" last month. It's in the regulation that VW was supposed to comply with.
The EPA rules also contain a NTE "not-to-exceed" limit, about 2.5x the test limit. They don't have much of an enforcement budget, so testing is pretty rare, but if the car exceeds the NTE limit, the manufacturer has to issue a recall to fix it.
Well, they pretty clearly f'd up by designing tests that differ so hard from actual usage. Given that you only have to sample a small set of the cars, it shouldn't be that big a deal to have it go through longer, more representative one, even if it allows greater intra-test variation. (Perhaps compensate by weakening the standard, since it's more accurate that way.)
>These four car makers involved in the report differ from VW: they do not have a defeat device in their design.
I think it's a difference of degree, not kind. They all optimize narrowly for one test at the expense of real-world usage. VW just optimized a lot more narrowly.
It's like compilers. There is optimization by recognizing cases that might occur in programs. And then there is optimization by recognizing ... entire functions featured in specific benchmark programs, or those entire programs themselves.
Imagine a compiler that recognizes when you're attempting to compile Firefox from source. It checks your internet connection speed and hashes all the source files. If your Internet connection is sufficient, and the hashes match, this nifty compiler would "optimize" the compilation by downloading the matching binary from mozilla.org.
ccache hashes the preprocessed input to gcc, hashes it, and pulls out a previously compiled object file if possible. You can use it together with distcc to farm out jobs to other machines; whether you pick their caches or yours depends on the order of ccache and distcc on the command line.
Fast compilation is a worth goal, but it's not fundamentally different from optimizing other kinds of programs: it involves good selection of data structures and algorithms, smart caching, that sort of stuff.
Compiler optimization (in the sense of optimizing the emitted code) is a research area of its own with millions of man hours invested.
Oh my. I've worked in those car test and simulation SW environments (and a cross compiler for a weird engine),
and now I'm working on compilers and compiler optimizations in a different industry.
You cannot compare that at all.
With engine simulation and control you have real-time loops, with the biggest problem being the latency. You reduce latency by getting better HW, better sensors, shorter distances, a better SW driver architecture.
You don't optimize by finding patterns or counting occurrences at all, esp. not at run-time. You always have to be prepared for the worst case.
Plus you always have HW + SW combined. Any major important SW feature is always doubled in HW for security. It's a different world.
With compilers there's no real-time trouble (GC's, ha!), mostly no loops, no worry about HW, cables, sensors or drivers. It's pure abstraction.
With JIT compilers it's a bit different, mostly a tradeoff memory + optimization overhead vs speed. More measurements and dynamic decisions.
But jit optimization in engine control? Nah.
And there's never a HW cross check. Signals, interrupts may arrive or get lost. The machine will not hard reset and reboot if a signal gets lost.
It does so in a car or in space or other industrial control environments.
The similarity is that optimization is taking place, and cheating is possible by recognizing a test situation, in order to "game" the test. Compilers can "game" benchmarks by recognizing them specifically and supplying hand-crafted code. The cases which do this have no applicability to non-benchmark code. Just like recognizing that an emissions test is taking place and behaving optimally for that test has no applicability to real driving.
Since I worked on this SW I have to clarify a bit.
Lab testing of course does emulate real-world driving conditions. Otherwise your tests will not be worth much. Either you play back a recorded existing course (RPS - road profile sequences) or you simulate it, simple steady state and transient cycles. We are automating reproducible test conditions.
But we are here only talking about the different emission tests, where we have 2 major philosophies. The european emission test tracks (NECD), which is also used in India and Asia, which is an highly abstract test with the engine alone driven by a similator simulating multiple real-world driving conditions (a few urban cycles) and the american test on a dyno with a real driver (with varying cycles), so that the american car manifacturers, who traditionally were not so good in designing emission-free engines could traditionally cheat by hiring the best drivers to pass the test.
So Americans have cheated historically the most, but apparently engine manifacturers which couldn't catch up with modern diesel engine development had to cheat also. First it is trivial to cheat, since the test has to be done only once. This is entirely a political game to get the NOx and COx values down, but don't harm their own industries. And esp. Americans seldom use scientifically based values.
And then you have cold starts in reality, which is mostly not tested in the emission cycles tests. Emissions will always be higher in the field than with a properly warmed up engine under lab conditions. So it's no surprise that different measurement methods will lead to different results.
I wonder what roads will be like in 20 years when batteries and motors reach peak efficiency and lowest cost of production.
I jog on the side of the road every day and I have to wonder what all these cars that cheat emissions are doing to my health and how I can't make the manufacturers pay for my future lung cancer.
I am on a passenger seat something like 3 times a week, in france, for the same 15 min rides (about 5 different rides in a southern city), and each time there is at least one car who emits a very foul smoke, so foul we immediately roll windows. Most of the time those cars aren't even so old.
I highly suspect french automakers to cheat too.
There needs to be many more random exhaust controls, or at least much tighter annual or biannual checks. I can't even comprehend why it's not the case already.
It's not even a climate change problem, it's a health problem. I caught a cold already 2 times this year, which was followed by a chronic cough that lasted 3 week after the cold went away.
"It's not even a climate change problem, it's a health problem": NOx being extremely dangerous for humans doesn't make it less of a problem for global warming.
Nitrous Oxide (N2O) has a Global Warming Potential 265-298 times that of CO2 for a 100-year timescale. N2O emitted today remains in the atmosphere for more than 100 years, on average. [0]
As far as I know, the NOX problems that were masked by the VW software cheating are not the kind of problems that everybody driving behind the car in question can smell.
As somebody who has driven only Peugeots for 14 years now, I can attest that the exhaust system of any Peugeot at any point in time is primarily occupied with rusting. I have to change some part of the exhaust every odd year. Most likely the smelly French cars you observed also have problems with rusted through exhausts.
Also: As somebody plagued with respiratory problems, I feel your pain. My health massively improved when I moved from the city back to my home town in the German woods.
As a Frenchman, I reckon French manufacturers are gaming the stats/cooking the numbers as well, but I don't share your view on the age of the cars emitting the foulest stenches. 90% of times there's an issue it's a 10+ year-old car and most often from the cheaper end of the price spectrum.
I have a feeling most carmakers have been fiddling with the emission software in order in pass smog. I feel it's just a matter of time and testing, and I don't think it will just be diesels. I looked into Ford's GT-350 and it's, I believe, the fasted stock vechicle Ford has ever produced. It passes smog with flying colors. Engineers claim is just good engineering. I hope that's the case.
I am for reasonable smog requirements. I know how hard I have to tune, and repair my 4 banger in order for it to pass every two years.
My biggest fear is the federal will tighten smog requirements to the point where most older vechicles won't pass. I am poor. I drive a 4 cylinder vechicle. My vechicles are always in tune. I consider myself a enviormentalist. I still hope I can afford to drive in the future? I know I will probally never be able to buy a new vechicle.
The issue appears to be, that lab testing does not emulate real-world driving conditions. So in the field, emissions are higher than in the lab.
These four car makers involved in the report differ from VW: they do not have a defeat device in their design. Their results are solely because of the lab-vs-road issue.
I think too !
Volkswagen (VW) cheated with its motor software and have a very high NOx emission and use a cheaper less useful catalyst in real world scenario.
Others have a higher NOx emission in real world scenario: Renault, Nissan, Hyundai, Citroen, Fiat, Volvo and Jeep, Mercedes-Benz, Honda, Mazda and Mitsubishi
Actually, I am surprised that agencies like EPA don't test diesel emissions in real road condition from the very beginning. Now the question is: whether VW's diesel engine is the only case, or it applies to all types engines from all manufactures in general...
There are a bunch of EVs on the market at reasonable price points, eg. Nissan Leaf, Renault Zoe. If you need something bigger you can get a Renault Kangoo or the Nissan NV200. Granted, those aren't very sexy cars, but it's not like Tesla is the only option. You can even get used electric cars already.
The transition would be very quick if fossil fuels were taxed according to their damage on health and climate change.
EVs would become comparatively cheap in a just few years, and their price would also come down very fast as the production of battery packs increased massively. The technology is already here, we just have to manufacture these cars and set up the charging network (already much cheaper than the fuel station infrastructure).
67% of US electricity is generated by fossil fuels, so for most people the price of electricity would skyrocket too.
Nuclear power accounts for 19%; those in a nuclear power plant's coverage zone would see EVs being cheaper than gasoline-powered cars, unless you factored in the nuclear waste storage problem. Those near powerful rivers can benefit from the 7% of electricity from hydropower dams.
Wind and solar together are less than 5% of US electricity generation and basically irrelevant as a source of power for our transportation needs.
EVs are not a panacea; cities are. Demolish the suburbs, build vertically, live within walking distance of things, and you won't need so much energy.
When does someone bring up the fact that the government's creation of too aggressive emissions targets is a problem? Massive, risk averse corporations are taking massive risks to achieve "compliance"... Maybe the bar has been set too high?
If it's impossible to make a labor-saving device that doesn't harm bystanders, then the reasonable solution is "do without that device," not "go ahead and let it hurt people."
But citizens and corporations stand united against most attempts to reduce our dependence on cars, so we've got to make do with what few concessions we can push through.
Apparently the consumers do. Its a popular signalling thing to say, but not relevant to car purchasers decision making process. That's one problem to be worked on.
Perhaps the overall net environmental situation would be in total better if X% of the testing budget were reassigned to propaganda to convince people that 100 hp cars are superior to 200 hp cars.
Another option is in my grandpa's day MASSIVE cheating happened with horsepower specs; it was like how air compressors and vacuum cleaners are marketed today. Cutting back on regulation in that area would likely result in cheaper 100 HP engines being sold as 200 HP engines (at the price of 200 HP engines of course) yet the environmental impact would obviously be only 100 HP.
A final option is nobody wants to acknowledge the management failure that naturally results from evaluating to a simple numerical metric for too long. All you eventually get is awesome numbers and everything else produced turns to garbage. One solution would be to dramatically change the metric faster than corruption can spread. In a distributed world, how about we wire up 1 in 1000 vehicles and track the unholy heck out of them and multiple the results by 1000 and grade on that criteria? Obviously in the long run we'll just corrupt the selection and manufacture process for those 1 in 1000 cars. But for awhile it'll work better. Once that system is powned, maybe switch the metrics back?
Take something like the "Smart cars", tiny one or two seaters that are designed for efficiency above all else.
Two problems:
* They're too light and too tiny. Get into a wreck with any other car on the road, and you will fare poorly.
* Performance is anemic, to the point where they can barely do 80PMH, which is standard cruising speed on the interstate, or do things like accelerate to pass.
There's also the ever-increasing safety requirements, which have resulted in (among other things) substantially heavier cars over the last few decades. Heavier cars require more power, which creates a bit of a catch-22 when it comes to reducing emissions and improving efficiency.
These safety requirements aren't even necessarily a straight up win for overall safety: the increased height of car beltlines reduces visibility and the increased vehicle weight (and resulting impact force) can worsen accidents involving non-vehicular objects.
An unregulated car industry is certainly not the solution (as proven by these diesel tests among many other examples), but overly strict regulation may only serve to improve regulatory avoidance by manufacturers. Sane, obtainable goals must be set, and both sides need to be involved in keeping things honest. It's absurd that these regulations were set without ensuring that they could reasonably be met and that they were actually being met. It reflects poorly not only on VW and the others, but also on the EPA and their counterparts.
These four car makers involved in the report differ from VW: they do not have a defeat device in their design. Their results are solely because of the lab-vs-road issue.
Its not clear this is any kind of problem. Reducing emissions in lab tests may very well result in car designs that have reduced road emissions. That was the hope when lab standards were introduced. Auto pollution in our cities has indeed gone down as (lab) emissions standards have been tightened over the years.