Hacker News new | past | comments | ask | show | jobs | submit login
In Soviet Union, Optimization Problem Solves You (2012) (crookedtimber.org)
35 points by vezzy-fnord on Dec 5, 2015 | hide | past | favorite | 14 comments



Speaking of that I recommend watching Pandora's Box documentary. It is fun and pretty well made (although the quality on youtube is rather low).

Here is where they start talking about Gosplan -- the central economic planning department:

https://www.youtube.com/watch?feature=player_detailpage&v=h3...

It is funny to hear how they tried to control things. At some point during Stalin's time they planned how many people to arrest and where. No matter if they were guilty or not. The it talks how they tried to modulate various controls -- they measured success by amount of raw material consumed, so all of the sudden they ended up with oversized couches, and trains were being run for thousands of miles empty just to burn the fuel so everyone can get a bonus during the years' end. Then they started to fix prices for everything. That ended in disaster of course and so on.

Then have to like the taxi driver driving past Gosplan and saying "how they hell do they come with such ridiculous plans".

EDIT: They also mention Victor Glushkov, the father of Soviet Cybernetics. Here is a documentary about him as well. It is putting it in good light as if Soviet Cybernetics used in planning would be successful. It is a propaganda film. But it is fun to watch:

https://www.youtube.com/watch?v=lMS1hBhV2-4

He talked about paper-less office and economy back in the 60s.


So yet another case of Goodhart's Law, this time perhaps before Goodhart formulated it?


Cosma Shalizi's article is correct, as far as it goes (as far as I know). But some of the commenters here did not read the whole thing. He adds, capitalist free markets can't optimize either: 'If it’s any consolation, allowing non-convexity messes up the markets-are-always-optimal theorems of neo-classical/bourgeois economics, too. (This illustrates Stiglitz’s contention that if the neo-classicals were right about how capitalism works, Kantorovich-style socialism would have been perfectly viable.) Markets with non-convex production are apt to see things like monopolies, or at least monopolistic competition, path dependence, and, actual profits and power. (My university owes its existence to Mr. Carnegie’s luck, skill, and ruthlessness in exploiting the non-convexities of making steel.) Somehow, I do not think that this will be much consolation). [...] 'Both neo-classical and Austrian economists make a fetish (in several senses) of markets and market prices. That this is crazy is reflected in the fact that even under capitalism, immense areas of the economy are not coordinated through the market. [...] ' The conditions under which equilibrium prices really are all a decision-maker needsto know, and really are sufficient for coordination, are so extreme as to be absurd.(Stiglitz is good on some of the failure modes.) Even if they hold, the market only lets people “serve notice of their needs and of their relative strength” up to a limit set by how much money they have. This is why careful economists talk about balancing supply and “effective” demand, demand backed by money.

'This is just as much an implicit choice of values as handing the planners an objective function and letting them fire up their optimization algorithm. Those values are not pretty. They are that the whims of the rich matter more than the needs of the poor; that it is more important to keep bond traders in strippers and cocaine than feed hungry children. At the extreme, the market literally starves people to death, because feeding them is a less”efficient” use of food than helping rich people eat more.'

The article has another weakness, which is that Shalizi does not seem to understand how capitalist corporate planning works in real life. Increasingly, whole supply chains are balanced to point of sale events and other signals of incipient demands. And those whole supply chains do not do open market exchanges, the members are contractually bound. It's not optimal, but it's good enough. And all of that could be done without prices or money.


Red Plenty (the book mentioned at the top of this) is truly great and well worth tracking down.


Seconded, and Backroom Boys [1] by the same author, is very enjoyable, too, even if Red Plenty is really in a league of its own.

1: http://www.amazon.com/The-Backroom-Boys-Secret-British/dp/05...


Thank you for the recommendation! Just ordered and excited for it to arrive.


The Amazon page for Francis Spufford's Red Plenty, http://www.amazon.com/Red-Plenty-Francis-Spufford/dp/1555976... , currently features as 2nd comment one that starts by saying:

"I happened to grew up in Soviet Union and actually met some of the people mentioned in this book ..."

(and goes on to praise the authors accuracy.)


Am I the only one suspicious of his claim "It will not do to say that it’s enough for the planners to approximate the optimal plan...This route is blocked."? It seems to me that in all my experience, computational complexity even for approximations rarely matches up with practical computation required. Computational complexity is worst case, doesn't understand the concept of "good enough for our application" when it's not mathematically defined.

For example, what's the computational complexity of training a deep neural network? Probably something pretty horrendous, even if you say it only has to be approximate within a factor of the optimal weights, and you allow it to fail some percent of the time, etc, etc. You could probably write an article about how training neural networks in the lifetime of the universe is impossible even in theory, if you defined "training a neural network" by starting out with the problem of finding the optimal weights, and then relaxing the requirements. But that entirely misses the point that training a neural network is not fundamentally about finding the optimal weights, but finding some weights which are good enough, which is measured in terms of real world performance in comparison to the alternatives.

Likewise, it seems like the discovery of the mathematics of linear optimization was mixed too strongly with the real-world problem it was trying to solve. The question is whether it is computationally feasible to outperform market based economies using this technique, and that's the only question that's really make-or-break it for the math side of things.

(Of course I may have misunderstood something, feel free to correct me)


Approximation of integer programming problems (what the author is talking about more precisely) is actually pretty well studied. From Wikipedia:

> NP-hard problems vary greatly in their approximability; some, such as the bin packing problem, can be approximated within any factor greater than 1 (such a family of approximation algorithms is often called a polynomial time approximation scheme or PTAS). Others are impossible to approximate within any constant, or even polynomial factor unless P = NP, such as the maximum clique problem.

> NP-hard problems can often be expressed as integer programs (IP) and solved exactly in exponential time. Many approximation algorithms emerge from the linear programming relaxation of the integer program.

It turns out that shortest paths finding (via Bellman-Ford), dynamic programming and linear programming are all interrelated. Figuring out how your linear program can be represented as some other problem, like shortest paths finding, directly yields to generic approximation.


I haven't read the book, but I believe one of the largest problems that centrally-planned institutions (like large corporations) face is that the planners tend to remove redundancy too much (in the pursuit of minimal cost).

For example, let say you have chain of 50 products that need to be made into a product. If each of these is produced by one factory and supplied into another, and each factory is late for a week, your new thing is one year late!

This was a frequent problem, that the planners tended to designate one producer of some product, in order to save costs. But the product wasn't good enough or late, and then you had a cascade of failure ending up in shortages of goods.

So there seems to be a tradeoff between redundancy and cost. What is optimal? Would you think it's optimal to say, send the cheapest possible rocket to the orbit, with no backup systems?

However, in the free market, you cannot control redundancy. It just happens through freedom. That pushes it out of the cost optimum but makes it lot more robust, and winning in the real world. A good example of "worse is better" indeed.

I also don't think the actual planning problem is very difficult. There are multinationals that are larger than some state economies and still can do that. So I think traditional economic textbook explanations of central planning failures are wrong, because they ignore the tradeoff.


One example of this is the Toyota Just-in-time system, where i believe they have a number of suppliers, some of them mom and pop producers in a basement or loft somewhere, that are gathered up and delivered to the factory as needed.

And i think you find a structure somewhat similar on Germany as well, where quite a bit of the parts manufacturing is done by smaller companies dotted around the nation.

As for traditional economic textbooks being wrong by ignorance, no surprises there. Check out Steve Keen's book on the topic, Debunking Economics.

Also, as best i can tell the soviet system was pretty much a perversion of what Marx was musing about back in the day. I think he even told Lenin that the latter was barking up the wrong tree.


Yeah, I agree with you, I am sure some places and configurations deal with the problem better.

And I am fan of Steve Keen too.


If you want redundancy in your solution, why don't you simply include it in your models that you solve?


Of course you can include it in models, but the problems is in the planners who build the models. They get it wrong, because they are human. They will risk to save costs to appear better to the upper management, and so the model will be skewed. That's why, for all its problems, it's actually pretty good idea to trust in the inhumane system which is the free market.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: