Hacker News new | past | comments | ask | show | jobs | submit login
We've lost our respect for complexity (wilsoniumite.com)
105 points by Wilsoniumite 25 days ago | hide | past | favorite | 62 comments



The thing is, historically we never really had much respect for complexity.

> We talk about complex subjects all the time. Medicine, politics, economics, sociology, morality and more

For our entire history we have being talking about complex subjects we barely understood and, like proud primates, we were always very vocal and cocksure about our view points.

We experienced two cultural revolutions, first in the world of ancient Greece, subsequently in the modern era of scientific enlightenment. One can think of the emergent scientific method as effectively learning to respect complexity. Being humble about assertions that cannot be validated again and again, in different conditions, by different parties etc.

The tangible result of this new trait (respecting complexity) was pretty amazing. But only for a precious few domains. Our default mode, being 100% sure about things we have little clue about (who needs hallucinating AI when you've got humans), continues to be the prevalent one.

Whats worse, we now have what has been called "the pretence of knowledge" [1]. We know that real knowledge is powerful, so why not pretend we have it when we actually don't? This leads to a random mix of (typically self-serving) opinions coupled to the superficial use of scientific tools. A pathology most visible precisely in the above list of really complex topics.

The risk is that as the challenges of our own complex societies mount, we will undo also what we have achieved, and effectively go back to become stochastic parrots, unhinged from complex reality.

[1] https://www.nobelprize.org/prizes/economic-sciences/1974/hay...


Political action is fuelled by beliefs and ideals for the future. As a subjective observer, no one can attain a complete image of reality, or "the thing in itself". All we observe are appearances (phenomena) from our subjective point of view. In the process of a wider societal dialogue, we can exchange these points of view and eventually come to a rough approximation of the objective state of things.


> In the process of a wider societal dialogue, we can exchange these points of view and eventually come to a rough approximation of the objective state of things.

By and large discussion about politics in civil society seems to me riddled with factional language and lenses, with lies and hostile, bad-faith accusations, rather than resemble some Rawlsian disinterested dialogue.


> we can exchange these points of view

the quality of that exchange is indeed the key catalyst of whether a "convergence" happens. For matters concerning the physical world we have (largely) learned not to let subjectivity stand in the way, even when it concerns very complex phenomena. For anything more directly related to the human condition we are in far more primitive state.


In other words, this suggests nearly all the population are posers/pretenders some of the time…?


Humans are social creatures, and social dynamics are as important to us as laws of physics. For better or worse, in every-day life, being right is usually less important than being liked. The basics of survival in the real world enter common knowledge, so that e.g. you know you shouldn't eat random mushrooms, or foul-smelling stuff, nor should you swim in pools of funny-colored liquids should you encounter one outside. Beyond that, you're more likely to die from being a contrarian than from common beliefs being wrong.

As people experience it, the truth about objective reality matters only if being wrong can bite you in the ass. It matters when you're working with physical stuff, where being wrong could be dangerous (perhaps indirectly by ruining your boss's profits). There's a reason science and engineering leap forward during times of war - all the social and political games, all the fraud, all the weird beliefs about the world, suddenly stop being important when the enemy is at your gates, and your survival depends on whether your soldiers' rifles can shoot straight, and the missiles they fire can hit their designated targets. At those times, being wrong about complex aspects of reality is deadly, so it's okay to defy preexisting social dynamics and get these things right.

War isn't the only situation like this, of course. The economy sometimes produces such states, however briefly. A new invention delivering real value requires being right to develop and improve... until the point improvements get more expensive than marketing, at which point truth stops mattering again.

Not everyone considers truth as a terminal value. For those of us that do, living in a world where social dynamics are more important than being right, becomes extra challenging.


Not really, that would imply they are pretending. There aren't enough people with a genuine respect for truth for that to be a factor. It is more that any signal of knowledge will be copied mercilessly if it affords any respect. If people with bookshelves get more respect, then everyone will have a bookshelf and nonetheless be confused if they meet someone who reads a lot of books.

The default assumption is that a person has no idea what they are talking about because the base rate success of that stance is rather high.


Its all us, most of the time lol. From the blog post:

> All I can do is continue looking for those people who have that respect for complexity. Some find them boring, or indecisive, or just wrong for not buying into some extreme.

Ignoring complexity and still making decisions is a key survival trait. The trick in modern society is imho to know when complexity really matters.


> One can think of the emergent scientific method as effectively learning to respect complexity.

Which I'm reading like it's the ONLY way we know of how to "respect complexity".

> The tangible result of this new trait (respecting complexity) was pretty amazing. But only for a precious few domains. Our default mode, being 100% sure about things we have little clue about (who needs hallucinating AI when you've got humans), continues to be the prevalent one.

Which means if only everybody in other areas learned to respect complexity as we enlightened scientist do, everything would be great in these other areas too.

And so on...

This is classical enlightenment scientific positivism that is fuelled by technical prowess of the contemporary society.

It's only natural to assume that all these wonders in science can be replicated if we only applied the scientific method for everything, and if it's not working we need to just work harder and eventually find the right set of hypothesis and reductions to prove the necessary theories that will render break through in other areas. What's wrong with this thought?

Well first in philosophy, science is a set of assumptions and simplifications that limit the scope of search to what's measurable and ignores qualia. This is the first simplification. [1]

The second is reduction, which is responsible for the wonders in physics. You break things down, you study the components of reality in an isolated and reproducible way in hopes to understand the larger picture by means of decomposition. It's just that science isn't direct about how to put things together after reduction. In most areas it's ok, things are simple enough we can make do. But it's being catastrophic in areas like quantum physics. In fact the calculations start to skyrocket in complexity so fast that reductionism itself is put in question, specially when geometric objects like the amplituhedron are known to exist that simplify the calculation quite substantially but isn't prone to investigation by reduction. [2]

Which puts in question the assumption that science as you put in your comment is the correct way to manage complexity in all areas. This positivist approach to science is put in question by many other philosophers, including Karl Popper [3].

[1] https://platosmirror.com/galileo-galilei-measure-what-is-mea...

[2] Donald Hoffman, The Case Against Reality: Why Evolution Hid the Truth from Our Eyes ISBN-13: 978-0393254693, ISBN-10: 0393254690

[3] https://www.sobider.net/FileUpload/ep842424/File/12.two_crit...


> Which puts in question the assumption that science as you put in your comment is the correct way to manage complexity in all areas.

One important aspect of the scientific method is that the outcome "we don't know" is perfectly acceptable. Not sure there are other approaches that have this humility built-in but happy to be enlightened (pun :-)


Why are you so sure everybody else's don't accept "we don't know" as an acceptable answer? What makes you think only scientists are humble enough to accept such outcomes.

I believe you are romanticising science. It's done by humans it's prone human error just as much (reputation and money comes to mind). And even if it wasn't, if was done 100% correctly all the time, there are fundamental assumptions and simplifications needed to make the current scientific method work that's known to be incompatible with some domains of study, as I mentioned before.

And to answer your question, philosophers are the other kind of category of thinkers that would fit that description.

So by all means try to apply science to everything. It's been great so far. Just don't put too much expectation on it. It's disrespectful and unfair to science.


> What makes you think only scientists are humble enough to accept such outcomes?

In your previous comment you equated the scientific method with naive positivism that purports to explain "everything". While this is a recurrent pathology, especially during times of major scientific advances that embolden people to make ridiculous assertions (from Laplace to Wolfram), its not really a defining part of science.

I don't see science in any way as adversarial to philosophy. But it does have a track record of having diffused as a working thought framework whereas philosophy is still a rather elitist affair.


> In your previous comment you equated the scientific method with naive positivism that purports to explain "everything".

I never did that. You should try to read what others write with as much care are you employ in writing stuff yourself, including these elitist philosophers you are writing off so easily.


> read what others write

You started with this: "Which I'm reading like it's the ONLY way we know of how to "respect complexity" before embarking on tearing down strawmen.


> We experienced two cultural revolutions, first in the world of ancient Greece, subsequently in the modern era of scientific enlightenment.

Oh yeah, I remember them well!


The rise of complexity, and its collapse, defines every civilization for which we have historical records. Given that our society seem hell-bent on repeating the pattern, I question whether we ever had respect for it. It seems that we have nothing but arrogance to proclaim "this time we'll get it right" as we layer on the next layer of complexity.

[1] Joseph Tainter "The Collapse of Complex Societies"


Complexity has always been less respected the more it is separated from the experiences of the average person; or more personally, your own experiences. Unfortunately, that's the very nature of complexity itself. Dependencies increase instead of decrease in the hopes of scaling, saving time and money, or even providing a slightly better product. As positive as those efforts sound, they also make the failure cases worse.

The only people to respect with regards to complexity are those dealing with inherent complexity -- especially when they're building more digestible abstractions, and those working to reduce complexity due to having a substantial understand of it. It can be rather difficult to distinguish between inherent complexity and incidental complexity though.


I largely agree with this post except for the reference to "juvenoia". Case in point: the new US administration doesn't contain a lot of terribly young people, but they have also thoroughly lost the respect for complexity. All those "deep state" bureaucrats who at best don't seem to do anything useful and at worst dare to contradict you? Fire them all!


There is a balance though, because large organisations tend to ossify, yet the environment continues to change.


You’re making a straw man of their argument.

They understand what the bureaucrats purported to do, but don’t believe that government bureaucrats can manage a complex system and that better outcomes are obtained by allowing individuals to optimize in a complex system than attempt to centrally manage complexity.

I’d go so far as to say you have it exactly backwards:

It’s people who advocate for government bureaucratic management that don’t respect complexity by hanging onto the belief a complex system (eg, an economy) can be managed by a handful of credentialed experts.


This is a very extreme stance. It also is a false dichtomy. A government bureaucracy (a needlessly pejorative term, btw....) can do things and individuals also can do things. One does not exclude the other. It gets interesting when asking what things are best done by government and what things by individuals. The non-respecting of complexity is happening when one thinks that it is one or the other. And the government is involved in any contract anyway because if one side does not do what it obligates, the judicial system can be used to enforce it.


> This is a very extreme stance

It doesn't seem extreme at all. If you've ever worked on a technical project and been given a deadline by someone not in the project who also isn't technical, you'll know the problem with central planning.

> The non-respecting of complexity is happening when one thinks that it is one or the other

This isn't to do with complexity, except that generally the government (or any centrally managed thing) is worse at managing complexity than the government. The government managing something isn't because it's too complex for the private sector. It's because the private sector would have adverse incentives with society-level bad outcomes if it attempted it. E.g. private prisons without sufficient enforced regulation.


tangential but doesn't deep state mean a secret group of leadership? does it apply if one is a known public servant


'Deep state' could also be called 'permanent state' and means those unelected entities which hold power through changes of government.

Victoria Nuland is an example. She worked for six presidents and ten secretaries of state over 30+ years, always promoting the same war agenda. She has worked in concert with her husband Robert Kagan, who has worked as an adviser to several governments and think tanks. These people and organisations are all part of the deep state, aka security state, aka military industrial complex, as forewarned by President Eisenhower on his departure from office.


that sounds so cool better that than be starved out and slowly die like me while watching someone else use your SSN

so it's like lobbyists? or does rit have to be a specific industry


If we take this definition, which is a lot less conspiratorial than the association I've had with the term, it refers to influential longstanding politicians that kept a low profile and out of the media for the most part.


No — “deep state” can refer to the permanent bureaucracy operating according to their own agenda, in coordination with NGOs, etc.

That’s the “unauthorized networks of power” which run independently of democratic control, eg by coordinating non-prosecution of crime despite citizen demands for more criminal prosecutions.

> A deep state is a type of government made up of […] unauthorized networks of power operating independently of a state's political leadership in pursuit of their own agenda and goals.

https://en.wikipedia.org/wiki/Deep_state


does this assume democracy is real? as in the people have a say in the policies so anything to the contrary is a deep state?

i've never had a say over my life let alone politics so it's a very foreign concept like having agency


> state's political leadership

The political leadership of the US is elected; by definition, any network of power which isn’t beholden to that is “deep state”.


so lobby groups would be considered deep state? or rich people?

but how can something be deep state if it has existed since the inception of the aforementioned state. rich people have run the world since humans came into existence

is it like a on paper not in reality thing?


A democratic form of governance isn't required for a Deep State.

Let's rename Deep State as "Under State" for the purposes of this conversation to make it easier to understand.

Whatever is the government (the "State", like in "Head of State") you can see is the "Above State". Presidents, Prime Ministers, Kings, Governors, Legislatures, Courts, the government that supposedly runs the country, this is the Above State.

The Deep State, the Under State, operates under the Above State for their own goals and benefits accountable to noone, the only time they come up to the sunlight is to correct an Above State that dares to harm or defy the Under State. You can almost never see the Under State as a common citizen, but you can probably feel their influence if you pay attention.

"Deep" in Deep State hence refers to where they are figuratively located in a given country's political mechanism.


the hard oart for me to grasp is since the day of the theocratic state influence groups have been a thing

how can it be a separate thing if it has been a part of the state since its inception?


It's separate in the sense that the individuals involved are mostly employed clerks, scribes, middle management, and other such "common" public servants who serve for practically life careers without input from anyone, as far as the Above State side of things go. In the Under State, those same people actually run the government because they are the people who do all the heavy lifting.

The Above State with its Presidents and Secretaries and Legislators are all replacable and expendable, they're just an abstraction and obfuscation layer for the Under State to put their desires into action.

Occasionally you might catch glimpses of them when they play their hand too strongly, like the Iraqi Weapons of Mass Destruction(tm) narrative pushed by the Intelligence Community (the Under State) with State Secretary Colin Powell (the Above State) acting as their mouthpiece. On rare occasions a Deep State agent might turn coat and blow the secrets as was the case with Snowden.


i think i'm getting it so where does lobbying and paying for campaigns come into play?

i've been in a cult my entire life (to give you an idea i live in a house full of fanatics rn) so the thought of voting or having a say seem very alien to me

why do people care so much about that?


>why do people care so much about that?

Because governments are usually accountable to someone or something.

For example, in most monarchies it's royal blood or even a god. In democracies like the US it's the voters.

Deep States are more powerful than governments (they are the power) and not accountable to anyone and arguably not even desired by the people governed, that is why people care.


Complexity is important when a subject at is core is complex, I'm mostly against complexity for the sake of complexity. Too many subjects are getting complex for the sake of it, or due to organic development.

There is an even greater beauty than complexity when very complex subjects can be reduced to simpler ones without losing their power, it requires a greater understanding of complexity than what is usually seen.

As an illustrative example : QED: The Strange Theory of Light and Matter, Feynman


A trillion dollar marketing industry condemns enquiry and threatens mild inconvenience as the terrible fate awaiting anyone who errs from the path of simplicity. The word "just" has a lot to answer for. "Just do it". "It just works". Complexity is an acquired taste. It is somewhat rebellious to celebrate and indulge in it.


I feel like "Complexity" is the wrong word here.

Although I agree with the notion that deep knowledge of complex systems has decreased over the years, I'd argue that the problem isn't the lack of interest in gaining deeper knowledge that's the problem, nor a perceived lack of respect for expertise.

I'd argue its more to do with the nature of how large codebases morph over time as they're retrofitted to do more usercases. Its a lot easy in the short term to just hack in the functionality you need into an existing codebase, instead of rewriting the entire thing from scratch every time. This comes at the cost of maintainability and leads to a lot of headaches in the long term, as systems that where never designed to work together clash in unexpected ways.

I don't begrudge anyone for not wanted to dive into codebases like that.


> I feel like "Complexity" is the wrong word here. [...] I'd argue its more to do with the nature of how large codebases morph over time as they're retrofitted to do more usercases. Its a lot easy in the short term to just hack in the functionality you need into an existing codebase, instead of rewriting the entire thing from scratch every time. This comes at the cost of maintainability and leads to a lot of headaches in the long term, as systems that where never designed to work together clash in unexpected ways.

Rich Hickey separates complexity into two parts[0], accidental and intentional (IIRC). Intentional complexity is complexity that stems from the domain of your problem, there isn't much you can do to manage it, DNA splicing will be complex no matter what you do, for example.

The other part is accidental complexity, which is complexity you could deal with, but for whatever reason choose not to right now, as you touch on in your comment. This eventually snowballs into a huge spaghetti monster as you pile hacks on top of hacks.

There is a (famous at this point) talk from Hickey where he dives much deeper into how he sees complexity, which happens to be a really good pitch for Clojure, but I first saw it before I knew Clojure, and the perspective applies to software in general, so no Clojure knowledge needed.

[0] - "Simple Made Easy" - Rich Hickey (2011) - https://www.youtube.com/watch?v=SxdOUGdseq4


There is a part in the Life of Brian, where Brian "helps"/intervenes in a complex assembly/manufacturing operation, which results,instantly, in chaos and destruction. The scripting and effort to film that short piece must have required a true understanding of complexity, and what happens when it isn't recognised and respected. Unfortunately, complexity, and the respect for profesionalism, is weaponised, to produce, mountains(litteral) of junk, that then has to be payed for once, and then again when it proves to be junk, and then a third time to get rid of, and fourth and fifth, pehaps to get strait.


In academia, respect for complexity still exists, but industries prioritize simplicity to enable mass production, reduce costs, and make things easier for people to understand, remember, and use. That’s why logos are becoming flat, with simple shapes and colors, buildings are often bland rectangular boxes, and apps have fewer texts and buttons on each screen.

As with almost everything, we need to find a balance between simplicity and complexity.


I feel like the reason for the change in design trends was that for most of human history, ornate details and complexity required immense labour and signified wealth.

Then with modern technology, we could print or weave any pattern you want on fabric effortlessly. Instead of a stone mason chiseling away all day, you can pour plaster in a mould and have all that same detail.

Now that detail is cheap, simplicity is on trend.


You're right, although once something isn't impressive to do it converts to a fashion-related trend. Fashion doesn't often have a practical benefit, so a lot of it is just "I want my stuff to look different to my parents' stuff, and that makes fashion cycles, where people's fashionable floral wallpaper now isn't too different from what their grandparents had.


Folding Ideas is one of the best channels on YouTube- glad to see it brought up here!


In academia, people had respect for complexity.

In industry, nobody seems to use big-O notation.


> In industry, nobody seems to use big-O notation.

Ironically, it's because big-O notation is too theoretical and doesn't respect real-world complexity. Big-O is used to see when the operations are of sizes out of normal use and big-O doesn't represent hardware complexity (like latency differences, size limitations, or caching).


A gentleman knows is someone who how to use big-O notation and doesn't.

The only times I have used big-O in my career has been job interviews. It is my belief that it's mostly used to weed out people who didn't take computer science classes, and make them feel bad regardless of their real world skills.

It's the same thing with database normalization. It has come up twice in my twenty year career. Once in an interview, and once during an acquisition when someone from the purchasing company was I believe trying to "Gotcha!" me. 3NF just doesn't come up in normal day to day work.


N is usually small in practice.

Analyzing how expensive it is to do something often costs more than just doing the thing.


I'm sure there's a sampling bias here. Many intractable methods are never adopted by the developer community exactly because they're intractable, which means such methods won't be encountered often. Like an ant line avoiding obstacles. Only pioneers/people developing new methods encounter them.


It is not "complexity good" versus "complexity bad". It is all about legibility, and legibility emerges from how complexity is managed.


I do not respect complexity. 99% of complexity is not necessary. I respect simplicity. Complexity appears naturally, there's nothing respectable about it, it's just the way world evolves. Simplicity takes lots of effort to achieve and even more effort to maintain.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.


So you don't respect semicon, airplanes, electricity transmission and distribution, running water... What do you respect then?


Complexity may be your enemy, but you should respect your enemy, or it will defeat you.

Disrespecting complexity usually goes "that complex thing is a mess, let's make something simple instead", which usually results in the "simple" thing being useless, and often not that simple. Respect complexity, especially if your goal is to remove it.


Rather than accuse you, I'll just refute you thusly:

In order to achieve perfection, you must understand what should be removed. To understand what should be removed, you must respect the complexity.

Respect is not acceptance, even though it's very easy to confuse the two.


Did you read the article? The author's and your points are entirely tangential.


Seems like this might be the same as the current popular disdain for experts and being educated.


It is about the number of connections in a graph.


>That’s not been entirely without good reason, many things were promised then that have failed to materialize. From flying cars to world peace

i mean, Flying cars has been a reality for quite some time now, they just go by another name, "helicopters".


I don't think we were promised those things.


An idiot admires complexity, a genius admires simplicity - Terry Davis


>We are constantly asking our brains to grasp at complex topics and distill down at least our own perception of them to something manageable.

Considering "DON'T TREAD ON ME"[1] is a meme that goes back all the way to 1776, the human desire to simplify what is needlessly complicated is nothing new and certainly nothing unexpected even today. This is especially so for Joe Average who more than likely is simply way too busy already with more pressing affairs to daily life.

>Now, when many may challenge that notion of consistent progress of the world toward a brighter future, letting somebody else do it just doesn’t seem so right.

This too is "older than history"[2]. A common (misattributed?) quote from Mark Twain goes that "If you don't read the newspapers, you are uninformed. If you do read them, you are misinformed."[3]

It has always been preferable to have a healthy appetite for critical thought and to be skeptical of everything. Critical thought is how you form your own opinions, your own beliefs, your own ideals and goals in life, your own identity. And indeed, the discipline of science is predicated on putting forth hypotheses and testing them; that is questioning how the world works and whether our current understanding stands up to questioning.

What was ridiculous was the deference to higher authority without regard. Noone has the time nor capacity to question and understand everything, especially Joe Average, but to then fundamentally give up your own freedom of thought and defer it to "some group of people who would make it their life careers and ... trust them" is what was and is ridiculous. The widely derided NPC meme starts becoming less of a meme if you trust someone to tell you how to think. Around these parts we call that programming: Telling computers how to think and what to do.

>we now live in a time ... that understanding the full complexity of every topic that might cross your path is not only possible, but somehow, expected.

Personally as a millenial (age mid-30s), I grew up on a healthy course of being taught to think for myself. Essentially, that is a natural conclusion to being taught that we should strive to truly understand the world around us.

>Still, clear answers don’t seem ahead of us

The answer is actually simple: Use experts (not "experts" like some pundit on television or a political office holder, actual real experts out in the field who know their stuff and speak objectively) as a potential guide (not an authority!) towards striving to better understand the world around you.

We quite literally have a limited capacity to care[4], we must pick and choose what we concern ourselves with even as we strive to understand. Logically then, we should first strive to understand the world around us because that's what is immediately relevant to our daily lives.

Progressivism today cares so much about things that are faraway and irrelevant to your daily life, and because you have a limited capacity to care that leaves you unable to care for the people and things in your life. That is simply stupid.

Understand the people around you, not some faceless entity on the other side of your continent/island or even the planet you hear on social media. Understand the society around you, not some construct you will probably never even travel to.

Understand the world around you, your life, and you might realize that life and the world are actually pretty nice.

[1]: https://en.wikipedia.org/wiki/Don%27t_Tread_on_Me

[2]: https://tvtropes.org/pmwiki/pmwiki.php/Main/OlderThanTheyThi...

[3]: https://old.reddit.com/r/QuotesPorn/comments/exkeku/if_you_d...

[4]: https://en.wikipedia.org/wiki/Dunbar's_number


We're recreating a demon haunted world that will foster cargo cults. The algorithms we interact with are inscrutible, so we make up rules of thumb and assign agency where there is none. E.g. In 5 years there won't be a way to truly know how to minimize your personally-customized prices at the grocery store, but there will be tons of cargo cult opinions (put your phone in airplane mode, walk through iles 3 times before you pick up an item, pick up a generic version before the name brand, ...).

Algorithms and AI are the new divine agents to appease with ritual.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: