If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. The only reason we have Docker on our machine is to test the images we deploy (just to make sure they build and run).
You have experienced what happens when web developers cosplay as software architects. That's when you get "microservices". It's a travesty, and it has ruined an entire generation of software developers who now believe that partitioning and autonomy are just myths.
So, as a blunt statement - your experience is typical. That is, it's average. That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the bell curve doing what it always does.
> So, as a blunt statement - your experience is typical. That is, it's average.
Definitely a fair judgement and would agree that this constitutes poor architecting on many many companies' parts including ones that I've been a part of.
> Every one of our 100+ components maintained by our dozen person team can be run and tested on its own.
Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team?
More directly, what's wrong with a monolith at the dozen person team size?
> Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team?
> More directly, what's wrong with a monolith at the dozen person team size?
Great question. It's honestly a hard question to answer because the real answer requires tacit knowledge, but I'll try anyway because you asked. I'll link a couple articles of longer form writing that may help to answer as well.
The benefit ultimately is productivity. That productivity comes from autonomy, which comes from partitioning.
Each component can be worked, tested, and deployed on its own. We never need to spin up more than one at a time, which means any time we are working on our code, we are typically working on 1/400th of it, and we know we are because we have 400 separate GitHub repositories. We can't accidentally get peanut butter in our mayonaise. Each part is independently built and tested.
When it's time to upgrade Rails, we can upgrade one application. This is 1/20th of our Rails code. That's a smaller batch. We know from basic flow principles that (see: Principles of Product Development Flow, Lean, etc.) that small batches are better for productivity.
We rarely have merge conflicts. We don't do pull requests. We branch when we need to. We are often working on a brand new, never been deployed project. We don't have our tests running in CI. Our tests run in less than a second in most projects on our laptops. With those that have UI, our tests typically run in less than 30 seconds which include thorough Capybara (UI Interaction) testing.
When we are working in any application, we only need to consider its direct efferents when making a change to any interfaces it exposes. We can trace those changes through easily when we need to make them. Because we practice software design, we don't often need to make large sweeping changes, but when we do, we can do them methodically without disruption.
The "worst" part is that some things are tedious. But we can automate tedious things. We can invent around tedious things. We used to have to deploy 15 web applications any time we had a style change. Now we deploy one application that supplies the CSS for all our applications via SSI (server-side-include). If we need to deploy 70 back end components, we can script that (with manual verification at each step). We can even spread that out across a couple team members and do it as an ensemble. It takes about an hour to deploy everything. We maybe do this once a month or so (when something highly afferent changes -- and guess what, we structure our design so that afferent things change as infrequently as possible).
Most things we work on we work until they are "complete". That is, they rarely change after that. We have components that have been in production for 3 years and haven't been substantially touched aside from Ruby upgrades and the like. They are still perfectly in control.
The list goes on and on.
Here are a few articles, feel free to poke around the others and ask any follow up questions.
Great that it works for you and your particular project.
Without knowing the details if your project and team, I’d take a monolith any day over 400 microrepos and the 70+ backend microservices. Our customers don’t care if it’s microservices or monoliths as long as it works and meets SLAs, so the question really is do I want to upgrade ruby or rails 400 times and deploy 100 services or just do it once in my monolith and deploy once?
Need to upgrade a gem that’s used in 100 of those repos to address a CVE - no thanks. Worse, 10 of those repos were owned by someone who left the company a year ago and since there were no PRs, no one knows how to work in the repo or understand what it does.
That isn't actually the question. I know it's not the question because we're capable of it, it is infrequent, and it takes on the order of single digit person hours a month for that type of concern for our particular project. Also, I didn't say 400 anything but repos. See some of my other comments for what those repos consist of.
> 10 of those repos were owned by someone who left the company a year ago
That's not how it works. Repos aren't "owned" by someone. Each repo stands on its own, is clearly testable on its own. Every one of our teammates is capable of updating every one of our projects because there is consistency between them.
If upgrading Rails or any gem scares you, you have other problems to deal with. For me personally (I'm relatively fast on this team) I can update Rails in all 20 of our Rails applications in maybe 30 minutes unless Rails caused a setback (this is somewhat frequent between propshaft and turbo). If Rails caused a setback, we have two dependencies (rails-application-operational and rails-application-development) that we can either include a patch or pin Rails. All told, it's a quick process.
You know what we don't have to spend time doing? Tracing callbacks across 30 entangled ActiveRecord models. Or anything else that comes from entanglement. We also don't need to entertain palliatives like packwerk and the like.
How do you know that? Hint: you don’t. You’re not the first to tell us what we do is impossible, and you won’t be the last.
By the way, I know how it sounds. And I know why a person would fear it. And, I know why they’d be incorrect in their assessment, because, unlike them, I have significant material experience in both camps.
This whole business has had few conceptual ideas that stand apart from fashion and stood the tests of time. A lot of people look down on C now for various reasons, but it's still extremely useful and going strong 50+ years later. Even though I'm older than C itself in the last year or two using it intensively, I found new insights in how to arrange things and what to avoid to make things more robust and maintainable.
It's not likely that hundreds of "microrepos" on github and even rails are The Last Word that people will eventually converge on and be using into the 2100s. It sounds like each repo is its own standalone thing with its own deps. There are advantages and disadvantages to maintaining this kind of thing, some of the disadvantages will only become clear as the deps evolve and mutate. Although you may feel you are in a local optimum with this architecture in 2023, ahead of the game, energized by your belief in it, you probably are not finished evolving your understanding even on the scale of one or five years.
One way or another almost everything in this business is ephemeral, disposable and will be disposed of in favour of something else.
Do you know what does stand the test of time? Design principles that are based on the immutable laws of physics. There's a reason that I can quote Plato to support the notion of partitioning. There's a reason that the basic principles are used in the design of every single significant thing in the world. There's a reason Linux has as many modules as it has (for them, a single monorepo works, but multiple works too -- see any library or application that isn't in the primary monorepo but is distributed via a package manager).
Also, no one said "microrepos". Don't put the current fad's fixation on what I am describing. I'm simply describing structural design. Does thing X and thing Y need common behavior? Is it actually generalized behavior? That's what we have called a "library" since we've been building non-trivial applications out of software.
Are there two disparate business processes that don't need to share states, and you know they don't need to share states because you've done the design work? Those are two separate components. Pub/Sub exists for collaboration between those components as necessary.
I'm not trying to say that our current technique for all things is the end-all-be-all. That would deny the continuous improvement mindset (i.e., the relentless pursuit of perfection) that got us where we are. But, there are things that won't change. Smaller batches will always be (generally) easier to manage than larger. Taking something that has high afferent coupling and adding specializations to it will always be a mistake (See Fat Model, Skinny Controller).
The reality is that there are software design mistakes that are knowably mistakes. They've been known for some time as well and can be measured. Those are the things that we avoid and those are the principles that have guided where we are.
I know this sounds highfalutin, but there is someone on our team that has been doing this for over 30 years (by "this", I mean continuous improvement/seeking perfection) and has studied under many of the greats in our industry. We aren't just making this stuff up.
Nobody is denying partitioning is helpful. It’s the implementation details of that partitioning that cause the debate. IMO using repos as a partition boundary is an abuse of git. And IMO invoking Plato to make a point is not helpful. Argumentum ad antiquitatem. This goes for the “30 years” comment as well: it’s entirely conceivable that they’ve been doing things inefficiently for 30 years. More likely that there is such a huge sunk cost that they are heavily invested in justifying it.
> IMO using repos as a partition boundary is an abuse of git.
How so? Why don't we put our Rails applications in the Rails repository?
> And IMO invoking Plato to make a point is not helpful. Argumentum ad antiquitatem
What I am pointing to is a common thread. Plato, Christopher Alexander, Edwards Deming, etc. What I am arguing for is anything but common tradition. It's counter to what is current "common knowledge".
> This goes for the “30 years” comment as well: it’s entirely conceivable that they’ve been doing things inefficiently for 30 years.
Yes, entirely possible. And if we find a better way to do things, we will be first in line.
> More likely that there is such a huge sunk cost that they are heavily invested in justifying it.
As someone who has worked in monoliths and microservices, monorepos and github galaxies, I think it would be better if we could converge on a monorepo managed by something like Bazel where you can just create a new build target that lists its dependencies, and can work directly with the dependency graph via its querying mechanism. Using repositories to encapsulate dependencies and build scripts I think is much too heavy of an abstraction and leads to poor discoverability. When I have to deal with that I usually wind up treating it like a monorepo anyways by creating automation to check out all repos and manage syncing them.
> When I have to deal with that I usually wind up treating it like a monorepo anyways by creating automation to check out all repos and manage syncing them.
Yes! Absolutely. With one key difference: You can't commit to multiple repos at the same time. You can certainly have them all checked out locally and search across them all, and we provide scripts to do that. Example from the Eventide project: https://github.com/eventide-project/contributor-assets
This is an open source project. Our project is based on many of the ideas and techniques. They've held up for 7 years or so.
Eh? 400 components without any integration testing?? Who glues the sum of the parts together into a whole? This can’t be a single application. Sounds more like many completely independent applications to me.
Who said we don't do integration testing? We likely don't do it in the way that you would imagine, because we don't need to. We also don't call it integration testing, we just call it testing. We have a test environment where we deploy to to do final inspection. It is a common occurrence that we deploy a cohesive set of new components for the first time and they all work together, because each component is tested in isolation including integration with its afferents. Each afferent (library or other component) exposes diagnostic substitutes and we use dependency injection. You can read more about that here: http://docs.eventide-project.org/user-guide/useful-objects.h...
Also, I never said 400 components -- we aren't there, yet. It's ~100 components. It's just 400 repositories. 100 of those are deploy projects (they simply contain scripts, a dockerfile, helm values, etc.) most of which is boilerplate.
Yes, thought so as well. "Instead of a single search and replace we get to a flow state and create an automated script to do it for all our repositories. Due to flow state productivity grains are immense. Of course we will manually check 400 times if the script did correct work, but we enjoy that each time."
Usually, when you put quotes around something, it's considered proper to actually be quoting someone instead of trying your best to make them sound like a clown.
But, why don't I go ahead and give you a real quote.
Of our 400+ repos, 100 or so of them are deploy projects. The only time they need to change in batch is if one of our boilerplate scripts changes. This is infrequent, but when it does happen, we have a "project-scripts" repository that contains our boilerplate scripts as well as scripts to update them across all of our repositories. This doesn't always require individual verification, but when it does, it's typically a matter of installing dependencies and starting the application.
Many text editors are capable of doing "single search and replace" across multiple repositories. All of our repos are in a single project directory, so we have a root, just like anyone with a monorepo does. Rarely does that "single search and replace" touch more than a small handful of projects. So, except in the extreme case, you are only running a fraction of the tests or verifications when you make one of these "search and replace" changes. Compare this to a monorepo, where, unless you have specialized tooling to slice your test suite while considering your package dependency tree, you end up needing to run all of your tests.
If you are responsible and your project is in control, you run the tests on your laptop first, and then (typically) you push your commit and it runs in CI. Actually, more typically, you push your commit to a branch, open a pull request, have it run in CI, have it reviewed, then merge it, where it runs in CI again. Each time it ran in CI, it ran your entire test suite (usually).
We don't do most of that. Most of what every team takes for granted, we just don't do. So, when someone imagines having to do their typical process for getting a change integrated 100 times, they are right to fear that. But, they are creating a false equivalence. We don't do that, because we've eliminated all of that waste from our process. What we do 100 times (again, the once or twice a month that we do it), it takes a fraction of the offort what most teams do once to get a single change in.
There's a reason that Toyota ate the west's lunch when it comes to manufacturing. They understand just-in-time (one-piece flow) and small batch sizes. They know how to address problems at their root and systematically eliminate waste.
Most in the software industry know how to do one thing: apply the thing the latest dev celebrity said in their latest tweet or blog post.
Here's the difference between me and that celebrity: I'll tell you straight up not to do what we do. I'll tell you straight up it will take you years with guidance to get anywhere close to what we do. I'm just telling you it's possible and we don't actually need to live in software shanty towns our whole lives.
Someone got to say it so I will. Most of your comments comes across like you've gone off the rails and started replacing competency, not to mention humility, with narcissism. Being so cocksure and backing it up with statements that you need to be as brilliant as you to "understand" is such a laughingly large red flag that I'm not surprised people interpret it as satire. I really hope your colleagues are onboard with this and that this isn't your own dogmatic crusade dragging them along.
I don't read cockiness or narcissism in his posts. Having just started reading Clean Architecture by Robert C. Martin two days ago (and hoping to complete it tonight, then planning on re-reading it again), it seems to me that hmeh has learned and PRACTICED what is discussed in that book. I wish I were in their shoes!
Clean Code (What I assume you are referring to with 'CC') is 10 years older than Clean Architecture. I haven't read Clean Code, I just picked up Clean Architecture, but I venture to guess some of his thoughts have changed since then. I also assume that Clean Architecture deals with higher level issues than Clean Code. I may have to pickup Clean Code to see what he said in that.
Thank you. I saw your other post, but it’s gone now. If you want to find me, I’m in the Eventide project Slack and on GitHub (you can find links to some of my posts here in this thread).
Let's just say that that sphere of the software dev community isn't exactly universally seen as competent work, more like the orthodoxy of ~15 years ago that's now increasingly, and in my opinion rightfully, being put into question for its bad complexity and performance trade-offs when applied beyond textbooks and conference talks. And this is being diplomatic compared to others.
I'm certainly not saying there isn't something to learn from them though, I've read a bunch of them, but after seeing countless wasted debugging hours and projects devolve into a complete mess of patterns where you have to ping-pong between files, functions, and patterns providing little to no value is astounding. The option to do non-clever code is seemingly insulting to the self-professed architect, but it is in my own experience at least, vastly more maintainable and productive, not because it results in "clean" code - it's usually pretty dirty and nothing to write home about, but because reading even a page-long flat procedural function that's just doing its thing, is much more straight-forward than ping-ponging between some clever architect's conglomeration of decaying design patterns.
Having the "meat" of the code available with a minimal amount of abstractions/layers also makes it much easier to adapt according to performance requirements without having to propagate changes throughout half the application due to it being entangled in pattern abstractions. (on that note, it sounds like hmeh is trying to work around this problem with abstractions by doing a significant amount artificial partitioning. But that would be solving a mess you've created yourself in the first place). Finally it will still be dumb & dirty code a couple years from now, no need to have all those design patterns internalized.
People promoting these things will of course then just blame it on the dev and call them average, or lacking in experience, because they can't identify and combine the various clever patterns correctly, they "are doing it wrong", and that's where I believe the above narcissistic tendencies I called out comes from.
I see a difference between "Clean Code" and code design patterns and "Clean Architecture". It seems to me that most think I'm speaking of Uncles Clean Code book/ideas when I'm thinking about his Clean Architecture book/ideas.
> The option to do non-clever code is seemingly insulting to the self-professed architect, but it is in my own experience at least, vastly more maintainable and productive, not because it results in "clean" code - it's usually pretty dirty and nothing to write home about, but because reading even a page-long flat procedural function that's just doing its thing, is much more straight-forward than ping-ponging between some clever architect's conglomeration of decaying design patterns.
With all due respect, if you think you are talking about anything that I was attempting to describe, you are not. Everything that we do is based on simplicity and building the most simple things that we can. We may have different definitions of simplicity however. For mine, check out Rich Hickey's "Simple Made Easy" talk if you haven't seen it.
We simplify things by de-complecting or, avoiding entanglement. Having separate repositories is just one way to re-enforce the simplification that we put in place with design. We call that structural design (because it's the design of the actual structure of the application).
> on that note, it sounds like hmeh is trying to work around this problem with abstractions by doing a significant amount artificial partitioning.
I can guarantee you that none of the partitioning is "artificial". I think at this point that you have projected so much of your experience into what I am saying that it's impossible for you to see anything I say in any other light. This, by the way, is what I mean by tacit knowledge. I say X, and you either hear Q, or you hear XIENRSTIENRSTIERNST, either way, you cannot possible understand what I am saying. This isn't a negative reflection on you, or a positive reflection on me. It's simply the way that tacit knowledge works, and it's why most "internet arguments" are wastes of time.
> People promoting these things will of course then just blame it on the dev and call them average, or lacking in experience, because they can't identify and combine the various clever patterns correctly, they "are doing it wrong", and that's where I believe the above narcissistic tendencies I called out comes from.
And yet, it is literally lack of experience (and ability) that prevents people from doing things they cannot yet do. That's how expertise works. Pointing that out is not narcissism. A gardener does not have the experience (or training) necessary to do brain surgery. This isn't a mark against the gardener. The difference in our industry is that it is filled with expert beginners. People who think that because they make 6-figure salaries after their limited education and experience they can now understand everything there is to understand about software design. This just isn't the case. If a person sees that as a slight to a fellow human, that says more about them than it does about the person pointing to that reality.
> Someone got to say it so I will. Most of your comments comes across like you've gone of the rails and started replacing competency, not to mention humility, with narcissism. Being so cocksure and backing it up with statements that you need to be as brilliant as you to "understand" is such a laughingly large red flag that I'm not surprised people interpret it as satire.
Where did I say that, exactly? I’ve said nothing about intellect. I’ve mentioned tacit knowledge multiple times, because I have enough experience to recognize how ludicrous a lot of this sounds. It sounded that way to me at first until I experienced it and saw the pieces fit together. That’s how tacit knowledge works, especially when it is counter to average knowledge. It’s the same reason American industry smashed Japanese cars in demonstration instead of listening to them when they said large batches were bad (I’m simplifying here somewhat)
And which part, specifically sounds incompetent? I’m happy to discuss that.
By the way, it's decidedly difficult to challenge a common belief, dare I say, orthodoxy, without sounding insane, confident, or even cocky. Could I be better at it? Of course, and I work at it, but I also am not here to make friends. I'm here to provide a perspective that I believe is sorely lacking from the development community and is drowned out by the orthodoxy, its acolytes, and its beneficiaries.
> I really hope your colleagues are onboard with this and that this isn't your own dogmatic crusade dragging them along.
Given their tenure on the team and their other options, it’s a pretty safe bet. It’s not my crusade, btw, it’s our teams goal to manage a relatively complex project with a relatively small team.
Also, someone has to say it, so I will. You have now dropped this conversation, which was about the work, into ad hominem and straw man attacks. That’s one of the worst part about this industry: people’s inability to debate and discuss without attacking people or glorifying celebrities.
> into ad hominem and straw man attacks. That’s one of the worst part about this industry: people’s inability to debate and discuss
Let's see here:
> You have experienced what happens when web developers cosplay as software architects.
> That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the bell curve doing what it always does.
> because, unlike them, I have significant material experience in both camps.
> It’s hard to imagine worse advice.
> But please, by all means, continue to spread disinformation and keep us in the dark ages.
> Just read the author’s bio. This is a person that appears to have zero software design experience writing an article telling you to ignore software design and just respect your team configuration. I call this Conway’s Confusion.
That sets a good friendly tone huh? If you can't take push back, don't be an arse to begin with.
> And which part, specifically sounds incompetent? I’m happy to discuss that.
Your arrogance precludes a fruitful discussion. But I believe that this needs to be called out, if nothing else to nudge other people to also do it when they see it, or to, albeit much less likely, nudge you towards eating some humble pie.
> You have experienced what happens when web developers cosplay as software architects.
I agree that this is overly snippy to the point of being counter-productive. It represents a particular emotional frustration with the state of our industry. We knew what ended up being microservices (web API servers making web API calls to other web API servers) was a failure mode. We knew you couldn't just introduce network hops and call it "architecture". So yes, I'm annoyed about that such that I'm willing to call out nameless "web developers", but I understand how that can create discomfort.
> That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the bell curve doing what it always does.
Indeed, and the person I was responding to effectively acknowledged this. The sooner we can recognize that the skill curve/technology adoption curve are real things, and crossing the chasm takes hard work, the sooner we can stop leaving people behind said chasm. It's not easy. Nothing I said here is untrue. If it is, please point it out.
> because, unlike them, I have significant material experience in both camps.
Again, I'm pointing to a material difference. Most people who are attempting to refute what I am saying have never done what I am saying. They are doing it from a position of fear, uncertainty, and doubt, or worse. I've been where they are. I've fought the fight they are fighting and, thankfully, I lost, and was introduced to new ways of doing things and seeing things. If a person has done what I'm discussing, I would expect them to say that and tell me why it failed. Instead I get people telling me it can't work and I'm incompetent, etc.
> It’s hard to imagine worse advice.
> Just read the author’s bio. This is a person that appears to have zero software design experience writing an article telling you to ignore software design and just respect your team configuration. I call this Conway’s Confusion.
Putting these together because they are about the OP article and the OP. "hard to imagine worse advice" is hyperbole, but it is bad advice. You can't force fit concepts to teams. I mean, you can try, but you'll always end up with unnaturalness.
The actual OP's bio talks about their career. None of it mentions software design. If a person with zero surgery experience started writing about how to properly set up operating rooms, you can darn well bet they will be called a charlatan and called to the carpet. If they aren't people may die. Software isn't that serious usually, but it's not hard to imagine that there have been billions upon billions of dollars flushed down the toilet for the sake of poor software design that no one speaks out against.
> But please, by all means, continue to spread disinformation and keep us in the dark ages.
Yes, overly cynical and unnecessary. I made my point prior to this and I didn't need to add this.
> That sets a good friendly tone huh? If you can't take push back, don't be an arse to begin with.
Some was unnecessary yes, thanks for calling it out. The rest represents what I think is healthy push-back against an orthodoxy that causes more harm than good. We can achieve more as an industry and make our way out of the realm of hobbyists and into something more serious. Many of us call ourselves engineers, but nothing we do resembles anything that people who are actually classically trained engineers do.
> > And which part, specifically sounds incompetent? I’m happy to discuss that.
> Your arrogance precludes a fruitful discussion. But I believe that this needs to be called out, if nothing else to nudge other people to also do it when they see it, or to, albeit much less likely, nudge you towards eating some humble pie.
Just to be clear: you said that what I was saying sounded incompetent. I asked you about that, and you're telling me I appear to be too arrogant for you to tell me why I sound incompetent? Please, tell me what I said that sounds incompetent. Let's move past the ad hominem portion of the discussion now and talk about the actual substance. I'll do my best to refrain from hyperbole and unnecessary snark.
Answered somewhat what I mean by at least the "code smells" and idiosyncrasies (competency was probably too harsh with given the amount of info) in a reply to another user.
> Your arrogance precludes a fruitful discussion.
This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience as the answer to why your setup is not suspicious, but excellent. Not sure what can come out of a discussion when that's the answer to everything, well nice for you, but a setup that doesn't scale with employee count (and thus competency/experience/interest differences) isn't exactly something that you can go around bragging about.
> This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience as the answer to why your setup is not suspicious, but excellent.
I touched on this in my reply to the other user, but at this point, I believe you may just not know what tacit knowledge is. You continue to think that I am pointing to our own brilliance. I'm not. I'm calling a spade a spade. I recognize what it takes to gain particular types of knowledge (tacit, or subtle knowledge), and I recognize that it's this reality that prevents most conversations about techniques from being fruitful.
Each participant will put their own experience behind the meaning of their words (and worse, their conversation partner's words) and it will prevent them from recognizing what one another are saying. The only way to have a fruitful discussion is for both sides to be capable of recognizing when that is happening. Since, in my experience, most people aren't -- they'd rather die on their hill than recognize that the person they are talking to is simply on the same hill but sees it differently, or they are on a different hill that really is better, but it cannot be seen as such yet, because it is over the horizon of their knowledge.
I don't actually know if you are interested in understanding this more or if you joined the conversation just to try to put me in my place, but if you are, here are some articles that may help:
Once you can recognize that there is subtle knowledge you might actually see my pointing to it as attempting to keep the conversation from devolving into exactly the type of thing that it tends to devolve into. Or not.
> I believe you may just not know what tacit knowledge is
Not sure where you got that from, haven't said anything about it nor did I include that in my quotes.
I'm well aware of it. But if I have one comment on it would be that I see it more as something an experienced person (or someone with a natural knack for it) makes use of under-the-hood, the resulting quality of the output however can usually be recognized by everyone, not something reserved for the "blessed ones". Take the redis source code for example (quite a few years since the last time I read it though). The author clearly has this skill, but one doesn't have to possess that to recognize the code quality (and btw, iirc without any mentions of a pile of design patterns/methodologies, just "doing it", but to each their own).
So therefore I'm a bit suspicious of anyone that claims that their idiosyncratic setup is actually simple if you just have experience enough to be able to judge it. I'm not saying that everything can be obvious at first glance but the vagueness triggers my radar after being worn down by experiencing way too much over-engineering motivated by self-serving vagueness and/or word salads ("baffle them with bullshit").
That said, if a code base keeps requiring you to make the correct design decision using subtle knowledge, that's a fragile situation and something seems off. You've probably already made the code base too complicated, my prejudices (somewhat confirmed by the language used in one the articles you linked [1]) would be through a pile of design patterns inspired abstractions that now everyone needs to be able to juggle at all times.
[1] "For example, the Tell, Don’t Ask principle can’t be expressed directly in terms of Push, Don’t Pull, which has a more common name: encapsulation. And each one of these qualities reflects cohesion and coupling. And furthermore, cohesion and coupling are inter-related and affect each other. Afference and efference are kinds of coupling. Afference is inbound coupling, and efference is outbound coupling"
> Not sure where you got that from, haven't said anything about it nor did I include that in my quotes.
I got it from here:
> This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience ...
and
> Being so cocksure and backing it up with statements that you need to be as brilliant as you to "understand" is such a laughingly large red flag...
Given that at no point did I point to our "brilliance", I assumed you refererring to my pointing to tacit knowledge:
> It's honestly a hard question to answer because the real answer requires tacit knowledge...
> ... because much of what we do requires tacit knowledge to see the benefit of ...
So if I'm mistaken, I apologize, but please do point out where I claimed brilliance. I am certainly claiming experience, and expertise, but those are earned, as they are in any profession. It's also true that no one in this conversation (including you) other than me have actually seen our code, and therefore would be ill-equipped to judge it. That is, they do not have the experience of our code base. Rushes to judge it based on pre-conceived notions only reflect a lack of dilligence, integrity, and/or awareness.
> the resulting quality of the output however can usually be recognized by everyone...
Have you seen our code? Or have you seen a few mentions of some of the things that we do and you are using that to fabricate an idea of what our code is?
> So therefore I'm a bit suspicious of anyone that claims that their idiosyncratic setup is actually simple
You are of course free to be suspicious. A significant part of the experience necessary to judge it would be to actually see it. We are, at the end of the day, dancing about architecture. You cannot see our code, you cannot see our actual set up, which typically would mean that one would have essentially nothing to say about it. If you'd like to refute particular practices I mention, that's fine. But please, keep your presumptions to yourself, or at least ask them in the form of a question (e.g., "artificial partitioning").
> ...would be through a pile of design patterns inspired abstractions that now everyone needs to be able to juggle at all times.
We have relatively junior developers on our team. There are a small handful of common patterns that are used repeatedly throughout our code base. We strive to eliminate special (unnecessary) variatation so there are always exemplars and/or documented norms. We don't have 10 varieties of "service objects". We don't have callback hell. They do not struggle with these things. They struggle with other things, as they are relatively junior, but we support them. Again, frankly, you have absolutely no idea what you are talking about and you are continuing to attempt to make up for your ignorance (of our code base and what I am saying) with hubris and presumption.
I just saw this edit from you:
> That said, if a code base keeps requiring you to make the correct design decision using subtle knowledge, that's a fragile situation and something seems off.
You and I have two different ideas of expertise and software design and they are irreconciable. Maybe in 15 years you can look back on this conversation and recognize that there was always land beyond the horizon. Or maybe not. You are the doctor that is refusing to wash his hands before surgery because you still believe in bad humors. There are things you do not know, and you are stubbornly refusing to recognize them. You have probably worked on some pretty horrific code bases, and you can probably back up every one of the things you are projecting onto me with personal experience, but you have drawn the wrong ultimate conclusion. You have drawn the conclusion that software design does not exist and does not have consequences and that anyone who claims to do it is a charlatan. That may even be true much of the time (goodness knows that's what I'm saying about many people who claim to be doing software design). Judge me as harshly as you wish for making this assessment. Thankfully, we do not work together and we will be unlikely to cross paths again in the future.
> Given that at no point did I point to our "brilliance", I assumed you refererring to my pointing to tacit knowledge
I deduced it from your general tone. But you're probably right that the tacit was part of it.
> Have you seen our code?
No, but it was a general comment, hinted by providing the redis source code as an example.
> and presumption.
Yes, it was prejudicial, as stated. I hope I'm wrong.
> You and I have two different ideas of expertise and software design and they are irreconciable.
Agree.
> Maybe in 15 years you can look back on this conversation and recognize that there was always land beyond the horizon. Or maybe not. You are the doctor that is refusing to wash his hands before surgery because you still believe in bad humors. There are things you do not know, and you are stubbornly refusing to recognize them.
And here's why I said your arrogance would make it unfruitful. Because in the end you're of course objectively right, and I'm objectively wrong and will see my errors in due time. Sigh.
> Thankfully, we do not work together and we will be unlikely to cross paths again in the future.
Well, there is Poe's law. You can call my views and our team's culture extreme. We believe it is extreme from a statistical perspective. There's a reason that our team has the tenure that it has. What we are doing isn't out there typically.
But, I would challenge you to challenge anything that you saw as satire and try and wrap your head around how a sane, rational person could, with a straight face say "that's not a problem" and be absolutely right. Feel free to call something out specifically to me, and I will back it up. I'll warn you that you may still not agree with me, because much of what we do requires tacit knowledge to see the benefit of. And once you do, it's self-evident. A person must be studied in Lean and design principles and have seen the trajectory of several software projects over the course of many years.
See if you can understand what are the actual problems in software. What causes projects to fail and teams to have to hire 100s of developers to maintain a semblance of the productivity they had for their first year or two of development.
Try to record your setbacks in a daily work log (we all do this) and address root causes. Do this for 10-20 years and see where you end up.
Just to clarify, what are components referring to? Are they independent web services?
And what is the total size of the developers you worked with? Are the 12 just your team or the entire engineering org?
Edit: found from another comment
> We don’t call them microservices, because we aren’t web developers pretending to do architecture and recreating the mistakes of DCOM, CORBA, and Web Services. Most of them are autonomous event sourced components. The others are stand alone web applications that are stitched together with Nginx routing and SSI.
Components are either autonomous event sourced back end components (using Eventide) or independent web applications that are combined with Nginx routes and SSI. The UI is (mostly) server-rendered Rails. The users have no idea they are hitting 20 or so different web applications, but the developers sure feel the productivity boost of every application being a small application that is (relatively) independent from one another.
A dozen or so is the entire software development organization.
Edit: Indeed! If you have any other questions, ask away. Most people think what we do isn't actually possible, but it is.
You have experienced what happens when web developers cosplay as software architects. That's when you get "microservices". It's a travesty, and it has ruined an entire generation of software developers who now believe that partitioning and autonomy are just myths.
So, as a blunt statement - your experience is typical. That is, it's average. That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the bell curve doing what it always does.