When did on-call become so accepted and demanded from employers?
Currently I am "Release Captain" for a week: So I have to setup any releases and manage all the related tasks, do automated/manual testing of the release, release (enabling toggles and any config changes).
Then Backup to secondary and primary for a week: About once or twice I am asked to help with tickets.
Then for 14 days we alternate primary / secondary.
Thursday to Thursday is our deal. Every ~40 days I am in one of the above.
It's absolutely miserable.
I have never had this much time spent doing non-development related tasks. For 4 weeks every 1.5 months I can't have a life at all.
This just screams to me that we are forcing broken software/not complete software out the gate a building huge piles of technical debt that will never get the focus.
I remember a time when I would start at 9am and end at 6pm every day and never heard a peep about production issues unless the support engineers couldn't figure it out. Which maybe happened twice a year.
To make matters worse most things are not allowed to be touched in production with the risk of being fired for making changes. So if you want to "fix" any data or call xyz service you need high ranking approval. It's like being tortured!
> When did on-call become so accepted and demanded from employers?
As a 50 something year old software engineer. Its always been like this. I'm kinda shocked at how reluctant the new generation is to support the systems. Sure we'd all prefer strict 9-5 hours but most companies rely on software to stay in business and you need experts available in case things go wrong.
If you need experts 24/7, you should have had shifts that cover that timeframe.
Oncall is a source of so many ways for abuse, don't even ask me how I know. Saying that rejecting Oncall is denying support for you system is bollocks.
I'm happy that younger engineers mostly laught at that concept and leave. Once of the few lessons they are teaching us (the old pricks), especially in self care and respect space.
My observation so far is that businesses don't aim at 24/7 any more. Officially maybe, but not in practice - they accept the risks. Some downtime is less of the deal, if that means less staff and other costs.
It's a big shift in thinking, IMO caused mostly by people not opposing abuse more.
Perhaps it's time they consider lidar? Elon hasn't seen the WHY yet?
Tesla CEO Elon Musk has stated that cameras are the only sensors self-driving vehicles need, and that Tesla's electric vehicles do not use lidars. Musk has previously called lidar "expensive appendices" and "a fools' errand". He has also said that any company that relies on lidar for its autonomous capabilities is "doomed".
If you're traveling near Dayton Ohio or near Wright-Patterson Air Force Base ever I HIGHLY recommend visiting the National Museum of the United States Air Force.
They have a XB-70 on display and it's amazing how large the plane is! There's also a pretty good documentary on it from Discovery's "Great Planes" (https://www.imdb.com/title/tt2158498/) if you want to nerd out.
Let's see...
14 days on call every 3 months.
3 releases a week.
All the normal review work. PRs, design, etc.
SCRUM meetings.
Meetings before the meetings.
Crazy deadlines which were arbitrarily given by someone in another group.
Being up late to make changes because the business deems it too risky to do it during regular business hours.
Endless performance testing, e2e testing which always generates defects that arent really defects but still need 30m of my time.
Upgrade this or that to the latest. version because xyz no longer supports what you have.
Endless security vulnerabilities that need to be upgraded.
Pipelines that need to be upgraded or fixed just to get a release.
Failing tests which need to be investigated.
Hundreds of configuration points which control process flows.
Never ending lower environment problems.
RTO.
Constant fear of being laid off.
Definitely no way anyone could be burnt out.
I actually want to work on my own projects during the weekends if I can. That somehow brings me happiness compared to what I work on at work, which is bogged down by external issues.
My personal projects I have no one to report status updates to, no one to tell me "I'm doing it wrong" no customers to support, freedom to mess up. Lol
I full agree these things matter us. These endless paper cuts tatter us as we go.
Still I think the real suffering is less about these little indignities, & more a cosmic sense that the business doesn't get it, doesn't see the real work, doesn't care. It doesn't seem like businesses fundamentally believe in us, our craft, or our talents. Performance review even when going great tends to recognizes us only in blunt vague generalizations.
As craftsmen we all too often feel alone & separated from the org when we are doing are best acts. Hard finicky stuff pulled together by hook, crook, and a couple dashes of wit. Finding excellent libraries and tools to offload hard aspects of the task.
The company can then be merry that they've built a great product, hopefully. But it feels like camaraderie - through good times & bad! - would come from sharing such incredible work & time, but the organization doesn't fully see. The faint distributed sensory network of the discorporal org miss the best parts of truth our code wroughts out.
Yeah, the burnout isn't usually coming from the coding (though it may be in some cases), it's from all the pointless bullshit that gets in the way of said coding. In every job I've gotten burnout from I've wanted to keep coding, and I would be happier if I could actually get to do so. But then either the business side got in the way, a dependency on another team got in the way, etc. So the frustrations just mounted up and up, and the feeling of hopelessness set in.
Yes! I’ll add on—endless requests from the suits to account for every task in a way that fits into one of their stupid (and usually new) buckets. Random 11th hour emergencies from poorly-vetted implementations. Having to fight leads and PMs of other teams to take ownership of or be flexible about _anything_. Tasks that are more about optics and saving face than about making anything better. Architects who “don’t write code anymore” but expect to be able to dictate how others approach the craft.
Wow, you very accurately explained my life, except i'm on call even more than that.
And I think somehow you understated the security patching.
What drives me crazy is I'm always behind, and when my manager asks me to do something, I'm always having to argue for higher priority things that are beyond obvious.
But if it's not his top priority, I get to feel bad by repeatedly getting asked in multiple meetings about whatever his highest priority topic is.
That's what drives me nuts. It feels like my managers way to get me to do things is to waste so much of my time, that I end up giving in just to not have any more of my time wasted.
>Wow, you very accurately explained my life, except i'm on call even more than that.
I'm sorry, it has to be one of my least favorite things about this career. Nothing worse than getting calls off-hours when it's only you. Feeling marooned is never a good thing. I am currently hoping my phone doesn't buzz today! Hope it's quiet for your sake!!! I don't deal in anything lifesaving or absolutely critical, but it sure does feel like it while you're on call.
>And I think somehow you understated the security patching.
Oh yes I do. 8+ critical CVEs on one microservice, you upgrade the libraries/dependencies with the CVEs just to find the new version has 2 other CVEs. Worse yet, you upgrade the dependency and it breaks other functionality (generally from internal dependencies that haven't been upgraded since they were created, LOL). To make matters worse, all of this is inherited and often we had nothing to do with the initial development. I get it, they need to be fixed so there aren't other bigger problems, but it's death by 1000 paper cuts.
>What drives me crazy is I'm always behind, and when my manager asks me to do something, I'm always having to argue for higher priority things that are beyond obvious.
I just wish they would give focus/time to preventing on-call/customer issues so we could all be more rested/fresh/prepared to handle all the changes in priority! Make things so we don't have to spend large amounts of time fixing them. Actually evaluate failure scenarios and address them. Being extremely intentional with our time as a team. I am currently dealing with EOY reviews and displaying my value so I don't have to potentially move for RTO (I despise being in an office and was so incredibly happy to find remote work). 6 years of working remote and suddenly it's a problem from leadership?! I struggle to find the energy to argue lately. Again, it's not even software development related! Weird times.
I haven't done any end of year work, or prepare for the new year work.
I've worked remotely for 16 years, and now they want me to RTO when I haven't worked in that office in 16 years, and literally no one I work with on any topic is there at that office. I work for a global team.
My team is gone home by the time the office opens...
If they make me go into the office alone, then I should stop having any meetings before my office time, which would mean I have zero meetings. No communication with anyone, alone in n office.
Let alone pending layoffs.
And yet, I have to continue to keep my service up and functional with features for my customers, while balancing everything else.
I'm not sure I've felt this down about my career in the entirety of it. What is the point right now?
I have been since the first people at my previous employer told me it would replace all the programmers.
Then I found out the developers assigned to working on the low/no code parts had a component where they could write Java inside their low code process. There were hundreds of lines of code in each business process in the low code parts. They were writing giant methods in their process wired up by the true low/no code part.
To make matters worse the tool did not support unit testing of the custom code.
It ended up being like dozer mapping entity to entity and then passed along to custom code. All wrapped in some crummy UI. It produced a jar file which would be deployed anyways.
Maybe the tools are better now. We had some Tibco product at the time.
The more I mess with SBCs the more I lean towards this method.
The M720q's or M710q's are two of my favorites.
As an added bonus the chargers even work with my lenovo laptop(s).
Having owned many raspberry pis, a handful of rock pis (4A, 4B, Zero and S), Mango Pis, Khadas VIM4 and other SBCs (I have a problem haha)... I can say without a doubt the two shining stars the Raspberry Pi has are their distro/software and general ecosystem.
Just upgrading and dealing with downloading/upgrading the Raspberry Pi is a dream compared to the Rock Pis. The Khadas is a little better, but still I have a hard time upgrading it and sometimes certain images just won't even work from the company. Sometimes when trying to upgrade Rock Pis with eMMC you will find 2 or three different ways to install the freaking image on their own wikis.
There's also the "security" aspect for me, I really don't trust some of these distros (maybe I shouldn't worry??). A random image someone put on their website with broken instructions and messed up repos/kernels doesn't exactly instill confidence to me.
I will say the one GREAT thing from Rock Pi is their rock pi x. But it has an x86 processor and I can just install any distro. I wish they were easier to purchase and had more memory!!!
Damn, I was hoping the old PoE hats would continue to work with this one.
Looks like the PoE pins have moved around. Another 20+ USD on top of the board cost...
Kinda disappointing, but the upgraded SD speeds will be nice.
Was also hoping they'd boost the core count or memory for some models, I know it's kinda against their targeted group, but would have been nice to have the options
This is amateur hour!
Let me know when you own all the flavors of the MX Keys, Mechanical, Mini, "Regular". All of the microsoft designer keys, a few Vortex keyboards (Core 40%, Cyph3r), some Dvorak keyboards, apple ones, custom made ones..... lol
It really is an addiction, it's a shame I too keep coming back to the MX Keys Line (Currently in love with the Mechanical w/ Cherry switches).
I have never had this much time spent doing non-development related tasks. For 4 weeks every 1.5 months I can't have a life at all. This just screams to me that we are forcing broken software/not complete software out the gate a building huge piles of technical debt that will never get the focus. I remember a time when I would start at 9am and end at 6pm every day and never heard a peep about production issues unless the support engineers couldn't figure it out. Which maybe happened twice a year. To make matters worse most things are not allowed to be touched in production with the risk of being fired for making changes. So if you want to "fix" any data or call xyz service you need high ranking approval. It's like being tortured!