I don't remember the exact project or where I read it but one project added delay (using sleep function) to depreatected methods in order to discourage people using them. After every release, they increased the sleep time so that at one point it became impossible to use it without noticing it. Although I don't know if it is possible to do this for new code trying to use deprecated function and have old code use non-delayed one, I think it was a nice trick.
Slightly tangent, this reminds me of the story of the game developer that would allocate a few tens or hundreds of megs of RAM early on in the project and not touch it. He knew that as the project was nearing completion, they would get squeezed on available RAM. Then he could just go and delete a single line of code and, bam, tons of free RAM.
This seems like a slight variant of SDD: dickhead-driven-development. Clever, but your coworkers still hate your guts.
There's a real reason to do something similar if you're developing for a memory constrained devices with a API someone else will use.
If you have objects your API functions return that API consumers will use, you want to make sure the objects don't suddenly grow in size, since that would eat into the memory that the consumers expect to use. So, you'd pad those objects with unused fields, and then if you need to add a new field, you can just use that padding area.
Something similar applies if you're sending objects over the wire, since you may not want to increase the size of the message later.
I knew someone who did this with humans at a bigcorp.
He’d politick to transfer in new teams for the sole purpose of building a supply of cannon fodder. Staff up in good times, and purge fodder when layoff targets came to protect “his people”.
Back in 2000 we were mandated with a big push to eliminate all shared excel documents and turn them into real database driven products. There was this one department that had a huge excel database that was bringing the network to its knees. Around that time I had discovered that you could create a function in excel with the moniker of a null character (alt-255). We had used that for playing pranks on one another. Someone had the bright idea to put a function into the code that invoked a slowly increasing pause. That function was sprinkled all throughout their code and no one knew because you can’t see a null character.
A few months later that department was practically begging us to convert their excel document into a database project.
I read this, and thought to myself that a hacker would love to find that in code through analysis tools. They could then replace the delay with malicious code, and no one would be the wiser for quite some time.
Doing stuff like this seems creative and awesome at the time, but it breeds vulnerabilities something fierce. It also creates a nightmare for maintenance.
I would suggest a different approach of figuring out how much that Excel doc was costing the company every month, how much the company would save if the doc was converted to a real data service and Web front end, and then present the comparison at a meeting with management from that department - give them a chance to sign on before you take it to more senior management.
In this scenario I would argue that deleting a function abruptly without verifying impacts is significantly worse than gradually adding a delay. Usually when function deprecations occur impacted dev teams are notified on a shared distro of what's coming in future releases. Library maintainers at large companies typically don't have the bandwidth to investigate each project that uses their library to determine if deleting a function would detrimentally impact a production environment. What a deletion does do is block the downstream dev team in this hypothetical from deploying to prod as they investigate why their builds are suddenly failing (or crashing in prod!) and refactor an alternative solution. Do you really want to work in an environment where other devs feel empowered to break your builds and sprints ad hoc?
I assume they are welcome to use the old version, and this comes after an ignored deprecation warning. If not, then this deletion should come after a forced migration by whatever library team enforces a one version rule.
That broken build won't be pushed to production and affect users, but invisibly slowing down a production service and hoping someone notices via monitoring will probably pass tests, build fine and be pushed to production where it will affect end users. I absolutely want to work in an environment where the build fails rather than people play childish tricks to punish my users because I used a deprecated function, and hope I notice.
While an interesting tactic, I don't think it's a great idea as it may mean a poorer experience for the customer rather than just the development team.
100% evil and pretentious. Not every dev team has the free time to go rewriting everything to the whimsy of library devs that can't help themselves from incessantly shuffling everything around every week. This is how you encourage people to never update and keep security vulnerabilities in the wild.
It is especially evil since on many systems old methods are more performant, because they were meant to be used on old limited hardware (where cycles and memory really mattered).
So by making those slower... you 're really digging a hole.
There are of course cases where the opposite is true too, like all things in life.
We're talking about transitions where the library authors deliberately want to (and have good reasons) to migrate whole organization over. It's not they who are pretentious in your scenario ;)
I think that really depends. If there are performance, compliance, or maintenance requirements for the new versions where this is being implemented, and devs are updating to, then it could be considered reasonable. You wouldn't be seeing these delays if you continue using old versions of the library. Freeze your dependencies if you don't want your dependencies to change.
This is a failure of the dev team not evaluating that libraries are stable enough for their organization. If you can't keep up with the iterations of the libraries you're using then you shouldn't be using those libraries in the first place.
Here, a “shitlist” tracks existing uses of deprecated behavior within a large code base. It prevents new uses of deprecated behavior without disrupting legacy code. This is a temporary measure used to facilitate refactoring large code bases.
This is a good example of "Organization as Code", where you are trying to influence a non-trivial change (often "cultural") by code.
Frequently it's better to change behaviour this way than say, holding meetings and presentations.
A typical example is "we want our developer to write more tests" but there are few existing examples of test code to look at and when you write a test it takes forever to run, so you fix this by fixing the underlaying issues (which is bad or non-existing code), rather than making developers attend TDD presentations and just talking about a "test culture" for example.
I've experienced this first-hand. Joining an existing project and being put on test-writing duty very quickly turned me into our team's chief advocate of writing testable code.
If you don't want people to use deprecated code, it's essential to document what the replacement should be. In JavaWorld, this can be done through Javadoc. Just tagging a method or class with @Deprecated and leaving it there is the sort of thing that makes me want to hurt people.
Your comment made me think of the adage: "Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live"
I don't know about my whole brain, but my programming brain totally is.
Now that I've been programming for a nontrivial number of years, it's happened enough that I no longer trust future me to understand anything clever that I do, so I spend much more time structuring code to require minimal context. Failing that, ample documentation, because future me will appreciate having an essay to read much more than current me wants to write it.
This is depressingly familiar. I can't tell you the number of times I've got an error, searched, ended up on StackOverflow reading the answer, full of stuff I have no knowledge of. Get to the end, it was answered by me. More than twice.
Seriously.
I'm now more obsessed with documentation and the kind of code structuring you mention and have come to hate, viscerally, projects that lack documentation.
As a potential aside, I once took a course in ASP. We had a bit of time before the class started and the tutor had a chat with a couple of us. Some huge government projects were in the news for having failed and gone well over budget. He told me he'd worked for the company behind them and that they used to obfuscate the code just by stripping out all the comments so the government had to keep going back to them for work.
I think of that any time I hear "but the code is the documentation"…
The version I've evolved, for teaching students why I care about style:
You need to communicate clearly with a number of coworkers, the most important of whom is future-nerdponx. Sometimes future-nerdponx is smarter than you, and sometimes future-nerdponx is dumber than you, but either way they know where you live, and their misery is your misery.
Sometimes I write my future self emails (well, logs), especially when I'm carefully stepping through knowledge domains or decisions I don't understand.
It's so helpful when I do it that I probably ought to do it more, and if there weren't so much overhead I'd probably do it everywhere.
This is literally one of the topics covered in the article and the author advocates for the same solution, documenting the alternative in the error/deprecation message...
No. But when it's not explicitly mentioned as support, it can appear as an implicit contradiction to readers. We also want the discussion about an article to build on what the article proposes, and explicitly tying comments to the article help with that.
Agreed. There's been a lot more "commenting without reading the article" here recently than in the past. So when someone comments on an article without anything in the comment that refers back to the article and merely offers the exact advice that was in the article without further analysis or insight:
1. That gives the appearance of a user who hasn't read the article
2. Even if the user did read the article I'm not sure what their comment adds to discussion of the article. For comparison, imagine participating in a conversation with a few friends in real life, friend X says "It's a good idea to write tests for your code," then friend Y replies "It's a good idea to write tests for your code." Usually the way people discuss things is to reply or offer addendums to the original idea. It's not a rule, it's just good conversation.
In this case, I think the generous interpretation is that the poster simply missed that section of the article. The generous interpretation basically requires one to conclude he read it, because otherwise he wouldn't make the connection to deprecation from the title alone.
The article in question is usually not the topic of conversation on HN. It is better to consider it to be the launching point for the conversation. Some will discuss the article, certainly, but you can expect the conversation to branch out very quickly to related items (especially if those items are mentioned in the article). While it's probably not a good idea to discuss the plight of African elephants in these threads, drawing attention to a minor, but specific point in the article is perfectly cromulent.
In this case, the post has value. It does two things. First, it draws attention to a subtopic that was only an aside in the article. (One I happened to miss the first read-through.) Second, the phrasing in the article is somewhat general and vague about this, and the post is relatively more specific and direct.
It's a shame people don't actually read the articles because, assuming the article is actually insightful, you can have better discussions after the community has actually read the article.
And in the case where people don't read the article and just read the headline, many comments have nothing to do with the article, and end up discussing a less interesting misconception of the submitted article's title/headline.
It's great to be generous with our interpretations of comments here on HN. But I would like to push for a higher standard of discussion, which has always been the draw of this community, particularly for technical articles (and this is actually a high quality technical article!). So I don't think I'll ever be happy being generous to a low value comment that re-states an opinion from the submitted article without adding anything or even without adding much.
This is the practical usage, it is really language dependent, though.
C++/Wikipedia uses the definition of the original poster, while Java's @Deprecated annotation literally has a `forRemoval` boolean value attached. Personally, I am used to how it is used in semantic versioning where it is just a helpful hint for what the next release's breaking changes may be while introducing the migration behavior if possible in this release. This allows for clients to incrementally migrate instead of making it all or nothing.
Deprecation without an obvious replacement is pretty toothless, because people will keep using the deprecated thing for lack of an obvious replacement. This is how we ended up with disasters like the Python 2/3 transition.
It's better to use a Shiterator, which can yield pieces of shit on demand, lazily enumerate them just in time, and generate infinite streams of shit, instead of allocating all your shit up front, and having your shit together in one place.
That's elegant dungtional programming but it's not really relevant here as you are supposed to check if new crap matches any of the enumerated fecal matter; how would you do that against an infinite stream of diarrhea? I'd leave this to the type system if possible. And of course use a colon-delimited list for the occasion.
Brilliant. It's a really good compliment to strangling certain parts of a large code base.
As always the key is getting the information to the right people at the right time, and making it more difficult to make the wrong choices then the right choices.
Exactly. What's nice about this approach is it factors in the "behavioural economics" of developers. As the author puts it...
> At the end of the day, everyone needs to get work done, and if they see a code-path already being used from 10 places in the code-base despite these soft warnings–it doesn’t seem crazy to introduce another.
Even the best person with the best intentions will take a short cut under some circumstances, meaning to fix it later then getting distracted before that happens.
The idea of deprecation and death of code assumes that those using it can easily stop.
That’s not always the case.
When Flash dies Jan 12, 2021, you’ll see what I mean.
There’s been so much misinformation about it, but there’s an OS datetime check in the Flash viewer code, browsers will disable the plugin even for some older versions not just new releases and have or will remove PPAPI and NPAPI support, and Windows already has an optional update to kill Flash support that will be part of regular updates in Summer 2021.
Even with three years of warning, it’s not ok.
It’s an extreme case of deprecation, being attempted by the best in the business, but it is inherently bad, and there’s hardly a good way of doing it.
At this point if you/your company is still using Flash I find it hard to sympathize with you. How did the company come to the conclusion that using Flash was a good business decision? It's been largely unsupported/disabled in mainstream browsers for _years_ now. They also knew for 3 years it was officially going to be deprecated in 2021. Sounds like despite that they continued to develop on the technology anyway?
Sure. Can't have it both ways though: you're welcome to choose to implement on a soon to be deprecated tech stack but you don't get to bitch about it or expect the maintainers of said stack to suddenly change their minds about deprecating.
> My point is that deprecation is by its very nature going
> to hurt something that isn’t prepared for its demise.
TFA describes techniques for deprecating code paths that do their best to notify and prepare the folks working with that code.
You're describing the end of Life of Adobe Flash that hits at the end of 2020.
Deprecation is different from EOL (Flash has been deprecated for some time now), and deprecating/EOLing a creation tool/product line has little similarity with tooling to deprecate internal APIs gracefully.
Although I do think that you and TFA agree on how difficult it is to deprecate functionality that folks rely on, it seems like you're talking about two wildly different facets of that statement.
This is an excellent way of using your (hopefully) already existing testing and CI infrastructure to change a social problem into a technical problem. As we all know, technical problems are by far the easier of the two. :)
We used this technique to great effect at my previous day job, though under the more neutral name of "ratchet".
A similar kind of automated mechanism is required in distributed systems that allow for rolling upgrades. New functionality in upgraded nodes can't break not-yet-upgraded nodes and legacy behavior in not-yet-upgraded nodes has to be tolerated in upgraded nodes but only until the entire system is upgraded and then it is prohibited. Doing this wrong results in some really hard to fix production states.
On a somewhat related note: One of the biggest things I've noticed is that simpler pieces of software are less likely to be deprecated, whereas complex codebases quickly become legacy and abandoned...
... windows ? MacOS ? Literally every Adobe software ? Literally every large audio / video / 3d création software ? Every large game engine ?
I don't know, I notice pretty much the opposite in the field of software I know - smaller software come and go, but the large ones were there 20 years ago and will likely still be there in twenty years
Those large ones are large and important enough to receive substantial rewrites/overhauls when necessary. Those rewrites can be very expensive, and are completely infeasible for most software projects that don't have huge backing behind them. For small and medium projects, (code) complexity is a leading factor in maintenance, and a huge amount of unmaintainable software gets abandoned regularly, or simply falls behind and is abandoned over time.
The maintenance of giants is tied to their importance rather than the complexity of their codebases. SQLite is no less likely to be around in the future because it is simple.
Codebase complexity also isn't just about size. A huge, well-written codebase is still more likely to stick around in its current form without requiring a total overhaul.
There is a lot of inertia driving maintenance on some of these large products but there is also a ton of deprecation taking place internally relative to them.
I still write new applications in Win32 (or x64). If you want to ensure your app works on as many environments out of the box as possible this is the way to go. In fact, you shouldn’t even depend on DLL version of C runtime as they may not exist on the environment.
On Windows, almost every "new" API is built on top of Win32.
Obviously, if you're writing a simple CRUD application, you'll never need to go down there. (You'll probably make a browser application or an Electron application.)
But if you're doing something unique, like a utility, or trying to hook into the OS to make it do something new, you gotta work at the lower level APIs.
And, yes, I shipped a product that included a Windows driver this year. It required working at the Win32 level in user-mode to communicate with the driver, and to test the driver. Windows recently added a new API to Windows 10 that eliminated the need for a driver, but we still had customers on Windows 7. Even then, that new API is technically Win32.
If you're targeting Windows, win32 is the stable platform. Microsoft keeps presenting other options that aren't as good, and don't do as much, and keeps walking back to win32. It even works on other platforms easily(ish) through wine/proton/etc.
Everything I do is cross-platform so I use Qt, for which version 1.0 is from 1996 and which has really not changed much since then : https://www.qt.io/blog/2018/05/24/porting-from-qt-1-0 ; I write new apps with Qt on a very regular basis.
Yes, yes I would. I'm pretty sure that Win32 will be supported in 30 years, without much maintenance effort, compared to whatever new hotness that is popular today.
In a similar vein but on a smaller scale we have a number of tests for deprecated behaviors in our application code, and whitelist the existing code based on serializing the file name, method name, and the ordered list of parsed token types of the method.
That way minor alterations like changing a string or numeric value don't remove the method from the whitelist, but alterations to the logic of the given method require you to fix the issue while you're in there poking around already to pass CI.
For the Java Developers here, have a look at archunit. We use this to discourage certain behaviour but you can also use it to test and whitelist the usage of shitlist items => https://www.archunit.org/
> Make sure that a certain datastore is only read from in a certain context ... Ensure fallbacks for all uses of a secondary data-store ... joins between tables that have no business being joined
I wish linters / typesystems were extensible enough to do this kind of domain-specific checking. There's a new generation of static analysis that's much more focused on architecture or business rules and is less about code-in-the-small concerns like class methods or operator compatibility.
I think a key to doing large, multi-developer projects, is true modular design, with opaque APIs, and each module with its own project identity and lifecycle.
Not a particularly popular stance, as it means a lot more overhead in each project.
Even in a single-team project, I've ran into the problem that I'm refactoring some old code while others keep adding to it. Some way to have to cause failing tests sounds like a great idea.
Another cool way you could do this: Ruby methods can ask for their caller_locations. They work like this:
# Source
$ cat example_for_hn.rb
def foo
bar
end
def bar
baz
end
def baz
(c1, c2) = caller_locations.first(2)
puts "Parent caller: '#{c1.label}' in '#{c1.path}'"
puts "Grandparent caller: '#{c2.label}' in '#{c2.path}'"
end
foo
# Demo
$ ruby example_for_hn.rb
Parent caller: 'bar' in 'example_for_hn.rb'
Grandparent caller: 'foo' in 'example_for_hn.rb'
So, you could define a method decorating class method like so:
module Shitlist
def shitlist(method_name, whitelist)
original_method = instance_method(method_name)
undef_method(method_name)
define_method(method_name) do |*args, &block|
call = caller_locations.first
passes_whitelist = whitelist.any? do |label, file_pattern|
call.label == label && call.absolute_path.end_with?(file_pattern)
end
unless passes_whitelist
fail "Shitlisted method! Permitted callers: #{whitelist}"
end
original_method.bind(self).call(*args, &block)
end
end
end
and then extend classes with it to use the decorator:
class Example
extend Shitlist
def not_on_shitlist
qux
end
def baz
qux
end
def qux
puts 'Only some methods can call me :)'
end
shitlist :qux, 'baz' => 'shitlist.rb'
end
If I run this example (full source: https://git.io/JLOdV), the non-whitelisted caller throws an error:
$ ruby shitlist.rb
Only some methods can call me :)
Traceback (most recent call last):
2: from shitlist.rb:44:in `<main>'
1: from shitlist.rb:25:in `not_on_shitlist'
shitlist.rb:13:in `block in shitlist': Shitlisted method! Permitted callers: {"baz"=>"shitlist.rb"} (RuntimeError)
---
Of course, you might not want this hijacked method with tracing inside something performance critical. You could always configure the implementation to be a no-op in production.
I'm more curious about how things like shitlists are implemented in different languages.
Most of my experience is C#, where calling deprecated code triggers a warning.
Considering that C# warnings will fail in CI, how would someone do a C# shitlist? Would it require some kind of #pragma, that would stick out like a sore thumb in a code review
I think if warnings can't do the job, you'd have to go the route of writing a test that greps the codebase for usages. Maybe a custom linter could also do the trick.
The ability to grandfather in existing violations, though, is something most build tools are likely to have trouble with.
Maybe another way to do it is to have warnings fail CI, but grep out the known offenders from stderr.
None of that sounds elegant to me, but it _is_ a shitlist.
To an extent this seems to be related to more general automated detection of refactor targets/"code smells" (ironically) for which there exist tools already (e.g. linters indicated in the article) - I like this simple "list" approach though.
Love the concept, hate the name. To me this seems like the recycle bin concept. Allow the code to exist temporarily, but encourage permanent deletion. Perhaps a better name could be a "screamlist" or "needstochange" list.
All arbritary rules like this just makes the process a mess.
Also the same people doing the shitlist will most likely be the dug in senior ones responsible for all the shit code being there in the first place and include the wrong things.
We use Bazel’s visibility settings this way. More often for controlled beta rollouts than for deprecation, but same principle. You can make it a compile time error to import a package that isn’t for you.
Yes, this is exactly what we do inside Google - it works even better when (a) everyone uses Bazel for everything and (b) all the commonly-used languages have dependencies checked at compile-time.
I love that C# supports deprecation as a language feature and allows to mark an obsolete method as only usable by other deprecated methods, which is the same as the shitlist approach here in the end
Shitlist driven migration/refactor, maybe. If it's development it means you're always deprecating something and that's a significant organizational smell.
Maybe we have different views of what "changing" means?
Every software is different, so take my words with a grain of salt, but personally, if it's always changing its behavior/interfaces/architecture (as opposed to just growing or a healthy mix of both) to me that sounds like either:
- It's software that runs with a highly volatile and niche functional target (scrapers, certain bots and business/operation research come to mind).
- It's a snowball of technical debt asking for a rewrite that was never given a green light.
Sometimes an evolution is needed and then shitlists have a place. If you guide development through shitlists it means you always have shit to get rid of. Maybe you have a ball of shit, in that case?
The article shows ways to prevent new dependencies on your deprecated library/service. This is half of the problem. The other half is efficiently removing the existing dependencies.
Your team could change the other teams' code to remove the dependencies. This is usually inefficient. You will waste time learning their code. They may drag out the code review process for weeks or months.
Some teams may refuse to accept your changes and use your service deprecation as political capital to demand more headcount. They may even lie to their managers and claim that the dependency deprecation is justification for a rewrite that they want to do.
There is a technical solution that can help with this social problem: AUTOMATICALLY EXPIRING DEPENDENCY APPROVALS. Configure your library to allow only existing systems to use it, and make them all break on a certain date. Then, instead of forcing the other team to move, they have to move or their build breaks. And if they want to delay turndown they must convince you to change your code. Without automatic expiration, they can delay turndown by simply ignoring you.
Some teams may wait for the dependency expiration, lie saying they didn't know about the turndown, and then demand that you delay the turndown and give them more time. You can work around this with a two-phased turndown. First create a new version of the library that allows only existing clients. Give the library a hideous name so code owners will want to remove it. Example: Deprecated_LiBrArY_YoUr_BuIlD_WiLL_BREaK_oN_20200601_LOL_Were_sERIUS_YOLO_exxtensuns_COme_frun_SVP_DaniELs_OnLY. Then set the existing library to expire in a week and email all users. They can easily switch to the new hideously-named library and in the process acknowledge that they know that their build will break at the specified date.
TLDR: Use expiring white lists so you won't get ignored. Rename your deprecated library to something hideous to motivate code owners to migrate away from it.
If you do the above, your prod servers will always run in "deprecated allow" mode because developers continue to use (and add new) deprecated functionality. That's what the article is attempting to tackle.
Prod should always be running in allow mode anyway for the same reason that prod builds compile out debug asserts and static type checks. Once the code passes review and makes it into prod you've already lost and spitting out warnings on every function invocation will just piss off your ops team.
Okay, so first you have to have a whitelist of "allowed" usages of depreciated functionality for this to actually work. Anything that makes it into prod goes on the whitelist. All dev environments, QA, and your automated testing run with fail mode on and trying to push code that adds to the whitelist should set off alarm bells. From there it's up to the team / business if it's worth doing it anyway.
Change the name to BannedList or something and keep it as is. No profanity and developers get failing tests immediately when they consume a deprecated API. The core principle no matter what you call it is a good one.
I feel like the profanity rule could go either way. If your team has a culture of casual profanity then go for it. Something like "listen here you little shit" can be lighthearted and funny in the right context. Profanity is a smoke test for anger which is the thing you need to actually avoid.
One can't always introduce a multitude of shitlists into different places in the code. This can impact performance and readability; and perhaps even more importantly - if the language used is not high-level with reflection capabilities, you will be counting on callers' participation in calling the semi-deprecated API with truthful "usage key" for looking up on the shitlist. Who's to say nobody reuses authroized keys for expediency?
I will probably remember Sirupsen forever for his move of breaking Golang projects by renaming his github username. This led to go modules freaking out because different project used different name casing for the dependency. He has a popular logging library and go modules are angry when you rename github usernames..
If you have built a module-enabled Go binary yesterday with a specific version of an external dependency, you will get that exact version of the dependency tomorrow when you rebuild (IFF you have version-controlled your go.mod and go.sum, which you should). And if someone tries to be sneaky and move the release tag, the checksum will no longer match, and your build will fail.
If you want to upgrade the version of a dependency, that is doable (with plenty of toll assistance), but it requires a positive action.
Theer is, however, no good solution for the "first use". But, that is pretyt much standard.
That is probably to some extent correct. Although this would bite you on a new build, not for anything that is already running in production. My understanding is that that is/was not necessarily the case for leftpad.
Yeah, the suggestion is that you should run a caching mirror or manually vendor your dependencies.
Other unconventional ideas about go packaging is that new major versions should have a new package name or be available in a subdirectory with the old version existing in perpetuity, and the minimal version selection approach to dependency conflicts.
I'm sure these things work well inside Google, but at least for me the package management situation is a big turn off for the language.
