I’ve written a nontrivial anount of Perl code in my life, admittedly almost none in the last decade. For all its obvious flaws, I always liked the language, and am happy to see it getting attention and moving forward.
Having said that, I think this might be too fine-grained.
First, opting in to an experimental feature could be a one-liner, “use experimental feature ‘try’” or similar. There’s no point in punishing your valuable beta testers beyond that with a second line that’s entirely redundant with the first one.
The larger problem is the versions. This basically requires someone to update their script headers all the time if they want to keep getting new features. Probably not much of a problem currently, but might be if releases get more frequent. I personally like the “edition” concept of Rust a lot better: get over with all necessary breaking changes in one swoop, but then begin a new (hopefully long) era of backwards compatibility. That makes it easier for users to maintain their code, is probably easier to implement, and also easier for users to learn, since there are fewer sets of rules.
Lastly, I hope they also focus on tooling around installation. One paradoxical problem with modernizing Perl is its historic success: every variant of Linux or Unix already comes with an ancient version of it. I know many experienced Unix people hate this trend, but there’s a reason some of the most actively evolving language ecosystems install more and more of their binaries into the user’s home directory. Maybe that’s already the case for Perl, I wouldn’t know.
All that said, I’m not following Perl that closely anymore. These are just some really quick observations about very complex topics. I don’t claim to know nearly as much as the people who made these decisions, and it’s great to see that things are happening.
> The larger problem is the versions. This basically requires someone to update their script headers all the time if they want to keep getting new features. Probably not much of a problem currently, but might be if releases get more frequent.
But to use that new feature they'd need to modify their code anyway, so this isn't really an issue in practice, is it?
> I personally like the “edition” concept of Rust a lot better: get over with all necessary breaking changes in one swoop, but then have a new (hopefully long) era of backwards compatibility.
How is this any different from having a repeat of the python2->python3 fiasco (which, AFAIK, Perl developers are trying to avoid)? Making piecemeal (if needed) changes is much easier than having to update a ton of code and that is even more important when that code wasn't touched for a long time.
EDIT (i put it here since i already got three replies on the same thing): i understand that you can mix two different files with different "editions" but it still makes it hard to update these files themselves.
> How is this any different from having a repeat of the python2->python3 fiasco (which, AFAIK, Perl developers are trying to avoid)?
AFAIUI you can mix Rust 2015, 2018, and 2021 crates; so a developer can update their own crate to 2021, while not having to completely re-write the dependencies which are still Rust 2015. With Python 2 -> 3, my understanding was that everything had to be updated recursively.
That said...
> I personally like the “edition” concept of Rust a lot better: get over with all necessary breaking changes in one swoop, but then have a new (hopefully long) era of backwards compatibility.
What they describe actually sounds somewhat similar:
> At some point in the future, the PSC may decide that the set of features, taken together, represent a big enough step forward to justify a new baseline for Perl. If that happens, then the version will be bumped to 7.0. If this happens, Perl 7 will still be backwards compatible with Perl 5 by default – you'll have to put use v7; at the top of your code to use all the new features. Think of use v7 like Modern::Perl and similar modules.
So the 'use <feature>' is going to be similar to Rust's "nightly unstable features", but actually being stable; and 'use vN' is going to be like Rust's Editions.
It's just that they don't think they've accumulated enough features to release a new Edition yet.
Your last two paragraphs describe Rust’s process differently than I would in comparison to this suggestion.
First, significant new language features are introduced all the time, even in stable, all of them enabled by default (indeed, there’s no way to turn them off).
Nightly’s #![feature(…)] is strictly for experimental stuff that is buggy and/or will change until stable release. I’m sure you know this, but all of those are intended to be backwards compatible! I think that’s a big difference.
Incompatible changes in Rust never happen with “features” (as you know, stable doesn’t even have those), but only with “editions”. The goal of an edition is to carve out a large chunk of future design space, like introducing some keywords for planned features, or rarely, fixing really annoying inconsistencies that require a breaking change. They feel a lot more proactive (we want to do this, we need an edition) than retroactive.
Yeah, the crucial trick is that Rust is a single language but the Editions system lets people keep older syntax in source code, knowing it will be transformed into the single abstract syntax of that language.
Rust editions are not versions of the language, they're just syntax.
New features of the language in a new library or compiler version work fine in all editions unless they require syntax which isn't in your chosen edition.
Rust 2015 edition is only restricted to the syntax from 2015, all the features of today are available unless they're gated off somehow by syntax. For example the "async" keyword didn't exist in Rust 2015, so, you won't be using that from 2015 edition even today. But even though in 2015 you certainly couldn't go around evaluating u32::checked_div() in a constant, it works just fine in Rust 2015 edition today, because it's not gated by syntax.
> the Editions system lets people keep older syntax in source code
Perl 'features' are lexically scoped, and could be applied in a finer-grained style, if you like. Shutting something off locally is done pretty commonly (e.g. to silence an ignorable warning).
The Perl devs have a deprecation cycle to remove mis-features, even if that breaks backwards compatibility. They don't, as far as I know, make much effort to line that up with major version changes[1], it's more a matter of making sure there's a long period of warning.
[1] The difficulty of course, was that for a long time the language now known as Raku was squatting on the next major version, "6", hence the need to jump to "7" to avoid confusion with Raku.
> With Python 2 -> 3, my understanding was that everything had to be updated recursively.
You can mix "Python 2" and "Python 3" code, but the problem is (was) that the semantics in Python 3 surrounding string handling changed so much that in reality this was tricky to accomplish, and Python's dynamic untyped nature didn't help either.
But it was definitely possible to write libraries that worked with Python 2 and 3; I've made a few. It was pretty tricky though, and often led to bugs.
For all the buzz about python3 fiasco, we're pretty much migrated now and py2 gets dropped from supported versions and system repositories. Perl devs would be in a better situation with a "fiasco" like that. Ruby went through its own big breaking changes at least 2 times before as well. So... if people have good reasons to stay, a big change won't drive them away.
> For all the buzz about python3 fiasco, we're pretty much migrated now and py2
Python 3.0 was released in 2008, the fact that 14 years later 2.7 is still widely available (and often the default python, e.g. in my openSUSE Tumbleweed system here) means that the "buzz" was more than warranted.
Also of course things would migrate, it isn't like there was any choice on that matter (and projects like Tauthon that tried to provide a choice had several "Pythonistas" attacking it).
> it isn't like there was any choice on that matter
There was another choice. This is what I mean: however bad you think py2->py3 was, people are using py3 now unless they've got really good reasons not to and are prepared to suffer. When it came to perl5/6, it turns out over time most people choose "neither".
The “neither” choice happened with Python 2/3 as well. The current re-emergence of Python as a machine learning/data science languages is basically a new trend. Long before that, before the Python 3 transition, there was a significant amount of buzz around everything from web frameworks to desktop GUI applications. The way I remember it, almost all of that attention was killed during the transition, when people tried Python and found a significantly lacking and confusing ecosystem.
it was a trend made possible by numpy and scipy being built for a decade before the phrase 'deep learning' was coined. it had just the right mix to blow up: easy to read, easy to write, batteries included.
> there was a significant amount of buzz around ... desktop GUI applications
As someone who is still supporting multiple desktop GUI applications in python, what did they pick instead? I'd love move off Python, but nothing else seemed to have the proper maturity.
Only if you count "not making a choice" (ie. staying with Python 2) as a choice. But this isn't really practical since everything else is pushed towards Python 3.
> Perl devs would be in a better situation with a "fiasco" like that.
Well, the hang-up in the roll out of "Perl 6" (now, Raku) gave people ammunition to shout about how "Perl is dead", but the central trouble was a lot of people wanted to spread that word. Myself, I think the success of a weirdo outsider language was making some insiders very upset, and they were fighting back any way they could.
Supporting older, variant behavior isn't really causing problems for the devs, from what I hear, when it does they go after it with the deprecation cycle.
Yes, but in some domains (for example bioinformatics) py2 is still a requirement for many packages. It really hasn't gone away. Ruby was far more elegant in its upgrades and nearly everyone upgraded their packages because the needed changes were far more minor. I have code from ruby 1.8.7 that still works with ruby 3.x.
> EDIT (i put it here since i already got three replies on the same thing): i understand that you can mix two different files with different "editions" but it still makes it hard to update these files themselves.
It does not, though? You have to update the entire file (really crate) at once, but that’s really not much of a challenge as it’s near exclusively syntactic, and syntactic changes are easy to make, just fix compiler errors.
I feel like you didn’t go through the Python migration yourself and just go off of the echoes you got from it, but the issues in the Python migration were not “updating code is hard”. It was really the easiest part, and could mostly be done mechanically (which incidentally `cargo fix` provides for when migrating between editions).
The challenges of the Python migration was:
- You had to wait for all your dependencies to be updated before you could migrate yourself, commonly this was mixed with API updates while at it especially at the start, this is not an issue in Rust because there is no dependency between crate editions, I can use edition 2018 and depend simultanously on edition 2015 and edition 2021 crates.
- Many changes were semantic in nature, there was no way to check them statically, instead they were runtime changes (in types or behaviour), which made ensuring they were correct a lot more challenging; editions do not cover or allow for this.
The Python migration wilfully included fixes to long-standing semantics issues of the language (though whether all were improvements, or whether they went far enough remains a matter of debate), and this as well as the incompatibility are what made it an issue.
If it had been entirely syntactic and a per-package or per-file thing it would have been much less of an issue (not an entirely non-issue as things like library contents are a runtime concern in python, but still…). And that’s what rust editions are.
Think about applications, not one-off scripts. It should be easy to always stay on the latest language version, with good tooling to migrate your code, and importantly, the ability to do it piece by piece. (Python 3 was all-or-nothing for the entire ecosystem.) Also, many migrations are trivial in practice, like renaming an identifier to avoid the new keyword.
What this gives you is high confidence that when inserting a few lines somewhere, that fancy new for loop simply works. No need to scroll to the top if the file and remember which exact release of the 27 in the last year introduced the feature.
> It should be easy to always stay on the latest language version
With perl you can upgrade your language version whenever you like, and do it reasonably safely, because there's a lot of emphasis on backwards compatibility.
Perl may actually have a "always gimme the latest features" option, but I don't know what it is, because things like that aren't really that popular in the perl world-- we want old code to keep working the way it always has.
Yes i know, in fact i thought about it when writing that line but i thought it'd be obvious that the issue still remains, especially if said files are too big. It might be easier but you are not really solving the problem as well as opting in to the new functionality when and if needed.
I hope we can agree that Python 3 was qualitatively different because all of your libraries had to migrate first.
Now, your new argument depends on how hard it is to migrate a single file. If that’s sufficiently easy, you simply do it. Migrating to the last Rust edition, I had to fix maybe a handful of things in an entire crate; most files didn’t have to be touched at all.
Certainly, Python 2 to Python 3 was harder due to all the dependencies and having to convert the entire program.
My argument isn't really about migration is about not having to migrate if you don't need the features while not losing anything - by opting in to the new features/changes you can use whatever you want without wasting time on changing your code to do the same stuff just in a different way (unless you decide that this different way is actually better and worth doing, but that'd be your choice and not something indirectly forced on you by the ecosystem you decided at an earlier point in time to rely on).
Python supported piecemeal changes with backported Python 3 features using the future imports. They were essentially the same thing as what Perl 7 is espousing minus the promise of a breaking cutoff.
Perl had the could be bytes, could be characters fiasco in 5.8, but they didn't include a bunch of other difficult changes at the same time. 5.8 could be a bit rough around unicode, but 5.10 was pretty solid as I recall, and you can always use bytes; in a scope where you need bytes.
Good to know, but maybe it’s not my fault that I assumed an official post by the Perl Steering Committee on perl.org would show the best solution.
> Exists. Wouldn't be Perl if one hadn't the choice between multiple solutions.
That’s great, but you might notice that you’ve given me, who is new to these approaches in Perl, exactly no information on where to start.
> It is evident.
Yes, I was very transparent about the extent and recency of my experience. Luckily, that combative attitude to a mostly interested/positive comment isn’t representative of the Perl community as a whole, or there really wouldn’t be any users left.
> First, opting in to an experimental feature could be a one-liner, “use experimental feature ‘try’” or similar. There’s no point in punishing your valuable beta testers beyond that with a second line that’s entirely redundant with the first one.
It is. "use experimental 'try';" works already.
> The larger problem is the versions. This basically requires someone to update their script headers all the time if they want to keep getting new features.
This is intentional. The largest strength of Perl is that, barring significant security-type issues, a script written in 1995 will still run with the version of Perl you've installed. If you write a script with "use v5.36", you would change that version only if you intend to modernize the script with features from a newer version, and determine that such features don't break the script. This is harder to determine for some features than others, for example applying the 'unicode_strings' feature to an existing script written without it is rather perilous.
Users do not "want to keep getting new features"; they occasionally decide to make an effort to upgrade the Perl interpreters on their servers and personal machines because they want some new features.
A "use v5.36" directive is going to be very opaque for many users, but it doesn't need to be understood to serve the purpose of determining the minimum Perl version to install in order to run a certain script very effectively.
If user = server administrator writing simple scripts, you are probably right. If user = a team of devs, I don’t agree. You often want to use new language features just to simplify a piece of code, and you don’t want confusing rules around when you can and can’t use them.
What confusing rules? Given the promise of perennial backwards compatibility, the rule that if you want to use a feature you have to upgrade to the appropriate Perl version isn't confusing.
Neither is keeping development environment up to date and letting Perl versions on servers lapse until you install the latest Perl version because you have new Perl scripts requiring new features, or there are urgent security fixes, or you are building a new server or container anyway.
>>I know many experienced Unix people hate this trend, but there’s a reason some of the most actively evolving language ecosystems install more and more of their binaries into the user’s home directory.
The future is containers, which is basically increasingly thin images tailored to your application.
These days I struggle to find Perl on many EC2 instance I work on. Its always good to have something like Perl in your installation for some quick scripting work. But most don't have Perl installed.
>> I know many experienced Unix people hate this trend, but there’s a reason some of the most actively evolving language ecosystems install more and more of their binaries into the user’s home directory. Maybe that’s already the case for Perl, I wouldn’t know.
Seems like they could just lock all of the new features behind a “use x.x” line rather than going feature-by-feature. And they could include “use latest” or similar for people who don’t care about the risk and are fine just fixing things if they break.
No, that is what they are doing. "use v7;" gets you all the new stuff and that will continue going forward. They only listed what "use v7;" is the equivalent of in the article.
This is the right decision. The primary reason anyone still writes Perl is because they already have a lot of existing code and dependencies in the language. Backwards compatibility is a critical feature for a language in that position.
I doubt anyone would write a greenfield project in Perl when numerous other platforms are available. Raku is there if you like the design philosophy of Perl but don't need to worry about backwards compatibility.
I'll speak up for Perl. I learntbit voluntarily in my own spare time and actually get a lot of delight out of the language. I like that it is very unrestrictive. The sigils make sense when you wrap your mind around them and you miss them in other languages. It is excellent at parsing text.
I wrote a static site builder using Perl along with a very rudimentary templating system (https://soft.thran.uk for the curious). I've also found it very convenient for any sort of data shunting I find myself doing, it can easily manipulate and convert XLS, Json, other formats I encounter in my work.
I hope it doesn't go away. There's a mindset to Perl that clicks with me and just isn't present in many other languages.
A couple of the things that perl has over python and ruby... #1 is execution overhead, although that may have changed in recent times. #2 is that it's preinstalled on most linux/BSD OS's as a requirement for its plumbing, which makes it nice if you're trying to run a system on minimal dependencies.
It's ugly for all the reasons that people complain about it for but those are largely complaints and the rest of us have work to get done.
Oh yeah, and if you liked perl - I highly recommend learning AWK/SED. Even lower on the execution overhead (Can do shit in the processor cache and avoid memory allocation bottlenecks on enormous hundreds of gigabyte CSV datasets with a crappy processor in milliseconds kind of stuff)
Perl is excellent for data analysis in a terminal.
It is built to work with pipes, it's fast, concise, stable, and easy to extend. The REGEX integration is superb.
It seems to me that we have a couple of generations of programmers who turn up their nose at anything that isn't Python notation and then repeat vague arguments about Python's superiority. But like all dogmatic systems, there is no agreement among the zealots.
Python is far from perfect; I see many problems in the Python ecosystem. There remains much to be said for concision and expressivity in other languages. In any case, by the time one has written enough Python, one sees that the "purity of notation" is a myth. And the DSLs within Python (Pandas, Numpy) offer their own pain.
awk, sed, grep, C, C++, Java, and Perl are excellent tools.
Python wins out the popularity contest because of ease of readability that I think we can take for granted. Perl is true wizardry, I mean 'sigils' and stuff you need to know in order to make it work... Plenty harder to get at first glance for sure but it's truly occult in a lot of ways. I think that's kind of cool.
The way I see it is that you can do interesting things in pretty much every computer language written out there, it's just some are more arcane than others in some ways than others.
I remember when python started getting big and all the old perl heads were complaining about the forced indentation hahahaha
That makes collaboration hard. The freedom I have in putting my thoughts into code, uninhibited by syntax, types and conventions makes it nearly impossible for others to understand and modify my work.
The freedom to flexibly put thoughts into code can be used to make code more readable, or less, it depends on the programmer. A significant code base in any language can turn into a hot mess
The perl coding team I was on made readability our #1 priority. We didn't use tricky or out of the ordinary syntax unless it was absolutely necessary (like unusual regex, etc), and in those cases we would document what was happening with comments and maybe a reference to the appropriate man page. We had a style guide, for a consistent look and feel. We paid attention to naming and commented blocks of code, so the reader would not need to run an interpreter in their head while skimming through the code. We'd seek feedback from each other. It worked out well. We had no problems reading our code.
We used the flexibility and expressiveness of perl to make our code more readable. We had the freedom to tailor the code to what we were doing, instead of the other way around.
Anything that "... depends on the programmer" is doomed to fill up with the output of a lot of programmers who were insufficiently disciplined. At some point the majority of the work becomes avoiding or working around or firewalling that output on your own projects. Telling people to just "Get Gud" at programming is a non-solution. The more I do this the more convinced I become that using a robot (compiler, linter, static-anlyzer) to enforce discipline is a good idea and the more difficult it is to escape that robot's gaze the better. Anything else is just going to cause massive pain later.
> The freedom to flexibly put thoughts into code can be used to make code more readable, or less, it depends on the programmer. A significant code base in any language can turn into a hot mess
If one learned how to collaborate on projects which, I think, the perl language does not incentivise.
(edit)
> The perl coding team I was on made readability our #1 priority.
I'd love to read more about that. Have you by chance documented your experience somewhere?
> If one learned how to collaborate on projects which, I think, the perl language does not incentivise.
I tend to think that things like high level documentation, organizing libraries and functions into good abstractions, comments, and naming make a bigger difference than the programming language chosen, within reason. The goal can be to make things as easy as possible for all developers as opposed to using every feature in a language.
There's a bunch of developer bad habits that can make collaboration hard. The processes and standards around development seem to make a bigger difference than the choice of language. So many coders don't bother with high level documentation, comments, or decent tests, nor do they spend the time to refactor their code as they go along to clean up readability issues. So many people just don't seem to want to bother, regardless of language. Often you have to fight management for time. I've seen atrocious and unmaintainable code in many languages.
> I'd love to read more about that. Have you by chance documented your experience somewhere?
My experiences grew to leading a team that were automating the reporting and deployment of complex configurations on a few tens of thousands of network devices, things like routers, switches, and voip, with all kinds of functionality for reporting, auditing, deployments, dry runs, debugging, etc. Perl is well suited to parsing and manipulating complex text configurations and outputs over ssh and interfacing to other related infrastructure applications.
We were able to process devices an order of magnitude faster than HPNA (java) and ansible (python). Our code was easier to read than code using these other systems, and more sustainable. We were able to handle complex projects like mpls cutovers, 802.11x and qos migrations and upgrades, etc. We'd translate complex network architecture standards to idempotent configuration management code. We also did a lot of reporting and compliance auditing. Some of it on short notice, most of it long lived.
Our project scripts read like pseudo code, a mix of perl and templating. The flexibility in perl helped us focus on representing our domain space as efficiently as possible. Our customers, who might be network engineers, managers, or compliance officers, could even read their project scripts, following the business logic, in this case the network architecture standards and/or reporting/auditing specifications.
Several of us were senior engineers, and there were always several junior engineers. Senior engineers tended to work on back-end library modules and more complex projects, at least getting them off the ground. Junior engineers were mostly contractors working on less complex projects. We made things easy for ourselves, tagging to-do notes by name and category in our source code. Everything was grep'able. Our processes were not in our way. Senior engineers would do code reviews, for readability, test coverage, and maintainability, usually just adding clarifying comments during a screen share and a phone call, maybe seeing a better way to arrange something, and be a second set of eyes looking for gotchas. We'd pay attention to the depth and complexity of our abstractions, try to keep user and developer documentation close to the relevant code, and have consistent naming for variables and functions. Seniors more frequently checked up on juniors. We'd discuss things as a group when need be. Customers were responsible for signing off on correct integration test outputs. I bet that well over half our bytes were comments, documentation, and tests.
My email address is in my profile, if anyone is more curious.
> It worked out well. We had no problems reading our code.
That's not the relevant test, even for fairly large teams. More important is whether someone from another team (or more critically, a brand new hire) can read and modify the code.
That said, based on what you're describing, folks from outside your team ought to get comfortable pretty quickly too .
Teams do have conventions. There's code review. There are linters and pretty printers. There's even a standard set of default linter rules in perlcritic, taken mostly from Damian Conway's Perl Best Practices.
Perl has no problem from flexibility not shared by C, Common Lisp, JavaScript, Ruby, or Ada.
Writing in code is the same as writing in your native language. You are NOT writing it for the machine - you are writing it for the others like you. You are telling a story, and yes, it's hard.
This is true. The downside is that a couple days of not worrying about the understandability of your code can lead to a spiral that quickly gets out of control. Plus it makes expanding a dev team significantly harder since you really need people all on the same page of what readability means. Yes you can do things like coding standards, but the more free form the language the tougher such standards are.
That doesn't make a language bad, but it does change when and who can get away with using it. Same as the argument that comes up every time people talk about Lisp.
Is there any branch of engineering where "a couple days of not worrying" does not have the potential to be "lead to a spiral that quickly gets out of control"?
Languages with tooling that will yell at you if you are sloppy make it harder to get too out of hand. Part of why I've started learning rust is between the compiler and the linter I'm forced to ask if I'm being sloppy or not.
The languages that truly let you do anything you want flexibly invariably lead to incomprehensible ivory towers. It is why managers hate Lisp, it's why Perl is joked as write-once read-never.
The issue with the "DSL" is the third letter in the abbreviation: LANGUAGE. A language isn't just a specification of syntax. A language is a collective shared understanding between a sufficiently large number of people. Without that, it is a dead language.
Socially in programming, programming languages need to align to real-world concepts, because that is what enables the language to be understood by new adopters. And converging on those shared meanings is hard, because designers and lingo makers can't anticipate all the variations of understanding people will have.
Learning a language is hard. So every DSL I've encountered invariably forces me to be the compiler: what is this being translated into language-wise and what does it convert to. That is a big load, so your DSL better provide power and convenience to deserve it.
DSLs also run headlong into NIH/creator's advantage. The people/person that make the DSL and implement it will know it FAR better than any adopter will. And documentation is always lacking. So what is blindingly obvious to the core practitioners is very much not so to the noob.
You are limited by the average intelligence of the programmer at a cultural level. The lone mad scientist may produce amazing ideas, but all the groundbreaking technology happens because an army of people can be coordinated to implement something.
Lisp is basically a DSL factory, and everything is done via DSL. Can Perl7 do that better than Lisp? Is that a good or bad thing?
Well, it doesn't matter. Perl5 is what it is. Perl6 was a social failure. I doubt Perl7 will change anything.
> The languages that truly let you do anything you want flexibly invariably lead to incomprehensible ivory towers. It is why managers hate Lisp, it's why Perl is joked as write-once read-never.
I've yet to see that happen in a Lisp project. On the other hand, inflexible languages like Java lead to abominations like the Spring framework, where tons metaprogramming happens at runtime via reflection.
The thing about compile-time macros (e.g. Lisp macros) is that they can be debugged at compile time. For example, if you see a macro invocation, and you don't know what it does, you can just expand that macro right in your editor, and look at what code was generated.
People are always worried about DSLs, but most projects end up with their own DSLs. They're just often built out of functions, methods, objects, etc.
I think the real issue with understanding a project is learning the concepts, architecture, and abstractions they use. Knowing e.g. Go really well doesn't save you from having to wade through the giant pile of code used to express the program's intent. Abstraction is a way to get that complexity filtered down as close to the problem's inherent complexity as possible. Yes, you'll have to learn the abstractions, but if done reasonably well, that will still be much easier than trying to understand the non-abstracted version.
> People are always worried about DSLs, but most projects end up with their own DSLs. They're just often built out of functions, methods, objects, etc.
I wish there was a kind of super-upvoting that added one to the font-size of the comment you're upvoting, because I would absolutely super-upvote this comment.
> Abstraction is a way to get that complexity filtered down as close to the problem's inherent complexity as possible. Yes, you'll have to learn the abstractions, but if done reasonably well, that will still be much easier than trying to understand the non-abstracted version.
Absolutely. It's possible (arguably, too easy) to build a bad DSL, but the point of a DSL is to surface the inherent complexity of your domain so you can grapple with it on its own terms. Not using a DSL doesn't mean that complexity goes away; it just means you have to grapple with that complexity using coarser tools. You usually end up with a DSL-lite of types and functions that work in your domain, often with warts due to poor interactions with the host language's feature set.
For a lot of reasons, I'm a big fan of eDSLs, which aren't too much different from the DSLs-lite we usually end up with. But overcoming that impedance mismatch with the host is not easy. (That's one of the benefits of Lisp, I think -- very low impedance mismatch to overcome!)
I've gotten somewhat good at doing eDSLs in Java(!), where you want to keep things mostly idiomatic, unconstrained Java while still adding the novel behaviors you need for your domain solutions. But you're always going to have to wrangle the complexity of your domain, whether or not you involve a DSL in that effort.
when i first heard that lisp was supposed to be good for making DSLs, my first thought was 'gross, i'll just stick to writing plain lisp thankyouverymuch'
but i think that's a misunderstanding- it's not a suggestion to make a mess with reader macros or whatever, its an observation that common lisp _naturally_ builds up to form the language constructs needed in your domain, simply by virtue of providing the tools and syntactic flexibility needed to construct them
another thing that seems to put off newcomers to cl is the parallax between the promise of a smooth, polished dev environment, vs the nuts-and-bolts-exposed nature of the core language itself (eg the infamous eq/eql/equal/equalp situation)- but i think this can be explained as how cl is _not_ a scripting language, and you are expected to paper over these edges as you define the system
cl gives you the best tools in the business to turn itself into a dev interface so clean it might as well be a DSL
CADT (the "Cascade of Attention Deficit Teenagers") is about rewrites. Jamie particularly doesn't like the habit of throwing away old bugs you didn't fix on the rationale that if they were really still a problem you should take the time to install the new program that doesn't fix them and report them again...
I don't think rewrites were on the map for Perl, except arguably for Raku (Perl 6 as was) and I don't think Raku's developers were intending to throw away bug reports and just begin over next year, indeed even though Perl 6 failed in its original goals Raku seems to still be going.
I wouldn't classify any of the spectrum of language evolution strategies for popular languages as CADT, even Python where Python 3 was needlessly incompatible this wasn't part of some larger plan it just fell out from what they wanted to do and was then mismanaged. Getting to a place where strings aren't just "some bytes, who knows if they're human text, good luck" is difficult enough to justify a lot of pain on the route there, notice how C++ has tried and failed several times already.
Jamie was thinking mostly of things like GNOME where this was (is?) a big problem.
Sure, Jamie is concerned about the practical consequences. If your response to Jamie's 2019 bug report about X in WordCount was to close the ticket because you rewrote WordCount as WordCounter even though you've got no reason to think that might fix X, that's CADT. You are wasting his time. Don't use a "bug tracker" this way just write "This is broken garbage, I don't care" so that people know what's up.
I don't see that nonsense with the programming languages. Even something egregious like provenance. There's a problem, neither C nor C++ make any actual sense on a vaguely modern computer as described if pointers are just machine addresses. So the compilers invented "pointer provenance" to explain why what they actually do is reasonable - but no such fix is actually endorsed in the Standards. A defect report was raised in, I think, 2003. But it was never repaired. Nobody is under the impression that C++ 11 magically means this doesn't count, they didn't fix the problem, it's still there, and in C++ 14 and C++ 17 and C++ 20. It's even there retrospectively (this seems to be controversial for some reason but it's obviously true) in C89 we just didn't realise.
Under CADT the C++ committee would say, well, C++ 20 is new, you need to write a new defect report if you claim it's still broken. But they don't suffer CADT, their problems are altogether different.
> the steering council simply wants to to avoid being bitten by Zawinski's "CADT" model
They're being bitten by the bikeshedding model instead, actively encouraging the use of the language like it's 1998. You want to live in 2022? Fine, just type
I hesitate to say "they're bikeshedding" since triviality can be highly subjective.
For me, backward compatibility and a smooth, discretionary upgrade path is greatly preferred to trends in language popularity. Therefore, I appreciate the extra deliberation they seem to be taking. Of course other people have different preferences.
To suggest they're bikeshedding implies there's some bigger problems being ignored. Can you elaborate on what those may be?
I see. Still, somehow I don't think trying to crow-bar "Perl11" onto machines against end user wishes is the right approach. There could be a backlash.
I worked on a site conversion of a 14 year old Perl site about a decade ago. The creator had build a database out of the file system leveraging Perl's parsing capabilities. Still one of the fastest sites I've ever touched in my career but the code was really hard to follow.
I find that the ideal place for Perl (for me) is smaller one-offs. When used for a huge site it can get really messy.
I used Perl in gradschool and it still has a special place in my heart. You're right: for reading / writing files it is excellent, and I've never seen a faster tool for those tasks either (okay, unless you want to write in C )
Hah, the former. My work isn't that arcane; I've a habit of saying XLS when I mean all things Excel. But there are compatible CPAN modules for either format!
If you're interested in the story behind why some languages and specs (PHP 6 and IPv5 and a few others) appear have skipped a major version number I wrote a little bit about them: https://blog.mclemon.org/missing-version-numbers
It only appears to have been skipped when looking from today's point of view, since many references to Perl 6 have been scrubbed from the web and replaced with its new name. But for those who were following the saga back then, starting with the announcement that the next major release of Perl would be Perl 6, followed by the many announcements of cool new features that next major release of Perl would have, culminating with its long-awaited Christmas release, it's hard to say that it never existed under that name.
The OP "What happened to Perl 7" doesn't say anything about Perl 6!
I'm still confused about what happened to Perl 6! "replaced with it's new name" -- googling, that's "raku". So... what was going to be Perl 6 is considered a different language, and not particularly compatible with Perl. But "Perl 7" is meant to be less of a departure?
It is weird OP left Perl 6 out of the story of "what happened to Perl 7". I guess it's just too painful?
Long ago (~year 2000), Perl 6 was announced with great fanfare. Larry Wall set out to start producing a bunch of specs ("Apocalypses", "Synopses" and "Exegeses"), while a bunch of people set out coding. Then two things went wrong.
First, the Perl 6 development had bad project management. According to the reports, the culture attracted experimentalists with interesting ideas, but didn't get along with boring people who wanted to do things like releases and steadily approaching functionality. So there were a number of projects, a number of which were abandoned. Things dragged on, and nobody was cracking the whip. It took 15 years for Perl 6 to finally announce its first official release.
Perl 6 also turned out to be too different from Perl 5. Different syntax, and no XS, which meant every Perl module interfacing with a library wouldn't work even if some sort of automatic conversion, or compatibility mode was possible. XS is tied to Perl 5's internals, and Perl 6 was an effort to make a new language from scratch, without using the old interpreter.
Meanwhile, people started abandoning Perl 5 reasoning that since Perl 6 was in development, 5 would eventually die, and the lack of compatibility would mean current efforts on 5 would be wasted.
It took a very long time but finally it was decided that Perl 6 wasn't really Perl 6, but some other, Perl-like language, and renamed to "Raku" to signal this. But by then it was already too late, most everyone had already moved on, and by the time when Perl 6 was finally released it was an extremely niche thing very few people had any interest in.
Meanwhile, Perl 5 development continued in the background, and they decided that it'd jump version from 5 to 7 to avoid confusion. Perl 7 is the direct descendant of Perl 5, with the same syntax, new features and very few deprecated things.
> It took a very long time but finally it was decided that Perl 6 wasn't really Perl 6, but some other, Perl-like language, and renamed to "Raku" to signal this. But by then it was already too late, most everyone had already moved on, and by the time when Perl 6 was finally released it was an extremely niche thing very few people had any interest in.
I agree with the rest of your comment, but I would like to emphasize the ordering of these events: first "Perl 6 was finally released", and only later was it "decided that Perl 6 wasn't really Perl 6 [...] and renamed to Raku to signal this". It was too late not only because most everyone had already moved on (by the reasons you mentioned: why would anyone invest in Perl 5, when the migration to Perl 6 would throw away all that investment?), but also because Perl 6 had already been released under the "Perl 6" name.
It's a pity really, lots of people worked hard on it, and I think things could have worked out better by just better messaging. Doing the "Perl 6 to Raku" renaming early on might have salvaged things.
I think this is an important lesson -- technical skills are great, but communication is extremely important.
Meanwhile, Perl 5 development continued in the background, and they decided that it'd jump version from 5 to 7 to avoid confusion. Perl 7 is the direct descendant of Perl 5, with the same syntax, new features and very few deprecated things.
Did you read the FA? They decided that, "Yeah nevermind: we're not going to call it Perl 7 ", until some future time where it may make sense to do so to somebody again - but maybe not, who knows? Backwards compatibility, everyone!
I'm not sure if I quite understand the Perl community problem of seemingly never, ever wanting to commit to something new. It's the metaphorical problem of not being able to get across the creek, because you imagine your feet glued to the stepping stone you're on.
> Did you read the FA? They decided that, "Yeah nevermind: we're not going to call it Perl 7 ", until some future time where it may make sense to do so to somebody again - but maybe not, who knows? Backwards compatibility, everyone!
I admit I only skimmed it, I was busy at the time. But it's not too incorrect anyway: Perl 7 will be a direct descendant of Perl 5, whenever that happens.
> I'm not sure if I quite understand the Perl community problem of seemingly never, ever wanting to commit to something new. It's the metaphorical problem of not being able to get across the creek, because you imagine your feet glued to the stepping stone you're on.
I don't think there's a general "Perl community". I mean it's a thing that exists, but as I see it, it's a smaller and tighter group than the general set of people using Perl. There were lots of people that used Perl for whatever needs they had and were never regulars at conventions, Perlmonks and other places, and as a result had little influence and weren't up to date on what the core people were doing.
Many of those also had no interest in committing to such a thing. Eg, they were writing stuff for their day job, and had little interest in participating in the development of the language.
I think quite a few people are confused by the line about it becoming Perl 7 "sometime in the future". I think it is unlikely that that is the distant future, myself. I think the main hang-up is they'd like to get the new built-in object system (Corinna) working.
If you're a "move fast and break things" kind-of guy, watching the perl devs in action will annoy you, because they're committed to not breaking things, and if they have to move slow to avoid it, they will.
On the other hand, if you want to be sloppy with your own perl code, or work with CPAN modules that move faster, that's up to you.
> It is weird OP left Perl 6 out of the story of "what happened to Perl 7". I guess it's just too painful?
Actually, it is not weird when you consider what the article is about. The announcement of Perl 7 was not about Perl 6, it was about Perl 5. Perl 6 had been already been renamed to Raku almost a year prior (Oct '19, according to https://en.wikipedia.org/wiki/Perl). Well, OK, it was about Perl 6 in the sense that newer versions of Perl 5 could not be called Perl 6, but that's about all. Raku/Perl 6 is a related but different language (hence the rename).
I suspect the Perl Steering Council was writing mostly to people who are focused on Perl 5. The announcement about bumping Perl 5 to version 7 created some controversy (centered on what the changes to Perl 5 would be for the bump to version 7). This led to the new governance structure discussed in the article, but it also kind of left people in the Perl 5 community hanging. A lot of them were thinking things like "OK, so are we going to bump to version 7 now, or not?" I think that is what they were trying to address. And anyone who is focused on Perl 5 already knows all about Perl 6/Raku, so there is little reason to bring it up in this article.
It's a matter of personal taste I think - I might change it to say "Now called 'Raku'". Also it's worth remembering that at the time I wrote the article there were plenty of blog posts and articles about Perl/Raku floating around. I wanted to briefly summarise it and use it as a way to look into the other ones which had been bouncing around in my head.
I still remember a wonderful presentation by Damian Conway a number of years ago about all of the great ways Perl 6 could turn into whatever domain specific language you needed it to be. It was beautiful, I was awestruck. I've always enjoyed Perl as a language. But I walked out of that presentation thinking "That was so beautiful, and I don't want it anywhere near my business." Because the last thing I need is software written in a language I can't hire anyone else to maintain.
At some level that's what I think has happened to Perl in general. I never liked Python much, until I got forced to use it on a new team. Now I'm convinced that it has a really distinct advantage -- there aren't too many ways to write Python, so an experienced Python developer can figure out code pretty quickly. But Perl programs are frequently art pieces that take a lot of effort to truly grok.
The problem is, as appealing as building a DSL (in or out of another language) is to engineers, turning your source base into a pool of pidgins where there are no native speakers and all new hires or people moving between parts of the code base will need to learn a new language is not an advantage. It is completely unnecessary friction.
We all get this when we encounter jargon-crazy groups, but somehow forget it when it comes to code.
OTOH an application does have its own language. Someone said "When you devised an application you wrote a hundred words [(functions)] or so that discussed the application and you used those hundred words to write a one line definition to solve the application. It is not easy to find those hundred words, but they exist, they always exist."
I think for non-trivial programs it is not avoidable to have to learn first the terminology of the domain and the terminology of the program (how things were named when it was not dictated by the language of the domain).
However, hacking the syntax of the language is a whole different story, and perhaps that's where DSL-ready languages go wrong. That's ironic coming from someone that just quoted the author of a language that has no syntax to speak of, and macro powers that are sometimes compared to Lisp macros. Just use them wisely, one would say, but that kind of wisdom comes with experience.
The folks responsible for that are still growing and improving their own ecosystem. It's a Perl-family language but it's different enough that they stepped away from the name so both Perl and Raku can flourish.
I think Perl fell victim to a huge change of fashion around the 2000s.
'There's more than one way to do it' and the whimsical, deliberately slightly obtuse design of Perl used to be what people valued.
Then something happened (perhaps it was how people started building much larger apps in 'scripting' languages, to use an out-dated term) and I think to most people under 35 deliberately having 'more than one way to do it' sounds like an atrocious idea and something they don't want anything to do with. Same with linguistic whimsy in their language design - they don't want to have to look up why a method to do with objects is called 'bless' for example.
Look at the success of languages like Go, which are the polar opposite of Perl's philosophy.
Perl is a great language if you are a single developer. I think there is some "reverse network effect" for Perl, where people might enjoy writing in the language but nobody enjoys having to maintain Perl written by someone else. So basically you don't encourage others to use it.
> I think there is some "reverse network effect" for Perl, where people might enjoy writing in the language but nobody enjoys having to maintain Perl written by someone else.
I think you might be right. Perl is a lot of fun to write, and it's still my language of choice for log parsing and automating sys admin stuff, but while it's possible to write Perl in a way that's easy for others to maintain and understand most people don't (less fun?) and for small projects it's even sometimes faster to rewrite something than mess with what someone else left you.
Actually... now that I think about it, is it ever a joy to work with someone else's mess in any language?
> Actually... now that I think about it, is it ever a joy to work with someone else's mess in any language?
Not really, but not every language is equal. Wen developing for my own sake, I enjoy Scheme or Haskell, but I would rather maintain code written in C#.
Interesting question, maybe computing was still a wildlife space to explore and 'more than one way' was felt like a challenge in creativity.. then the field decided it had to become industrial (the term that comes to mind thinking about all the professors talking about UML, diagrams, build swap-in packages like ICs) and suddenly, standardization is the only way.
C# keeps accumulating "more than one way to do it with every (too frequent) update.
A guy who is an expert lets say C# 2.0, would have no way to just read
code for C# 9 that took advantage of new features and syntax.
It can be procedural, object oriented, functional, in a limited way aspect oriented, it can be data oriented in a way and all of those at the same time.
Hmmm I'm not sure I think so? JavaScript is a pretty simple and compact language. Referencing object fields yes... but that's because of a uniform model of arrays and objects afforded different syntaxes. That's not really what I mean.
> I never liked Python much, until I got forced to use it on a new team. Now I'm convinced that it has a really distinct advantage -- there aren't too many ways to write Python, so an experienced Python developer can figure out code pretty quickly.
This is why I picked up Python many years ago. I had been using Perl 5 for my scripting and system administration needs. While it worked just fine any project larger than a few hundred lines needed a lot of discipline to keep maintainable. Perl encouraged too many line noise shortcuts, reading unfamiliar code was too often an exercise in looking up uncommon operators.
It's elegant when writing but frustrating when reading. Python not only read a bit more like it executed but didn't lend itself to unreadable shortcuts. If your code is elegant to write it tends to be straightforward to read.
When Perl 6 was still Perl 6 the DSL stuff sounded interesting until like you I realized it would just turn large projects into unreadable messes. It wouldn't help the small Perl scripts be more readable nor would it help the large projects be more maintainable.
"It's elegant when writing but frustrating when reading"
As a person who used to use Perl 5 rather extensively back in the day for all sorts of things, I fondly remember the joke that Perl is a "write-only language" :)
I'd joke about Perl 5 being write-only until I had to go read my own code from only months prior. I would get so mad at myself for making it so hard to follow but then for expediency I'd go and do the same damn thing in the current batch of code. Then months later curse myself. I've opened Perl files that I swear had bitrot but nope, not corruption, just a bunch of sigils and weird operators.
beware, when reading django faux-metaclass based declarative ORM it feels not very far from a bad DSL. In the end you have no clue about what's what or the meaning of things.
This is one thing I'm certain of after 40 years of programming: successful programs will need to be read more often than they are written. Language features that make programs cool or fun or "easy" to write are generally counter-productive to readability. Even things like this that are meant to make a program "expressive" are in fact just gimmicks if they are unique to one program.
Features that make a language expressive in a consistent way are very valuable though. Python is a good example of this, even people who don't know anything about Python can generally read a Python program.
I totally agree. That's why I loved it for me, but hated it if I ever had to deal with someone else's code. And why I never use it any more, unless I'm doing a bunch of text processing on my own workstation.
you've also described the reason lisp ultimately failed IMHO - it's selling point is creating first-class DSLs, but you can't hire anyone experienced in the DSL you've got...
There is no such thing as a software shop with no local DSLs, in any language. All the in-house abstractions constitute DSLs.
Without syntactic abstraction (function and data only), things are worse, because those abstractions have to be coordinated together into proper usage scenarios, and those are written by hand in multiple places. So you're looking at eye-glazing repetitions.
Oh, and just because a language doesn't have macros doesn't mean you won't see any syntax-level DSL. You may find textual code generation in a project in any language. C++, Go, Java, you name it.
Understanding a large amount of unfamiliar code is a challenge no matter how it is organized. Whether it uses syntactic abstraction is a minor factor, and it is not an iron clad truth one way or the other whether that helps or hurts. If you can learn a complex API where you have to take several kinds of objects through a multi-step workflow, you can probably learn a syntactic construct that does it for you; especially if it is documented and has test cases.
You will never hire anyone who is experienced in the vocabulary of your code base, unless it was someone who was previously laid off.
The old friend, you can always rely on to do quick scripting work. I've written non-trivial quantities of Perl in the past, and maintained other people's Perl code too. Contrary to popular opinion, I've always found it easy to maintain.
I still call upon Perl to do
open(FILEHANDLE, $file) or die;
while(<FILEHANDLE>){
....
}
close(FILEHANDLE)
Sort of work.
These days I do good deal of work in Python. Python itself has been heavily Perlified over the years, especially Python3. Its fame seems to have come moving away from Pythonic principles, to Perlic principles. They add lots of features, even syntactic features, libraries and even things like typing. Python isn't the small, minimal core of ecosystem anymore.
Every single release of Python gives me the Perl feel from the old days.
So I kind of don't miss Perl at all, after all we managed to turn Python into Perl.
Use the debugger: `python -mpdb` is a very similar experience to `perl -d`, so much so that it feels like a gift from an earlier perl-to-python emigrant.
(I showed my co-workers `perl -d`, they later discovered and showed me `python -mpdb`)
I don't really see it either, but maybe a few things like collections.defaultdict (like Perl's normal associative arrays) and sysconfig (very similar to "use Config;").
Python was always deeply and extremely perl-inspired.
Each line of the Zen of Python is the negation of one of the Perl's architectural principles. And when your entire identity is denying something, you can't be anything but similar to that thing.
I still love Perl as essentially a low-level abstraction layer for C. I can get so much done at the system level without having to compile a bunch of code every time I make a small change.
I've heard that Python (and maybe some others) are slowly replacing Perl as a "subsystem" of Linux, but I doubt that will happen before I retire. So, until then, TIMTOWTDI!
None of this makes sense IMO. The point of Perl 7 was not to allow breaking changes. It was to send a signal that Perl was alive and that the Perl 6 messup was behind them. Then two years later they announce that there's still no Perl 7 because they're still working on Perl 5.
They are not working on perl 5, neither a perl 7. The biggest excitement might be the design of their new object system. But if you look at it and compare it to the design of their OO designed in 2002 (for their perl 6) it's a serious throwback. not even talking about their inability of a proper implementation.
in the meantime I've implemented that, plus types, plus proper signatures, plus unchecked arrays when safe, plus an integrated ffi plus tons more features.
their is no development, only maintainance. plus lot of internal fights. plus a CoC, which was used to eliminate the only remaining developer, because he dared to critice the stakeholders which blocked progress, and still continue to make the codebase worse.
For all his agreeableness, maybe Larry Wall left behind a chaos of unsolved semantics. Perhaps discussions they never really recovered from. I do not know the story, and this is speculative only.
From what I've gathered, Larry was simply focused on Perl 6 (which became "Raku" in Oct of 2019). It seems to me that he just wanted to let others handle the new development in Perl 5 rather than being directly involved all of the time. I don't think he has done much at all with Perl 5 for quite a long time.
I chose to build in Perl because of its ubiquity and committment to backwards compatibility.
I was extremely frustrated with existing projects having dependency issues and frequent breakage and wanted to avoid that at all costs.
Perl's flexibility has allowed me to develop my own coding style, which is basically Java-like, and I rarely have trouble figuring out what something does, even months later.
I think Perl is vastly underrated as a language, and its suffers from repeat-speak of people who have only seen poorly written Perl or have never seen a well-managed Perl project.
One of my favorite things about Perl is that there are 20 years of code samples on the Web for it and they ALL WORK because Perl has not introduced breaking changes since 5.000.
Unfortunately most of that doesn't really matter for any non-trivial project, because you'll almost unavoidably need a module for something. A lot of the time you'll find out that it's bit rotted 5 years ago, and the bug tracker has a 7 year old issue nobody seems to have read.
It's even worse for anything that interacts with libraries or APIs -- those change over time, and often nobody does the work to repair compatibility.
So I think at this point it's fair to say it's dead. Even if the core language is technically functional, the ecosystem is rotting.
Perl doesn't need an ecosystem any more than the Almquist shell does. It's just a nice tool that's omnipresent on UNIX systems. It's definitely going to be useful for shell scripters for a very long time.
Yes, most of them relatively minor though, which is why the OP has never stumbled across one.
(I found one once-- it turned out one coder had invented his own hash slice syntax. It wasn't supposed to work, but it did, until a particular upgrade...).
In general, Perl has been traditionally committed to backwards compatibility, but not fanatically so-- there is a deprecation cycle that can be used to remove the more problematic things.
No, those are "real" breaking changes in that a script running in one version of the interpreter could break if it runs in a later version, without any modifications to the script itself. But in fairness, they're a very conservative definition of breaking changes - e.g. the output of the `--version` flag changing, or some Unicode characters classes changing.
I found Perl suffered from dependency issues as well. Not the language but the modules you tend to want to use. Especially when they're underpinned by c libraries. I'd preference operating system libraries then fall back to Cpan. Over time it got harder to maintain older applications as the libraries dropped out of repositories. Have you found anything similar?
Seems like you're jumping through a like of hoops to write thing in a Java-like fashion when you could be using Java, which has a much healthier ecosystem.
Unfortunately, yes, people are still building new systems in Perl. In my experience, it's been because existing infrastructure that new systems need is already written in Perl.
I'll note that if you use a strict subset of Perl, and write it well, with lots of unit tests, it's bearable to use. But it falls massively short when it comes to anything concurrent or async. And if you stray into the "clever" subset of Perl, frankly it becomes hell. The ecosystem is also pretty much dead, it's not unusual to find bugs in packages where the issue tracker hasn't been responded to in 10 years, and the issue for the bug you're interested in has been languishing for literally years.
> The ecosystem is also pretty much dead, it's not unusual to find bugs in packages where the issue tracker hasn't been responded to in 10 years, and the issue for the bug you're interested in has been languishing for literally years.
Yes. Very much this. Anecdotally, I'd estimate that, outside of the big, well-known modules, about 25% of the CPAN modules I try to use have bugs that render them unusable and unresponsive maintainers.
> And if you stray into the "clever" subset of Perl, frankly it becomes hell
Yeah, this is the thing... it's very very easy to go down a one-way street to an endless hell of incomprehensible line-noise code and "clever" tricks that are incomprehensible and unmaintainable.
On the plus side, having dealt with the hell of "clever" perl early in my career (and, ahem, maybe having been guilty of writing some), it has beaten into me the absolute necessity of writing the simplest, clearest, not-clever code possible. In some other language.
My experience with all of these has been that they've been frustrating to work with, and general support for them in the broader ecosystem is lacking and mutually incompatible.
> those modules are probably abandoned and you shouldn't use them anyway.
Yes, that's the point, the ecosystem is now lacking because of the number of abandoned modules (even for quite common stuff).
Mojo::IOLoop is quite lean and straightforward to work with, but it's quite minimal. If I'd pick something, I'd probably pick this one as the decision to use Mojolicious was very rewarding.
AnyEvent is harder because it uses closures everywhere but it works really nicely otherwise. It's not endorsed by some in the Perl community because the author insists that it doesn't need a license and chooses to develop it and track bugs on his own infrastructure. EV is developed by the same author and it requires a compiler, it can be plugged into AnyEvent. They're both very good quality.
IO::Async is a different event loop brought to you by the people that do not endorse AnyEvent but are involved with Perl 5 development. I haven't used it, but it has a large number of active bugs on RT, plus past criticism from the author of AnyEvent.
There's also POE which we use but it's possibly on life support and you probably wouldn't want to use it unless you want to live in callback hell, so I fully understand your frustration if you've actually tried it. We use POE in a critical part of our infrastructure and has worked fine, but that part is quite frustrating to maintain. Futher criticism of this event framework could be found here:
As a sidenote, I'm curreny working with Ruby's Async:IO which is a breath of fresh air compared to other event loops that I've worked with in the past.
Perl is very well suited for certain tasks (not large software systems, but programs that process data). It is also one of very few languages/ecosystems where you can expect your code to work after >10 years. This is why I sometimes use it, for example my fs consistency checker (https://github.com/jwr/ccheck) was written in Perl specifically because it's a long-term tool and I would like to be able to run it on any system in 15 years.
Compare this long-term approach with the fires in Python or (heaven forbid) Node ecosystems, where things break all the time.
"not large software systems, but programs that process data"
I disagree. Perl still works fine for large systems. I understand why people choose other languages. It depends on what your needs are and what skillset works best for you and those you work with.
I've also used Perl for small utility programs (basically as a "enhanced" shell script). The problem with Docker or virtualenvs is that they greatly increase the installation complexity of your script.
For example, two years ago I wrote a small (50 LOC) Perl script to convert between two text format. On Unix you could install it by just putting the one file in /usr/bin. It would run in less than 1 ms, which was critical because it was repeatedly invoked by another (legacy) program. It was mostly a bunch of regex so it was natural to write it in Perl (or awk maybe but I don't know awk).
The python equivalent would have been at least 30x slower, significantly longer, less portable etc. Using docker just for this script would have been overkill. The only real alternative IMO would have been to make a statically compiled Go or Rust binary.
I do use Docker for freezing dumpster fires like CSS toolkits that need to be built using node. And that works well to a certain extent, but there is still the complexity of Docker, having to set it up when you want to use the tool, having to deal with changes happening in it over the years, setting up filesystem sharing, etc.
Whereas with Perl it's easy: any UNIX system (and I do mean UNIX in a wide sense, as I've run perl on systems like HP-UX, Solaris, Unicos, IRIX and others) will normally have perl5 installed and you'll be able to just run your software. If it doesn't have perl5, installing it is usually very easy and you don't have to deal with horrors like the python2->python3 transition.
Speaking as a London-based freelancer specialising in Perl, I can tell you that the number of companies developing new systems in Perl is tiny (like, maybe, half a dozen).
Until four or five years ago, there was still plenty of maintenance work to be had, but even that has pretty much dried up now.
I'll qualify that by some observations re the Ruby job situation in London which seems quite steady. Rails jobs in London in Indeed, for example have hovered around the 250 mark for many years now whilst Ruby jobs advertised are still higher than Golang. I just received an email from a recruiter listing 21 Ruby roles so it's not quite Perl yet. I also notice a high percentage of "Who Is Hiring" roles here on HN feature Ruby/Rails so maybe Ruby's fate will be as a middle-tier language with adoption just below the big guns - Python, JS, Java, C# and PHP.
Yes, the trend of usage metrics don't look promising. It's basically only used for Rails and the heyday was 2010-2016. People are much more likely to reach for Django/Python or Javascript backends, especially with the ubiquity of React. Ruby jobs are going to generally be maintenance of older projects.
Will take awhile for Ruby to reach that state (if at all), 15 years at least I'd say. The reality is many companies are built on Ruby and rewrites are hard. Stripe, Shopify, Github, Gitlab and the list goes on. Perl never had this kind of traction afaik - yes many scripts were written on Perl but not many .com companies were based on it.
But I'm not arguing Ruby is well past it's peak - that is (sad, to me) fact because I happen to think it's a gorgeous language but it is what it is. I guess I'll have to get used to Python/Node or maybe switch to low level which is an old and perhaps unrealistic old dream of mine.
> Perl never had this kind of traction afaik - yes many scripts were written on Perl but not many .com companies were based on it.
Were you around for the first (pre-2000) dotcom boom? From where I was sitting (freelancing in London) it sometimes seemed that all of the first wave of dotcom companies were writing their apps in Perl.
In fact, I've often thought that a lot of Perl's current bad reputation stems from this generation's CTOs and dev managers being people who suffered writing those Perl web apps at a time when no-one knew how any of this stuff was supposed to work and careful design went out the window in the rush to get things to market before the bubble burst.
I think Perl lost to Ruby because of Rails. Whilst Perl had a couple of web frameworks - Catalyst, Mojolicious and Dancer - they never gained anywhere near the traction of Rails. I think this was because Perl's early success in the 90s was based on CGI.pm, not frameworks. Catalyst gained a little adoption at a few big companies but it was hobbled by dependence on mod_perl which had a much riskier memory mode for hosting companies than mod_php. Perl was also a bit late shedding its mod_perl legacy by which time Ruby had Rack and Python had WSGI. Despite the brilliant work of Miyagawa it was game over by the time PSGI and Plack appeared.
> I think this was because Perl's early success in the 90s was based on CGI.pm, not frameworks.
Exactly, the second wave of dotcom software was largely written using frameworks that built on the mistakes we'd been making in the first wave.
And because, so many programmers had memories of horrible experiences trying to beat Perl/CGI applications into submission, it was probably too late for Perl even then - although a large number of excellent Perl tools (Catalyst, DBIx::Class, Template Toolkit, Moose, PSGI to name just a few) were emerging at about the same time as Ruby on Rails or Django.
Yeah, A lot of those early web apps were total spaghetti garbage hardcoded to a table layout, so if they survived most of them were rewritten from scratch. Of course perl programmers' tricky tendencies didn't help, but also say ASP and ColdFusion etc have similar reputation issues.
Zoopla. It's been a couple of years I left, but I believe they still use Perl for the core business logic. There are wrappers/docker images built on top of them, but Perl is doing all the heavy lifting AFAIK.
IME, big older companies have Perl code chugging along, but not very visible. Part of this is also because there's the perception that it is hard to attract developers if the company mentions Perl any where in the job description.
Agreed. It's VHS trumping Betamax all over again. Few things irk me more than Pythonistas crowing over their world dominance but if you look at the top 3 languages - Python, JS & Java - it's clear that language design counts for nothing.
I think it's largely because of Ruby's deep association with Rails. This is nothing against Rails, per se, but so many people only think of Ruby in association with it that they just never reach for it in other contexts. Personally, I think Ruby is a great utility scripting language, really good for the sort of small projects that many people would probably have used Perl for two decades ago.
(Having said that, I suspect that the Ruby community's love of "first, install a Ruby version manager" does it no favors when it comes to adoption for these sorts of small projects, although the same is arguably true of Python.)
I'm semi-retired. I have a part-time contract maintaining a system I worked on a few years ago and I pick up other bits of freelance work from time to time.
And I've been prototyping a few project ideas (in Perl) to see if any of them could be a sustainable source of passive income. Of course, if any of them grow to the extent that they need a development team, I'll need to rewrite them in a more sustainable language.
Amazon's language for storefront UI templates is Perl [1]. When I joined, I couldn't help but laugh that poor developers are dragged through the hell of 5 leetcode interviews to end up working on a UI framework that is 25 years old.
To be fair, HTML::Mason was probably the most powerful templating toolkit at the time (Text::Template was there too, but from memory Mason was almost everyone's goto).
Perl seems to still be fairly popular with the OpenBSD folk, since a Perl installation comes with the base system. It seems to be pretty much the scripting language of choice when simple Shell scripts don't do the job or are too slow.
a) iCalendar and timezone pre-processing for an Arduino alarm clock; Arduino libraries aren't up to the task (and neither am I! recurring events with recurring exceptions is kind of complex), but cpan iCal libraries and timezone libraries seem to work.
b) personally monitoring script, checks a bunch of stuff and sends email (well, cron sends the email, really) if problems persist for long enough
c) something to fetch my DSL sync speed and status (uses Net::Telnet), which feeds into the monitor script, but also used to adjust bandwidth limits for fq_codel to reduce bufferbloat.
Not new, but I'm currently working on a freelance project modernising the front end / UI of a bioinformatics platform that's written in Perl. Also I wouldn't say it's in maintenance mode, there are new features being developed for it all the time.
Maintainance can mean: reuse most of the business logic and port the software to another MVC framework, so it's easier to maintain and add new features, as opposed to keeping it running on life support.
This seems like a whole lot of bending over backwards to avoid breaking a handful of edge cases that could be cleaned up on their own without too much drama.
I would kind of prefer the features be enabled by default if you use a high enough version and if a module breaks you just fix it or dial back the version number at the top of your script until it is fixed.
But I'd also run all of the test suites on CPAN against every new feature that comes out and fixing or at least tagging anything that breaks. Maybe even list a "maximum tested version number" for each module and emit a warning if you exceed it.
Cue all the "Perl was my first real programming language but I moved to Python years ago" comments. I wish there were more comments about interesting stuff people are doing NOW in Perl. I'm actually new to Perl and I really enjoy it. Python seems so boring.
It says Larry Wall isn't working on Perl anymore. When did that happen? It looks like the Perl6 thing was announced in 2000 and around 2010 Larry Wall deleted the Perl page from his blog (it's been a 404 link ever since)
Agree completely. Larry Wall is pretty brilliant, but he shouldn't be doing graphics design.
Actually, I tend to dislike cutsey-poo cartoon icons, in general. If you want to pick an animal totem, there are plenty of public domain photos of actual animals courtesy of the NSF and such...
Yeah, this is basically what Free Pascal does too. The default language mode is actually kinda obsolete (it is more of a Turbo Pascal 7+), but you can change dialects with the $mode directive (with most common being "objfpc", an extension to the default "fpc" mode that adds more advanced features and "delphi" which is used for Delphi compatibility to allow sharing code). In addition to that many new features are enabled with additional directives, like $modeswitch (e.g. "$modeswitch advancedrecords" enables using methods and management operators in records and "$modeswitch prefixedattributes" enables attaching custom attributes to classes, properties, etc to be accessed via the RTTI later) and $scopedenums (so that when you have an enum like TFoo = (Bar, Baz) it wont create Bar and Baz globals like in classic Pascal, but TFoo.Bar and TFoo.Baz - the fact that it isn't a modeswitch is most likely for Delphi compatibility as AFAIK Delphi doesn't have a modeswitch directive - or mode directive for that matter).
It does require having a litany of switches at the top of each source code but it beats having old code break (though i do think that in some cases the FPC devs go a bit too overboard - e.g. attributes would be a syntax error anyway).
I never understood this argument. If you have code that only runs on perl 5 then install perl 5. If breaking changes objectively make a language better then break it! Nobody is forcing you to migrate to perl 7 and you can always just maintain binaries for perl 5 for legacy use.
The amount of time spent on this seems absurd and could have been spent getting a perl 7 release candidate ready. It's sad to see OSS eat itself alive so often over a desire to keep something that people just assume is desired.
I’m disappointed in the way this Perl 6 v Perl 7 debate is developing. Perl 6 modernizes Perl with e.g. concurrent and reactive programming, built-in grammars and a reimagined regex syntax superior to legacy PCRE, named function arguments, and gradual typing.
Perl 6 was designed to be the successor language to Perl 5.
The only technical grounds for Perl 6 being metaphorically sidelined was because Perl 6 lacked the startup and runtime performance characteristics of Perl 5 — fixable problems.
Proponents of Perl 5 often contend Perl could’ve escaped developer mindshare loss without Perl 6 in the picture. But to claim Python, Ruby on Rails, Clojure, Go, Rust and JS/TS never would’ve gained serious developer mindshare had _Perl 6_ not existed seems very myopic to me. All the brilliant new languages and web frameworks launching were bound to erode Perl’s early established dominance regardless.
Things would be different now if Perl 6 was at least as performant as Perl 5. Perhaps then it would’ve become popular years ago amongst Perl users to switch all greenfield Perl code to Perl 6. Then “Perl” would’ve become synonymous with modern language features.
I think more than anything it was an error in the messaging.
Perl 6 was treated as the successor of Perl 5 -- and that was the mistake. It meant Perl 5 started dying, since people assumed that Perl 5 would be soon dead, and Perl 6 had a new different syntax. And then it took 15 years to happen, during which Python and others ate its lunch.
I think a more successful strategy would have been to make it clear very early on that Perl 6 would be some sort of long term experimental project, and that Perl 5 would be expected to be a thing for a long time still.
If in 2015 Perl 5 still had a thriving ecosystem, and there was a demand of Perl-like but better, then Perl 6 could have been more successful. But in the current timeline it's a successor to an almost defunct language, and isn't such an attractive proposition.
In 2000, for all intents and purposes, Perl 6 was the successor of Perl 5. And one of the reasons the project was started, was because Perl 5 was already dying at that point. Not only losing the web server competition to PHP, but also from internal battles, close to civil war.
The internal battles ceased for a while when the Perl 6 process was started. But around 2008, it became a war between Perl 5 and Perl 6. To squelch that, the sister language meme was born. But that just meant a cease-fire, rather than peace, and the dissent and resentments continued to fester under the blanket of the sister language meme.
Until 2019, when the rename of Perl 6 to the Raku Programming Language happened. Factions within Perl 5 lost their common enemy, and fighting could start all over again. Which caused at least one pumpking to resign.
To mark Perl 6 as the cause for Perl 5's demise, is incorrect. Perl 6 was one of the effects of Perl 5's demise.
Meanwhile the Raku Programming Language continues to build a programming language of the future. You can check out the Rakudo Weekly News if you want to stay up-to-date on developments https://rakudoweekly.blog
Have you used Raku before? The new and improved regex syntax [1] IMHO completely obsoletes legacy PCRE. Writing regexes in other languages feels like stepping into an ICE vehicle after driving a Tesla: so crufty and old and obvious legacy.
Raku’s built-in grammars make parsers trivial to write. I effortlessly created two [2] — one for reordering fstab entries, and the other for converting human-readable LUKS offsets into cryptsetup sectors — on a lazy afternoon. Grammars in Raku make this second nature.
Then you have Raku’s multi-dispatch. It is more capable than Erlang/Elixir pattern matching:
# a list with at least one element, extracting the head and tail
multi sub tail(*@list ($head, *@tail)) { @tail }
multi sub tail(*@list) { @list }
[3]
==> tail()
==> say(); # []
[3, 4]
==> tail()
==> say(); # [4]
# pattern matching with arbitrary guards
multi sub user($name where { is-valid-user($_) }) { $name.say }
multi sub user($name) { "invalid name: $name".say }
sub is-valid-user($name)
{
# notice the additive character class in this regex: “letters plus digits”
# fail the match if $name is root
try with $name ~~ /(<+:Letter +digit>+)/ { $0 ne 'root' or fail }; $!.not
}
user('name'); # name
user('1234'); # 1234
user('root'); # invalid name: root
class Coordinates
{
has $.latitude is required;
has $.longitude is required;
}
class City
{
has Str:D $.name is required;
has Str:D $.state is required;
has Str:D $.country is required;
has Coordinates:D $.coordinates is required;
}
my $latitude = -37.840935;
my $longitude = 144.946457;
my Coordinates $coordinates .= new(:$latitude, :$longitude);
my Str:D $name = 'Melbourne';
my Str:D $state = 'Victoria';
my Str:D $country = 'Australia';
my City $melbourne .= new(:$name, :$state, :$country, :$coordinates);
my City:D $sydney = do {
my Coordinates:D $coordinates = do {
my $latitude = -33.86514;
my $longitude = 151.209900;
Coordinates.new(:$latitude, :$longitude);
};
my Str:D $name = 'Sydney';
my Str:D $state = 'New South Wales';
my Str:D $country = 'Australia';
City.new(:$name, :$state, :$country, :$coordinates);
};
# deeply nested argument deconstruction
multi sub melbourne-or-bust(
City:D $city (
Str:D :$name where 'Melbourne',
Str:D :$state,
Str:D :$country,
:$coordinates (
:$latitude,
:$longitude
)
)
)
{
my $gist = qq:to/EOF/.trim;
Welcome to the city of $name. It’s located in $state, $country.
GPS coordinates: $latitude, $longitude
EOF
$gist.say;
}
multi sub melbourne-or-bust(City:D $city)
{
'This isn’t Melbourne.'.say;
}
melbourne-or-bust($melbourne); # Welcome to the city of Melbourne ...
melbourne-or-bust($sydney); # This isn’t Melbourne
> Perl 6 was treated as the successor of Perl 5 -- and that was the mistake. It meant Perl 5 started dying,
Perl 6 took a long time to make, but how much did that matter? What was Perl going to do about Rails, Clojure, Go, Rust, JS/TS, and more? The world of programming languages used to be a lot smaller than it is today.
Perl is great for working with text based data and stuff that is a pain to do in bash.
It is also great at stability (ie. code written 20+ years ago, still works on modern systems).
So for me, something that I'll write, put in some cron somewhere and forget, is all written in perl... at the same time, my colleagues are rewriting old python2 stuff, because modern distros come with python3 only. Some of them had to fix stuff even for 2.6->2.7 python versions.
I haven't seen frameworks in other languages rival mojolicious for rapid prototyping, web scraping, and good event-driven support. It's super easy to make websockets and even quickly have unit tests. Mojolicious still has 0 external dependencies and is rock solid and very scalable. The simplicity is just very nice compared to, for instance, frameworks in which lots of code generation is involved just to get started.
My MO is to start off scripts in bash, but once I get to about 3 functions in or need to do some text processing (but no need to drop down to Rust), I rewrite the script in Perl and go from there.
On the flip side, I would never get a Perl script and rewrite it in Bash.
Perl has a fast, powerful well-integrated regular expression engine, with full unicode support.
In general, Perl tends to be much faster than the competing scripting languages (which is why you never see people talk about benchmark numbers, they'd give you the wrong answer).
I don't know about "must use", but now and then I write a short script for some kind of text processing. I've also used it fairly recently for a script pulling data out of SQL Server, mostly because I already had the DBI::Sybase module installed.
Yes! The committee approach does not seem to work well. The “benevolent dictator” is really often just a solid CEO type: sets the vision, acts as head salesperson, and has the final word on all important decisions.
What happened with Perl is that, effectively, the “pumpking” held that power but when Sawyer X actually used his discretion to announce Perl 7 there was a mutiny by jealous and bitter collaborators. They felt they should have some say in the matter and formed this Perl Steering Committee. As the brilliant Reini Urban has pointed out elsewhere in the comments, this group are fairly mediocre talent wise and mostly just spend their time sniping at each other.
Reini’s cperl or Will Braswell’s rperl are both worthy projects that in a more perfect universe would have long ago supplanted the now rudderless mainline perl interpreter.
> Perl 7 there was a mutiny by jealous and bitter collaborators
Look, I hate to harsh on Sawyer X-- he's done a bunch of good stuff for perl over the years, and I hope to see him around again-- but his Perl 7 push was just a mess. He was getting frustrated about things, and tried to plan and push through some big changes before anyone could complain, but he wasn't really that clear on what the big changes were supposed to be-- he left a couple of weeks to figure it out after that big announcement, and even the inner cabal he had talked to about this stuff first seemed more than a little surprised.
One of the major things we got out of all this is it made it clear we needed some work on improved processes and transparency and such, and that's actually happened. There's a steering committee that makes a point of publishing its minutes, and an RFC process to talk over proposed changes. Some of these changes are in fact actually happening, and a number of them are discussed in this v5.36 annoucement (e.g. subroutine signatures are no longer experimental).
This actually seems like a really bright crew in charge, and they're making very sane decisions.
There's really no reason to think that there's some great benefit to breakage-on-upgrade: its a solution in search of a problem.
An anecdote of my career is that some programming languages attract different kinds of programmers. I've dealt with Delphi in the beginning of my career and while there was nothing wrong with the language, it seemed to attract a lot of bad programmers writing really poor code.
And Perl was the opposite of this anecdote: the most brilliant engineers I've met in my career were Perl monks, as they would call themselves. I respect Perl for that :)
Having said that, I think this might be too fine-grained.
First, opting in to an experimental feature could be a one-liner, “use experimental feature ‘try’” or similar. There’s no point in punishing your valuable beta testers beyond that with a second line that’s entirely redundant with the first one.
The larger problem is the versions. This basically requires someone to update their script headers all the time if they want to keep getting new features. Probably not much of a problem currently, but might be if releases get more frequent. I personally like the “edition” concept of Rust a lot better: get over with all necessary breaking changes in one swoop, but then begin a new (hopefully long) era of backwards compatibility. That makes it easier for users to maintain their code, is probably easier to implement, and also easier for users to learn, since there are fewer sets of rules.
Lastly, I hope they also focus on tooling around installation. One paradoxical problem with modernizing Perl is its historic success: every variant of Linux or Unix already comes with an ancient version of it. I know many experienced Unix people hate this trend, but there’s a reason some of the most actively evolving language ecosystems install more and more of their binaries into the user’s home directory. Maybe that’s already the case for Perl, I wouldn’t know.
All that said, I’m not following Perl that closely anymore. These are just some really quick observations about very complex topics. I don’t claim to know nearly as much as the people who made these decisions, and it’s great to see that things are happening.