Looking at the first 10, none of them are asking for Perl specifically, they're generic job listings with a point "N years of experience in Bash, Perl, Python, Java..."
EDIT: since it was silently changed: This was originally titled and submitted as "Are banks still using Perl in 2020? The answer is no." (still visible in the URL slug)
This is hardly a surprise. No matter if you like a language or not, you have to acknowledge that we don't develop software in a vacuum and we cannot ignore market trends. Don't fall in love with your technology [0].
(I won't comment on the Python 2 rewrite though)
I like Perl a lot. Out of the usual, interpreted dynamic languages, it is still my favourite. It's fun, incredibly expressive, got some things right early on (CPAN) and the community is probably the most entertaining out there.
It also caters to a kind of developer that no longer exists, or at least is in the minority now.
I will keep it as a sort of dirty pleasure for myself. Among the likes of Forth and Lisp.
> It also caters to a kind of developer that no longer exists, or at least is in the minority now.
The one developer I met who liked Perl was quite the character. He thought in a certain way I envied. Once he rewrote a bash script into Perl just for fun (and to confuse the rest of us). As a gratuitous side effect, the Perl script cut execution of the same task from over 1m in Bash to a matter of seconds.
I've resolved to learn it myself now, even if it nudges me more towards a mode of thinking. There's still plenty it can do.
RHEL's distribution of Python 2.6 is still supported as part of the OS. As the author noted, the scripts were written to be forwards compatible with Python 3 for when the bank upgrades to a newer version of RHEL.
Seems to be common amongst pythonistas. I've run into their tired bashing of perl for 20+ years.
I use perl daily. And Julia. And C. Python more like weekly.
Always select the right tool for the job. Changing code that works, and is critical should not be undertaken lightly. Most especially not of irrational dislike of the current implementation language.
Not at all. I rather wrote this to illustrate how critical legacy software is created, operated and evolved over time, but it seems that readers can't get past immediately defending/bashing Perl.
Yeah, I'm not convinced. Not in the slightest. JPMorgan has something like 250,000 employees. Just think of how many divisions and departments there are. It might be that someone ran a report at a high level over some major projects and said, "We're not using it." I guarantee it's still there in pockets still being used and still being maintained.
I think you’ll need something more than a feeling, to disagree with the person tasked with finding uses of Perl at JP Morgan on how much Perl is used at JP Morgan
Also it sounds like you didn’t read the piece as this comes right at the start:
> search covering repositories with 50M lines of code and thousands of servers
1. You notice how he said "Investment Bank". Guess what, that's not their entire business....not even in the slightest. Not even the most profitable. When I was there, they sold a significant portion of their investment services to buy more commercial, which is in an entirely different line of business. They aren't connected. Their systems are separate.
2. I worked there. I understand the absolute magnitude of how large their code bases are. The legacy software, the business processes. 1000s of servers, is nothing in a company that size. The division I worked for was a dust spec compared to the size of JPMC, and we had something like 300 servers on our own. We were a rounding error on the division's balance sheet. I should say, a division, of a division, of a division of a Line of Business. Multiply that by orders of magnitude and you begin to understand the size of their operations. Let me put this into perspective. A quick line count of the project I'm on, with 4 people and started 2 months ago (with npm packages granted) has over 500,000 lines of code. How big do you think JPMC's code bases are?
3. Do you honestly believe that a company that size tasked 1 person with just "removing PERL". Most of it was not being used anyway, and he "just did it." He just "searched" the repositories. How exactly do you think that worked? Do you think an EVP, a chief, just tasked 1 person with search "the codebases" across multiple lines of business? He might have done it for a division, a part of a LOB. I'll give him that, but not JPMorgan as a whole not even the the entirety of the investment arm. Think about how many legacy systems use PERL everything from legacy systems, to commercial ETLs. The amount of business knowledge held in those scripts, and making a mistake, even a minuscule one will cost potentially millions of dollars. What did he search? Did he do a Git search, or SVN, or CVS, or event maybe VSS? How many file and network shares? Did he have access to everything? All the devops stuff? What about all the things that are maintained by contractors and consultants? How about those?
Author here. Definitely great questions. I would like to reply but this is touching sensitive information that I don't think I can publicly disclose. ^^
What I can maybe say, is that the scope is touching JP Morgan, the corporate and investment bank with maybe 40 000 employees. It is much broader than you think.
I concur. I'd have to reach out to my former colleagues but it seems possible that someone rooted out Perl in one or more departments or even a division but not for the entirety of JPMC. Ten years ago the word I heard was Python and people had shifted from Perl for new things.
This guy may have gotten Perl out of his part of the JP Morgan world, but JP Morgan still actively uses Perl. It’s not as big as it once was, but there are teams still developing and deploying Perl.
This is what I figured. My experience of SW development at a merchant bank is siloed teams using their own development tools to support their banking operations. There was never a company wide mandate on using/not using some technology.
That's not the case in JP Morgan. Development is very centralized.
There are like 3 to 6 major platforms used inside of JP Morgan, depending how you count (the oldest one is deprecated and being migrated away which will never complete, while the newest one is nowhere near working and will not have anything running for a long time).
The largest one regroups maybe 30% of all the developers and developments in the firm.
The Perl system was underlying on of these platforms, and has clients for use in all other platforms, so it was quite ingrained across the entire company really.
Reminds me of a quote from the Learning Perl book:
"There's a joke among Perl developers that the next economic crash will be caused by a bug in someone's Perl script. Even then, all those redundant economists will have at least one employable skill."
JPMorgan is only one bank; it's still used in plenty of banks. What is the point of this article anyway? Proving 'perl is dead' as it says in the epilogue?
Im a Junior dev and I actually have been learning basic Perl over the last year. I think Perl is still useful in shell scripts and most of the time its faster than awk and sed. But I cant justify using it instead of Python for scripts or applications. I cant think of any advantage to using perl instead of python in 2020. Even if there is an advantage, any coder can read and write python without learning python whereas Perl scripts are a liability, few can maintain them. I would love to use Perl I think its powerful and especially I see use cases for it in data munging. But at the current moment it needs a killer app that can offer some clear justification over python.
Because the servers ran on RHEL 6, so the application had to run on python 2.6.
Servers will be upgraded to RHEL 7, so the application had to run on python 2.7 too in prevision of the upgrade very soon.
Then servers will be upgraded to RHEL 8 maybe 5-10 years later, so the application better work on python 3 too.
It is not glamorous but it's the practical way to make critical software work and continue working for some period of time. I think we can all agree that we don't want our banks to break every 3 months accidentally due to the newer release of whatever.
Not in JP Morgan, apparently. As mentioned in the post, the code was barely touched since 2006. The rewrite was needed presumably so that the code could be better maintained (or hot patched, if the emergency arrived). And there was, presumably, more Python devs at the firm than Perl devs.
I call bullshit. Every macbook on this bank (and elsewhere) has several perl programs running daily and supporting the jobs of the people who own them.
JP Morgan doesn't support macbook. I've only ever seen one employee in my time there with a macbook, who's had to go out of his way to procure it and he had regular issues because it was unsupported, issues like DNS not working.
I'm not surprised that Perl is being phased out, but was JPMC really that big on Perl? IIRC, I worked at JPMC in NYC over 10 years ago and IIRC they never had the kind of centralized Perl infrastructure like Morgan Stanley did -- though after the BankOne acquisition, things started getting streamlined a bit.
Nothing against Python, but I'd have expected the choice of a little more (statically) typed language for such a critical application (also considering future maintenance). Any reasoning behind why?
> Are banks still using Perl in 2020? The correct answer is no.
> There is no more Perl at JP Morgan, the largest investment bank in the US. The last Perl code was decommissioned in 2019.
I see the situation at JP Morgan now extrapolates to all banks globally.
I am glad that JP have done something sensible here, though.
> Nope, not python 3. Think 2018-2019 so no python 3 in sight.
What. That was literally last year. We all knew Python 2.7 was going to be EOL on January 1st 2020.
I know a lot of these legacy banks/financial institutions seem to operate in a time-shifted bubble a good few decades in the past, so maybe that explains it?
> I am glad that JP have done something sensible here,
I doubt that.
For over 20 years people have been pinning their hopes on new languages and "paradigms" to make things better. Things haven't progressed a lot with that strategy.
I blame this on how software architects don't, and programmers can't, and because of that they all hedge their bets on new language X with feature Y, hoping it will make all the hard bits go away.
It's akin to how in the creative writing world, writers sometimes dive deep into the grammar and vocabulary of a particular natural language whereas the real challenge is thinking up new and entertaining situations and characters, and then filling that in with words in whatever language you happen to be proficient in.
Literate programming has been a concept for a long time now: write a document that explains, understandably, what the software is supposed to do, then accompany it with some source in whatever language the programmer is most comfortable with.
In a perfect world, that would be called "doing you job". In our current world, there is no money to be made in "doing your job" and it will be heckled as slow or outdated; praying for deliverance is much more profitable.
The too long, didn't read: Turing and Church proved that, fundamentally, all languages suck, so no new language will ever save one from one's own incompetence. Maybe it's time we all stopped blaming or praising "the language" wholesale for the various inadequacies and blind spots in our branch of industry.
Yeah, JP Morgan doesn't use Perl, so nobody uses Perl? I would not take that bet.
None of the banks I worked for uses Perl to my knowledge, but that doesn't mean they don't use Perl at all, and certainly not that no other bank uses Perl.
But is there a problem if banks use Perl specifically? I'm no Perl fan, and I certainly wouldn't use Perl (or Python or NodeJS for that matter) to run systems handling money, but there's a lot more than just direct money handling going on at banks.
> We all knew Python 2.7 was going to be EOL on January 1st 2020.
Note that this just means that the Python people won't support Python 2.7 going forward. I think you will be able to get Python 2.7 support if you pay for it for perhaps literally the next hundred years.
It's not even python2.7, they went with 2.6. A version that reached end of official support in 2013. All because that's the default for RHEL 6.
It's not hard to build RPMs of newer and/or supported versions of Python. You might even find it in EPEL. I get being risk averse, but if you're going to make the change, move to something sensible that your developers will want to work with.
>>> It's not hard to build RPMs of newer and/or supported versions of Python.
It's actually the hardest thing to do and to maintain long term. I'd say from experience that building your own is the #1 thing that was plaguing legacy systems in the bank and preventing to maintain them.
Developers built perl/python/whatever manually and distributed (RPM or whatever). It's de facto unmaintainable because it cannot be built again. Sure thing is, there were special build flags set and some extra libraries/plugins embedded into the build, but who knows? There is never any instruction left. Even if there was, these are complex software with huge web of dependencies, it's probably impossible to get everything to run/build 5-10 years later. (DLL hell is a major recurrent issue on Linux when upgrading the distro and some old libraries are gone due to googlecode/sourceforge dying).
Hence the solution to ensure that simple critical scripts work and can be maintained in the far future when required, is to rely on the /usr/bin/python from RHEL (and store everything in source control). Don't build your own interpreter.
I know a lot of these legacy banks/financial institutions seem to operate in a time-shifted bubble a good few decades in the past, so maybe that explains it?
Decades maybe not necessarily, but years definitely. It boils down to the very real risk of being sued for damages if something's wrong with a payment.
I'm currently in a project which aims to re-write the front-end of a library of banking components - from AngularJS to Angular 9. The product is fairly simple to use, but still it's being delivered a year before AngularJS's end-of-life which will be after the 30th June 2021, because the time window for ensuring this thing works according to the banks' use cases must be as wide as possible.
In general i think it is fine that banks and similar places are years behind. It is not exaclty the kind of institutions i want to mess around with bleeding edge, barely tested and undocumented technogy.(not referring to Angular here)
The problem that I have seen in the enterprise/goverment world when I was in it, is them not keeping up from behind. E.g. They might be reasoanbly recent for a new project, but then dont update for a decade or so.
There is this recent introductory talk about enterprise frontend stacks on why it makes sense for them to stay quite far behind (compared to the pace in frontend dev) the latest developments that I found pretty interesting.
> I'm currently in a project which aims to re-write the front-end of a library of banking components - from AngularJS to Angular 9. The product is fairly simple to use, but still it's being delivered a year before AngularJS's end-of-life which will be after the 30th June 2021, because the time window for ensuring this thing works according to the banks' use cases must be as wide as possible.
Is this in the states? I get the impression the regulatory landscape is much more hostile to innovation and tends to favour incumbents (vs the UK, recently, where challengers have been supported a bit more).
I know right. Banks are so risk adverse now, the biggest thing people are worried about is breaking things and downtime. Running on 10 year old platforms isn't a problem as long as there is no outage. No wonder they're getting killed by Citadel.
> I know right. Banks are so risk adverse now, the biggest thing people are worried about is breaking things and downtime.
Yeah, it's funny though that the ones who've stuck to their aging mainframe systems and haven't invested in maintenance and reducing technical debt have the worst uptime.
No bank is getting killed by Citadel. The numbers are on a different scale completely. JP Morgan makes over $40B a year. The entire proprietary HFT industry globally makes a fraction of that. Citadel almost certainly less than 5% of that number in a typical year.
I dont believe this. Maybe it's no longer used for "core banking services" but companies the size of JPMorgan have subdivisions that are as big as small/medium sized companies.
There must be a lot of software written in <whatever language> that is still used inside the company.
I never worked at JPMorgan but a lot of the investment banks went towards a single platform for everything. JPMorgan had Athena, Bank of America had Quartz, and they both use Python. Goldman Sachs had something similar whose name I forget (slang?)
At least in 2011 when I last worked in the industry, this sort of infrastructure was replacing the ad-hoc data-munging scripts that people used. It was even slowly eroding some of the Excel-based workflows designed by less software-engineery teams. 10 years later, I'm guessing it's only doing more than it was back then.
I work at JPMC. Athena is a huge platform, but not everything is on it (for example the team I work on isn't). I very much doubt there aren't small perl scripts and tools still hanging around somewhere. But probably no large customer facing is running on perl anymore.
That's true. Several companies I know are pursuing such endeavors. It's often even battle between Java & .NET as they are trying to focus on a single stack.
The comment you reply to happens to hit the nail on the head. Some perl was slipped deep into some platform in the 2000s.
While client software could be python, java or C# (usage has evolved quite a bit over the decades), they ultimately called and depended upon a bunch of Perl on some servers.
For some reason people like to compare perl to cobol. Perl fell out of usage more like lisp or tcl, mostly a failure to deliver on some of its promises which proved to be market advantages for others. The influence of latin is obvious.
Let's hope the windows 7 phase is over. I personally think perl 5 was a mistake, perl 4 was more fun and minimalistic ... just like javascript 5 vs typescript.
Including many that seem pretty clear Perl is involved in a substantial way: https://jpmc.fa.oraclecloud.com/hcmUI/CandidateExperience/en...