Hacker News new | past | comments | ask | show | jobs | submit login
Citigroup plans to hire 4k tech staff (businesstimes.com.sg)
135 points by aruanavekar on June 7, 2022 | hide | past | favorite | 161 comments



Had an offer from Citigroup a few months ago, hope this is useful for anyone approached, this is from my interview/offer experience and from talking to folks at Citi privately:

- They claimed remote was impossible, after I told them it was a red line there was suddenly an exception. They did claim this exception required a lot of approvals but take from that what you will.

- The projects they wanted me to work on were genuinely ambitious and impactful, not just something thankless.

- They claimed to want to start open sourcing large parts of their tech from 2023 onwards.

- It's definitely a different world from the rest of tech, people seemed to care about the things they worked on, but the way things were reasoned about just seemed a bit different.

- The people I met there are genuinely nice people (at least in Tech) who I'd be happy to sit down and have a coffee with anytime.

- Even the very specifically designated tech places had debt coming in from the rest of the company (primarily in process when they had to touch other teams), they did say they had C-Suite buy-in to go solve that but as I didn't join I can't comment on how strong that buy in really is.

- Politics is everywhere, people will fight you not on technical merit but because they want to claim responsibility for something even if its inferior to the product you could build - if you don't have the support to push through this I suspect you'd get stuck here.

- Corporate structure is confusing because everyone's titles are massively inflated.

- Claimed to have internal open source in the Division I interviewed for (didn't expect that).

- Offer for 2 YoE UK was VP £190k/year with no stock


I did a job interview at Citibank a number of years ago, for an IT security position, and the thing I remember most was walking through the office and seeing a well used copy of Computer Security For Dummies on every desk. That reminded me of a past position at Chase where our group’s head of IT security was a former bricklayer, with no technical background, who was never without his copy of Computer Security for Dummies, and was forever trying to find out where we were “hiding our NFS”. We always tried to tell him that we didn’t use NFS and he would always counter that he didn’t buy it, and he knew we had one and that we were hiding it from him. Ok.

Chase and Citigroup are like the McDonalds of financial services, everyone works there once, the talented people leave, the ones who stay generally lack the confidence or ability to do likewise, and they are a PITA to work for because they live in constant fear of being out shown. I really wouldn’t recommend working at any of the big banks, they are thankless soul crushing jobs, and you’ll forever be working on what everyone else was doing ten years ago.


>Chase and Citigroup are like the McDonalds of financial services, everyone works there once, the talented people leave, the ones who stay generally lack the confidence or ability to do likewise, and they are a PITA to work for

Thanks, this explains the experience I had interviewing with Chase.

I have never ghosted an entire company before, let alone walked into a bank branch to tell them to get their computer doctors to learn to be respectful in a job interview.

On my end, I'd hoped it might be a cultural fit since I think many big tech companies have insider threats because they don't allow people to truly take time off -- it's common in the finance world to force someone to take a vacation at least once a year for fraud prevention.

(The idea being your scam requires your constant presence, it will fall apart.)


I will say, having worked at Capital one in the past, that it is the outlier here. They have a really strong tech focused culture and from my experience, rarely outsource their tech teams.

To drive this point home, they're creating a whole new business based on the expertise they developed with Snowflake.

https://www.forbes.com/sites/martingiles/2022/06/01/capitsl-...


> you’ll forever be working on what everyone else was doing ten years ago.

This. I've spend over 20 years working in banks. DB, BAML, UBS, Barclays, BNPP to name a few.

They all suck for good tech.

In the 90s and 00s it was a cool place to go. Cash was flowing, algo trading was the new hotness, everyone was making good money and building cool shit.

Post 2008 crisis that all went to hell. No more cool shit, only regulatory knee jerk bolt-ons, turd polishing, and budget cuts.

It's never really got any better since then.


> They claimed to want to start open sourcing large parts of their tech from 2023 onwards.

My background is enterprise, finance. I talk to enterprises frequently. Question this as hard as you can. They will never act like Google, Facebook, Netflix. In short: don't work for the OS side of enterprises. There is none.

"large parts" - which? In corporate "large parts" is different from tech people.

"from 2023..." - well, this tale is a running gag at least at my company. Telling this since 2017: "Next year!" There are a lot of processes involved, finance fears of losing intellectual property, decision makers are usually business people who know next to nothing about tech people thinking and reasoning.

Banks fear of code injections, have to deal with regulators, don't want to deal with an OS community, because costs - that's my experience. Ask them about these topics and you can sense how realistic these claims are.


I'm in enterprise, insurance industry. Our witticism for future functionality is "Any decade now!"

Enterprises are slow, they have high inertia for many reasons and tech can only affect that so much.


I don’t know if it is the case with Citi, but it may not be that titles are inflated, but that titles are different. I’ve spent most of my career on Wall Street (i-banking and investing) and VP is a mid level position, and it comes before director. Whereas my mom works for Google and VPs there are quite senior.


If having thousands of vice-presidents is not title inflation, what would qualify as such?


There are historical reasons for the proliferation of the title- every city and town used to have their own bank or several. These were independent businesses, each one having a President and maybe a couple of Vice Presidents. When mergers and acquisitions happened and all these single branch banks got rolled up to mega corps they kept the nomenclature because that’s what people were used to- a manager at a bank branch was a Vice President because that’s what it always meant, but the title then got used for anyone at the same level.

I’ve also heard that at least at some point the title “vice president” carried some form of regulatory meaning and was the minimum level required for sign off on some pretty basic things, but I’m having trouble confirming that and it may be wrong.


> I’ve also heard that at least at some point the title “vice president” carried some form of regulatory meaning and was the minimum level required for sign off on some pretty basic things, but I’m having trouble confirming that and it may be wrong.

I have heard the same, although phrased as ‘the lowest position where someone is allowed to speak for the company’.


That's the cause of the title inflation

But it doesn't mean it's not title inflation just because there's a reason for it


It's not inflation because the same proportion of staff have that title as they used to (is the claim).


The reason banks have so many Vice Presidents is that to bind a company to a contract, you need to be a corporate officer. A Vice President is a corporate officer but a director isn't. If you want to issue loans, etc, you practically need this authority. If a relatively low level employee in one part of the business is a VP, that creates pressure to make lower level people in other parts of the business VPs also. Some of this may be historical, but this as I understand it is why almost everyone at a bank is a Vice President.


> A Vice President is a corporate officer but a director isn't.

Director is typically above Vice President in financial firms. I’m not sure I follow your reasoning.


Officer = legal responsibility

Director = high level manager (of other managers)


You think that the vice-presidents are officers and the directors above them are not? Do the VPs lose their legal responsibilities and powers when they are promoted?


Directors are also officers, just more senior than VPs.


No, Directors are also corporate officers (and more senior than VPs).


I viewed this more as a quirk of American IB titles rather title inflation. The path has always been Analyst > Associate > VP, and you can make VP in under 5 years.


> what would qualify as such?

“Senior Engineer” with 4 years experience.


Just that the convention is different. It is just the third rung on the ladder in the finance world. You might manage a small team of junior staff.


You don’t need to manage anyone to be VP. If you manage people you deserve a director title! More than 25% of Goldman Sachs employees are VP, for example. And of course the idea of title inflation can be rendered meaningless in any situation saying “it’s just a different convention”.


It depends on the bank. In some places VPs can manage a lot of people without being promoted to Director.


You’re right. Let me change that to “if you manage a group of VPs you deserve a director title”!


For most east cost financial companies VP is usually a mid level position. Many companies talk about VP roles as “senior” and “leadership” positions but then have poor role clarity and organization structures that make everyone in the organization behave 1-3 levels lower than their title suggests. Decisioning at these companies is usually reserved for a higher level of VP, SVP, EVP, etc.

Big west coast tech companies do seem to reserve VP for a select few. As they start to mature and their equity driven compensation gives way I’d be surprised if they don’t start to inflate titles as well.


This is right. It’s just different, not inflated.


ISTM the definition of title inflation should be something like “titles frequently connote a significantly higher level of responsibility than the title holder actually has”.

If the title communicates incorrect information it isn’t merely a different convention; it actively causes confusion.


That is a great salary for the UK - did you apply for a UK role, or did you apply for a US role working remotely, and thus got a US-level salary? I'm at the executive director level at another bank and get paid only slightly more than that in the UK - looks like I need to start interviewing.


What's crazy is just a few years ago that TC was only available for IBD. Even if the trajectory is only down after the 1st year bonus, you can hop off after the paycheck. Now that money can only be found in tech.

200k is roughly ICT4 at Apple Ldn which is like 10 years of experience. GP definitely an outlier. Maybe their profile fits a narrative that the bank is pushing.


TC, IBD, ICT4, Ldn, GP… I simply can’t parse this comment.


Tc = total compensation (salary, bonus, stock, benefits)

Ibd = investment banking division

Ict4 = more commonly IC, individual contributor level 4. Not a manager.

Ldn = London I think

Gp = grandparent poster (comment 2 levels above the referenced comment)


I'm fairly sure that's total comp - so with a 1st year bonus guarantee. After that first comp cycle, we all know which way the bonus will go...And then you're fighting the rest of the team for that 2-5% merit budget.

If that's just salary - I have no idea what Citi are doing, it's way out of market.


That’s a very good VP level UK salary, at least compared to JPM.

However, I wouldn’t go back to an IB if I could avoid it. Fundamentally tech is just a cost centre and they have very top-down corporate cultures. Everything is intensely political.


I’d posit that most VPs and some EDs at JPM aren’t on that salary.

Looking at the data, that salary is a massive outlier, and I know SVP salaries at Citi aren’t at 190 for technologists.


It's total comp. So salary plus bonus. It's high but I've seen higher at JPM in terms of total comp at VP level. If we're talking pure salary then yes £190k is base for a seasoned ED.


I guess I'm severely underpaid then....


I know that JPM are trying to compete with the likes of Citadel, so perhaps IBs are increasing their offers for new tech people.


  Politics is everywhere, people will fight you not on technical merit
  but because they want to claim responsibility for something even
  if its inferior to the product you could build - if you don't
  have the support to push through this I suspect you'd get stuck here.
In my meager experience this is not unusual in large(r) organizations. At one point at megacorp there was a big push to get me to buy into some atrocious docker orchestration thing as I was part of a high velocity team. There were two teams working on this internal monstrosity and each manager was quite happy to promise entirely contradictory things to promote their own status within the company. I've never been so fast to turn someone's name into a verb.

Another shining moment was when the security/compliance department cobbled together a Hashicorp Vault proof-of-concept and lobbed it over the fence at one of the ops teams. The ops team didn't have the bandwidth to deal with it and the security team refused to maintain it. It was ridiculous enough that veeps and directors got involved.

  Corporate structure is confusing because everyone's titles are massively inflated.
This has been my experience at most tech companies I've been at.

  Claimed to have internal open source in the Division I interviewed for (didn't expect that).
Careful what you ask for. ;)


goes doubly for banking sector. I can't tell you how many 'vice presidents' are in my rolodex, it's pretty much everybody that isn't a teller.


> I can't tell you how many 'vice presidents' are in my rolodex

Yes I have seen this. This person was boasting about being a "Vice President" and then I see almost everyone is a VP. What is up with this?


Suppose you are the CEO of a bottle cap company in the Midwest. Whose call you going to take, a VP at Goldman or a junior banker at Goldman?


Vice President at financial companies is just a mid-senior title. Similar to L4/L5 at Google. There is no expectation of managing people.


My resume and LinkedIn original had "vice president" as my title (as a bank SWE). It seemed to be detrimental to finding a job at a tech company, so I've long since replaced it with just "senior software engineer" to much better results.

One FAANG hiring manager outright told me he found it very intimidating.


Sure. The first time around at megacorp I left shortly before a reorg that demoted my manager to IC status. He was still a manager when I returned but a manager with no direct reports.


One bank in Australia (NAB) is also doing this inner source thing.

https://www.itnews.com.au/news/nab-to-innersource-some-of-it....


> cobbled together a Hashicorp Vault proof-of-concept and lobbed it over the fence at one of the ops teams.

I have seen this twice now. Vault is trivial to get going and get a POC up, but it is a lot more work (ten times?), getting up properly deployed with working HA, a good structure and good processes in place so it does not become a steaming pile of spaghetti. I probably wrecked my career at one firm spending all the time doing it properly, but it was probably time to leave when they started a project to replace it with Password Manager Pro. :)


This was I think more an issue of integrating it with other bits of infrastructure. Things like access control provisioning namespaces. The folks who were on the receiving end of the security team's bullshit were quite capable of getting vault setup but bogged down by mediocre management.

I think we ended up using KMS instead (much to the chagrin of everyone).


(No offense) We're paying in-experienced new to industry engineers way too much money. Thanks for sharing.


If you ask me, no employee has ever been paid too much money for anything ever. It's a job, making money is the whole point, the more the better. No such thing as too much for any level of seniority/responsibilities/skill.


If you pay people more than they produce (across the board), the company fails. That qualifies as too much to me.

If you pay people far above their value add, most would consider that unfair, and vice versa for being underpaid.

I don't think we should consider people being underpaid or overpaid as being just in a moral sense. Obviously everybody would prefer to be overpaid for themselves.

Of course high comp in tech is more a function of supply/demand of labor, and the industry still being new. It won't be this way in 20-30 years, as people continue to trickle in and correct the supply imbalance


> Of course high comp in tech is more a function of supply/demand of labor, and the industry still being new. It won't be this way in 20-30 years, as people continue to trickle in and correct the supply imbalance

Except demand has continued to expand, and I believe it will keep expanding. Salaries are high because of what technology can do for a company. Your type of argument has been made since I started writing software in the 90s. All that's happened since then is software has relentlessly eaten every other industry.

The classic widget selling company had material input costs, labor costs, delivery, etc... for many companies now the marginal cost of adding the next customer is almost zero with the primary input cost being labor.

Salaries can certainly be a bit high right now while also staying on a long term trend where tech salaries outpace other careers.


It's inevitable. Just because it takes a long time doesn't mean it won't happen.

Higher level abstractions = less labor required (e.g. serverless), plus additional supply of labor = lower costs.

Wages are always defined by the equilibrium point between supply and demand of labor, never by the value add of the job.

Simple thought experiment, say you have a button that makes you $1m dollars every time it's pressed and it can be pressed only 100 times per day.

How much do you have to pay somebody to convince them to press it each day?

Now say you have the same button but only 1 person in the world can activate it. How much do you have to pay them to convince them to do it?

The answer to these demonstrates the principle in quite an obvious way.


> Higher level abstractions = less labor required (e.g. serverless), plus additional supply of labor = lower costs.

I've been hearing this about new technologies also since the 90s - going to make programmers obsolete/need less programmers. What ends up happening is what's required expands. I'm sure there's some quippy law here.

> Wages are always defined by the equilibrium point between supply and demand of labor, never by the value add of the job.

Your analogy is only focused supply while keeping a fixed demand. In fact, 'value' drives demand. The reason why demand keeps outpacing supply is the value of technology is enormous.

Will the demand side eventually slow? On a long enough timescale I have to say yes, but a lot of people have gone broke betting against the pace of technology.


> I've been hearing this about new technologies also since the 90s - going to make programmers obsolete/need less programmers. What ends up happening is what's required expands. I'm sure there's some quippy law here.

We used to have manual QA now most of the testing is done via automation, there used to sys admins handling all the servers now its all cloud which can be handled by programmers/dev ops people.

Maybe in the future we will have code as a service where most of the code is written automatically while few programmers oversee it.


>” What ends up happening is what's required expands.”

I’ve found that abstraction doesn’t scale with Business Logic. Abstraction is good, absolutely, but the sheer number of strange edge cases, special processing rules, and “the client refuses to change their process whatsoever” only seems to increase.

I sense this is why programs keep growing in scope despite all the wonderful Dev-tools, abstractions, and platforms we now have.


Depends on the business. Bespoke logic maybe, but rules based logic such as tax considerations can be abstracted away by APIs etc. Look at the difference in LOE of starting an ecommerce business in 1999 vs 2022 for example.

Also when you consider something like Docusign, you can have a pretty much feature complete product that requires little labor to maintain. You can always expand the scope of any product, but we started from 0 with web based/app software just a decade or two ago. Much of the common and widely useful programs will be largely perfected and require less active development as the years go by.

Same thing with github. You can continue to add features, but the core value of the product is already written and in a serverless world, with appropriate automation, can be maintained by a small team.

I really don't understand how somebody can think labor needs in software will never contract... the main point of debate should be on what a realistic timeline is.


i think the opposite might happen. coding would become like reading and writing with every desk job involving code.

high level positions would require the same training and experience they do now with full stack added to them.

like conveyor belt work becomes checking widgets for defects managers busy themselves with maintaining code. The code will grow endlessly creating more work than doing it by hand.

the company db has a crazy amount of data that you will need to be able to query to be able to develop new insights. if it turns out to be useful you wrap it into the software or a separate app

eventually you want the video data in the db with all the machine learning trimmings.

there is no end to this but we wil fail to train people to the desired level


Company performance is so separated from employee input, and value add is so difficult to measure for most positions that I don't trust this model as a useful guide. It's like an oversimplified model from an outdated econ textbook that has nothing to do with how things work in the real world.


Company performance is exactly defined by employee input from the start. Later on people can coast on input from past employees, sure.

But the company in the end is a result of the aggregation of all current and past employee input


Citibank is well into the "Later on" part then. They have 200k employees and are in a heavily regulated industry, it's not some startup where you can make a meaningful difference with a cool MVP that you developed over the weekend.


Being overpaid and underqualified in a job you hate is an intensely sucky place to be in. It's not just about vague morals.


I'm not sure how much can be gleaned about my competency and experience from just YoE (I jokingly refer to it as "bum in seat time" sometimes)

In any case though, that comp seems pretty typical nowadays, folks I know at Apple are on 250k and that's considered normal.


> folks I know at Apple are on 250k and that's considered normal

As in UK based Apple employees?

5 years back you would only get to that level as a massively prolific contractor. Maybe I should move back to the UK! :)


Yeah, high Senior / low Staff at Apple nets you 250k based on folks I know


Sure 250k is normal, but at a senior level. 2 YoE would never get someone 190k salary (in the UK).That's just nuts.

If you told me 190k for a senior role I'd totally buy that.


OP didn't specify whether that includes bonus or not. Regardless, for 2 YoE, this has to be a huge outlier. I'm inclined to think that this more an exception to the rule - I have a bunch of friends working in London finance tech, none of them making even close to that figure.


> folks I know at Apple are on 250k

The pressure on them is quite high. I'm not sure I'd like to work for Apple.

If I got offered a job, I'd add smooth scrolling and SGR 53 support to Terminal.app and quit immediately after it's released.

Maybe fix text colors beyond what xterm256 offers.


Those 3-5 years at Apple could be lucrative for you. Go for it!


No pressure is non-negotiable. I work fine under pressure, but I hate it and will burnout.


2 YoE in what, specifically? I have _20_ years across a whole bunch of software and hardware areas, and would love to know how to get that kind of number.

I see Citi do not post salaries on their job adverts, so we can't comparison shop.


Mostly Go / Kubernetes and some Architecture stuff as well. Main focus has been Security, then Infrastructure.

If you want to check what I've written for my experience my LinkedIn has it: https://www.linkedin.com/in/intunderflow


Interesting. How should I parse "Assistant VP (Internship)" - that sounds like you were an assistant VP and an intern at the same time? Or was that "with responsibility for interns"?


Title didn't mean much at all. Officially I was an Assistant VP on a fixed term contract instead of an Intern, but this was over the traditional internship months and between University years.

Main differences were:

- There was no interview, I spoke to a Director for 15 minutes who happened to be visiting a professor I knew at University (sheer luck/fluke).

- Building a team of full time employees as an intern is pretty unusual. That team went on after my internship.

- Really open-ended (I was just dropped into the Data Science team and told "do something useful").


I think OP meant 2 years experience at that senior level based on other comments.


2 years total at time of offer, I'm just coming up on 3 now (a few months away)


> Offer for 2 YoE UK was VP £190k/year with no stock

This sounds higher than Google UK?


Depends wildly on the team / level surely, much more than YoE? I'd have guessed Google/Deepmind can pay anywhere from 80k up through 400k based on the role.


Are you talking about US dollars? The salaries in the UK are much lower.


No that’s definitely consistent with what Google pay in GBP in London.


Google UK pays an L5 which is Senior Software Engineer around 200K GBP including stocks [1]

[1] https://www.levels.fyi/company/Google/salaries/Software-Engi...


I don’t get how that is inconsistent with the root comment? If the median senior earns 200k, then it’s reasonable that someone junior would earn 80k and a top engineer at DeepMind would earn more.


Wow that is honestly much higher than I was expecting. I was thinking back to Dan Abramov earning half of that at Facebook.


"They claimed to want to start open sourcing large parts of their tech from 2023 onwards."

I work for a different financial company that said something similar about contributing to open source. That was 10 years ago and still nothing. So take this skeptically.

"Politics is everywhere, people will fight you not on technical merit but because they want to claim responsibility for something even if its inferior to the product you could build - if you don't have the support to push through this I suspect you'd get stuck here.

Corporate structure is confusing because everyone's titles are massively inflated."

This is also true at my company.


I had a job offer like this with grand visions and promises of greenfield freedom, with the goal to compete with someone big in the payments sector who was and has since eaten their lunch, by WorldPay years ago.

No product ever came of it and others left early from good salaries due to the endless political infighting that handcuffed the entire thing.

Same thing happened again more recently with a company that has to remain nameless

I'm not saying every opportunity in institutional fintech is like this but initial appearance and promises are easily wiped away once middle management get a smell of it, far more so than in other industries.


Is that your offer (with 2 YoE) or someone else? Is that usual do you think, or do you have some golden stuff on the cv?


This is my offer.

I suspect golden stuff on CV / strong interview round. (I'm now a Staff Engineer on 2.5 YoE)


More power to you, but that sounds like a pretty inflated title to (not staff) me.


Probably is honestly.


I imagine it was HR BS. They had to call you that to pay you X. You are worth X. But they needed to match the title to it.

From what I have read Staff Engineer is mostly architecture/meetings/people management, and being across a very large team. If you are doing that it might be Staff Engineer, but if you are mostly coding it probably isn't.

Either way it doesn't matter. It is just another title.


I think this different (as said by other poster) or inflated title is on purpose. i.e. to get customers trust.

Imagine Citi’s customer talk to their VP!


This is absolutely a technique they use at branches and for customer support calls. The majority of people respond very positively when they think their problem is being handled by a Senior Vice President.


> Offer for 2 YoE UK was VP £190k/year with no stock

I'm assuming you have a PhD and this was a quant role?


BSc and no


Banks are where tech careers go to die. Expect to be caught in a myriad of "initiatives" that are all fluff. Remember, if you're not making the bank money, you're costing them money.


I live in NYC so many of my friends and colleagues have had tech jobs in the well known banks. Here are some of the data points I've collected.

Person 1 says there is a lot of politics, ass-covering, and throwing under the bus

Persons 2 and 3 says back in the 80s and 90s there were a lot of exciting projects, but now it's all maintenance work and making sure money train does not stop.

Person 4 says at his company they tracked how much money your bugs cost the firm, and at the end of the year if that number is too high, you're fired.

Person 5 corroborates what Person 4 said, adding that no one is allowed to touch production -- you touch production and you might cause an outage in some part of the company you never heard of, next thing that happens is Security shows up at your desk with cardboard boxes telling you to pack your stuff.

Person 6 says for the same reasons no one is allowed to touch the base classes you inherit from -- people just make their own copy of the base class and make their changes to it. The code base is littered with many copies of the same file each different in its own way.

Person 7 flat out told me "you are too nice, you will get eaten alive at a financial services firm"

Person 8 says he was not allowed to talk to or collaborate with a colleague because they worked for competing managers, they had to leave the office to collaborate


> Person 4 says at his company they tracked how much money your bugs cost the firm, and at the end of the year if that number is too high, you're fired.

As someone who almost never ships a bug but is a bit slower because of that, I would love it ;)


Anecdotally, 80-90% of my bugs are business logic “errors” that came about because either 1) the client didn’t tell us about the special edge case that is now happening, or 2) the person who wrote the requirements didn’t capture the business logic quite right (usually the reason is because of #1, but not always).

Very few of my tickets are directly from programming mistakes, but I do own up to them when they happen.


Given

> Person 1 says there is a lot of politics, ass-covering, and throwing under the bus

It sounds like you have to be good at politicking or it doesn't matter if you write bugs or not :(


I did a relatively quick stint at Citibank for about a month. It was the lowest point in my entire career, I'll never do it again.


dilbert-esque anecdote please?


Want one? I'm not OP, but I hope this is enjoyable.

Ok... I was hired as a DevOps expert by a bank. They had this "user friendly" click-click platform nobody was actually using for automation. I mean, they were trying, but there's a reason modern stuff is Configuration as Code.

I wanted to use Jenkins and a bunch of Python scripts.

The local "tech expert" shot that down, told me that Jenkins was for "builds". I was like, ok, in that case let's use something else to trigger the launch of the scripts. While that was in the pipeline, I started working on the Python scripts.

I worked a bit on them, got them working form my workstation. Then during the "review", I was asked which version they were. I said, Python 3. Nope, the servers only have Python 2. Ok, I'll make them backwards compatible. Finished that, then I was told that we can't run unapproved Python scripts (!!!).

At that point I figured, you know what, what if I write just some local automation to cut down on my tedium, and maybe I share the scripts with the team later on.

I asked what I could use for sure, and they told me: whatever I had pre-installed on the workstation.

So I wrote a bunch Powershell scripts that controlled the IO of plink.exe (SSH client bundled with PuTTY) to send shell commands to servers. I basically reinvented a dumb version of Python fabric...

And I programmed that using Vim bundled with Git Bash, because that was the only decent "simple" editor that I had access to (actual devs were only using company sanctioned Java IDEs with limited plugins allowed, i.e. Eclipse).

My scripts ran well, but when it came around to sharing them with the team, I was basically told that it would be more prudent to keep my head down and let people do what they'd always been doing (i.e. deploying manually...).

I left shortly after that.


To be fair you could have gathered the requirements and proposed your solution, wait for the approvals and then start the coding work instead of working on the solution.


I did try all of that, was basically told no, use the official solution, and meanwhile deploy hundreds of applications manually on Linux servers, using shell commands.

And for the official solution, I kid you not, they had me fill in a huge Excel spreadsheet, print it, have my manager sign it, then attach the PDF scan to another Excel I was supposed to email.

And that was for a review by the architects team. One step out of tens of super similar steps (including the print and scan part).

I presented my project as a small, "skunkworks project", to try to get approval on the side of the main solution, to have some automation so I wouldn't go crazy.

If they were better organized and frankly less political, I would have tried to help them more. But at every step the impression was that they didn't want me there, I was a troublemaker...


Can you elaborate?


Story please!


I know quite a few people making insane amounts of money working at banks. Sure it’s not glamorous, but their careers are doing just fine. I would agree it’s not a great place for fresh grads and extremely junior developers. Way too easy to pick up the bad habits and negative aspects of the industry.


I’m increasingly convinced banking IT is one of the trades of software. There are glitzier gigs. But you’ll need connections or student debt to land them, and their net ROI won’t be as high ten years out.

On the other hand, your HN peers won’t look down on your cereal-bowl ad-targeting (granted, fabulously compensated) gig at Google.


>I’m increasingly convinced banking IT is one of the trades of software.

Banks an Insurance have just really bad marketing, but fraud-detection (aka ML.."AI"), analytics..it's all really interesting, and if you have a good manager who shields you from the internal-wannabe-politicians you can produce software that you can be really proud of (enough time to test/plan etc).

And hell, Mainframes are such a interesting field no one talks about (but one could learn some really good lessons).


> if you have a good manager who shields you from the internal-wannabe-politicians

Unfortunately it's hard to find out in interviews if your manager is worth their salt.


Yeah true....like always..


I'm curious what those people are working on.

As a former bulge bracket investment bank software developer, I certainly did not make anywhere close to insane amounts of money. As a "vice president" 10+yoe senior SWE equivalent my TC was probably a tad bit higher than the higher end of a FAANG junior SWE TC. This is in NYC. Ditto for my colleagues.

The only kind of SWE I can see making great money at a bank are those working in the algo trading and maybe the quant space, but those are highly niche roles.


Depending on what you do though, decent money can be had there for not a lot of skill.


In my city there's a few service centres for banks, and the interview process for engineers seems to be "can you spell Python?". I'm sure there are good people there, but there's so many bad people, that having the name of a bank on your CV is often seen as a negative.

On the other hand I know someone who is a DBA. A few years ago she decided to switch to a bank, and the work-life balance is much better than any other company she has worked for. This year they let her work remotely from another EU country for 3 months.


If the skill level there is so low it's an opportunity to exceed their very low standard and still finish the workday in a couple hours and then enjoy the rest of the time or use it to work elsewhere/build a side-project/etc. Seems like an amazing deal to me.


You can do that, yes, but it’s surprisingly tricky and stressful to manage.

Bear in mind that most of the people there won’t actually realise that they’re low skill level. In fact, they probably think they’re high skill level because they’re working in a bank and banks are elite, right? That is a toxic combination.

So actually you spend much of your time fending off requests to change your work in stupid ways or making futile attempts to avoid the codebase turning into spaghetti code.

It’s not the case that you can just turn in two hours’ code and everyone’s happy with your day’s achievement.

It is exhausting to have to argue the case for every minimal piece of good software design (eg validating inputs, designing minimal interfaces, avoiding god classes).

Eventually you give up and/or they think you’re stupid because you find their spaghetti code hard to understand. The app becomes increasingly slow, hard to maintain and full of weird bugs. Everyone wonders how this could have happened and the rewrite begins.


> In fact, they probably think they’re high skill level because they’re working in a bank and banks are elite, right?

Over last two years I had to work with few people who worked in banking as C# developers, they all had 8-12 years of experience and they wrote worst, most outdated, unreadable and untestable spaghetti code I've ever seen. First week on the job I've added StyleCop to all our projects because I just couldn't stand people being unable to press one shortcut to format their own code(or install an extension to do that for them). This way it wasn't me pointing out ugly code, it was the pipeline, 'pls fix'.

Obviously there are people and people, but so far I've only noticed that developers who worked in banking institutions usually have a lot of experience in that one or two particular companies and they barely can do anything of value on their own.


100% true.

I did a small stint at a mid tier bank and this is exactly how it went. You try to advocate for better methods and you constantly hit inertia and a deep institutional belief in years of service.

"You use git? Interesting but our director (15 YOE) says it's a waste of time. Do you claim you know more than him? You've only just started here!"

In the end you could easily do the assigned work in 10h a week except you are hobbled by bad practices, poor infrastructure, cheap leadership (you want an IDE? our division didn't approve the expense) and so so much Legacy code.


It's also hard to see all these problems when you are in the middle of them. Finance culture is toxic in ways you don't notice until you leave.


"It is exhausting to have to argue the case for every minimal piece of good software design"

So don't do that. It's rarely worth it. I'm happy to bill $150 an hour while a couple of colleagues argue vehemently about directory structures on S3.

If you care that much about every minimal aspect of software design, then you need your own personal project. Landing on a team that all have the same good sense about software development with the ability to express that is just pure luck.


You're right, that's the way to look at it. If you can be transactional, it's ok and maybe you can get away with a 10hr week.

But it's still tiring to see code being merged and thinking "that's going to make X, Y and Z harder for me to do". Or being asked/told to make bad changes. Or being the one who has to do the emergency fix for the non-null assertions causing runtime exceptions in prod (which you had flagged in review and been told "the compiler is wrong, we know they can't be null").


Also expect hostility between teams. Everyone is competing against each other for their fluff to be noticed. Everyone is just looking to get promoted without caring if they're actually delivering anything useful. It just has to have the appearance of bringing value.


Apologies, but this is one of the first times I’ve heard that certain industries in tech are better or worse then others. Is there a place to read up more about this?

Thank you. c:


That seems naive? All sorts of industries have nice side/nasty side, and it's often very dependent on both which company and which job title and even which individual manager you get. This is why Glassdoor is a thing.

At the end of the day it's all anecdote, but the plural of anecdote can be opinion even if it's not data.

The cost center heuristic is good: is the work you're doing an important part of a product being sold? If not, you're just a cost like office cleaning, and will be treated and paid accordingly.

Apart from banking (process-driven, often boring, can be well paid) the other big stereotypes are consultancy ("body shop": cheap and nasty; higher end consultancy can be hugely lucrative but very political) and games (almost uniformly appalling, only work for a games company if you really want to be able to tell your friends you do so)


Meh I worked at a mobile bank startup and I learned a lot and grew a lot. Was a great point in my career.


They are hiring 2 techs to service their 4k TVs in the conference rooms.


They will compete in the 4k demo compo


You joke but some of these parties really need the funding. I wouldn't mind a Goldman sachs sponsored 4k compo and whatever prods may result.


I agree, that would be something


Angry upvote.


This made me laugh.


Ah traditional banking, where tech teams are disposable, seasonal and underpaid. Good luck to anyone who trusts a traditional bank with their technical career.


Exactly this. However most people's tech skills are React, some-other-backend framework and whatever newest database is being peddled right now. These people are easy to hire for around a 140k base for a bank. For pure tech companies of course the wages would be much higher.

Anyways I personally wouldn't be able to work in a place where software is not their core business.


Wow, I had no idea they had this many purely tech staff:

"Citi currently has more than 30,000 software engineers."


Something that SV engineers seem to not understand is just how much software is private software (also how much of it is just plain old boring "enterprise" Java or C#).

It's not glamorous or pays as well as SV startups/unicorns but it's out there and accounts for most of the worlds software.


Too bad they couldn't have a couple of them improve that unusable old loan management program that lost them $900M.


I’ve heard Jamie Dimon brag on more than one occasion that JP Morgan employs “more programmers than Microsoft.”


As a customer, I've been very happy with Chase's software, apps, and even their ATMs.


As a customer of Chase and CapitalOne, I hate having to use any of Chase's apps.


What are you comparing it to?


Another quote "it's a tech company with a banking license"


They even had a tech company within citi. https://www.citigroup.com/citi/news/2009/090120e.htm


This was actually fairly common in super large international conglomerates. At their size it made sense for financial reasons. I think these days the way the tax code is set up it kinda doesn't anymore. I've seen a lot of these IT departments sold off.


wonder how many of 4k are new-hire vs replacement staff


I wonder how many those are part-time contractors or outsourced.


And outsourced to the crappiest of contracting firms like FDM or Infosys.


Somewhat unrelated but Citi's software has been ... not very good from my perspective as a customer. The app is crashing/down frequently, as is the website.

Which seems pretty smelly to me from the outside.


It is by far the worst website I've ever had to use regularly. It's painful even when it's working.


When it's so bad on the front end, just imagine how awful it is on the back end.


Incoming class action lawsuit from the existing 1080p staff for impossible beauty standards.


Citigroup will suck the life from your body.


My experience: If big corporations build up staff quickly, they decrease staff quickly as well.


This year has already seen plenty of layoffs and far more will come. Some companies even rescinded job offerings. The economic situation isn't suited for hiring, especially not so many people.

But, even worse, it's a bank.


Banking is ripe for a tech disruption, there's honestly a lot of things that are higher tech in banks than outsiders realize. There's a very good chance in 5-10 years a tech forward bank becomes the dominant player by a lot. An already large bank could pull this off, but I'd say that's less likely since they have to completely redefine their internal culture.

The economy also isn't a monolith, different sectors grow and shrink differently. The interest rates were so low that VC was way more attractive, but as rates tick up more traditional banking products will likely become more attractive. Banks took a beating in 2008, because it was bad bank policy that crashed the market, that isn't the case this time.


Slightly late to this thread - I heard a rumour that Citi were using Clojure, anyone able to confirm whether this is true for the SG dev team?


In what countries/regions are they hiring? If they need such a large workforce, maybe you can negotiate a nicer wage than normal for a bank.


I thought we are leading to recession ?


Warren Buffett once said that it is wise for investors to be “fearful when others are greedy, and greedy when others are fearful."

Might apply here...


[flagged]


>Their brothers are directors in other orgs, cousins run the contracting agencies they hire.

I always suspected this. I know someone who got into VP position (inflated position of course) with his so called "contacts". He is the most useless person I have seen.


It’s hard to get insight into. Hard to see anything at all from the outside, and on the inside you will suspect it. But the depth will be impossible to know unless you are accepted into those circles or know someone that is


Just curious, are you of italian extraction by any chance? Grew up around NJ?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: