This is the first I'm hearing of Pivotal. Are they a cloud provider or something? I'm having a hard time figuring out exactly what they do from their website.
They are a super trendy development and cloud consultancy firm based in San Francisco and New York. They have all the right words and everything. Pretentious start time of 9:06 am, signalled by a gong. All pair programming on iMacs. I visited their very shiny office for a tech talk. Two employees (they call themselves Pivots) used the exact same language to describe how good the strict schedule is. I struck up a conversation with a gentleman wearing their logo. He seemed visibly taken aback when I asked what his role is. Turned out to be C-level and giving the presentation.
I did a couple consulting stints at Pivotal, working on the CF team. They have a very particular methodology that evolved from the earliest days of agile (Rob Mee was a friend of Kent Beck back when XP was being worked out). I walked in there as a very experienced engineer and a healthy amount of skepticism... and it turned out to be a fantastic learning experience. I've since taken the Pivotal process (yeah, even the weird stuff like pair programming) and used it effectively to manage subsequent companies.
I don't love everything about Pivotal (neither Ruby nor Go make my favorite-languages list) but then, not everyone at Pivotal does either. But overall they do agile right, and they have a lot to show for it. You can mock 9:06 but also note that people go home at 6pm and have lives. The schedule has its upsides.
I think it's pretty clear that these strict agile processes work well for some and poorly for others - some people do better in a highly structured environment and others do worse - it's a bit like remote work on the other end of the spectrum.
The criticism and pushback comes from the negative experiences devs have had when management pushes a one-size-fits-all approach.
I've been involved in more than 5 failed agile methodology process improvement initiatives. I've seen great engineers reduced to a tiny fraction of their former productivity and seen other "hopeless" engineers get a lot better.
It's a great process for reigning in "great" engineers that produce a ton of technical debt. It's a fantastic approach for making your engineering team more stable and product delivery more predictable.
> It's a great process for reigning in "great" engineers that produce a ton of technical debt
So any positives can safely be attributed to the new process and any negatives are clearly just the new process shining a light on existing problems.
Gotcha.
And the parent poster was wondering why people were so cynical.
> making your engineering team more stable
Can you expand upon what you mean by more stable?
One of the main selling points to management is it becomes much easier to switch developers between teams.
> product delivery more predictable.
In what sense? And through what mechanism?
It does reduce risk on short term deliverables but that's a very narrow interpretation of predictable.
It's certainly not the case as an external customer - trying to nail down an xp team to a fixed deadline more than a few weeks away is an exercise in frustration.
Much of the process revolves around trying to ensure everyone on the team can tackle any problem in any part of the codebase. This is accomplished by doing all work paired, rotating pairs daily. Similarly the consultants assigned to your team are regularly rotated in and out to different projects during your engagement.
Specialization is highly discouraged, so the result is a mix of people who can kinda accomplish most things eventually, and a rush to find external resources when you need even moderately specialized knowledge about components in your stack.
Outside of your oddly combative use of the term cog, this is an accurate portrayal of an integral aspect of their process: distributing knowledge, capability, and reasponsbility across the team.
I am no capitalism apologist. The interoperability of technicians diminishes our marketplace bargaining power, thus diminishing labor's leverage over management/capital. At this point in my life, I have accepted this trade-off in exchange for
1) mutual code-review,
2) nights off on-call,
3) opportunities to learn new tech,
4) a product roadmap that can be prioritized without a freaking gant-chart to slot e.g. language-dependent work into language-adherent technicians' backlogs.
While these and other factors make my role less stressful, and my team more effective, I do concede that knowledge dissemination diminishes tech workers' bargaining power.
You've gotta keep in mind their product is not the day to day engineering work while you're engaged with Pivotal, but the transformation of your team.
If they're successful, you're left with a team who adheres to or even enjoys the rules and rituals laid down by Pivotal, and you end up with that team I described who can "kinda-sorta mostly get anything done... eventually... with the help of a few external vendors..." From a business perspective, this isn't a terrible place to be.
If they aren't successful, oops, your entire team churned during your engagement with Pivotal, but don't worry -- they're happy to help out with your hiring processes.
Considering the idea I was trying to convey with the word "cog" was "trivially replaced by any of the dozen others we have laying around"... yeah, at least a minimal level of specialization is a "precondition for non-cogs".
My experience also was that they have a huge snob factor there. I was doing consulting for a database company when I visited their office. In the meeting I was announced as someone when knew our company's product, to which one of the people responded "Oh, I think we already know how <product> works". I wanted to respond, you guys have no f-ing clue how it works, but couldn't for decorum reasons.
I'm sorry you had such a disheartening experience at Pivotal; it's uncommon — in general, Pivotal people tend to be very warm and kind, and not snobby at all.
When I joined Pivotal six (seven?) years ago, I assumed that I'd stay maybe two years, tops, but the people were nice, and that's what has kept me here.
Also, the comment you heard ("Oh I think we already know how [Database X] works") may have not intended to be snobby but rather a nod to one of the developers there: if the database company was InfluxDB, and the office you visited was the NYC office, then there's a good chance that they were referring to one of the Pivotal developers, John, who was one of the early developers of InfluxDB along with Paul Dix and Todd Persen. [1]
Having worked with Pivots in their SF office and in smaller satellite offices, the Kool-aid isn't nearly as strong away from their main offices.
But, I do really like a lot of their processes. Pair programming isn't for everyone, but I think it's a better way of building software. Now that I'm not doing it full time, I constantly miss the benefits of it.
Their strict schedule is kinda nice for work life balance, but it also means working with them was a little strange. Our whole team would go out for happy hour at 5, but they would stay until 6, even with almost nobody else in the office.
TL;DR "So Pivotal decided to employ both a stick and a carrot. The stick is a mandatory morning meeting at 9 a.m., where your absence will likely be noted. The carrot is the breakfast buffet, "sort of a prize to get in,""
I'm glad I read this. Now I'm sure to never apply for this company
I run the Pivotal office in Singapore. Breakfast is wonderful, but it's hardly mandatory.
Standup at 9:06 is important, though. In order for pairing to work, you need to have everybody there at the same time.
The iron bargain is that you work a rigid eight hour schedule, but then you go home (or wherever) and don't think about work. No overtime or crunch mode, ever.
It's not for everybody, but many folks really really really like it.
Everyone has different levels of self-control. Maybe in an ideal society the consultants with a great deal of it would work from home/for themselves while the consultants with less would get the breakfast buffet and the gong.
I keep reading that US companies complain that they dont get skilled people and I always think its because they are not ready to pay a good salary but looking at the list the salaries look good. So is there really a skills shortage?
The skills shortage in my experience is at the management level and above. The number of people that aren't remotely qualified to even be an "IT Guy" that are in management roles are astounding.
Besides consulting and software development (Labs), their main product is PCF (Pivotal Cloud Foundry). Basically it helps combine the best bits of cloud centric, automated, containerized tools - without locking you in and letting you be portable across AWS, Google, Azure or private clouds.
The big thing here is that there is a massive opportunity of large companies still managing large monolithic apps with slow, expensive horribly inefficient infrastructure and deployment practices. Pivotal helps (especially bigCo.'s) rebuild their whole app infrastructure as well as transform their culture of software development. The biggest thing, according to their case studies, is that both speed of development as well as the ratio of 'ops' headcount to 'developer' headcount can be significantly improved.
Thanks for this description. So it seems to me that they are consultants who come in to a non-engineering focused company and kind of "train" their engineers to work in a certain way on their own tools and then run a subscription model on those tools.
Pretty smart way to do it - and I can see how it would be really aggravating from a power-user/10x engineer perspective.
They basically make a Java-based application abstraction layer that sits on top of AWS, Azure, VMWare, etc. Basically a platform for managing storage and compute across multiple on-prem and cloud providers. It’s real enterprisey stuff — as in, it’s probably not even interesting to you until you’re big enough to run in a half dozen data centers and within all the cloud providers.
For Cloud Foundry, Java is just one runtime target, provided by build packs.
You can run whatever you like, even Windows stuff. You only need to have (a) build pack(s) (aka runtime layer) and a stemcell (base operating system).
CF is like Heroku on steroids or like Lamda+API Gateway+Services for more complex applications.
And Pivotal provides one product of Cloud Foundry (
Pivotal Cloud Foundry), that can be deployed to various Cloud Providers (such as AWS, Azure or OpenStack). There are other vendors, like IBM (Bluemix).
I guess, you mixed it up with the Spring ecosystem, which surely a big driver in the adoption of Cloud Foundry. Pivotal is the core maintainer of the ecosystem.
And no, you don't need to be a big enterprise to run a CF.
I don't foresee companies continuing to pay for the ability to switch cloud providers. Mostly because I doubt the pricing differences between providers will ever be significant enough to justify moving.
I also suspect that providers will not keep feature-competitive with one another. It seems like every month, AWS is releasing new tools, and there's no way that other providers are maintaining this pace. My suspicion is each provider will specialize in non-overlapping domains. So unless your company will always be running a lowest-common-denominator site, you'll eventually encounter a vendor lock-in feature.
Well, a lot of what PCF does is help you map the services available in AWS / Azure to an internal service catalog as defined by an enterprise architecture group. This is really powerful in the “small enterprise” space ($1 billion - $5 billion) because that’s often where the EA value proposition becomes strong enough to worry about.
This is the level PCF plays at; and the level they’re useful, but in my experience Pivotal doesn’t know how enterprise companies develop software. Their scrappy pair-programming Dojos don’t really build systems the way those kinds of companies need to design system requirements.
So in my mind, Pivotal’s consulting services are a bunch of startup guys coming in to the enterprise world telling enterprise developers how to build java services. They’re smart and know what they’re doing, but they’re totally out of touch with the kinds of internal controls, reporting metrics and architecture management that larger companies often require.
I absolutely disagree. Pivotal understands the "needs" of the enterprise world and they prove them wrong at every level. Its that "enterprise world" that needs to change in order to survive or at least to be able to hire new people and to manage the ever growing complexity of enterprise architecture. That's whats referred to as "Digital Transformation".
Its just extremely hard for the old "enterprise world" to adopt to change that is inevitable.
Funny, I see the opposite trend. Technologies like kubernetes enable immutable infrastructure that’s not locked into a single cloud. There’s no reason you couldn’t build a system that regularly and automatically buys the cheapest capacity, sets up a kubernetes cluster, and deploys the latest build to it.
Do they make it easy to migrate from tools like AWS EMR, SageMaker, or ElasticTranscoder? It's one thing to be able to migrate VM images or microservices, but in my experience, the big draw to AWS is the features they offer beyond that.
I worked at a place that had recently switched over to microservices in the Cloud from a monolith on premises. Rumors were we were spending a lot of money on our cloud providers. I heard other rumors that pricing got less competitive after awhile because the size of our bills implied that we were locked in. I'd guess that you could get more competitive pricing if you were able to easily switch between cloud providers.
Platforms like this also allow you to choose between using Amazon RDS or a homerolled Postgres DBaaS in the same way. You can make a strategic/tactical decision to implement certain parts of your infrastructure yourself vs outsource the capability entirely.
But again, this is really only useful if you’re big enough to have many application groups developing for a bunch of different business units. I have yet to see a good use case for PCF outside enterprise IT departments.
The main benefits of PCF are not cloud vender lock-in. Where I saw it used, they weren't even on AWS, but using PCF to migrate to microservices. Microservices were written with in Java with Spring Framework, another pivotal product. Pivotal consulting also taught them how to do pair programming, agile, and other "modern" software techniques.
On the contrary, many major companies, especially finservs have multicloud strategies to avoid lock-in or concentration risk (a real concern of regulators). Platforms like pcf and K8s allow for (easier) porting between clouds rather than building directly to cloud provider api's.
The wildcard here is acquisitions. Large companies make a lot of acquisitions, it can be one of the only ways to grow some companies. Each acquisitions brings its own potentially locked in cloud ecosystem.
Large competent software companies like Facebook and Google have still not completely integrated Instagram and Waze respectively for example.
At least when I was working with them last (6-9 months ago) Java had by far the most platform support as a language. I wasn’t talking about the language the PCF code was written in, but rather the development language you use to write services. I realize they support a half dozen plus languages, but Java was by far the best supported in my recollection.
In April 2013 VMware, and its parent company EMC Corporation, formally created a joint venture (with GE) called Pivotal Software. All of VMware's application-oriented products, including Spring were transferred to this organization.
I'm surprised nobody mentioned Gemfire and Greenplum (one of their acquisitions) - both rather important data platforms.
I also had to do work with Pivotal HD, which was their custom Hadoop distribution. Not sure the status of that these days, they may have nixed it - it's been a while since I worked on that platform.
