I consulted at Pivotal on and off over the last several years and got an up-close look at the process. This a heroic feat.
In a short matter of years, they took a consulting services company, adopted some poorly designed abandonware from VMWare, and rebuilt it into a product that now serves some of the biggest companies in the world. Pretty much everything has been rewritten and almost all of it is available in github.
CloudFoundry is not sexy software. It's basically Heroku that big stodgy companies can run in-house. Startups will never use it; they can use actual-Heroku. The crazy thing is that Pivotal supports these huge enterprise software installations inside other people's data centers, often without direct access. It's really, really hard work.
I really hope this makes a bunch of my friends rich - they deserve it.
I know some folks at Pivotal as well. Congrats to them! I'm a bit depressed looking through some of the comments on here. It feels like people don't understand or care what Pivotal has done and is doing and are writing it off because of the tech rather than the idea and implementation.
To sell, you need to speak the language of your target audience.
Pivotal's audience is enterprise customers, so their website is naturally filled with enterprisey messaging - high on 'outcomes' for decision makers and low on nitty gritty details.
HN's audience is not really the same (broadly speaking). A landing page that shows you how simple it is to deploy a python function into production on Pivotal Foundry would probably be a better sell here.
Interesting... I was watching a bunch of Cloud Foundry talks on YouTube around the 2010-2012 era (my memory is hazy).
What happened to it? Why did VMWare abandon it? As far as I remember it was written in Ruby / Event Machine. As I understand it, Heroku also has large parts in Ruby, so that isn't a problem by itself.
How did Pivotal get involved? Were they hired by VMWare, and then they took it over?
Why / how did they rewrite it?
I had some ex co-workers who consulted for Pivotal around 2005-2006, and I have seen their name pop up from time to time. Pivotal tracker seems somewhat popular, and then I remember their name popping up again around Cloud Foundry.
It's nice to see that they turned it into something. It does seem like a nice success story for "open source cloud". This was around the same time that OpenStack was getting going too, but it seems like the success of OpenStack has been more mixed.
* EMC bought VMWare
* EMC bought Pivotal
* EMC moved all the enterprisey software to Pivotal (including CF & Spring)
* Dell bought EMC
* Dell/EMC spun off Pivotal
I arrived shortly after CF moved to Pivotal, so much of this is lore: VMWare had been working on CF for years; they started a 2.0 rewrite which went out of control (with a huge expensive team of primadonnas); EMC basically pulled the plug and gave the software to Pivotal, who built a new team more-or-less from scratch (the only remnants of the original team were ops people). When I was there we were still doing a lot of "forensic programming" or as they say at Pivotal, building context.
When I say "pretty much everything has been rewritten" I mean it has evolved like that; they didn't sit down to rewrite CloudFoundry (Pivotal does not do rewrites as a cultural axiom). I mean that piecemeal pretty much every bit of code has been altered in significant ways. A lot of the Ruby has been swapped out for Go for performance and maintainability reasons. It was pretty funny to watch a large body of die-hard Rubyists get excited about static types (come to the light!).
As a consultant I was really a fringe player; I'm sure there are people reading this who are/were much more involved and could tell the story better.
The thing I found most interesting is that this project should have failed. A HUGE incredibly complicated body of enterprise software with near-100% team turnover? I would have bet against it ever working. But all that pair programming and rotation and writing stories and backfilling tests etc just eventually ground the problem down. It was expensive as hell and it took years but it looks like a success story now. I don't know of any other big takeover project like this that worked. It's a huge credit to the people working on it, and yes - to the "Pivotal process" that seems to irk so many people in this thread.
Wow didn't realize it was so tangled. Considering all that turnover, it's even more of a miracle! Thanks for the recap.
I agree with the philosophy of "grinding it down". Writing quality software is largely about doing the straightforward thing, and not regressing, for years on end... At a certain size, it's too big for any heroics to make a difference, and you have to rely on plain old "engineering" (tests, prioritization, etc.)
> How did Pivotal get involved? Were they hired by VMWare, and then they took it over?
Pivotal Labs got bought by EMC in 2012, who also owned VMWare. A year or so later EMC spun out Pivotal Labs combined with VMWare and some other companies(Green Plum) in a new company under the Pivotal name.
Another super important part of Cloud Foundry is the underlying VM automation layer called Bosh [1]. It used to be extremely high learning curve, but its also very powerful. Cloud Foundry could not exist without it, but it can be used for automation of other packages of software as well (the CI/CD tool Concourse for example)
Cloud Foundry is VERY mature nowadays, is completely opensource and is part of the Linux Foundation. You can view it as Linux, which is the vanilla opensource version and which has different distros (CentOS, Ubuntu, ...)
Cloud Foundry is the vanilla opensource version, and there are many (commercial) distros available, like Pivotal Cloud Foundry, IBM Bluemix, SAP Cloud Platform, etc...
ah my bad I meant more the work practices at pivotal (labs?). They seem to be using a very specific flavour of agile with pair programming etc. I'd be interested to dive more into it.
Which is a bit misleading as far as Spring is concerned.
Spring has never been abandonware to the Java community. It is probably the most used framework for enterprise - I've seen maybe 10 jobs looking for Spring experience vs the JEE crap that it has mostly replaced.
I know it gets a bad view on HN - a lot of FactoryBuilderFactoryBeans and a rather large kitchen sink of APIs.
But the framework itself is well written, great docs, lots of resources.
I've been involved in a dozen different projects where the architects hated Spring and instead went to some newer hipper lang like Flask, Django, Rails, Finatra / Finagle. Spring has a learning curve, but once you get it, it is far more powerful and stable than any of these alternatives.
We do not control and may be unable to predict the future course of open-source technologies, including those used in our offering, which could reduce the market appeal of our offering and damage our reputation.
This is a really cool risk factor that I've never seen before in an S-1, exciting that more big public companies are willing to accept the risk reward that comes with contributing to open source:
If open-source software programmers, many of whom we do not employ, or our own internal programmers do not continue to use, contribute to and enhance the open-source technologies that we rely on, the market appeal of our offering may be reduced, which could harm our reputation, diminish our brand and result in decreased revenue. We also cannot predict whether further developments and enhancements to these open-source technologies will be available from reliable alternative sources. If the open-source technologies that we rely on become unavailable, we may need to invest in researching and developing alternative technologies.
Their quarterly reports always have similar wording, to wit:
"If open source software programmers, most of whom we do not employ, do not continue to develop and enhance open source technologies, we may be unable to develop new technologies, adequately enhance our existing technologies or meet customer requirements for innovation, quality and price.
We rely to a significant degree on a number of largely informal communities of independent open source software programmers to develop and enhance our enterprise technologies. For example, Linus Torvalds, a prominent open source software developer, and a relatively small group of software engineers, many of whom are not employed by us, are primarily responsible for the development and evolution of the Linux kernel, which is the heart of the Red Hat Enterprise Linux operating system. If these groups of programmers fail to adequately further develop and enhance open source technologies, we would have to rely on other parties to develop and enhance our offerings or we would need to develop and enhance our offerings with our own resources. We cannot predict whether further developments and enhancements to these technologies would be available from reliable alternative sources. In either event, our development expenses could be increased and our technology release and upgrade schedules could be delayed. Moreover, if third-party software programmers fail to adequately further develop and enhance open source technologies, the development and adoption of these technologies could be stifled and our offerings could become less competitive. Delays in developing, completing or delivering new or enhanced offerings could result in delayed or reduced revenue for those offerings and could also adversely affect customer acceptance of those offerings."
"If open source software programmers, most of whom we do not employ, do not continue to develop and enhance open source technologies, our development expenses could be increased and our product release and upgrade schedules could be delayed."
A few months ago, I was working a contract at Idaho National Laboratory as a senior DevOps engineer. I recommended they try Cloud Foundry since they were doing a lot of things manually and had a pretty immature CI/CD pipeline; additionally, I was involved in a PCF project at General Motors. INL runs VMware vSphere, which is the platform to run PCF on in-house. I got approval to bring in Pivotal and, through mostly my teammate's and my hard work, we managed to get it running. Props to the helpful sales guy and architect we were assigned. They were available to answer some very detailed questions and did not disappoint with their knowledge and enthusiasm.
PCF is very solid. The only issues we ran into were some confusing configuration settings and your basic federal government bureaucracy nonsense. Once we got all the IaaS boxes ticked, it installed without issue and worked beautifully.
I'd never want to install or maintain open-source Cloud Foundry as it's a nightmare of complexity; however, PCF is a different animal because of its streamlined installer. I definitely recommend giving it a go if you're looking for on-premises PaaS and you've got the IaaS layer to support it. You can also demo PCF within a single VirtualBox VM.
It's interesting how much the filing seems to revolve around PCF. I played with PCF a few years ago and was really unimpressed. More recently I've been consulting with firms using PCF and the devops guys there seem to really like it and Bosh, so it may have gotten better at least.
That said, I just can't understand why you'd ever pick PCF over Kubernetes. Sure, I get one deploys application code, and the other deploys containers. But after using the buildpack system for a while, I really think containers are the better solution (assuming you have the CI/CD to keep your custom containers up-to-date with new base images).
So it'll be interesting to see what Pivotal does there, and how they position PCF, especially now that they have their own branded version of Kubernetes[1].
Right. Similar with Amazon. Both seem to recognize that K8S is going to take over the market, so they've hedged against that threat.
And having someone provide a more enterprisey K8S is helpful. The version churn right now is pretty bad, and not something F500 companies want to deal with.
> We are focused on subscription sales of our platform. Since announcing PCF in November 2013, our subscription customer count has grown rapidly to 319 as of the end of fiscal 2018. Our subscription revenue was $95.0 million, $150.0 million and $259.0 million for fiscal 2016, fiscal 2017 and fiscal 2018, respectively, representing year-over-year growth of 58% and 73% for our two most recent fiscal years.
Their pricing was based on application instances and the list price was ludicrous (several hundred dollars) but most companies got good discounts.
The thing that bothered me with the model was it disincentives modern microservice architectures. One big monolith cost less than lots of smaller components.
This is still the case. Even after enterprise discounts, the pricing is outrageously high. I know of at least one well-known enterprise customer where IT has mandated a company-wide "get off Pivotal" policy because TCO is just too damn high.
I hope for Pivotal's sake that the eye-popping price is not crucial to their growth forecast. If it is, I think 2019-20 will be difficult for them. They just don't have the market share to force these prices down customer's throats.
They tell us there that 319 customers paid $259 million between them in fiscal 2018 - so on average each customer pays $800k/year. They'll have outliers in both directions from that average - But my gut feel is they probably don't have many customers spending less that $80k/year. (Same as I'll bet there's not too many $8mill+/year ones).
Account size and pricing are two different things.
The question is: are the $800k customers getting good value for that subscription, or are they looking at projected costs and thinking "we better switch to a competitor before this gets out of hand".
The YoY growth rates suggest they're at least capable of convincing new business that it's good value.
(Though in enterprise sales, even amazingly poor vendors often get to re-bill for several years, until the exec who signed off on the subscription moves on and it becomes politically possible for people internally to admit to each other it was a stupid decision to sign up in the first place... So perhaps 2 years growth here is only telling the "sales capability" side of the story, not the "ongoing value provided" side...)
Right, that's what I've witnessed anecdotally: strong ability to sell, usually as part of a top-down "digital transformation" project with tons of agile development consulting and multi-year sales cycle; then a few years of abysmal results, covered up by the project sponsors, then eventually a purge once the sponsoring exec is replaced.
And the other similar version: tightly fought procurement between two opposing vendors with different internal "sponsors". Winning vendor's ability to deliver is stymied at every opportunity by losing sponsor or their internal supporters. Then the few years of abysmal results happens in spite of winning vendors capability and efforts - same outcome plays out.
I interviewed with Pivotal Labs this week. It went well and they wanted to move forward until I asked for my current compensation. They balked like I was asking for the moon. I coincidentally had an offer from another firm for a 15% raise, so I know I wasn't asking for too much. Pivotal declined when I wouldn't budge, and I took my 15% raise. I start in 3 weeks.
I had a similar experience a couple years ago. The interview was pretty hard, 6 hours of pair programming in Go, a language that I have only passing familiarity with. The interviewer seemed impressed that I completed the entire task with time to spare. Then it got to talking about salary and that's when things seemed to go sideways. I make top of market for my area, because I've been doing this a long time and I have a history of success so past employers have rewarded me for it.
I think the thing is that in consulting, the equation is "margin = billable rate - salary". The higher the salary, the less profitable you are. Feels to me like Pivotal mostly hires mid-level people to hit the sweet spot of experience/skill/salary.
It's a bummer because I went on to work with a company that engaged Pivotal Labs and I loved working with the Pivots.
That person sounds really poorly-adjusted, agenda'd, and borderline-hateful.
Not only are many of his claims dubious, unwarranted, or hyperbolic, but there are a few gems like this:
> Remember when the CEO globally emailed a picture of his ugly newborn and drugged up post-birth wife to the entire company because he thinks we all submit to his cult of personality?
It's not uncommon for that to happen. People share pictures of their new kids to their departments and whole companies all the time. Usually it's because new kids are a joyful experience, for, y'know, humans in general. CEOs do it too--for cynical reasons or not--and raging against that, to say nothing of the criticism of the "drugged-up" mother (epidural anesthesia is incredibly helpful and medically essential in tons of births), displays a somewhat shocking failure to grasp typical corporate-people behavior in America at best, and an axe-to-grind willingness to engage in ad-hominem slander of people's families at worst.
Even if any of his claims have merit, I have a ton of trouble believing anything from someone who talks like that.
Geez, incredibly negative and even bitter. Sure, the deployment of Cloud Foundry is complex, but we're running 6 (OSS) clusters in production with hardly any problems in the last 2 years. The quality is not nearly as bad as being described here. The author seems angry overall, and especially about sales people making more money, which is simply a necessary evil of selling to enterprises.
Full disclosure: I am currently employed by another member of the Cloud Foundry Foundation.
The only two experiences I had with Pivotal were with RabbitMQ (a very well designed product) and with Pivotal Tracker - I tried it about a year ago. I felt like I was fighting the software; they were extremely stubborn about adhering to Scrum 'best practices' that may not work in practice.
The thing is, their flavor of Agile development absolutely does work in practice, and it is hands down the most reliable way to build great software I have ever seen. However it only works if it is pretty much 100% adhered to, hence the stubbornness.
We’re currently considering PCF and their methodology. The VP pushing it has said that the cost means it has to be a success. Or, to be more clear, management knows their necks are on the block if they screw up a very overtly expensive culture change.
Not sure how honest the metrics around success will actually be...
I got to work with some Pivots at their office about a decade ago and learned to use Tracker from them. It's the only system I've ever seen that works well and gives predictable results over time.
Almost every company I've been at has used abhorrent alternatives like Asana, Jira, etc. and they've been consistently awful experiences.
I tried and eventually got tired of Jira, Trello, Team Foundation Server, and Pivotal Tracker.
The one tool that I've been loving for the past year is http://clubhouse.io. Such a well-designed product and extremely flexible (I think it could fit most teams' requirements around Agile/Kanban/etc.)
Their consulting practice is similar. They won't sign on unless your people adopt their methodology 100% (pairing, etc) Guess it's nice to be able to be that choosy about clients.
I worked for a company (subsidiary of Siemens) who adopted that horrendous Scrum methodology.
It was the absolute __worst__ year of my life. I had to pair with people non-stop even when __fixing bugs__. There were zounds of stupid rules which did not make sense whatsoever.
Perfect example of forcing a methodology on a company where it will never work.
I really like Pivotal products though. RabbitMQ is superb, and although Spring Boot is really slow and has its problems I use it on projects where performance is not important.
It's not Scrum. It's Extreme Programming. It works fantastically well, even when you're __fixing bugs__. The whole idea is that you need to be paired so that two people can understand and problem solve around the bug. You end up with twice the understanding that you had before. It's worth it.
Here's a radical thought - maybe Extreme Programming (I still think this is a genius marketing name :) works for some people and doesn't work for others?
After trying so many of these methodologies I've come to realize that people just work differently. It's self-defeating to try to push Extreme Programming or whatever on someone who prefers and is efficient at working alone.
Maybe adopt some of Scrum/XP/whatever that can be adopted at a high-level, but trying to push it through a team composed of people with different personalities and different ways of working just defeats the entire purpose.
My thoughts exactly. I just __hate it__. I was super bored, super frustrated, and it was a complete waste of time. I resigned after a year of struggle.
It worked extremely well results-wise for the few months we did it. I went home at the end of the day mentally exhausted, though. You are essentially having a conversation for 9 hours straight, 5 days per week.
I could have done 10x work working alone. I speak from experience. In my current company we use Kanban, and I am super productive. We only do things which are useful for us.
I dunno. Sometimes it's easier to simply stare at a problem & noodle your way through it. That's hard to do when forced to jibber jabber. Is the ol' stare & noodle a pair-friendly activity?
I haven't done pairing myself, but I suspect it's highly variable based on the two specific individuals. Certainly I've noted smart and capable people I work with well, and some that I don't. Not every smart/capable pair is going to mesh. Just because I respect you doesn't mean I can work well with you, especially in such an intimate set up.
It works better though, if you start from zero and hire people that drank the Koolaid.
It's hard to shoehorn it into an existing team where some portion of the team doesn't believe the premise. For example, some people (introverts, Asperger's spectrum, etc) will just never be comfortable with pairing.
Pivotal doesn't have this problem because developers that don't believe wouldn't ever apply for a job there.
But, that doesn't help Pivotal in their consulting practice. Very few of their clients would have the same advantage.
No it __is not working fantastically well__. I speak from experience. There are also numerous studies (I can link to them) which detail why pair programming wastes time if not used for the right problem (mentoring or super-difficult task).
Yes. Pair programming works well if someone is being mentored or you are facing a very hard problem. In other cases it is a waste of time. You can read a lot of studies about this topic.
Ok, I was working some time with Cloud Foundry as sysadmin/devops. It's big fucking enterprise piece of software which nobody know exactly how to install it and maintain. Their guide on site is obsolete, it's almost unrealistic to understand in nutshell how it works (compared with Kubernetes/Mesos-Marathon). Only info on Github pages were useful.
CF even can't even survive unplanned reboot of DC, after such incidents you literally will get parted cluster and no one know what to do.
This is the first I'm hearing of Pivotal. Are they a cloud provider or something? I'm having a hard time figuring out exactly what they do from their website.
They are a super trendy development and cloud consultancy firm based in San Francisco and New York. They have all the right words and everything. Pretentious start time of 9:06 am, signalled by a gong. All pair programming on iMacs. I visited their very shiny office for a tech talk. Two employees (they call themselves Pivots) used the exact same language to describe how good the strict schedule is. I struck up a conversation with a gentleman wearing their logo. He seemed visibly taken aback when I asked what his role is. Turned out to be C-level and giving the presentation.
I did a couple consulting stints at Pivotal, working on the CF team. They have a very particular methodology that evolved from the earliest days of agile (Rob Mee was a friend of Kent Beck back when XP was being worked out). I walked in there as a very experienced engineer and a healthy amount of skepticism... and it turned out to be a fantastic learning experience. I've since taken the Pivotal process (yeah, even the weird stuff like pair programming) and used it effectively to manage subsequent companies.
I don't love everything about Pivotal (neither Ruby nor Go make my favorite-languages list) but then, not everyone at Pivotal does either. But overall they do agile right, and they have a lot to show for it. You can mock 9:06 but also note that people go home at 6pm and have lives. The schedule has its upsides.
I think it's pretty clear that these strict agile processes work well for some and poorly for others - some people do better in a highly structured environment and others do worse - it's a bit like remote work on the other end of the spectrum.
The criticism and pushback comes from the negative experiences devs have had when management pushes a one-size-fits-all approach.
I've been involved in more than 5 failed agile methodology process improvement initiatives. I've seen great engineers reduced to a tiny fraction of their former productivity and seen other "hopeless" engineers get a lot better.
It's a great process for reigning in "great" engineers that produce a ton of technical debt. It's a fantastic approach for making your engineering team more stable and product delivery more predictable.
> It's a great process for reigning in "great" engineers that produce a ton of technical debt
So any positives can safely be attributed to the new process and any negatives are clearly just the new process shining a light on existing problems.
Gotcha.
And the parent poster was wondering why people were so cynical.
> making your engineering team more stable
Can you expand upon what you mean by more stable?
One of the main selling points to management is it becomes much easier to switch developers between teams.
> product delivery more predictable.
In what sense? And through what mechanism?
It does reduce risk on short term deliverables but that's a very narrow interpretation of predictable.
It's certainly not the case as an external customer - trying to nail down an xp team to a fixed deadline more than a few weeks away is an exercise in frustration.
Much of the process revolves around trying to ensure everyone on the team can tackle any problem in any part of the codebase. This is accomplished by doing all work paired, rotating pairs daily. Similarly the consultants assigned to your team are regularly rotated in and out to different projects during your engagement.
Specialization is highly discouraged, so the result is a mix of people who can kinda accomplish most things eventually, and a rush to find external resources when you need even moderately specialized knowledge about components in your stack.
Outside of your oddly combative use of the term cog, this is an accurate portrayal of an integral aspect of their process: distributing knowledge, capability, and reasponsbility across the team.
I am no capitalism apologist. The interoperability of technicians diminishes our marketplace bargaining power, thus diminishing labor's leverage over management/capital. At this point in my life, I have accepted this trade-off in exchange for
1) mutual code-review,
2) nights off on-call,
3) opportunities to learn new tech,
4) a product roadmap that can be prioritized without a freaking gant-chart to slot e.g. language-dependent work into language-adherent technicians' backlogs.
While these and other factors make my role less stressful, and my team more effective, I do concede that knowledge dissemination diminishes tech workers' bargaining power.
You've gotta keep in mind their product is not the day to day engineering work while you're engaged with Pivotal, but the transformation of your team.
If they're successful, you're left with a team who adheres to or even enjoys the rules and rituals laid down by Pivotal, and you end up with that team I described who can "kinda-sorta mostly get anything done... eventually... with the help of a few external vendors..." From a business perspective, this isn't a terrible place to be.
If they aren't successful, oops, your entire team churned during your engagement with Pivotal, but don't worry -- they're happy to help out with your hiring processes.
Considering the idea I was trying to convey with the word "cog" was "trivially replaced by any of the dozen others we have laying around"... yeah, at least a minimal level of specialization is a "precondition for non-cogs".
My experience also was that they have a huge snob factor there. I was doing consulting for a database company when I visited their office. In the meeting I was announced as someone when knew our company's product, to which one of the people responded "Oh, I think we already know how <product> works". I wanted to respond, you guys have no f-ing clue how it works, but couldn't for decorum reasons.
I'm sorry you had such a disheartening experience at Pivotal; it's uncommon — in general, Pivotal people tend to be very warm and kind, and not snobby at all.
When I joined Pivotal six (seven?) years ago, I assumed that I'd stay maybe two years, tops, but the people were nice, and that's what has kept me here.
Also, the comment you heard ("Oh I think we already know how [Database X] works") may have not intended to be snobby but rather a nod to one of the developers there: if the database company was InfluxDB, and the office you visited was the NYC office, then there's a good chance that they were referring to one of the Pivotal developers, John, who was one of the early developers of InfluxDB along with Paul Dix and Todd Persen. [1]
Having worked with Pivots in their SF office and in smaller satellite offices, the Kool-aid isn't nearly as strong away from their main offices.
But, I do really like a lot of their processes. Pair programming isn't for everyone, but I think it's a better way of building software. Now that I'm not doing it full time, I constantly miss the benefits of it.
Their strict schedule is kinda nice for work life balance, but it also means working with them was a little strange. Our whole team would go out for happy hour at 5, but they would stay until 6, even with almost nobody else in the office.
TL;DR "So Pivotal decided to employ both a stick and a carrot. The stick is a mandatory morning meeting at 9 a.m., where your absence will likely be noted. The carrot is the breakfast buffet, "sort of a prize to get in,""
I'm glad I read this. Now I'm sure to never apply for this company
I run the Pivotal office in Singapore. Breakfast is wonderful, but it's hardly mandatory.
Standup at 9:06 is important, though. In order for pairing to work, you need to have everybody there at the same time.
The iron bargain is that you work a rigid eight hour schedule, but then you go home (or wherever) and don't think about work. No overtime or crunch mode, ever.
It's not for everybody, but many folks really really really like it.
Everyone has different levels of self-control. Maybe in an ideal society the consultants with a great deal of it would work from home/for themselves while the consultants with less would get the breakfast buffet and the gong.
I keep reading that US companies complain that they dont get skilled people and I always think its because they are not ready to pay a good salary but looking at the list the salaries look good. So is there really a skills shortage?
The skills shortage in my experience is at the management level and above. The number of people that aren't remotely qualified to even be an "IT Guy" that are in management roles are astounding.
Besides consulting and software development (Labs), their main product is PCF (Pivotal Cloud Foundry). Basically it helps combine the best bits of cloud centric, automated, containerized tools - without locking you in and letting you be portable across AWS, Google, Azure or private clouds.
The big thing here is that there is a massive opportunity of large companies still managing large monolithic apps with slow, expensive horribly inefficient infrastructure and deployment practices. Pivotal helps (especially bigCo.'s) rebuild their whole app infrastructure as well as transform their culture of software development. The biggest thing, according to their case studies, is that both speed of development as well as the ratio of 'ops' headcount to 'developer' headcount can be significantly improved.
Thanks for this description. So it seems to me that they are consultants who come in to a non-engineering focused company and kind of "train" their engineers to work in a certain way on their own tools and then run a subscription model on those tools.
Pretty smart way to do it - and I can see how it would be really aggravating from a power-user/10x engineer perspective.
They basically make a Java-based application abstraction layer that sits on top of AWS, Azure, VMWare, etc. Basically a platform for managing storage and compute across multiple on-prem and cloud providers. It’s real enterprisey stuff — as in, it’s probably not even interesting to you until you’re big enough to run in a half dozen data centers and within all the cloud providers.
For Cloud Foundry, Java is just one runtime target, provided by build packs.
You can run whatever you like, even Windows stuff. You only need to have (a) build pack(s) (aka runtime layer) and a stemcell (base operating system).
CF is like Heroku on steroids or like Lamda+API Gateway+Services for more complex applications.
And Pivotal provides one product of Cloud Foundry (
Pivotal Cloud Foundry), that can be deployed to various Cloud Providers (such as AWS, Azure or OpenStack). There are other vendors, like IBM (Bluemix).
I guess, you mixed it up with the Spring ecosystem, which surely a big driver in the adoption of Cloud Foundry. Pivotal is the core maintainer of the ecosystem.
And no, you don't need to be a big enterprise to run a CF.
I don't foresee companies continuing to pay for the ability to switch cloud providers. Mostly because I doubt the pricing differences between providers will ever be significant enough to justify moving.
I also suspect that providers will not keep feature-competitive with one another. It seems like every month, AWS is releasing new tools, and there's no way that other providers are maintaining this pace. My suspicion is each provider will specialize in non-overlapping domains. So unless your company will always be running a lowest-common-denominator site, you'll eventually encounter a vendor lock-in feature.
Well, a lot of what PCF does is help you map the services available in AWS / Azure to an internal service catalog as defined by an enterprise architecture group. This is really powerful in the “small enterprise” space ($1 billion - $5 billion) because that’s often where the EA value proposition becomes strong enough to worry about.
This is the level PCF plays at; and the level they’re useful, but in my experience Pivotal doesn’t know how enterprise companies develop software. Their scrappy pair-programming Dojos don’t really build systems the way those kinds of companies need to design system requirements.
So in my mind, Pivotal’s consulting services are a bunch of startup guys coming in to the enterprise world telling enterprise developers how to build java services. They’re smart and know what they’re doing, but they’re totally out of touch with the kinds of internal controls, reporting metrics and architecture management that larger companies often require.
I absolutely disagree. Pivotal understands the "needs" of the enterprise world and they prove them wrong at every level. Its that "enterprise world" that needs to change in order to survive or at least to be able to hire new people and to manage the ever growing complexity of enterprise architecture. That's whats referred to as "Digital Transformation".
Its just extremely hard for the old "enterprise world" to adopt to change that is inevitable.
Funny, I see the opposite trend. Technologies like kubernetes enable immutable infrastructure that’s not locked into a single cloud. There’s no reason you couldn’t build a system that regularly and automatically buys the cheapest capacity, sets up a kubernetes cluster, and deploys the latest build to it.
Do they make it easy to migrate from tools like AWS EMR, SageMaker, or ElasticTranscoder? It's one thing to be able to migrate VM images or microservices, but in my experience, the big draw to AWS is the features they offer beyond that.
I worked at a place that had recently switched over to microservices in the Cloud from a monolith on premises. Rumors were we were spending a lot of money on our cloud providers. I heard other rumors that pricing got less competitive after awhile because the size of our bills implied that we were locked in. I'd guess that you could get more competitive pricing if you were able to easily switch between cloud providers.
Platforms like this also allow you to choose between using Amazon RDS or a homerolled Postgres DBaaS in the same way. You can make a strategic/tactical decision to implement certain parts of your infrastructure yourself vs outsource the capability entirely.
But again, this is really only useful if you’re big enough to have many application groups developing for a bunch of different business units. I have yet to see a good use case for PCF outside enterprise IT departments.
The main benefits of PCF are not cloud vender lock-in. Where I saw it used, they weren't even on AWS, but using PCF to migrate to microservices. Microservices were written with in Java with Spring Framework, another pivotal product. Pivotal consulting also taught them how to do pair programming, agile, and other "modern" software techniques.
On the contrary, many major companies, especially finservs have multicloud strategies to avoid lock-in or concentration risk (a real concern of regulators). Platforms like pcf and K8s allow for (easier) porting between clouds rather than building directly to cloud provider api's.
The wildcard here is acquisitions. Large companies make a lot of acquisitions, it can be one of the only ways to grow some companies. Each acquisitions brings its own potentially locked in cloud ecosystem.
Large competent software companies like Facebook and Google have still not completely integrated Instagram and Waze respectively for example.
At least when I was working with them last (6-9 months ago) Java had by far the most platform support as a language. I wasn’t talking about the language the PCF code was written in, but rather the development language you use to write services. I realize they support a half dozen plus languages, but Java was by far the best supported in my recollection.
In April 2013 VMware, and its parent company EMC Corporation, formally created a joint venture (with GE) called Pivotal Software. All of VMware's application-oriented products, including Spring were transferred to this organization.
I'm surprised nobody mentioned Gemfire and Greenplum (one of their acquisitions) - both rather important data platforms.
I also had to do work with Pivotal HD, which was their custom Hadoop distribution. Not sure the status of that these days, they may have nixed it - it's been a while since I worked on that platform.
Seriously? Dell owns 50% of Pivotal with the rest owned by companies like General Electric and Microsoft. The valuation is $3B. They aren't coattailing anything - it's an actual business with actual customers and actual value.
The ownership breakdown is so unusual compared to typical VC-backed startup IPOs. Pivotal has had an interesting run with their history of acquisition and spinoff. Also the Pivotal (cloud infrastructure software) vs Pivotal Labs (consulting) split.
Are you talking about Gross profit? I'm not an accountant, but by the looks of their financial data, Pivotal is consistently losing money. I'm talking about Net loss 163 million.
Spring is probably the only thing. All the rest of the stuff on the tech blog and Netflix OSS is heavily AWS centric, even the container platform Titus.
Spring Cloud includes a lot of the Netflix OSS stuff though like Zuul and Archaius.
In a short matter of years, they took a consulting services company, adopted some poorly designed abandonware from VMWare, and rebuilt it into a product that now serves some of the biggest companies in the world. Pretty much everything has been rewritten and almost all of it is available in github.
CloudFoundry is not sexy software. It's basically Heroku that big stodgy companies can run in-house. Startups will never use it; they can use actual-Heroku. The crazy thing is that Pivotal supports these huge enterprise software installations inside other people's data centers, often without direct access. It's really, really hard work.
I really hope this makes a bunch of my friends rich - they deserve it.