Move to the cloud, all fine and everything, as long as you're an US company.
Can't speak too detailed because of NDA, but at work we cannot use innovative tools like Google Docs, Google Mail for Business, Dropbox, Office 365 and the likes (and for our privacy-conscious clients, no AWS, only Azure) because the US government can compel the providers to turn over our confidential data to them. We're stuck with Lotus Notes 8.5 and Office 2013 because our IT wants to migrate to Office, but our client NDAs prohibit us from using cloud stuff with their data (again, fears of the clients of US industrial espionage. Snowden actually proved them right).
The NSA and the US government effectively killed their cloud business for non US customers with their spying crap.
That's more likely to do with the cloud in general or even more likely with platform specific compliance issues rather than anything to do with the US.
EU's data protection directive and various local legislation like the DPA don't have an issue with the cloud, but are quite stringent on the fact that the data doesn't leave the EU so you are restricted to EU data centers.
Google and Microsoft have services for EU customers that more or less ensure that the data never leaves an EU data center (MSFT guarantees it, google slightly less so) and as far as infrastructure goes most if not all PAAS/IAAS providers have a European data centers.
https://aws.amazon.com/compliance/eu-data-protection/
I know for a fact that at the least British, German, Dutch and Italian companies and governmental organizations even in heavily regulated industries like health care and defense use both Azure and AWS for both corporate IT and production (and if we count less regulated verticals such as banking, financial services, and telco then everyone and their mother within the EU uses Azure, AWS, and many other cloud platforms).
So no while there is some issues with some US companies and services in regards to privacy especially around the safe harbor rulings there isn't any global or even meaningful resistance to the use of US services providers and software.
P.S.
Also the fact that you use Lotus Notes and Office 2013 also has very little to do with US phobia, you could've used Exchange just as well, and office 2015 just came out and very few organizations have actually migrated to it.
P.S. 2
You've likely already broken your NDA by admitting you signed one or that one exists, as the existence of an NDA is the first thing that is by it.
Not to mention that while you didn't gave any technical detail the fact that you've mentioned the restrictions your company and or organization is under is also most likely a breach of any potential NDA as well as organizational opsec protocols.
That hasn't been my experience, I don't know from where you're getting your facts from. But I'm involved with two very big German companies and I've heard of many others that are banning US-based services and it's not enough for those services to have servers in the EU. There is no such thing as AWS or Azure, their usage would be unthinkable. Everything happens on their own infrastructure and installing Skype on your computer for example can get you fired. The reason is quite simply fear of industrial espionage and the Snowden leaks made things a lot worse.
From actually doing security and compliance(ISO, PCI, FCA) related work within the EU, as for getting fired for installing Skype, you can get fired for installing any unauthorized software in a company with a strict computer use and computer security policy.
I use skype to communicate with German clients including in the banking sector like ING-DiBa, so anecdotal evidence aside not every company has some aversion from US companies not to mention that Amazon is pretty much handing out free AWS credits in every large accelerator in Europe (If you want anecdotal German examples then Deutsche Bank's innovation hub in Berlin) including fin-tech and health care specific accelerators and the participants are eating them up (and if you want bigger companies than AWS's case studies have quite a few of big German clients https://aws.amazon.com/de/solutions/case-studies/all/ including Siemens health care solutions and Software AG, and Nuremberg's Airport so 2 regulated industries, and the 2nd largest German software house).
With the exception of 1 startup in the current Barclay's accelerator run in London every company in IIRC is running on AWS because of that, heck some companies in the the MSFT accelerator across the hall are using AWS even tho that Azure is pretty much complementary.
Big companies always are slow in adapting new technologies but it doesn't mean that there is some generic no cloud or non US policy in it, yes if you are handling data protected under local and EU data directives you need assurances but if you can achieve those or flex them enough to maintain compliance you will be able to use them.
Yes if you are giant org that has it's own data centers and 100% control over all of it's assets you won't be jumping on the cloud bandwagon and you are more likely to deploy a "private cloud" in house which is just a fancy word to say that you will have more modern resource management and deployment infrastructure, and sure the likes of Rackspace are still considerably more popular in Europe (as they are in the US) than AWS as far as managed services go mostly because they've existed for much longer and they still offer traditional types of managed infrastructure / data center as well as cloud-ish products.
You are also way way over estimating the impact of the Snowden leaks on the industry.
When avoiding US Services it's not about the whole company. At least 4 DAX companies research departments I know about enforce GPG Crypted mail transport since Snowden. Everything else bounces. The same companies asked us to exclude the US and the UK out of sensible data routes. I know that's not gonna do it, and there's a lot more to do but... I know that you know why I can't go into further details.
Research departments have always had different operational procedures, 4-5 years ago i did a project for a company called Interhyp(financial services mostly finance management loans and mortgages) in the building across them there was a Siemens facility which had cell jammers that leaked through the street if you were too close well tough luck.
Different departments and subsidiaries will operate according to different procedures based on the specific threats and requirements, this happens even in already highly regulated and restricted fields for example Lockheed Martin'a Skunk Works and Boeing's Phantom Works operate on a completely different level than their civilian and even military BAU aerospace departments when it comes to operational security and secrecy.
Not every company, and not every department can and has to operate using the same ruleset, departments that are relatively sensitive or can afford to work under stringent rules may do so, departments that can't or don't really need too won't, life isn't binary there's more than 2 ways to skin a cat ;)
Depends where and for what use, some company in the US got hit in the rear because of that, but if you are working in the defense industry you might get exempted, heck in South Africa there was a scandal this year when they jammed the signal inside the parliament even tho it was illegal.
http://www.news24.com/SouthAfrica/Politics/Reports-of-a-cell...
At least with Android you can view and compile/install from source right? I thought that was one of the main benefits of using open source for security; that you can both verify the source is secure and also that you can be sure the source you're viewing is being used.
I suppose you also need to trust the build system and compilers though.
How many companies would (not to mention could) you think buy OEM phones and build their own Android OS from scratch including device specific and baseband drivers? I would bet that it's 0 even government agencies opt-out to certify existing devices.
If you work in a place that is too sensitive to accept phones there is a much simpler way to go around it and it's that you aren't allowed to bring in phones into the facility in such cases you usually leave them in a locker at reception.
Other policies might only prevent phones from being used in places that deal with sensitive information such as labs and R&D departments or meetings which touch sensitive matters.
I've worked at a facility that while allowing phones it didn't allow phones with cameras so around 2009 I still have an old nokia phone which while had a camera initially it was easily removable, if you didn't had a phone without a camera then you could simply do a follow me to your roaming extension before shutting it off.
I don't think that any company that sees phone tapping as an actual realistic threat would filter phones based on their manufacturer yet alone the OS, they might issue their own phones which are connected to the companies MDM or require BYOD devices to be pre-screened but saying Android no, iOS yes or vise versa unless it's has to do with specifics like MDM compatibility wouldn't be a realistic scenario.
> How many companies would (not to mention could) you think buy OEM phones and build their own Android OS from scratch including device specific and baseband drivers? I would bet that it's 0 even government agencies opt-out to certify existing devices.
Well, I certainly expect Amazon to be able to do so. Google actually has left them no other choice in the long term than to re-implement everything because a shitload of stuff has been moved into Google Play Services.
Ad PS2: My NDA thankfully covers only client-related matters (which kinda makes sense).
The point of why no migration has occurred (or, actually, the migration underway has been paused) is the massive level of uncertainity, which has been fueled even more by the recent EUGH ruling on Facebook.
I know of many big deal companies in the USA that work with many big deal European companies. I can assure you that this is a real deal. Big companies that have CISOs and care about legal compliance in fact DO care if their proprietary data is stored in the USA, and under what circumstances.
I posit with empirical observation and experience that there is "global or even meaningful resistance to the use of US services providers."
>> You've likely already broken your NDA by admitting you signed one or that one exists
In my opinion and I'm not a laywer, you aren't violating an NDA by stating an NDA exists unless the NDA explicitly states to not state that it exists. He did not disclose anything about the NDA or parties signed to it or any content that the NDA covers.
I think these types of concerns may eventually be addressed by better and more consistent use of encrypt-in-place in application design. If your data is well encrypted, theoretically it should be safe from the NSA even if they get their hands on the files.
Encrypt-in-transit is well along in terms of culture and technology. "SSL everywhere" is something that, I think, most folks believe we can achieve. Google, for example, has said that eventually Chrome will start marking all HTTP connections as insecure.
But how many hosted apps being designed today will encrypt all their stored data from the start? I would guess almost none. Even though that is where the smart folks go to steal data: where it's resting. Just sitting there for anyone to grab once they get through the perimeter.
What if all data was encrypted at rest by default? It's an idea worth thinking about.
I'm pretty sure most people have thought about doing this; the issue is that with our current technology, the overhead is too large to be practical. At least, that was my understanding in the matter when I was doing some cryptography related research.
On the other hand, a lot of research is striving towards that goal. A method for fully homomorphic encryption was very recently discovered, and although it is still way too slow to be practical, the fact that we finally figured out a way to do it after several decades of searching is a huge step.
Even if all data is encrypted at rest, you can still learn a huge amount from access patterns. You need some kind of Oblivious RAM (which could perhaps be better named Oblivious Disk) where the cloud server doesn't even know the metadata (e.g. which disk sector) it's serving, let alone the content. Cryptographers know how to do this in theory, but the overhead is pretty big.
Most likely nothing to do with US companies or the "cloud" just specific compliance concerns, the whole sentiment sounds like it came straight of the lips of a circa 2007-8 CTO/CIO of a heavily regulated vertical.
Most real world work is done in privately owned data centers. Hipster's love for AWS or the like isn't going to change that. Netflix rocks on AWS but that doesn't change the reality of data control concerns for most companies in or outside the US.
Yes it is, but it doesn't mean that AWS and similar solutions aren't used for development, DR, and other usecases. When a company uses AWS or Azure it doesn't mean 100% of it's infrastructure is invested in it.
Surely you can mitigate that by using encryption at every phase of your operation? That would be a good idea even if the US government were trustworthy. It's also a good idea if you're running everything on-site; if someone were to gain access to your systems, they get nothing.
If you're using stuff like the IoT Hub by MS, you can't use encryption other than HTTPS, and even if you could, deploying a secure encryption solution to IoT devices is a nightmare.
I have about 5 years experience with both AWS and Google Cloud (App Engine, Google Compute, ..). I agree that AWS has more services to offer than Google. Initially Google took the route of the App Engine and fell behind AWS. However, recently its Compute Engine and other related services like Container Engine, Datalab (IPython, Jupyter as a Service), Big Query, Dataproc, Pub/Sub and more started outshining their AWS counterparts. Google's network is way faster than AWS. AWS network is too slow and flaky. For anyone taking the route of microservies, Google Cloud offers a huge advantage. Google new cloud console (which is still in beta) is sleek and sexy. Google's cloud services are a pleasure to use compared to AWS (other than a few pain points). I recommend small and medium Cloud-based shops to relook into Google Compute again. Google Cloud is adding services to its arsenal at a super fast rate (much faster than AWS). Just take a look at their engineering blog: http://googlecloudplatform.blogspot.com/. In a year I see google Cloud overtake AWS in terms of the capability.
Have been running a behemoth of a web application on AWS for 3 years on multiple regions, using ec2, s3, elastic beanstalk, elastic transcoder, sqs, dynamodb, cloudfront and rds. Have been pretty happy with both performance and cost effectiveness.
That said. I have deployed some newer / smaller stuff on Google Container Engine. Used Cloud SQL and Storage services along with it. I think Google has a solid offering right now. Performance is great and tooling / management interfaces are, IMHO, better than AWS's. I think it has a bright future as well.
Also for us Google Compute Engine turned out to be better for our use case (cloud encoding, c.f. https://www.bitcodin.com/blog/2015/08/how-bitcodin-utilizes-...). For it was nice to have shorter billing periods (just 10 minutes vs. 1 hour with AWS) and better interconnection bandwidth between the cloud instances.
In the most recent quarter it essentially matched the retail side of Amazon's North America business on operating income.
In North America Amazon generated $528m in operating income. AWS generated $521m.
AWS is growing extremely fast of course (78% sales growth year over year for the quarter). In three years it'll probably be spitting off $4 to $5 billion per year in operating income.
No idea where they got their figures from but maybe the shortfall is the difference between pure retail and other stuff, like order fulfillment, advertising, seller fees, etc?
It's just really bad reporting. The sales figure Amazon reports, is its share of revenue and does not include the merchandise value of what its merchants sell.
You can safely bet that Amazon's retail dollar keep, is about eight times the size of AWS right now (excluding their hardware and digital goods business).
Amazon handles about 40% of dollar figure sales of all merchandise on its site (and 45% of all units sold). Amazon breaks out the units sold figure, but the dollar figure is a ballpark guess from analysts that track Amazon. You can roughly double their sales to get the total merchandise value of goods moving across their retail site.
Amazing, but not surprising, that AWS is overtaking the retail operation in terms of profit and growth. For all the minor frustrations involved in running a big operation on AWS (good luck actually provisioning 1000 instances of a single type on a whim during peak hours), the service itself is nothing short of miraculous to someone who ran physical server farms a mere decade ago.
All that said, of course, Amazon's dominance will become a problem if no other competitors find a way to seriously compete. Certainly their newer services are designed to foster lock-in as much as they are made to provide simpler architectures and automated resource management.
But I sincerely hope one of the smaller players grows into a real contender as well. Amazon, at least at first, seemed like a disinterested player making a hedge bet. Their customers were not competitors. With Google and Microsoft the same can't be said.
Any players are good that encourage Amazon to do a better job. I really think AWS improved since Azure and GCE came along. Moreover, when I dealt with Amazon on calls and in meetings, any time we mentioned the Google or Azure words, they immediately wanted to proactively respond (and eventually did). We were a customer of both their competitors so this helped, but I give Amazon some credit.
Regarding lock-in, I think you are right but everyone seems to be doing this.
I think for better or worse, part of what Amazon has done has been adapting what they already were doing in-house into a product. Sometimes when people just come up with products to compete or "just because," they add in a lot of nonsense and architect things badly. Microsoft seemed to be going both routes at different times with Azure. I was an original user of Azure including the various data stores/data stores they asked for at the time. To say everything on Azure, whether the machines, data stores, services, libraries, or anything else were horrible would be an understatement (might be one of the worst major platforms I've used ever). Amazing how they turned it around when they were sure they had to compete stronger.
It's irony manifest that companies would consider moving away from vendor lockin in data centers they own to data centers they do not own yet still locked into one vendor's platform.
Competition with Amazon AWS won't come from where you expect. Amazon owns the public cloud, so whatever it is won't technically be a public cloud offering.
I'm only a small user of cloud providers, but find Google cloud engine much more user friendly than aws, so why is amazon still capturing the market? I don't see the downsides of GCE.
> * Google has earned a reputation for shuttering products with little warning. CTOs really dislike an ad-hoc forced migration.
Exactly why we smiled and said "thanks but no thanks" when their salesfolk hit us up; I can't trust that any given Google platform will still be there in three to five years.
I am a user of Google App Engine and found its performance unsatisfying. I can execute the same Java programs 10 times faster on my local machine than in an app engine instance with the same GHz and RAM according to Google. I.e. what takes 1 second locally (CPU-bound, no i/o), takes 10 seconds in app engine.
Also +1, App Engine is terrible. I worked for one of the largest App Engine deployments for startups. We were Python, Java, and Go and all were terrible. I've never felt like a company was giving me the middle finger as hard as App Engine. It's a terrible product and a shameful dev team.
There's a million things wrong with GAE besides performance, but if we just focus on that, where to even start? When I first took my last job using GAE, I was able to improve the performance of one of our apps in my 2nd week on the job from requests taking anywhere from 3-14 seconds to sub 1 second. That sounds good, but actually the original authors of the app didn't really do anything too bad, it was just they tried to write a standard web CRUD app in Django.
The answer to improving performance was mainly to never query their data store if you could help it. Solid advice for most apps, only the queries were completely unpredictable. Pretty much the standard Google answer for all App Engine is to do everything in memcached. OK, again, fine, but then memcached started being unpredictable. Google Answer - we'll develop private memcached to let you tweak it more. OK, fine, but it's still not as fast as when we run memcached on AWS.
Even when we ran our site with 99.9% cache hit ratio, we still had erratic performance. We next switched things to do more on the client and ran an Angular SPA app. Again we saw benefits of course, but still things sometimes were slower than we wanted. Simply Google's billing API requests and their slow servers serving requests tended to be the problem, which was often something we had no control over. And that was basically the theme of other problems we had. We figured out the requests problems at the beginning FYI and told Google, and they simply said they would work on it, but actually over time things often got slower.
App Engine is a terrible platform and most of the benefits can be replicated by various libraries out there for deployment, versioning, caching, etc. or through admin panels in other services like AWS. If you want to lock yourself into terrible APIs that do things like intercept your POST and re-issue it as a GET, go for it. If you want to start building your apps around billing patterns rather than proper architecture, go for it (how can I do y instead of x to avoid being billed?). I could go on, but yeah App Engine is pretty bad. It gets you going from dev to production somewhat quickly, but don't many frameworks + cloud providers do that now? The only real benefit I found which you could argue is that you alleviate some of the security challenges, but that's assuming you trust Google's environment and configuration (which you have essentially no control).
I try to avoid commenting on App Engine related posts because I'm afraid that, despite myself, I will end up writing a long angry rant and come off hateful and bitter. When I have a bad experience with a given technology or providor, I can usually stay balanced, recognize the history and context, and show gratitude for hard work and good ideas, even those that didn't pan out. Not so with GAE, their demonstrated negligence and arrogance preclude that.
See, there I go.
Reading your rant (and the extended version below) definitely gave me a misery-loves-company sort of smile though, so thanks.
I agree that is what the parent is talking about, however in the context of GCE, Google Cloud Services which include App Engine and a whole slew of other services are relevant. Given that if you've used them, there are integrations and even shared admin areas/dependencies, it is very relevant to talk about how other services perform in the Google Cloud.
While I agree, it seems relevant to mention the experience with it anyway because it seems some people's poor experience with GAE might have actually prevented them from trying GCE. At the very least it helps answer why some people use AWS instead of GCE.
Does GCE still require that you name (vs simply tag) instances via their APIs in order to create new guests?
That and the slightly weird authentication/setup process (rough memory of that, really) were the obvious downsides for me.
Obvious upsides to GCE would be fractional hourly billing and allegedly no-need to prewarm loadbalancers. And probably there aren't mystery settings that require you to email someone to ask them to be set, and there probably aren't features that you can only access through the GUI with API support coming later -- though that is just a guess :)
A downside to GCE would be if you wanted any of the various other services that AWS provides, since they provide a ton more. AWS also has really good service -- Google may also, but I haven't had direct experience.
Maybe this is preference or something changed since I used GCE daily, but I disagree on almost all these points.
* Documentation - On the surface, GCE docs always looked better to me until I realized how many mistakes there were and things out-of-date. Not to mention there was too much of the obvious documentation and not anything regarding subjects where I'd actually want detailed docs. Although Amazon's docs are trash, they have the benefit of everyone and their mother writing about how to do x, y, or z. Whether that's a benefit or not, it depends on the author.
* Console - The console was sometimes more responsive depending on the action, but for some things was worse. Overall I'd give this one to GCE as well, but I think the GUI was a mess in places. Perhaps that's more based on my total usage of Google Services, which are a superset of GCE. Regarding command line tools though, I felt these were awful, slow, poorly documented, and buggy. Again, maybe an update fixed some of this. Google has a really bad history with consoles. Amazon's GUI is 90sish at times, but it's been pretty good to me and the API just works.
* The answer to this one is really it depends on the machines, location, etc. GCE has some advantages here that are well documented, but performance of a running machine was never really a problem I faced on Amazon provided you select a good instance type.
* Can't argue here. Amazon does like to give out its share of freebies though if you are a big enough customer or paying attention to the community.
* There are a lot of blog posts about the problem is that these can be outdated and may not be using best practises.
* I mainly use the GCE console and I found it fantastic. Showing all VMs from across regions is itself a big improvement from AWS console. The project dashboard was a mess. The new beta console is better. I found Google's command line tools much more intuitive and very easy to keep up to date. They have consolidated some previously separate tools into a single gcloud utility.
When using the python bindings, you could not simply request N instances using image ID X. You had to come up with an (account) unique name for each one, like webserver-01 and webserver-02.
It is somewhat fundamentally anti "cattles not pets", which led to some weird code in libraries to try to pick IDs that were not used.
Which services are you comparing? We (sadly) make our pricing table for Compute Engine (GCE) look a lot like EC2. Just looking at rates misses the advantage of per-minute billing for certain workloads (e.g. Batch compute or autoscaled anything) and our "sustained use discount" which gives you roughly the same discount as EC2 reserved instances but without having to do anything. As I said to someone recently, our model isn't super simple to "understand" but it's way easier: we always give you the best price.
The same thing goes for GCS vs S3, general network pricing, etc.
Just curious, do you have anything to do with whatever infrastructure the Apps Script scripts run on? I normally like the Google Sheets app and find it really handy to collaborate with, but just a few days ago, I wrote an utterly trivial script for the first time in Apps Script to do some formatting changes on a small sheet automatically. This is like a 20 line script with one loop that runs through like 5 times, the kind of thing you'd expect should run in under 1ms on pretty much anything. I didn't time it when I ran it, but I'm pretty sure it took over a minute to run, certainly much more than a few seconds. What's the deal with that?
Can you tell me why your disk latency and throughput is so horrible, even compared to AWS? I've seen ioping spikes in the 7-8 ms range and it has far worse throughput than AWS.
Also any idea if your folks are ever going to let people set RDNS records for their IPs? It has been requested but no one at GCE seems to communicate.
Oh wow... I didn't know Google lacks the ability for customers to specify reverse DNS. I had assumed any full-service cloud provider would offer this feature.
If Google doesn't have configurable RDNS, they're not a "one-stop shop" for all hosting needs. For example, a customer wouldn't send transactional e-mails from a server at Google because the lack of RDNS would affect deliverability.
We are most definitely not a one-stop shop yet. We don't even have IPv6 on GCE (I recommend Linode for low traffic IPv6 to IPv4 bridges).
Moreover, (sadly) your example doesn't hold anyway: sending email is actually something all cloud providers try to avoid (SoftLayer in Brazil lately is the notable exception). So getting anyone to whitelist your smtp and other ports is usually an unpleasant experience.
Oh, this actually works great at AWS, yeah! We've been sending transactional e-mails from our servers at Amazon for a couple of years. They do require you to request the e-mail ports be unblocked, but once we made the request it was approved within 24 hours. As a bonus, the same request form also has a space where you tell them what you want your reverse DNS to be. Very handy.
Last year Microsoft also announced Azure support for reverse DNS entries. So I would anticipate Google getting on this "bandwagon" fairly soon too. Just my guess though!
We've got Persistent Disks aka PD (logically similar to EBS / network storage) that come in both a "hard drive" variety and an "ssd" variety, as well as local SSD.
I assume you're talking about one of the PD products, most likely PD-SSD and comparing to one of the EBS offerings (either the new one or a piops one)?
If so, unlike traditional EBS, PD scales throughput according to disk size [1]. You get 30 IOPS/GB up to 15k or so. As far as latency goes, what instance were you testing on? Your VM's I/O competes for cycles with your vCPU (this is true of all KVM-based virtualization), so if you spike up your CPU you can easily "starve" your I/O threads or vice versa.
As to your DNS question, I'll have to look into that. But where are you trying to communicate? (I mean clearly you found me, but I mean where else)
They are different for sure. AWS has the extra hoop of needing 1 year reservations to get fair pricing, whereas GCE gives you the good price just by using something for > 50% of a month. So from a startup POV, loving GCE pricing model.
I've been a big user of cloud providers, and left a startup a year ago that was a huge customer of Google (GCE, BigQuery, App Engine, Google Drive, etc.) and ran one of the larger GCE and App Engine deployments for startups. We ended up having a few fringe services in Amazon and moving a lot of new ones to Amazon. Eventually we decided to leave GCE as soon as we could migrate one of our main sites off of App Engine. I mention App Engine because it shares a lot of the same core issues as GCE in terms of pain points.
Why did we want to leave? Honestly, it was a combination of things starting with the hateful and broken App Engine platform (API, admin, everything) to the usual Google nonsense of cancellations, adding useless features, bad prioritization of bugs, lack of fulfilling SLAs, etc whether it was GCE or another service. After working with 2 startups that were based heavily on Google's cloud, I can't recommend them unless something dramatic has changed. I can go into more details in private and a few publicly but it would be a huge post so I'll mention only a few things here.
Suffice to say, I found GCE to be anything but friendly compared to Amazon at all levels. While Amazon has a confusing and sometimes horrible admin panel, it's been my experience as an AWS customer for a long long time that it just works most of the time. Most of the time sounds bad, but it is better than "rarely" or "never" we experienced with Google (more later). When something is broken, Amazon generally fixes it or even a primitive retry will suffice. If something was really broken, Amazon gave us free credit, while Google either lied or hardly budged on bills (even funnier since Google was one of our investors).
A related and huge problem we had was that Google tended to make platforms, libraries, features, and services very 1/2 baked. Google is perhaps good at ideas and maybe even better than Amazon at building a solid core, but there is absolutely no follow-through. Too many times we encountered libraries and services that were terminated or overhauled because something was fundamentally broken. Worse, we pointed out big problems months or years in advance along with other customers and the community but saw no action. Google's dev team would continue to pretend nothing is wrong until either terminating or going back to the drawing board. I've never experienced a large company burying its head in the sand as much as Google. I love Google search, but I refuse to use any of their cloud products with the attitude that I've seen top to bottom, from sales to dev.
Another very important point I think a lot of inexperience people neglect - AWS has great support when it comes to libraries and integrations. Pick x language and generally you can find at least a working version of a library for whatever service you are using, including new features. Google's cloud services in my experience are filled with broken, slow, horrible libraries, including the official ones. As an example, there was a really bad (probably still open) form encoding bug in App Engine with multi-part forms that's been broken since almost the start of App Engine (basically, anything non-Ascii got corrupted). It's never really been fixed despite multiple people giving Google tons of data, patches, and workaround. We even got on the phone with them about it several times. GCE has honestly not been as bad as App Engine in terms of fixing things, but it also lacks features and library integrations.
As an example of a more specific GCE problem, anything that has issues with multi-cast had to have tons of patches. There was a special adapter for ElasticSearch when we deployed it on GCE that constantly had to be updated, and if you read about it, you'll see it's mainly a GCE thing. Pretty much every service, tech, or library we used that was more serious had similar issues - there was always a workaround or feature you had to manually implement that you didn't realize you took for granted on Amazon. The deployment story just given the tools was always far better on Amazon for example, but that may have changed.
Then there's installing and developing against GCE. Most of GCE for the past few years was command line only. That's fine if your command line is well written and well-documented, but GCE is most certainly not. The command line didn't even really work on Windows and not even really with something like cygwin. You can laugh, but at my other company we did a lot of DirectX and Xbox development, and developing on other platforms didn't make that much sense. Unfortunately some people in our company decided not using AWS would be cool. Developing against GCE once you got things working was no treat and we felt we often had to write tons of new libraries to do basic things that existed somewhere in the Amazon stack.
As far as the documentation, Google's services are usually out-of-date and full of mistakes. You could argue the docs are better than what Amazon has, however Amazon has the benefit of the community at this point that generally can answer your questions quickly. Moreover, Google in its own documentation often brags how good their docs are, while simple things like broken links or deprecated features are usually all over the place. How hard is it to run a broken link checker at least? The community is a huge topic, but I'll add that support was so bad at some points that the official line was to ask on StackOverflow instead (fine, but you're a huge company with a tiny install base at the time, wtf).
GCE did (does) have an admin console in addition to the command line, but the early versions were very limited. The last GCE console I worked with was in-step with most Google admin consoles looks like an ASP 2.0 app. Most things in the admin console are far inferior to the command line and full of bugs. Even finding the right settings in the admin console are a complete nightmare and sometimes require you to open other Google Services that you would think have no impact on GCE, App Engine, or anything else, and yet do. They often re-arrange how all of this works, but then leave parts of the old sites still running, so you end up with crazy nesting, conflicting options and navigation, and randomness. Mostly you feel like some dev somewhere kept saying, "I don't feel like doing this, so I'll put this all here because I can." It all looks like a bad assignment done last minute in a computer lab for a comp-sci class.
I know I seem a bit harsh, but I really wanted to try something new besides AWS the last few years. I joined 2 companies that did exactly that (independent of me), and that ended up terrible for both. There really was not much I couldn't do on AWS that GCE could do, while AWS could do plenty of things that GCE could not do at the time. AWS was far more reliable, stable, and well-supported for us as well, along with my own startup, and my past companies. Given I was a customer of Google's for a long, long time along with Amazon's and little changed with Google, I doubt it is much different now. Maybe someone can correct me, but usually when we get into these kinds of debates, the person with high opinions about GCE and the Google Cloud in general has not been on it long and certainly has not hosted a serious environment on it. I find Amazon as a company somewhat repulsive like Google in various ways, but AWS for all its faults is definitely more popular for a reason still. I hope Google can push Amazon to make their platform better, but from what I have seen the last few years, GCE is half-baked and only suitable if you're throwing a few small or personal servers in the cloud without significant interactions between them.
Sorry about your experience. It sounds like you were mostly burned by App Engine, and the unfortunate situation of a migration from old to new consoles (we have a multi year deprecation policy in some cases). The old admin console was truly awful, however within just the last few months the final nail has been driven into its coffin: you can manage your App Engine custom domain settings and SSL from inside the Developers Console rather than go through Apps.
As someone on a side thread mentioned, we even have a refresh of the new console in Beta for the next month or so (want to make sure we didn't suddenly break some user flow). The integration between products is vastly better than it was 12 months ago and with gcloud (the CLI) and the Developer Console gaining maturity and scope, even three months ago the experience wasn't as good.
tl;dr: Your comment is fair, but things are changing (and have changed) quickly. Take another look with a free trial account?
I'm not OP, but I tried that in a bake-off a few months back. Setup went well, I got some free trial credit-- and immediately found my account turned suspended due to fraud concerns. Note that this was the same payment details I was using (and continue to use) for Google Apps for Domains.
If GCE can't get the simple things like billing handled, I've got little faith in their ability to solve the harder problems.
Curiously, a few months later (this week in fact) I got a message out of the blue telling me that my account was unsuspended. Rather obviously, I had already gone with AWS around mid-summer...
Sigh. Don't get me started on our abuse handling. That said, any system will have false positives and I'm sorry you got hit by that. I assume you didn't bother talking to a person (and I don't blame you).
I think my comment above still stands though: everything has improved a lot (particularly technically).
Dear boulos: I'm saying this to you but it is a general message to the Google Cloud division.
I'm an early GAE adopter, still running a system in it to this day. I've seen your kind come and go many times. Sometimes it's a well-placed engineer, sometimes a technically capable and empowered support rep, an evangelist, something. These people show up in the community, they're empathetic, responsive and openly talk about what goes on behind the scenes. Sometimes they blog (http://ikaisays.com) and go much deeper than the docs ever would. They make you think "google really is different."
And then they disappear. And then there's crickets for a while, maybe some corporate types putting out fires and helpfully pointing everyone at the issue tracker. And then it repeats. Consistency in communication matters a lot, especially when your users are putting their livelihoods in your hands.
People come and go. For App Engine in particular (before my time) both Kevin Gibbs and Jon McAllister who "founded" the product left in 2012. You can find most of these guys over here: https://quip.com/about/ ;).
1. Because Google (it's not just the Google Cloud console).
2. Good point. I'll see why that isn't the case (I agree strongly, it'd be nice to know if this hasn't been used so you can revoke it).
3. A lot of systems require a user/email to identify the "person" taking the action. So this one is a crazy auto generated one for you. You didn't say it, but I'd love to be able to name them (like "Default GCE Service Account" or "Service Account for that Binary I built for deploying stuff").
4. This is something Google Cloud is behind on and we're working on it. For Storage (and PubSub) there actually are much finer grained ACLs (and you could add them as just a VIEWER role, etc.). I don't recall if we yet support the "Bob can use GCE but not BigQuery". Expect updates here soon.
1. Didn't get that. IMO Enabling/disabling APIs should be a separate menu item from Credentials.
2. Ok.
3. Yeah. Naming is a start. I also want to know what does it have access to.
4. Yeah. It's quite messy. Also there are Service Accounts in "Credentials" and also in "Permissions". There are multiple "Google APIs service account" in Permissions. I have no idea what they are for and what does it have access to
The article didn't answered the how part. All but last two paragraphs are the things every programmer takes for granted, second last sounds like an ad and last stats.
But these sites do know how to get people to click their sites.
He committed a bad action. And this led to consequences with him getting fired and the show changing. Not sure why he needs to keep paying for it forever. If you don't agree with the punishment then that's certainly valid.
However just because you made a mistake doesn't preclude you from working for the rest of your life. And he's certainly worth the money for his following and trailblazing he's done in the industry, that doesn't go away because of a single incident.
Well, the question is if the firing is effective when he immediately gets hired by a competitor at a higher wage.
I know, it’s necessary for society to accept that everyone makes mistakes, and that everyone can change, but then the person actually has to use that chance and has to change. Just continuing business as usual is problematic.
(I wonder if a court could have sentenced him to a stress-management course?)
I'm not sure what you're expecting here because all the BBC could do is fire him. And I don't see what the problem is with any other company hiring him because he's actually worth it. The guy led the way for the biggest show on the BBC and that's no small feat. He still has all his skills, experience and following.
I think what you're looking for is criminal charges for his actions but it looks like the other guy didn't choose to file them nor did any prosecutor so that's that.
I bet on a big Hollywood movie a assistant producer screwing up an evening meal for a big star would be fired that day and probably black listed.
And he wasn't fired the BBC didn't take up a new contract at least in part due to lobbying by other media owners who don't like the BBC having a hit show.
Additionally is it fair that James may and Richard Hammond and the rest of the TG crew lost their jobs
No, it is not fair that the others lost their jobs.
And he was fired – the BBC instantly suspended the show and stopped even broadcasting all further recorded episodes. That’s as "fired" as it gets.
And it was not that the assistant producer "screwed up", it was that Clarkson insisted they would stay longer out, then they came so late to the Hotel that the hotel had no warm food anymore, and then Clarkson punched the assistant producer.
And how is this behaviour rewarded? With a raise.
(And the "that other group is behaving immoral, so I should too" defense is in no way relevant)
Well, to quote James May 'Jeremy Clarkson is a complete nob but I like him' And for the next useless tidbit, heard May was holding out for more cash from Amazon for the new 2016 series and got it.
Since when were corporations concerned with moral values? I mean this is Amazon, the company that infamously treats its employees like poorly. FWIW I used to work there and it wasn't as bad as the articles made it out to be, but I would not consider the company to be one that cares about morals or its employees much.
Since when are they not? I'd argue the radical majority of all businesses in the US operate morally. Big businesses are an extremely small fraction of all business in the US and the world. Almost all businesses stay in business because they maintain a favorable reputation. Good moral values (like not stealing from or lying to customers) is a per-requisite for most businesses to keep operating.
To the point: companies like Amazon or Comcast != most businesses. They're extreme outliers to put it mildly. Further, Amazon treats its customers extraordinarily well, and operates morally by its customers, which protects their reputation where they believe it counts.
Try opening a small business and aggressively screwing over your customers. You'll quickly discover how business and morality go together. The only time that isn't true, is when a business is protected by scale (which means it'll take longer for their business to collapse), or protected by government-enforced monopoly.
I don't think companies care about morals. I think they really only care about making money.
In Amazon's case, I know they go out of their way to treat employees well only because they need to. This makes sense, especially for a company that sells things online when ecommerce wasn't nearly as common, and so they had to convince people that buying online was safe and as easy as buying in person.
On the other hand, if companies were actually concerned with morals in and of itself, they would do things morally right that don't benefit them.
Using Amazon again as an example, did you know AmazonSmile is a thing? I mean, it's really great. But, if a company were solely concerned with charity, AmazonSmile wouldn't be a thing; instead they would just automatically donate for everyone. It's a thing because it means that only the people who know about it and care about donating to charity will use it, and their profits will only be lower for people who care about giving to charity.
Anyway, what I am trying to say is, do companies care about morals? Unless they are run by very nice people, no. For the most part, they only care about morals to the extent that it benefits them. Companies are just run by people, and I wouldn't even argue that most people have particularly strong moral values.
They're not amoral entities, they're run by people that make all decisions. Every person involved in a business has a moral compass of some regard, and every decision made flows from that.
If a business can get away with something and profit, they won't do it. See how I just claimed the opposite without any proof?
Saying you know plenty of scummy small businesses doesn't mean more than even a small fraction of small businesses are in fact scummy.
I know very few scummy small businesses. <- That's a meaningless statement and it's true. I'll argue that most businesses, and most people in general, are neither scummy nor looking to steal from, rip off, or otherwise defraud other people. The opposite position requires that one believe most people are bad, which is blatantly not true (if it were, civilization would be impossible).
Being scummy isn't necessary being bad. Price gouging is scummy. I wouldn't say it was evil, just self interest which in the end drives positive behaviour. Price gouging limits demand, and encourages supply.
Of course civilisation can exist with scummy businesses. I would say civilisation couldn't exist without companies seeking self interest.
He's definitely an asshole, and what he did was unacceptable and deserving of getting fired, but as far as celebrities go he's not that bad and he honestly handled it pretty reasonably. The fact is that he was going to get hired somewhere, and it isn't like this didn't hurt him at all.
Can someone shed some light on how much it actually costs to run your own infrastructure? Say that I use 5 large instances of RDS database and then around 100 large EC2 instances along with ELB's. This will roughly cost around $1M a year with AWS. Will running your own infrastructure be costlier than this?
I think this $50k number is way low unless you are talking bargain basement data center. Also, doesn't contemplate needed bandwidth. I agree with the expertise point, though.
Rough numbers:
Server = $5k, straight line depreciation over 3 years, 40 per rack
Switches = $6k, each, "", 2 per rack
Rack = $2k/month
Power = $2k/month
That works out to:
Servers: 40 * $138/server/month=$5555/month
Servers: 2 * $333/switch/month
Rent: $2000/rack/month
Power: $2000/rack/month
"Nowadays, startups just rent AWS servers and storage, and if they suddenly need an extra thousand servers, they just click on a link and put the cost on a credit card."
Where's this magic autoscaling link? Last I checked, AWS provided an extremely unfriendly API on top of extremely buggy services at prices that'd allow you to build your own data center with the money spent on a few months of renting VMs. They are indeed the 90's Microsoft of the cloud: incredibly expensive, buggy, and widely adopted by people that I assume burn cash to keep warm because they're so rich.
Add to that the possibility that one's account will be closed for no reason and without recourse, and you have the most unstable platform for apps imaginable.
> Add to that the possibility that one's account will be closed for no reason and without recourse, and you have the most unstable platform for apps imaginable.
We use AWS heavily and I considered the very same possibility. My solution: redundant cloud providers! We now have servers on standby at another vendor (Joyent), with live database replication to the standby servers. Something happens to Amazon, we'll be okay.
AWS and Azure are now household names in enterprise. Clients who once only allowed on premise IT solutions are beginning to bend to vendors will by using "the cloud".
This year's AWS re:Invent conference was totally focused on winning over the corporate enterprise to the Cloud. Compliance, enterprise-scale resources, and migration solutions were major major pushes in the keynotes and sessions. Day 1's keynote was more for the CEOs in the crowd than for the techies.
Can't speak too detailed because of NDA, but at work we cannot use innovative tools like Google Docs, Google Mail for Business, Dropbox, Office 365 and the likes (and for our privacy-conscious clients, no AWS, only Azure) because the US government can compel the providers to turn over our confidential data to them. We're stuck with Lotus Notes 8.5 and Office 2013 because our IT wants to migrate to Office, but our client NDAs prohibit us from using cloud stuff with their data (again, fears of the clients of US industrial espionage. Snowden actually proved them right).
The NSA and the US government effectively killed their cloud business for non US customers with their spying crap.