Don't use any service provider that does not align well with what is important for your business. If this includes guaranteed service availability and solid customer service and the company in question just can't provide these guarantees, well, don't use them.
It really is that simple.
I have not touched anything Google except for Search and Analytics for probably three years. I learned early-on where my business needs did not align with their offerings and we don't use those services. I couldn't be happier. From my perspective Google is a great company.
There are other approaches to all of the other services. We host our own email for a number of domains on different private servers. Easy. A license of MS Office on every machine isn't a problem if you have a real business, just like a license of the appropriate Adobe suite is almost a must. Tools like GoToMyPC, for the technically challenged are a no-brainer and they are cheap. There are a multitude of cloud storage solutions. And, frankly, for a lot of stuff, there's nothing wrong with hosting your own access-controlled FTP site to share files with your team (although a paid Dropbox account is oh-so-simple).
So, yeah, Google is a good company. Just stick to the stuff that works the way you need it to and you'll be fine.
>We host our own email for a number of domains on different private servers. Easy.
From what I've heard other people say, running your own mail servers can be described with a lot of colorful vocabulary, though "easy" not being one of them. Do you know how much it costs you guys to run your own mail servers (both in actual "server has power" as well as technical ops cost) compared to using Google Apps? I've heard numbers in the 10x - 100x range.
It really depends on how comfortable you are at Linux systems administration, and how ambitious you want your mail server to be. If you're going to have to rely on any (really, any) of the guides or howtos online, then expect to budget in many hours of problem-solving and knob turning.
We currently spend very little time each month on ongoing maintenance for our mail server, and we have a stack that includes good spam & antivirus systems, webmail, easy user administration, and hourly offsite backups.
But, I've got probably over a hundred hours of my time sunk into all of that, and there's still a lot more I'd like to do to the server.
So, from a billable hour standpoint, you probably can't justify hosting your own mail versus Google Apps (or just about anything else), unless you're specifically looking for functionality that Google doesn't provide.
You don't have to run it yourself if you don't want to.
Plenty of people will take that work of your hands for a reasonable price. Oh and they will give you an actual phone number you can call.
Huh. For some reason, I'd never considered that people might be willing to pay for a remote "on call sysadmin". We already admin a handful of mail servers, that's something we could add to our services.
When you have services like Intermedia.net doing a lot of the heavy lifting for you, it's insanely easy to sell mail as a service. I will say though, Intermedia is a LITTLE trigger happy with their routing security; one of our PC's had a malware infection, already quarantined and was pending removal. Our entire IP range was banned until it got cleaned up, and the machine itself was a DMZ'd test machine that we purposely infect regularly for "war games" etc, so it had no email clients, didn't even send any messages.
There's a reason for this. It occurred to me the other day, after dealing with a stupid hover.com problem, that it has become almost impossible for network admins at different networks to talk to each-other.
I have never, not once, in the five or six years I've been doing network admin work, been able to contact another network admin to report a problem. Instead, I have to go through incompetent and clueless frontline support first, and spend hours or days navigating that until the problem is no longer relevant anymore or I give up.
It became obvious that I wasn't the only one that had given up on contacting network admins when I recently had to deal with a spam issue (same spammer, multiple hosting providers, new technique) -- while I tried following the RBL rules, it was clear that other service providers entirely skipped the "notify the network admin, give them reasonable time to resolve the problem before nominating their block for the RBL" step. the victim networks all became listed on the RBLs within just a few hours of the first waves of spam.
It's just gotten to be too much trouble, nobody bothers anymore, and unfortunately that will have to include me from now on too. If I see bad behavior coming from another network, I won't any longer even try to contact anybody at the other network; I'll just ban their IP and move on.
Your post is spot on and I've felt every frustration you've outlined. I maybe should have clarified my post originally to suggest I'm not entirely suggesting this is a fault of Intermedia, as they don't know our internal systems architecture. But getting the issue resolved and being able to get the message relayed that the machine posed virtually no threat to an admin who could have done something about it was just as much of a chore as you've just explained.
Granted, the hilarious irony in this is that it was Intermedia who pointed out "there's an infected machine" on your network, so all of the wrangling around and sending tracert outputs, just to get a reply weeks later "This IP address is infected" and the resulting "That's what it was? That machine is just for testing, we know it's infected." was a bit of a grind.
Who cares what the multiplier is. If GA costs $1/yr and your own mail server costs $100/yr (just making up these numbers), it's still worth getting right. Just focus on the bottomline cost to yourself, and figure out if it makes sense.
For my personal mail I use Fastmail. $40/year for 10 GB of mail. I have far more faith in things getting resolved with either of those companies than were we to be using GMail and email is too important for me to not have that trust.
Does fastmail include phone support? Their support link takes you to a form and points to online documentation. This seems like it provides less support than Google.
Google Apps works great. It is an incredible product. Google Apps is backed by a 99.9% SLA and they almost hit 99.99% last year. Customer service has improved vastly over the last couple of years and Google now provides 24/7 phone support. Do you get that kind of support running your own servers? No planned downtime? Your suggestions are dated. FTP isn't an easy thing for most users..you still need local programs to support whatever filetypes you are opening.
Many businesses rely on Google Apps..over 5 million to date, including some very big and important companies with extremely smart and technical people making the decision to switch from running their own servers to Google's solutions. Their evaluations of these services go far beyond what anyone here could ever imagine..I know, we setup Google Apps for some very large organizations (over 25,000 employees).
It is a great set of services that can be extremely valuable to organizations small and large.
Remember, to receive prompt support for failing Google services, you need to contact Google via a story at a famous blog, a major social news site, or (ideally) The New York Times.
I have been a paying GAFYD user for ~ two years and wasn't even aware there was an 'account PIN'! I fear for most people this simple solution may come with the benefit of hindsight.
> "Please note that for your security, your Customer PIN will be updated periodically.
So even if I find it now and write it down, there's a chance that I still won't be able to access support at some point down the road if/when I need it.
Good. The more this happens, the more people will realise that contracting out your critical services to a company with virtually no support and a false reputation isn't a good idea. It also undermines "the cloud" which is a good thing IMHO.
Sorry but I have little sympathy and I don't want to hear the "well it works for me excuses" and the financial advantage crap.
Yeah, let's ignore the success stories and the financial benefits because it what is sort of a race scenario, Google's support failed. This seems like a bug and resolvable as noted by others in the thread.
Yes for data. Data is easily recovered, but under the circumstances that your cloud apps suddenly become instantly unavailable, how do you recover your applications?
You don't - you have to migrate and retrain everyone, which if you've used all the features in your cloud platform requires major process and tooling rewrites.
With an offline solution, you can shelve it and migrate slowly. When someone else hosts it, you're at their mercy.
You mean google apps? Well, you could keep a local copy of office around, it's not exactly rocket science.
When someone else hosts it, you're at their mercy.
And when you host it, you're at your own mercy.
Matching the availability of google tends to cost a little more than a hypothetical day of downtime would.
There are plenty valid concerns about hosting your office in the cloud. However, availability and TCO are normally the main arguments for google apps, not the other way round. Those infrastructure costs really add up when you're talking thousands of seats.
OK what about Google Apps Script, Forms etc? You're stuffed there. That's where the rocket science starts to kick in. It's not as simple as it looks. Oh and as part of the cost cutting, can you afford to splash out for Office company-wide suddenly out of the blue?
If you have thousands of seats, you should have the budget lying around to do it properly and a team of people to handle it. The TCO if you include risk is an order of magnitude lower if you manage it yourself. I've done the figures enough times for enough companies to know this.
Most companies operate "on the line". An event like this would take them offline permanently.
Well, no idea how you calculate that. Google would have to be down several days every year only to offset the staffing costs for running your own equivalent to google apps. And that's not even getting into hardware, licenses and the expensive windows-lockin that everyone seeks to escape nowadays.
I guess we'll just have to agree on disagreeing here.
I see no obvious chip. If cloud apps and data can become unavailable with no immediate support response, they shouldn't be critically used by a business that desires to stay in business, period. For non-critical apps the cloud could be fine.
Are people still surprised by Google's lack of support?
I think one of the messages here is to take note of any support codes (account #'s or PINs in this case) and store them outside of the service that requires them.
> I think one of the messages here is to take note of any support codes (account #'s or PINs in this case) and store them outside of the service that requires them.
These codes "will be updated periodically", so there's no guarantee you'll have the correct one if/when you need it.
I use Office 365 (Exchange Online) and made the mistake of using my own email address as the primary Exchange Online address. I was unable to access my email for close to two weeks. Other than that they've been better that expected.
What I'm saying is that unless you do it yourself these things will happen. A service level agreement can only be viewed as "this is what we promise as our best endeavour", but when it goes south it goes south, and you're at the provider's mercy.
I made the mistake of regarding"the cloud" as this "machine" that's impervious to human error.
I haven't read the article. First thing that I saw when visiting this blog/site whatever was a long pageover list with checkboxes and email input fields asking me to subscribe to something. There was no close button immediately visible, so I just closed the tab instead of proceeding.
Sorry, but if you really want your story visible, please stop treating your visitors like that.
Of course you could do that, I also do that sometimes, but after being welcomed in such an unrespectful way I just didn't want to bother, since I assumed the quality of the article is the same as "welcome" message.
markokocic I run the CTOvision site and I apologize for that horrid popup. I was experimenting with a new plugin that did not work right and I really screwed it up. But I pulled it down as soon as I saw that.
Google has a well documented history of it being next to impossible to talk to a real person unless you have an $X0,000 hardware search appliance or similar. As an individual, if you're not internet famous, you're taking your business into your hands using Google for critical services like these.
That said, not everyone looks at the level of support provided before they choose a service. So, if you get blindsided with this type of thing, you do a quick search to see if it'll get resolved quickly, realise it won't, change your MX records and learn your lesson / plan for a migration out of the service if/when you get your account back. Just like with any problem of this nature.
Usually it's just worth paying for something more expensive (and often less featured) for which you know that there's someone you can call when the service goes down in the middle of the night or you get locked out for some random reason.
edit: I'm really only being overly harsh because the guy who wrote the article is the CEO of a cyber risk management and security company, and should know better.
Something similar happened at a company I work for; the account of the CEO suddenly becomes "temporarily unavailable".
I take their word for it and 12 hours pass and the account is still inaccessible. We're a paying customer, so I open up a ticket and it is eventually passed on to the "technical team".
I give them a call but they cannot offer any additional details nor any timeframe whatsoever when this will be resolved. Another 20 hours pass and my requests for any update go unanswered. Finally, after a total of 48 hours the account is available again. Still no update from Google.
Having been completely cut off for two working days, the CEO is now (correctly) questioning their use of Google Apps so I update my original ticket requesting any additional information they have that could assist me in justifying staying with the service. They tell me this will be forthcoming.
A week passes, no update. I reiterate my request and 10 hours later I'm told that the cause was a "server issue"...
It gets better, even if you change the MX entries, there is no guarantee that GMail will deliver mail to those new servers because the domain may still exist in their system. So you may remian cut off from a significant number of people.
No, spindritf has a point. If a MTA is configured to consider a domain as "local" it will route emails internally instead of doing MX lookups. It's likely more complex in this case, but essentially it can mean that any emails routed through Google's systems (Gmail and Google Apps for Domains) will keep bouncing as long as the domain is configured, even if you've changed MX records to a different MTA.
Based on the quality of other Google products, I think it's safer to assume the inverse: Google's mail servers likely know about a domain's MX records and adjust accordingly without account configuration or external intervention.
Maintaining a system as large as the one they do without this feature would seriously suck for the mail admin - it really has nothing to do with the customer at that point. I wouldn't want to work that job if it did because of a) all the angry people it affected and b) the overhead in my job introduced by a bad programmer.
This is a fact seen in practice. You can check it if you want. Just create Google Apps account and configure mail without setting any MX records in your DNS. You will be able to send and receive emails to and from that account and regular Gmail account.
Not a complete counter example, but on a domain I run with external mail, gmail enabled but MX never setup, sending a email with google apps gmail, from me@mydomain to you@mydomain sends the email to the external provider
I believe spindritf is indicating that google doesn't use the "open internet" for gmail <-> gapps, it's all routed internally. If they find your domain registered within their system they skip the lookup.
> Is that speculation or do you know it will happen?
I had troubles a few years back with e-mail routing from GMail to mail servers for a domain that had previously Google Apps set up for it but was subsequently moved to another provider. They disappeared after deleting Google Apps.
Maybe it's not an issue any more, maybe it was just some weird caching problem, maybe disabled domains are handled differently but it wouldn't also surprise me if they had some system for handling what is essentially mail internal to GMail.
Exchange will ignore MX records and deliver directly to accounts that exist in AD. This is true for some other email systems as well like Mirapoint. I'm not sure about Gmail.
When I set set up my email account at google apps few years ago my MX records were messed up, so no mail could be delievered to my new google app address from outside gmail. However, my regular test gmail account was happily sending emails that were received on my apps account although MX records were messed.
Thinks might have changed in the meantime, but my guess is that Google still sends emails directly for accounts that it "knows" are hosted on gmail and apps and happily ignore MX records.
You just made this FUD up. Google uses MX records to send to Apps customers, because some of them run their mail through third-party services and proxies.
Imagine an events services company (like the one I am working for) which is 100% dependent in business operations from email and website availability during the event or marketing phase. It would be an assassination for us, and I thank the author for the warning. We shall not use Google Apps as a company for this reason.
I've been looking for a functionally equivalent alternative to gmail largely because I fear ending up in a similar situation. I'd like to host my email with a company that has remotely competent and caring support.
What are the best services?
And no thanks to cloud haters - I will not be setting up my own imap servers.
Rackspace Email (formerly MailTrust). The service itself is excellent (IMAP/POP/Exchange hosting, spam filtering as good as Google's, a 100% uptime SLA, daily backups), and it comes with Rackspace's well-known 24x7x365 phone, chat and ticket support. I've hosted my mail there for 7 domains for years.
When I was doing IT consulting before starting my company, we used to contract a great Florida-based company called AppRiver for hosted exchange (plus IMAP and POP, naturally) for some of our clients.
The service was excellent, and their customer support was even better -- always extremely friendly and very responsive.
I used to use fastmail but switched to google as their interface fell behind the times and I wanted features they didn't have. Perhaps I should look again.
Just for my own personal curiosity, has any company made a reasonable attempt at competing with Google apps core services? I know Microsoft offers similar services, but it's not that well "put together" the last time I looked.
You're probably not going to be able to compete with Google Drive integration, but if someone put together a well designed integration of inbox, calendar, and contacts with an API, they would do well.
Zoho actually has quite a nice offering of web based applications. They have mail, calendar, contacts, documents and various additional tools. I used it for some time, but switched back due to a lack of sync of contacts with my phone. But for a business, it looks promising, though not as flashy as what Google offers.
I've been using Apple's Server.app (Formerly OS X Server) for quite some time now at my home. Its definitely a bit of work to maintain, specifically on the CalDAV and CardDAV integration. But failure is something I would hardly attribute to the setup. It's certainly not any worse to use than configuring and maintaining Postfix/Dovecot/Roundcube and CalendarServer individually.
In what way would you suggest Apple's Server.app has failed (outside of the obvious hardware compatibility)?
I referred to MobileMe solution when pointed about Apple failure.
You are quite right on your opinion, but the question was whether is a viable alternative to Google Apps somewhere. And yet there is nothing on horizon except some distant plumes from Dropbox, and now I must add Podio and box.net.
This is not really a well thought out response. Most businesses don't have the resources to employ a system administrator to manage their Windows install, let alone manage all the other services required to run to meet the same level of functionality that "ready to go" services like Google Apps provide. Seriously.
Does anyone have a current best-of-breed list of web enabled applications (preferably open source) that you could run on your own server? Ideally, I'd like to put together a kit that is easy to install, either on your own server or on any hosted provider. Bonus points if the kit includes replication tools built in so that you can use multiple web hosts, and have your own redundant cloud.
Derek, this would be a cool project. If someone could work alongside a talented UX/Designer and bundle together some open source software with a nice UI, that would be quite useful. Roundcube seems to be taking a pretty good stab at this.
im not suggesting every google apps customer should do this, but if you had purchased google apps through a reseller, said reseller could have helped resolve this issue within a couple of hours, most likely. without a PIN. also, why wait so long to use another domain's PIN to contact support? why not do that on day 1? lastly, if you change your MX records away from google apps, your mail will route to wherever you point it. google looks up MX like everyone else, for each domain. every time you send an email to or from a google apps account, it goes outside of google and then if it is an apps/gmail account and mx is pointed to google's mx, it goes back into apps. only then. could support be better on google's end for normal paying customers? yes. so could every company's IT support. just go through a premier reseller next time. most charge the same price ($50/yser) and include support, especially for issues like this.
I wasn't aware of this PIN either. Even if you do change your MX record, and move your email elsewhere, you still get locked out of access to your old email, which can be important.
I have a (free) google apps account for my domain, but I configured it so it actually still goes through my own MX server. It took a bit of trickery, because google wants you to use theirs, but I managed to get around it. It works well for a couple of years now. My email always goes first through my own MX, and then to google.
I would still be pretty upset if they blocked me though. But I guess I can't expect much from a free service.
I am currently unable to login to one of my domains on Google Apps (i.e., www.google.com/a/example.com returns an error). When I try to register it, it says the domain is already registered. Fun.
It isn't my primary domain, so I haven't bothered to pester one of my Googler friends to see if they can do something about it.
Similar thing happened to me; it turns out someone else had somehow setup an account a couple years ago with my domain (I've owned it since 2000 or so). I was able to get control of the google apps settings for it (they give a couple options, such as adding a DNS entry). However, it was useless because the previous person had set it up in Russian and I couldn't find a way to change it.
I had the same issue and contacted Google Apps support using PIN of another account I already had. I told them the issue and they basically cleared out all the internal Apps data of the previous domain owner. After 48 hours I was able to set it up as new with Apps.
As I mentioned in the follow-on comments, no service is perfect but I think Google needs to try harder for instances like this. The account was disabled and was also the Google Apps admin account. In that instance, I think Google is at least obligated to kick-off an email to the Google Apps back-up email address and alert to the issue with a time-expiring form that triggers a support request with a slightly higher priority. This could all be automated I would think.
I realize Google is huge and they provide critical services to many of us (I continue to use Google Apps for several domains) but the "can't be bothered" customer service attitude can have a significant impact on operations and I wanted to generate some awareness about that. Looks like we succeeded on the awareness front.
Yet another great example proving google's bad customer support.
What is google trying to prove by such pathetic customer service? I have heard complain about almost every google service: admob, adsense, google app ....
I haven't had any experience with Google customer support, but here's an experiment. Go on Yelp, look up some highly-rated restaurants and shops in your area. You'll still find that many of them have one or two reviews about how terrible the service was, and how the product was awful, and the owner was a dick about providing a refund. Sometimes things go really wrong, and even the best will in the world can't fix it quickly. Only those stories make it to HN, because we all love a dogpile, and because people seem to have realized that bitching publicly is one of the best ways to get their problem solved quickest... although sometimes it's like faking a heart attack so you can see a doctor immediately, then asking him to take a look at a rash.
There's a big difference between a company providing bad service on one occasion, and a company actively blocking someone for days or weeks.
Don't imagine a restaurant providing the wrong food, burnt, and still charging full price. Imagine a restaurant taking the customer's iphone and keeping it in a safe for a month, employees ignoring them and telling them to leave. Are you really going to see many stories like that on Yelp?
Agreed. My biggest issue wasn't that an error occurred, but the fact that there was no recourse available in a timely manner. I tried to be patient and follow recommended support procedures (post to the forums, etc) but losing email for over a week because Google has an internal issue and couldn't be bothered to resolve it didn't sit well with me (thus the post to CTOVision).
Not being a part of the Google ecosystem is always a good thing for business.
I grief with the author for the loss, and really encourage to use Rackspace, which has more than enough human support options.
"No App services – Calendar, Docs, Drive, and all Google App services are completely blocked. These are not critical for this domain in particular, but would be high impact if denied for some of my other domains."
Email is the only thing that got broken AND mattered. And having all your eggs in the same basket has the implication that the owner of the basket can take your eggs as hostages, so you better be careful when you talk to them. Also, remember to carefully read the ToS every day because they can change without notice and f*ck you up.
Right, only one egg mattered. It doesn't matter what happened to the rest of the eggs. If Google only had the email egg, it would have been just as bad. It was not a problem of centralizing the eggs. Google had to take just one hostage to cause major problems.
Unless you have a suggestion of how to split the email egg into multiple baskets?
Google has similar attitude on providing customer support for all its services. A quick example, I have integrated my Sprint number to Google Voice, all my messages and phone calls go through Google Voice, for past few months I have stopped receiving text messages on my phone. I only get them through Google Voice. There is no support, Sprint has no idea, and Google is not reachable.
This whole discussion (post was written by my friend) and my personal experience says to be very cautious about dependencies on any service provider. A simple smart hack would be to auto forward 100% of emails to a different account just for backup, then have the domain registered at a non-google DNS so a fast change to an MX record could be made.
The reason is fairly simple. As a small business, I need my email to work. When I do it myself or outsource it to a small company, it works most of the time ... but not always. I can't afford that.
Also it simply makes no sense spending time setting up, configuring and maintaining email servers as a business owner when there are more important things to put your time in.
This is unbelievably not true. Installing dovecot or similar is super easy. There are many options.
Unfortunately, setting things up to the point where your email isn't instantly tagged as SPAM by Google's, Yahoo's, and Microsoft's servers which handle the bulk of the email you want delivered ... is a monumental feat of engineering.
Frequently, they just mark it as SPAM when it doesn't come from one of the big boy's servers. What's the point of sending an email if the recipient will never have a chance to read it?
Are you sure? I've been sending email from my workstations for years and never had an issue. Maybe if I'd let spammers relay through them for a day or two I'd get blocked. Even that would just be temporary. Let's see some proof that you need a big boy's server to successfully send email. The only real problem I can think of is that a lot of ISPs block 25 out.
While I agree that installing Dovecot, and perhaps Postfix, and some other tools can be rather trivial, I switched to Google Apps when I was world-travelling with my family and my VPS provider back in 2008 basically disintegrated off the face of the earth.
Google Apps to the rescue. I created a Google Apps account, update my MX records and mail was restored in about 15 minutes (plus any DNS propogation time). This made Google Apps a huge win for me. It was supposed to be temporary, but am still using to this day.
But, I still have a Linux box or 2 that could handle email within 5 minutes if needed.
I used to run my own mail server. The main problem is spam: even with aggressive filtering, my mail was still 90% spam. Getting SMTP working is trivial. Getting useful mail is significantly more difficult.
As the article indicates the same is not always going to be the case with google.
With my email I spent about 6-8 hours setting it up a few years ago and haven't had to touch it since. Never had issues with spam or deliverability. Any decent hosting company should be able to provide the same, might cost more than google though.
We migrated from Exchange (50 accounts) to GApps 2 years ago and I don't want to go back to Exchange. Of course costs savings were one reason but main benefit came from the easiness and level of maintenance.
To recap: In order to receive support from a human being for Google Apps the account holder needs her account associated PIN, but since this guy was locked out of his account he didn't have it available to take advantage of the phone support. So after a week and by using some trickery he got hold of someone on the phone and the issue was resolved.
I’d argues that Google Apps has reasonable support (bigger accounts, I assume, have direct support lines) and these occurrences are relatively rare considering the user base, but they get amplified rather loudly, especially when there are plenty of cloud haters, and Google haters (and competitors) ready to pounce.
There might be some risks or trade-offs involved but they’re much less pronounced than some of the comments on any “Google support sux” thread lead you to believe. I haven't had an issue with my personal Apps account but my case is statistically insignificant, same as the case discussed here.
I don't agree that their experience is equally as statistically insignificant as yours. To me, a company should be providing an acceptable level of service to all of their customers, and they must have systems in place to make sure that huge blunders don't happen, even in rare cases.
Your experience doesn't prove that Google always offers good support. The occasional stories we hear about Google are enough to prove that they sometimes offer terrible support.
Don't use any service provider that does not align well with what is important for your business. If this includes guaranteed service availability and solid customer service and the company in question just can't provide these guarantees, well, don't use them.
It really is that simple.
I have not touched anything Google except for Search and Analytics for probably three years. I learned early-on where my business needs did not align with their offerings and we don't use those services. I couldn't be happier. From my perspective Google is a great company.
There are other approaches to all of the other services. We host our own email for a number of domains on different private servers. Easy. A license of MS Office on every machine isn't a problem if you have a real business, just like a license of the appropriate Adobe suite is almost a must. Tools like GoToMyPC, for the technically challenged are a no-brainer and they are cheap. There are a multitude of cloud storage solutions. And, frankly, for a lot of stuff, there's nothing wrong with hosting your own access-controlled FTP site to share files with your team (although a paid Dropbox account is oh-so-simple).
So, yeah, Google is a good company. Just stick to the stuff that works the way you need it to and you'll be fine.