Would you be interested in a blog post about it? I'm a sysadmin/devops with 12 years in, and build stuff like this on a daily basis. I wouldn't mind sharing how the sausage is made.
I'm a PHP dev working with Apache on a daily basis, but besides .htaccess and some minor changes I can't really do much. I would love an article which explained in a simple way how to scale your server and really debug problems with it.
Last week, I literally spun down an EC2 instance and signed up for Digital Ocean because I couldn't figure out how the hell to make stuff work on EC2, but have lots of VPS experience. As a dev and not a sysadmin, it's much easier to go with what you know... but I want to learn.
What's the difference between EC2 and a VPS for you? Do your VPSs already have things installed or a GUI? I've used EC2 before as a single server, never scaling. The main difference was installing things that are usually pre-installed (like on Ubuntu's official desktop image). Is that it or is it more about the scaling?
And thanks for the reference to Digital Ocean. Never heard of them before. Seems great, might try using them :)
> What's the difference between EC2 and a VPS for you?
For me it that restarting an EC2 instance deletes all the local storage. I have had good success just getting a big ass VPS, and running the database locally, and pushing text backups to S3. It is trivial to manage, and in the real world, downtime is more likely to be caused by configuration wonkiness than hardware failures.
You also have to have a huge amount of traffic to overwhelm a 24core / 96GB ram server. Why not put off the managing the complexity until you really are doing 10M page views per day?
If you where having all your data deleted when you restarted your EC2 server, then something was VERY wrong. I'm not an expert, but I've used EC2 a little bit and I think I hit that exact problem.
The thing is that for whatever reason, data wasn't being written to the EBS (the virtual hard disks for use with EC2 instances) and was instead being written to the "ephemeral storage", a really big local data store that every EC2 instance has that is basically a `/tmp` directory. If the server restarts, everything in the ephemeral storage is destroyed.
EBS is a slow turd. Running a database on EBS is running a database on network attached storage. I know tons of people do this, but I have no idea why. Provisioned IOPS will work, but it is expensive.
Local storage on a $5 Digital Ocean plan will do 2000 IOPS, where as that would cost $200/month with Amazon Provisioned IOPS. I know it is not an apples to apples comparison, but it is worth thinking about. Running a database on a local SSD is a good option for many people, and it is not an option that Amazon offers.
Actually there absolutely is an instance type for this. It's the high storage io instance. Not cheap, but if you want local SSD it is definitely on the menu
> Do your VPSs already have things installed or a GUI?
Nope. I prefer straight-up Arch linux. ssh in and go from there.
To add some concrete-ness to the mix, I was installing ejabberd. When it came time to ping the server... no response. I did the exact same steps on my Digital Ocean VPS and everything went fine. I had done whatever commands EC2 expects to open the right ports...
I'm a large user of AWS but, in general, I never feel like I'm in a true VPS. My last experience:
Out of the sudden one of our EC2 instances could not connect to another, causing our HA solution to spun dozens of instances and eventually crash too. It was clear to me that the dest machine was behind some firewall, we went to the security group, the machine was supposed to accept any connection, from any port, any host. The instance itself had no active firewall.
Out of desperation I added the very own security group to itself. It worked for a few hours, then stopped again, I removed, it came back to work and still working (8+ months now)
This is only one of various mysterious events I've seen happening on AWS.
For EC2 security groups you have to open access to both the correct ports and protocols. Ports are a concept at layer 4 of the OSI model, while ping, or more correctly 'ICMP Echo Request', is lower down the TCP/IP stack at layer 3. So when configuring the security group, look for the option to choose Protocols, then enable ICMP :)
> A raw EC2 instance is identical to your average VPS offering.
See, you say that, but then you later say
> What were your security group settings?
I dunno, man, I just want a server. Every other VPS provider works, and I can set up my own iptables etc.
I'm not saying EC2 is _bad_, as I'm sure things being extra mega locked down by default is good overall. But I don't think it's fair to say that EC2 is the same as a VPS.
The solution to your problem was one Googling for "AWS ping" away, and I've had more significant setup differences between two VPS providers than having to configure the AWS firewall to allow pings.
At that point, I had already been googling for so long that giving up was the best decision. Also, when I said 'ping' I meant 'hit via a web browser' as well as ping on the command line.
Furthermore, I don't feel very comfy when doing 'sysadmin via Google,' who knows what stuff I'm screwing up?
I had never used Digital Ocean before. But getting going with them was the exact same as my previous Linode, Rackspace Cloud, prgmr.com, and every other VPS provider I have tried. I don't need to install special command line tools, or set up security groups, or generate .pems... I ask for a server, they send me an email with a password, I log in, change it, and set up my ssh prefs. Super easy, using the same stuff I use everywhere else.
The big difference between traditional VPS and IaaS services like EC2 (or Google Compute Engine, etc.) is that the latter has dynamic scaling (and a pricing model built around the assumption that you will use dynamic scaling) as a core feature; for traditional VPS-style work on an IaaS, you are likely to end up paying a premium for flexibility you aren't using (and possibly dealing with some attendant management complexity from the same source), but, other than that, IaaS should be a complete substitute for VPS.
I would absolutely love this. Can't get enough solid sysops info - as a dev I find the docs, tutorials, etc out there fairly esoteric. Anything written recently and actually finished helps a ton : )
I have quite a bit of devops experience myself, especially on Amazon Web Services and their CloudFormation service combined with Chef. I would be very interested in a blog post (or a series, go nuts!) and I'm considering doing a series of write-ups on our setup as well.
I would LOVE to see a detailed breakdown on how an experienced sysadmin would set something like this up. I've cobbled together systems before, but I've found that doing so in a robust way is difficult.
