Ask YC: Good resources for dev/staging/production config

[whalesalad]

Looking for some good information regarding keeping development, staging, and production separate from one another in order to balance development of new features and maintenance of the current release.

Any and all info is greatly appreciated. For the record, however, we're using SVN and Django. Three developers on the project.

[wastedbrains]

We use SVN, Rails, and Capistrano.

If you are using rails I highly recommend capistrano.

We each have our dev environment on our local machines including the DB.

We have a staging server which is at our place the staging server is the same OS (Debian), apache, ruby, rails, and updates as our production server. (We always perform system updates to this box and do test deployments before going to production).

We have our production servers off site.

deploying to either environment is fully automated with capistrano.

cap staging deploy cap production deploy

We mostly develop on the trunk, but when working on really large features that might take a few days or break significant amounts of the code / test, we then work in a branch. We often want the trunk in good shape in case any large bugs show on production and we need to start to work on fixes right away.

This works well for us, I am sure there is similar tools for Python and I do think keeping a separate staging server around your place is worth the trouble.

[iamelgringo → wastedbrains]

From what I've read. Capistrano is not Rails exclusive. You can use it to deploy other frameworks as well.

[whalesalad → iamelgringo]

Yeah, Capistrano is language exclusive, you can use it for anything. I have looked into it but never found it that exciting. I guess I need to revisit Capistrano :)

[tlrobinson → whalesalad]

Language agnostic?

[whalesalad → tlrobinson]

Garrr. Yes, that's the word I was looking for. Thanks :)

[iamelgringo]

I really like VMware's virtualization. Workstation is well worth the price. You can set up several server VM's on one box, and then use that for Dev, and Staging boxes if you like. And, you can set up a separate SVN repo on a different VM as well. If your box starts to bog down, it's as easy as copying and pasting the VM to another machine without losing anything.

And, if you want to deploy an entire VM to Amazon's EC2 service, Enominalism sells a tool to do that: http://www.enomalism.com/features/amazon-ec2-migration/

Also, it's possible to set up virtual networks between separate VM's, so you can model a 3 tier architecture and test your DB + Middleware setup if you like as well.

It's worth looking in to.

[nostrademons → iamelgringo]

+1 for VMWare.

If you're on-the-cheap, you may also want to check out VMWare Player and the virtual appliances. I had a Workstation license, but it was from nearly 4 years ago and didn't work with many of the newer Linux distributions (notably Ubuntu). So I downloaded Player and an Ubuntu Gutsy virtual appliance, and was up-and-running in about 5 minutes.

[tienshiao → nostrademons]

VMWare Server is free and allows you to create new VMs (and comes with the VM addons for various OSes).

It allows you to start up VM at boot time, etc hence the "Server" name.

The drawback is that it installs a bunch of services.

Microsoft's VirtualPC 2007 is (was?) free and more of a single package, but I found VMWare Server to be faster.

VirtualBox is also available and open source.

[graywh → nostrademons]

I've used http://www.easyvmx.com to create new blank machines instead of downloading the appliances and installed my own system from scratch.

[whalesalad]

Thanks everyone for all of your info. Let me explain our situation and it might help some of you guys understand where I am coming from and where I am trying to go, as well as others out there in the same situation.

Right now we're developing, as I mentioned above, in Django and use SVN for everything. We have DB's locally.

From the get go we knew that there would be two major versions of our website. "Phase1" was fairly simple, the initial creation of the site and initial deploy into production.

"Phase2" Was the next giant step, which was essentially (and we learned this as we went) going to be an entire redesign from the ground up. We pretty much touched every file, redesigned over half of our models, etc... So, we created a branch called phase2-development and continued working. This was a pain in the ass because we still had to run and maintain the older "phase1" code locally for bug fixes, etc.. as well as continue our future development of the "Phase2" code. We also ran into problems later on because we were fixing bugs in the trunk "Phase1" and not merging those as we went into our phase2-development branch. So at one point we spent an entire weekend merging code.

We just launched our "Phase2" stuff a month or so ago, however, it wasn't all of the functionality we wanted in phase2 so we refer to it as phase1.5. Our head guy (more marketing/business xp. than tech came up with those names, so for all of us you can think of it as s/phase/version/g)

So now we're about to dive fullspeed into actually completing phase2, and but we still need to be able to maintain and fix bugs on the production site. What should we do? I would rather not run two copies of the site but since the new development we're going to be doing is going to be even crazier than going from v1 to v1.5 so not sure if that's even possible. Should we just create anther branch and develop on that? I'm not even sure the correct questions to be asking here, maybe you guys can help me from what I've described :)

[jgrahamc]

What we've been doing is the following:

Source code management system (we use SVN) with two branches: development and production. Anything goes on development and we use the ElectricCommander system to do automatic build and test off of the development branch.

Code that's being released to production is manually merged to the production branch and ElectricCommander runs build and test.

To deploy we have a staging server that takes the output from the build system and we can use it for final testing if we want to do something manual (mostly we don't). Then the build system is instructed to deploy the code (which in our case is rsync over ssh to our servers).

The servers will pick up the new code by cycling virtual machines (basically all our servers are virtual and when the VM starts it picks up the latest code).

[tlrobinson]

I too am interested in this. I'm a relative newbie to this all.

At the moment, we have a simple script that pulls down the latest (or any) revision and deploys it on the production server. Our app is mostly client side, so there's not a lot of complex backend logic, which makes things easier.

I highly recommend having separate dev and production servers for anything but the simplest app. SliceHost slices or other VPS works well for this. Just make sure you get one with enough memory. We were regularly wedging our 256MB dev slice, which was a pain in the ass.

[jonvaughan]

With 3 developers do you really need to be working on Trunk and a Branch? Branch for every prod release but resist the urge to work on it unless an absolute disaster strikes, instead do regular (say every 2 weeks at most) from Trunk. Also - as the other commenters say - automate everything, and use a continuous integration tool e.g TeamCity

[jonvaughan → jonvaughan]

regular releases, I meant to say

[ropiku]

Slicehost has a articles about configuring a server http://articles.slicehost.com/.

I use git, Rails and Capistrano. Git is nice because everybody has a copy of the repository and in case the server is down or you are offline you can still work.

[nreece]

I've been using TortoiseCVS ( http://www.tortoisecvs.org/ ) for a while on some small-to-medium scale projects. Its working out pretty well for version control and branching.

[fleaflicker]

Roll your own. A lot of open source projects out there are overkill.