Hacker News new | past | comments | ask | show | jobs | submit login

Could someone point out to me (non superficial) differences between various service discovery projects that have popped up recently like,

[1]. etcd [2] skydock [3] consul [4] zookeper ( ok this is not new) [5] flynn

They all seem to be doing the exact same to me.

1. https://github.com/coreos/etcd 2. https://github.com/crosbymichael/skydock 3. https://github.com/hashicorp/consul/ 5.https://flynn.io/

Edit: flynn uses discoverd (https://github.com/flynn/discoverd)




Etcd and Zookeeper provide essentially the same functionality. They are both a strongly consistent key/value stores that support notifications to clients of changes. These two projects are limited to service discovery, there are a number of projects that depend on zookeeper (that theoretically could be ported to etcd) for various distributed state management such as Kafka, Storm, and Hadoop.

Skydock (Skydns) and Consul provide automagic service discovery primarily through dns. They use a strongly consistent backend (think Zookeeper or Etcd), to keep track of what servers are active and what applications they run. They then expose a DNS that routes through said servers. So lets say you had a client application that would talk to a node application that could be on any number of servers. What you could do is hard code that list into your application and randomly select one, in order to "fake" load balancing. However every time a machine went up or down you would have to update that list. What Consul provides is you just tell your app to connect to "mynodeapp.consul" and then consul will give you the proper address of one of your node apps.

Consul and Skydock are both applications that build on top of a tool like Zookeeper and Etcd. [1]

Finally all these tools, you can essentially consider "ops" tools to manage the lifetime of application, services and servers "in the cloud". What a developer ideally wants to do is just push code and not have to worry about what servers are running what, and worry about failover and the like. What Flynn provides (if I get it), is a diy Heroku like platform that makes use of tools like Etcd, Skydns, Docker(?) and others. With Flynn, I believe the goal is to radically simplify ops. Another project that I believe may be similar to Flynn is Apache Mesos.

[1] Technically Consul and Skydock, IIRC don't actually use Etcd or Zookeeper, but both implement an underlying protocol to achieve the same thing - Raft, but they could use Etcd or Zookeeper instead of implementing the protocol themselves.


Consul actually uses Serf rather than implementing it itself.

But a good overview, thanks.


Consul actually uses serf for communicating between edge nodes. The central servers (think your zookeeper cluster) use their own implementation of Raft for internal communication.


Flynn isn't a service discovery library by itself. It's more of a self hosted Heroku. Though, it does use etcd for service discovery.


Are you sure flynn uses etcd for service discovery instead of its own[1]? Looks like it uses Google Omega instead of RAFT which etcd uses. https://github.com/flynn/discoverd


You can read more about service discovery on the Flynn site: https://flynn.io/docs/architecture#service-discovery

discoverd is an API on top of another system. Right now, Flynn uses discoverd backed by etcd, though they indicate that this could be replaced with Zookeeper, etc. due to the mdoular design.


As I understand it Google Omega and RAFT are two different things. Google Omega is Google's answer to Apache Mesos. Omega would need a service like Raft to understand what services are currently available. In short, Google Omega is not a service discovery service.


Actually, I think Google Omega is the next-generation scheduler after their original Borg scheduler, which in-turn inspired Mesos.

The design of Omega is quite different than Mesos. I really do recommend reading the Mesos & Omega papers, they are quite interesting.

And yes, Raft is a consensus algorithm for keeping a set of distributed state machines in a consistent state. Mesos actually uses ZooKeeper under the hood for master election, which uses its own consensus algorithm (ZAB - Zookeeper Atomic Broadcast).


Very first line of discoverd's README:

    It's currently backed by etcd[...]


totally missed that. thank you.


Check this out for an excellent and detailed technical comparison of consul vs zookeeper/etcd/doozerd/chef/serf/skydns/etc:

http://www.consul.io/intro/vs/index.html

Also, discoverd is pluggable, but currently relies on etcd.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: