ElasticSearch is a complete, distributed search engine. It runs as a daemon, does sharing and clustering, provides APIs etc. Lucene is just a library. ElasticSearch uses Lucene internally.
And I'll say it again: Elasticsearch is one of the few things that lives up to its hype. It really is that good. Indexing, clustering, It Just Works. I'm very impressed with them and it's a refreshing change from other very hyped tech.
I actually have quite a few gripes with Elasticsearch. It's a pain to manage - there seems to be no way of killing long-running queries other than restarting it (so if you have users using kibana etc at some point someone will submit a impossibly hard query and kill the server), the quality of the search results just isn't that good and it takes a lot of customization to improve it - I've never been impressed with the results from any of the large deployments (e.g. github). Ideally it would come with good tokenization and synonyms etc out of the box rather than every deployment having to tinker with settings until it seems to work.
That said ELK is great overall and I would still use it for user-facing search, but I think there is still a lot to improve.
jordan, agree fully with your assessment. i have experienced significant challenges in keeping elasticsearch in green since it just loves to failover at times without prior notice, while seeming to work smoothly otherwise!
Elasticsearch is built on top of Apache Lucene and is very similar to technologies like Solr but with a nicer interface (really more like SolrCloud these days).
It is different in that it provides higher level APIs and distributed functionality on top of Lucene. For example, one Elasticsearch index might contain multiple Lucene indices spread across multiple machines, but your application treats it as a single "index".
