I would add that it is pretty easy to get started with on Rails as well. Using the Sunspot library for Solr is super easy to use and has been really flexible for our use cases. It is a solid piece of software.
Maybe it's just me, but over the years Solr and Lucene have become an impenetrable project. When I first started playing with Lucene many years ago it was very simple, transparent code backed by solid and understandable algorithms and data structures. Now of course, it was limited.
But now Solr has absorbed Lucene and it seems to do everything and the kitchen sink. The abstractions in Solr in my experience are mindbogglingly complicated if you want to extend its capabilities, because it does so many things.
I have yet to look at 4.0 but I hope they have removed as much as they have added. I think if the focus for the next release of Solr was to remove lots of unused, poorly factored, or complicated features it would be a huge benefit to the project.
Yes, a bit like Java itself, I see it as trending towards being an "Aircraft Supercarrier Battle Fleet" of functionality. (See also: the Hadoop ecosystem.)
But, it's free, powerful, and supported by a giant community... so once you get the hang of picking and choosing the parts you need, ignoring the rest until needed, you can get impressive results with minimal effort.
You can always continue to use Lucene if you want to write the bulk of the search engine functionality yourself, and just need an indexing+search library.
It does do a lot of stuff, mostly that only gets in the way once you start scratching around and realise there's a lot to figure out.
For me one place it could really be simplified is the basic config it comes with. When you first approach the project the config files are so complete that you just have no idea where to start.
Then again, once you figure out the basics you have an amazingly powerful search engine that performs admirably and can scale to massive loads.
You can use ElasticSearch (based on Lucene - http://www.elasticsearch.org/) if Solr/Lucene seems to complex. I wasn't very successful in getting a stable setup when I tested it (more than a year ago), but as you can see from their web page, they aim at making things easier.
Updating a single field in a document is a pretty big feature. It will require a much slimmer and less monolithic pipeline for big-data indexing from multiple sources.
I agree with some of the other new and exciting features already posted.. but I'm surprised no one has mentioned that the Solr Cloud was taken out of beta! That is pretty awesome and we will be looking at this feature closely for sure.
The spatial enhancements are pretty exciting. You can support a lot of interesting things when search can be polygon queries against polygon data. WKT is a pretty fatty format though.
Yeah, that's what I was thinking. Either way you (potentially) have a lot of data to push around, but encoding things as text that could largely be fixed width binary floats really bloats them up.
I'm really excited about all the new features in 4.0. The more I read, the more I can't wait to try it out on my dev server. The new features that are most interesting to me: spatial search improvements, atomic updates, custom scoring options, pivot faceting, speed improvements and Solr Cloud. All these have the potential to make searches on $work's website much more relevant and faster.
API is cool and dandy, but I think project should go in other direction. Lucene focus should be REAL support for languages, lack of proper dictionaries, lame stemmers makes it hard to use.
For me Lucene was always an NoSQL database and the rest of real search stuff I had to implement myself. (not to mention Solr mess).
It's fun and cool to work on some Java code architecture, but the real need lies somewhere else.
Basically, its a search engine; if you need to run one of those yourself, it's really neat.