AWS also forked ElasticSearch into their “OpenSearch” DBaaS. It caused some issues at my last job because OpenSearch limited us to a particular version of the NEST .NET library that was missing some newer functionality. Real bummer and feels like a step in the wrong direction given all we’ve accomplished in tech over the last 20 years.
It lacks so many improvements and advancements since the ancient version it was forked at, but because AWS already has an org's payments details, teams often refuse to look at Elasticsearch.
Even basic things like autocompleting queries have been WIP for half a decade now:
Teams should refuse to look at Elasticsearch. It's license is SSPL and they ship free and non-free features in the same binary. It's a ticking time bomb to run it in your company.
Also you can just keep your data in postgres and use paradedb and stop having to deal with dramatically more expensive infrastructure and the JVM.
The reality is that open search will be (if it is not already) more widely deployed and “battle tested” with bugs that production use raise resolved in it.
Opensearch has been great so far, no issues ever since deploying the very initial forked version. Neither of those links seem like dealbreakers, am I missing something? Is the idea that opensearch is not usable in production because of missing autocomplete?
> Is the idea that opensearch is not usable in production
No one said it's not usable in production.
> because of missing autocomplete?
We have an operations team that wants to do searches across 200+ fields for an embedded device's logs. The engine supports it just fine, but what kind of UX is it to expect them to do manual lookups of the fields available?
People with simple use cases of course can't imagine how important discovery features are.
Of course those aren't all the parity gaps, a random sampling of the ones I banged my head against:
- No Log Stream view, also critical for observability operations with any semblance of a reasonable UX
- No wildcard type, critical for machine generated logs having sane searchability. Searches are literally broken otherwise by false negatives.
- No nested fields in visualizations, can't visualize properly structured logs.
- Can't change indexes on visualizations, need to recreate the entire visualization.
- Can't use underscores at the start of a field name.
- Doesn't support auto refreshing fields which again, is terrible for embedding device logging
Elastic moved past basic search since the days OS forked it at, and now it's a genuinely nice choice for observability.
There's a literal report I wrote on the gaps there to justify going to Elastic before giving up on our slow RFP process. Every gap no matter how small is representative of what's wrong with OpenSearch: they don't have 1/10th the incentive to actually put comparable resources to Elastic behind it.
Especially when you have people lining up to make excuses based on the fact they're clueless about the gaps between them. Literal droves of people using it to provide a middling search experience to their users just don't see anything wrong with it.
Query autocomplete is a feature of the Kibana web interface, not of the ElasticSearch database itself. Which isn't to say that it isn't useful, but it's more of a niche utility than a core feature of the stack.
Maybe you're unaware OpenSearch covers Kibana's functionality via OpenSearch-Dashboards? Just like the rest of X-Pack under OpenDistro pre-name change
It's not exactly a niche utility for observability unless you plan on hand searching hundreds of fields. But of course see my other comment for a list of the other observability fumbles they've made.
Elastic chose a pretty great time to start to give observability attention, and OS didn't keep up there. Meanwhile search is becoming more and more focused on integrating semantic search (which Lucene isn't particularly excellent at)
What I'm getting at here is that there are use cases for ElasticSearch/OpenSearch beyond log collection and analysis; many of them don't involve Kibana at all.
