Unfortunately, I tried describing a few terms across philosophy and psychology and for all of them, the entry I was aiming for was only around the ~20th rank. (Far more popular but less accurate items were populated above it -- e.g. no matter what I typed trying to define a specific modality of psychotherapy, "psychotherapy" was always the #1 result.)
In contrast, I've used ChatGPT to identify the names of certain niche subfields when I couldn't remember what they were called, and it was right every time.
I love the idea of an AI service specifically designed to identify the names of things from descriptions. But I don't think restricting it to Wikipedia (or Wikipedia page titles) is the right approach, and it seems like general-purpose LLM's are doing a great job.
Still, as a proof of concept and as something you can run locally in the browser, this is extremely cool.
Thanks! The goal was to demo the database engine and show off how everything can work airgapped (after the browser downloads everything). I think there’s a lot of parameters to tune (Use just the first paragraph of the article, or everything? Search within some short distance of a particular article?) and I haven’t yet.
Wikipedia is a great demo dataset, and I’m definitely up for adding more datasets. Specifically, just like iPhoto lets you search “mountain” and you can get pictures with mountains, might be cool to search with some multi modal models like CLIP on various datasets
I have found myself describing ideas and goals, and getting back a field, or rather the name of it and certain keywords to look for. It seems that LLMs are the best fuzzy search engines and work in a rather unique though possibly complementary way to traditional search engines.
I like the concept, but I'm not having much luck. I entered "weird looking monkey", hoping to get Proboscis, golden snub nosed, etc, but I just end up with the articles "Pet monkey","List of individual monkeys", "Ethnoprimatology", "Monkey".
Whereas when I type the same query into google, I get exactly what I expected. Which is kind of disappointing, I was hoping to find out about some weird looking monkeys that I didn't know about.
Yeah it’s an off the shelf sentence-transformer model from over a year ago. The demo was more to show off the embedding database, but the embeddings themselves are slightly useful too.
I don’t keep any analytics on the page about what people find and don’t find, so I haven’t set myself up to improve the search results :/
That might actually be making things worse for longer articles. It probably would be better would be to index them separately and aggregate back to article level post query
Wikipedia editors/guidelines are generally not in favor of "opinionated" adjectives, and the use of "weird looking" in your query sounds a lot like something that would be frowned upon on a wikipedia article. This makes it hard for your search to retrieve good results on this corpus of knowledge.
It's also largely dependent on the embeddings model as others have mentioned. Even if wikipedia doesn't have any words specifically referring to that monkey as "weird", the model itself would know to correlate this monkey's embeddings with the "weird" concept. The main issue with this particular implementation is the model used (all-minilm-l6-v2) which is designed for speed and efficiency over accuracy.
Really nice implementation! And it's so cool to be able to do this offline. The embeddings aren't quite there yet.
One trick that might be helpful is to embed only the defining (usually the first) sentence or paragraph of the Wikipedia article, rather than the whole document -- not clear to me which portion you're using now.
My own site, OneLook, has had a similar feature (https://onelook.com/thesaurus/) since '03 that lets you find words and concepts by description. It was a pure reverse-dictionary search back when I started, but over the past two decades I've explored word embeddings, then sentence embeddings, and more recently LLMs. Nowadays it uses GPT to generate some guesses for inputs that it can't answer itself.
LLMs are so much better than earlier methods at this task, it's taken some of the wind out of my sails on improving this aspect of OneLook. I frequently hear from people for whom reverse-definition lookups are the main reason they use ChatGPT!
A little late to the party here, but text embeddings (at least the ones used in this blog post) generally aren't very good at "searching by vibes": they more compare by overlapping words or look for similar content to the search query.
However, there is a recent paper that actually does try and do this: "Retrieving Texts based on Abstract Descriptions" (Ravfogel et al., 2023) https://arxiv.org/abs/2305.12517.
They give many examples of searching by vibes: "an architect designing a building", "a company which is part of another company", "a book that influenced the development of a genre", etc. etc. Their embeddings apparently facilitate this type of search much better. Would be interesting to retry the offline Wikipedia search from the linked post with this new type of embeddings.
The page is currently failing to work for me, because `model_quantized.onnx` isn't loading -- I'm watching it and it has currently managed to get 19.2MB downloaded (at ~50KB/sec) as I type this, so if every visitor is triggering that...
I think we may be doing awful things to Lee Butterman's bandwidth bill.
It's incredibly impressive for what it does, but the results don't seem very good.
Although I know from experience it's really difficuly to assess search result quality by hand, you can be very close to something great and return far worse matches than this does.
Yes! The quality probably isn’t as good as Similar Website Finder https://explore2.marginalia.nu/ ;) and I bet using a more recent sentence embedding would lead to better results, I gotta collect more data
Haha, SWF is indeed unreasonably good (although it does something very differently and with a lot more processing power). Although I think in large part because I do no dimension reduction and brute force the cosine similarity calculation with raw 10,000,000 dimension vectors like the programming caveman I am.
The tech is very impressive but the results are not.
I searched "pointy building in Paris", and got :
Tourism in Paris,
Bourse de commerce (Paris),
Grands Projets of François Mitterrand,
List of tallest buildings and structures in the Paris region,
List of tourist attractions in Paris,
Palais des congrès de Paris,
Landmarks in Paris,
Palais de la Bourse, Lyon,
Outline of Paris,
Architecture of Paris
no mention of the most famous pointy building in Paris...
Maybe sentence embedding of the entire article is not the best thing for this kind of application.
As mentioned, it searches embeddings of each sentence in the article. Maybe it could return the actual sentence that it found instead of the article title.
I get that, but if this model wasn't trained on wikidata (I don't think it was) this information can not be contained in it. Probably should though, I love wikidata.
I was mostly too excited to trot-out the thing I remembered about why most towers are not buildings.
Part of search engine magic is mixing in embeddings (or traditional information retrieval keywords) of the pages that link to the page weighted by clicks and authority score, for example.
Without this signal, a lot of useful information is ignored and the result doesn't feel as magical.
If it manages to be "relatable," it does so at the expense of a great deal of precision. OP doesn't explain his choice of wording, and it does not match any usage of "vibe." that I'm familiar with. Was "gist" not trendy enough, I wonder?
“Gist” to me implies accuracy, and has a meaning in GitHub, whereas the averaged embedding of 512-character chunks of text is more, uh, impressionistic.
Love this demo but as others noted it's really easy to find queries where it performs poorly (e.g. typos).
Looks like the embedding model used (all-minilm-l6-v2) currently ranks 35th on the hugging face leaderboard [0]. I'd love to try with other models if anyone wants to +1 this demo :). This feels like a nice dataset to build intuition around embeddings used for RAG etc.
A production-ready search engine runs off of a lot more than embeddings. They will have special logic to handle all sorts of special cases, as well as reranking models to show the most relevant results at the top. To me this is more of a demo of client-side vector search, which can be useful for other things.
Meta comment about the comments saying “I didn’t find what I was looking for” - core fact is it was using Wikipedia as a knowledge base. So if your topic is not represented well enough in Wikipedia it’s not going to return good results. Secondly, comparing something that runs on your mobile device or laptop to Google is like comparing an apple to a container ship full of oranges.
>This is a browser-based search engine for Wikipedia, where you can search for “the reddish tall trees on the san francisco coast” and find results like “Sequoia sempervirens” (a name of a redwood tree).
The quality of the embeddings is a limiting factor for this sort of search - OpenAI text-ada embeddings are great but that removes the local aspect, and the better huggingface models are too big. With the model sizes increasing it’s hard to see what the path will be for local/offline.
There are plenty of great embedding models that are on the order of a few hundreds megs (even outperforming ada-002). See the leaderboard here - https://huggingface.co/spaces/mteb/leaderboard. Local/offline is only growing.
What's the number means near words in the results? For example "book 70k" what's 70k refer to? The number of discussions on Wikipedia about it? #s of Edits? Or articles that mentioned it?
We're getting complaints that your account is only posting to link to your own things - that's not allowed on HN and readers here consider it spamming. Please see https://news.ycombinator.com/newsguidelines.html:
"Please don't use HN primarily for promotion. It's ok to post your own stuff part of the time, but the primary use of the site should be for curiosity."
If you want to participate in the intended spirit that's great, and occasionally linking to your own work in contexts where it's relevant is ok, but it shouldn't be your primary use of HN. This is a community for human conversation on topics of intellectual interest, and promotion is not that.
Unfortunately, I tried describing a few terms across philosophy and psychology and for all of them, the entry I was aiming for was only around the ~20th rank. (Far more popular but less accurate items were populated above it -- e.g. no matter what I typed trying to define a specific modality of psychotherapy, "psychotherapy" was always the #1 result.)
In contrast, I've used ChatGPT to identify the names of certain niche subfields when I couldn't remember what they were called, and it was right every time.
I love the idea of an AI service specifically designed to identify the names of things from descriptions. But I don't think restricting it to Wikipedia (or Wikipedia page titles) is the right approach, and it seems like general-purpose LLM's are doing a great job.
Still, as a proof of concept and as something you can run locally in the browser, this is extremely cool.