Hey HN, exciting news! Our RAG framework, Cognita (
https://github.com/truefoundry/cognita), born from collaborations with diverse enterprises, is now open-source. Currently, it offers seamless integrations with Qdrant and SingleStore.
In recent weeks, numerous engineers have explored Cognita, providing invaluable insights and feedback. We deeply appreciate your input and encourage ongoing dialogue (share your thoughts in the comments – let's keep this ‘open source’).
While RAG is undoubtedly powerful, the process of building a functional application with it can feel overwhelming. From selecting the right AI models to organizing data effectively, there's a lot to navigate. While tools like LangChain and LlamaIndex simplify prototyping, an accessible, ready-to-use open-source RAG template with modular support is still missing. That's where Cognita comes in.
Key benefits of Cognita:
1. Central repository for parsers, loaders, embedders, and retrievers.
2. User-friendly UI empowers non-technical users to upload documents and engage in Q&A.
3. Fully API-driven for seamless integration with other systems.
We invite you to explore Cognita and share your feedback as we refine and expand its capabilities. Interested in contributing? Join the journey at https://www.truefoundry.com/cognita-launch.
I find it relevant to what I want to do next and put in some time to understand the application vs other stuff e.g. Langchain. And if my understanding is correct, what this tries to do is:
For a lot of typical web services, there're non-realtime batch-processing data processors, e.g. search engine's crawler and indexer, or database's OLAP system, Hadoop, spark, etc. Once their processing is done, they will output data in arelevant, easy-to-use form for real-time web services to consume, e.g. search engine's index, or a list of e-commerce's best selling items.
If we extend such analogy to today's LLM RAG application and compare it with an out-of-the-box Langchain or LlamaIndex implementation, we'll realize everything is in one process altogether. Of course, for demo purpose, they have to.
Cognita tries to fit in by splitting the process into real-time and not real-time parts, on top of existing LangChain and LlamaIndex, and comes with an API endpoint for each part and a web UI for user querying.
For my use case, I'm looking into setting up a very basic RAG-based internal doc QA app, to see if this helps with some of our notoriously bad wikis. So I'm likely going to use this UI and just shovel whatever simple LangChain or LlamaIndex implementation into it. I'm not that interested in the modular design. Honestly, I could see a couple of different ways each market segment approaches such a problem: for demo/mainly static document/low stake application, the need to periodically refresh vector-db is non-existent; for companies with enough engineering expertise, they'll likely put the data processing part into existing data processing framework; for the rest segment, they probably can also get away with putting the whole offline data processing into a very long python script, setup cron and call it a day.
---
I haven't look into RAG in a year or so, but my overall sensation is this: 1. the RAG layer (on top of vector-db) isn't technically difficult, vs say OS development, database development, etc, after all, text manipulation has been around since 60s. 2, since the whole LLM generation is very sensitive to prompt, an early, too rigid abstraction likely do more harm than good.