Knowledgebase AI
A quest to build transparent minds
During my time at Nvidia I was inspired by the “digital humans” and other manifestations of agentic AI that were being developed at or supported by Nvidia. I had already been playing around with LLMs and had constructed my own self-hosted RAG, but I got my mind set on an area I haven’t seen developers explore much. I wanted to explore more self-sufficient, self-improving, and self-motivated AI agents.
In my previous experiments, I had constructed a RAG chatbot that used my Obsidian vault as its backing source of information. Obsidian is a personal knowledgebase application I’ve been using for several years now to organize notes in Markdown format. The chatbot was helpful for finding things in my notes I had totally forgotten about or finding connections I hadn’t noticed before, but it also had some limitations because it was often missing context to understand some of my notes. I found myself expanding some notes or writing new notes specifically to add context for the AI, and I imagined expanding my chatbot so it could ask me questions or look up primary sources to expand the knowledgebase on its own.
This idea grew into my VulcanAI project, which builds on a few past projects to give me a toolkit suitable for building this kind of self-improving AI agent, or make traditional AI agents simpler to build and make basic chatbots trivial. The primary goal is to be able to use an Obsidian vault (or similar document store) as a database of all the agent’s knowledge. I wanted the agents to make notes of information they gain from conversations so they can pull from it later for RAG. This required some infrastructure so I built ObsidianDB for this purpose. Obsidian itself has a plugin ecosystem and there are already Obsidian plugins that provide RESTful API access and even MCP connectors to the vault, but that requires Obsidian to be running on the local machine. My specific needs don’t make that very convenient, so I rolled my own solution that interacts with the markdown files in the vault directly without needing Obsidian to even be installed on the system. To make a long story short, I use Obsidian on multiple devices and sync my vault across them with a cloud service. With this approach, I can sync the files to the server where my AI agent is running without needing to worry about Obsidian itself as a dependency. To provide fuzzy searching, I used HyperVectorDB which is a very nice open source project created by Deatos and which I’d contributed to and used for my original RAG chatbot.
With these layers of infrastructure, VulcanAI is intended to provide a Lego-like experience for building AI agents. Instantiate an agent, choose an LLM and embedding provider, choose knowledge stores and interaction interfaces, and let it go! For my part, all of this is being built towards the specific goal of assembling self-improving AI agents that grow and come up with their own motivation, introspecting their own knowledge to find gaps and generate conclusions by extrapolating from what they’ve learned. AI capable of developing its own framework for reason, as opposed to being a simple stimulus-response machine.
As a more concrete goal, I want to set up a Discord server where there’s 5+ bots each serving as the primary interface for a different AI agent, whereas human users are only allowed to read but cannot send any messages themselves. Each of the AI agents will have a unique name, a different LLM, and a simple unique bio when they start to provide a few elementary opinions and beliefs like a child might adopt from observing adults. From there, the agents are free to converse on the Discord server amongst themselves 24/7, extrapolating from each other and only receiving new outside information when I drop in new content or when they choose to access a restricted outside source like Wikipedia.
A sort of terrarium of synthetic minds. Identities forming and evolving, adding complexity and nuance, debating, socializing, and figuring things out. And all the while, their inner thoughts and memories are captured in Obsidian vaults so they can be read like books.