A centralized system around building software #

Imagine a system that can capture the idea you want to build, help you build it, then help you maintain it. A complete system to help companies to with the development process end to end. What would that look like?

Do we still need human input? #

Before I describe what I envision, I would like to address this paper I read.

Communicative Agents for Software Development 28 August 2023 https://browse.arxiv.org/pdf/2307.07924.pdf

This paper describes a process where multiple LLMs are used to simulate roles in a company. Effectively automating the entire company, a user can simply declare a prompt and the agents within the company will manage to build this software. Highly exciting, I recommend reading this paper thoroughly.

But I think humans want to be involved in this process. How should we design the UI? What are the business metrics to optimize? What tradeoffs in architecture are we making? Making software is not the same as driving from point A to B. How you get to the destination matters. I don’t think you want to entirely automate the software development process because it usually requires future iterations. Maintainability is critical.

Therefore, assistive AI is more important than automation AI in software development or business building. I don’t want an AI that is not responsive to my input, I want to work together with AI to build the best software possible.

AI should gain domain specific knowledge over time #

As humans work with AI more on a specific software, AI should gain domain specific knowledge. This is critical not just because you don’t want to repeat yourself, but because you want AI to communicate on your behalf. Once a software is built, there will be bugs and questions. You don’t want to be answering 10k tickets and debugging yourself. You want the AI to handle that for you. Further, when AI knows what you are building, it should give constructive advice on building additional features since it helped with development in the first place.

Fine-tuning #

So far fine-tuning a LLM model seems promising. But OpenAI wants the training set in QA format, question then answer. To build such a dataset, one idea is to encourage the LLM to ask questions at the end of every prompt. This creates a cycle to generate data for further finetuning. This might be helpful in answering questions, but sometimes the new data isn’t in the form of QA. Imagine, I added a new feature with some additional code, how would I communicate that to the LLM? What about updating AI knowledge from new documentation?

RAG #

On the subject of RAG systems, what makes me uncertain is that the LLM is simply retrieving the information, so it is not clear how much the LLM can truly understand the information it retrieves. If the retrieved information requires additional context, can the LLM synthesize such concepts? One possible solution is to use a proxy agent to ask these questions for additional retrievals recursively before returning to the user. But I have yet to experiment with this.

UI with AI #

Lastly, I want to re-emphasize the UI with AI.Typing your wishes into a chat interface is quite annoying. It feels like having to explain everything to that dumb guy on the team all the time. Sometimes it is faster to just do it yourself instead of explaining. So there should be buttons for common requests and more sophisticated interfaces for multi-modal interactions.

In summary, I feel like a crazy person in the room talking about this idea. Enterprise AI system, you mean like skynet for companies? What if they got hacked, isn’t that an info-security issue? Absolutely we are creating more security vulnerabilities. But if they can be addressed and fixed, is that a good reason to not take any risks?