I’ve been talking a lot about MemGPT and AutoGen lately, but I still think their potential is underappreciated. In this post, I will explain why I believe projects like AutoGen and MemGPT, combined with the increasing breadth and ability of small open-source models, could pose a significant threat to ChatGPT.
We Have No Moat
A few months ago, I made a video about a leaked internal memo from a Google engineer, which claimed that open-source models would become a significant threat to closed-source models for a plethora of reasons, including iteration speed, developer resources, innovation, rapid increase of open-source model quality, and more.
One primary reason for open-source LLMs being a threat to Google and OpenAI is that they can be fine-tuned inexpensively to accommodate any vertical use case. Here’s an important quote:
“Suddenly, anyone could fine-tune the model to do anything, kicking off a race to the bottom on low-budget fine-tuning projects.”
This is playing out in real time, but it took a different form than anticipated. Yes, incredibly performant and relatively small models are being created, with Mistral 7b being a recent (and incredible) addition. And these models are being fine-tuned to excel at specific tasks. However, what I didn’t see coming was projects like AutoGen and MemGPT, which are enabling the coordination of many fine-tuned models. This is the same architecture that GPT4 uses.
With the combination of agent teams, which perform better than single agents, and their ability to tap any fine-tuned model for a specific use case, we now have a quality similar to GPT4. But that shouldn’t be a surprise because, as mentioned earlier, GPT4 is actually (allegedly, as shown by internal leaks) 8 separate fine-tuned models working in coordination depending on the use case.
Cost Is The Biggest Risk To ChatGPT
There’s no doubt about it: GPT4 still reigns supreme when it comes to consistent quality, especially for the most complex use cases, such as function calling. But open-source is very, very close.
That last 5% or 10% quality gap will be ignored by developers choosing which platform to build their next AI app (OpenAI vs. open-source). OpenAI is extremely expensive, especially when considering sophisticated use cases like building agent teams. This week, OpenAI will likely be announcing a price reduction for their GPT4 model, but it still won’t be as inexpensive as open-source. With open-source, the benefits are just too great at this point.
The cost to run an open-source model is minimal, with the only thing needed being a GPU. And as more small models are released, this requirement becomes even cheaper. Then, you have all of the other benefits of open-source, such as transparency, ownership, privacy, security, and no platform risk.
What’s Next?
We will continue seeing new open-source foundational models released, especially with companies like a16z providing funding to open-source researchers and Meta releasing open-source AI rapidly.
From a technology perspective, I’ve been thinking about the ability of AI to generate synthetic fine-tuning datasets and fine-tuning models for specific use cases in real time. I suspect we are still far away from that scenario, given the time it takes to fine-tune a model, but at the speed AI is moving, maybe we aren’t that far.
This week, the Biden Administration released an executive order detailing how they want to handle AI, and it’s clear it was influenced by “AI Doomers” (people who think AI has the potential to destroy humanity). The EO detailed many ways in which training AI models will be limited, which is like banning certain types of math (huh?). Many people who believe AI will doom humanity work at closed-source AI companies, such as OpenAI. This is also known as “regulatory capture.”
The scariest scenario is a world where only a few of the biggest companies are allowed to create cutting-edge AI models. We have to protect the AI open-source community.
