If you are in the tech space, or even just someone that has turned on the television (I hope not!) in the past few days, you will have seen nearly a trillion dollars of paper value wiped off the market in a day because China came out with a faster chatbot than anything the west has yet conceived.
This would be pretty interesting in and of itself, however Deepseek has done this with three caveats:
One, they trained it much cheaper than anything previously done in the Western world. They claim to have done so for under $6 million USD.
Two, their model is currently the one with most parameters (671 billion) on the market (as of this writing). Edit: in my video I was corrected that o4 is a 1-1.5 trillion parameter model
And Three. The big one. The one that matters most: The model is open source.
So, people are rightfully losing their minds in the tech ogilopoly in America that we have allowed to run rampant. Someone did it for free. And now everyone can run a model on their own hardware (albeit pretty beefy hardware is needed.)
Not only this, but Deepseek released papers on the research and development of the model, and this is massive for the holistic advancement in the space.
If you are not in the world of Linux and open source systems and software, you might be saying "what is the big deal."
To which I would reply, this is massive.
Arguably, the "good guys" are winning, not because it was China that did this, but because they allowed the model to go out into the world for free, so everyone can use it. Those that would want AI models locked behind the doors of one company or country are missing the point entirely.
The news is calling this a "shock", however, it's been known for over a month in the LocalLLM community that this model existed, with many on Twitter cancelling their ChatGPT memberships in favour of the open model.
A couple thoughts on this:
There is no "AI" moat
Those that think that AI companies have a USP (unique selling proposition) would be mistaken here. Their UI and some tweaking is really all they have on the next guy, and we are dealing with a market that is going to be commoditized. A commodity market is one in which it matters not at all who you use or buy from, you get the same thing, just as you do with your ISP or phone company.
This is what I think we will see this year - it won't matter who you buy from or what model you run, they will converge. So - if the open source one is best now, why not run that?
There will be no "winner" in the AI space
I apologize for my language and judgement here, but people who believe that a country needs to "win AI" and control it are idiots.
AI, if it is to become what people say it is to become, is necessary to be in the hands of every man, woman, and child in the world. If it is to create the "paradigm shift" everyone is saying it will, then these models cannot be in the hands of just a few people. They need to be as widely spread as the internet itself. So, companies that are trying to keep this closed source will lose. I assure you of this, and the only way they wouldn't would be if the States regulates AI development. Open source will still win, just not in the technocracy that is the United States.
While the internet as we know it has some big "winners" that was not the intention of the internet - its open protocols allowed anyone to create, share, and talk to everyone else on the internet. The "openness" is what led to the widespread adoption and impact the internet has had, and continues to have.
Opensource means uncensorable
Because we have a model that anyone can run, we now have a way to audit, change weights of parameters, and remove censorship.
While people were laughing at the "censorship" of the Deepseek model (ask it about Taiwan!) this was not entirely true - if you run the model locally, the censorship is next to non-existent.
Even if there was censoring going on, on an open source model, anyone can change the weighting, and there is already a 32B parameter version on [[https://huggingface.co/huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated ][Hugging Face that is uncensored]].
Opensource Needs to win
As it always has since the inception of the computer - FOSS needs to win.
The reason is simple - the control of the means of production of idea and creativity needs to be in the hands of humanity, not in the hands of a few people. As with AI, we can augment many more people with the technology when it is in the hands of many more people, simple as.
The inefficiencies of current models is starting to show
We just keep throwing more compute at the problem, but what needs to happen now is optimization of the modelling. What I mean by this is that we keep throwing more powerful hardware at the model, but the model itself needs optimization (likely in a language that is not python). Those that start doing this optimization will see orders of magnitude increases in the "productivity" (if we can call it that) of LLMs. So, someone needs to write a Rust library and move away from the legacy libraries that have been running this research for some time.
It keeps getting more interesting
I have been in the space for a couple years now, albeit on the peripheries. The LLM space is interesting not in a "hype way", but in that I think we have a better mousetrap for search. Google could've had this in the bag years ago, but now we are seeing companies with under 20 employees create modelling that is breaking the stock market it seems.
The space is moving very quickly, and by next week this post will likely be out of date, but until then, it's good to see open source in the limelight for the time being.