Rephrased
Rephrased
Recently, David Sachs, a member of the new White House and PayPal Mafia, claimed on
the news that they have strong evidence suggesting Deep Seek stole OpenAI's outputs
to improve their models using a technique called distillation, which is against
OpenAI's terms of service. The irony here is striking—some might even call it
artificial super irony. You might remember how OpenAI gathered a vast amount of
internet data and copyrighted materials without permission, leading to lawsuits
from people like Elon Musk over Twitter scraping and authors like George R.R.
Martin.
Now, here’s a tech founder's secret: we often engage in questionable practices and
then ask for forgiveness later. Once a company gains traction, it’s hard to stop
it, with Uber and Airbnb being prime examples. So far, OpenAI has mostly succeeded
in its copyright infringement cases. In today’s video, we’ll dive into the
technical details of Deep Seek, how it bypassed CUDA, and see if it truly copied
OpenAI. It’s January 29, 2025, and you’re watching The Code Report. My conspiracy
theory? OpenAI has been Deep Seek all along, pulling off a clever marketing stunt
to assert its dominance. I wouldn’t put it past Chief Persuasion Officer Sam
Altman, who has a reputation for being deceptive.
Currently, OpenAI and Microsoft are accusing Deep Seek of distillation—where one
large, expensive model like GPT-3 is used to train a smaller model. They haven’t
provided solid evidence yet, but screenshots are circulating online showing Deep
Seek giving responses that seem unique to ChatGPT. This isn’t a smoking gun,
however, as similar content is all over the internet, and Deep Seek could have
gathered it organically. Microsoft, which provides much of OpenAI’s computing
power, claims they’ve seen large data extractions from the OpenAI API that may be
linked to Deep Seek.
In essence, Deep Seek is like Robin Hood, taking from the wealthy to help the less
fortunate. Generally, distillation produces better results than reinforcement
learning, where new data is fed into the model. Distillation isn’t controversial,
and Deep Seek has models distilled from LLaMA and Claude. You can even distill
OpenAI models, as long as you don’t use the API to create a rival model, which
seems to be the core of OpenAI's complaint.
Additionally, Alibaba just launched Qwen 2.5 Max moments ago. While it’s not a
reasoning model, it’s an open model that beats Deep Seek’s Claude and GPT-4 on
certain benchmarks. Another Chinese model, KIM 1.5, has also been released and
reportedly surpasses OpenAI’s GPT-3. We are witnessing a race between Chinese AI
models, while the United States lags behind. Meanwhile, Europe is focused on other
tech innovations, like bottle caps that can’t be removed.
Many have criticized Deep Seek for being heavily censored, but it’s relatively easy
to jailbreak if you’re skilled. Speaking of irony, last year, MidJourney accused
Stability AI of image theft, but that’s not the main issue now. Deep Seek has just
released the Jan series models for diffusion-based image generation. While the
quality may not rival Stable Diffusion or MidJourney, it’s another open-source
model that can be used commercially, which is a positive for humanity.
Interestingly, Deep Seek achieved ten times better efficiency than other models by
not using NVIDIA’s proprietary CUDA platform, opting instead for NVIDIA’s parallel
thread execution. This illustrates the ingenuity of the Deep Seek engineers.
However, a major concern about Deep Seek is that when you use it online, all your
prompts, data, and keystrokes are sent to China. If privacy is important to you, it
might be wise to avoid the internet altogether, or at least use it locally, as
shown in this video on my second channel.
The key trend here is that open source is winning. If you’re a developer, now is
the time to start creating products people will love. You can do this with a
fantastic open-source tool called PostHog, the sponsor of today’s video. It’s like
a Swiss Army knife for analyzing, testing, observing, and deploying better
features. Its product analytics tool can help you understand your customers and
build funnels, while its web analytics can replace Google Analytics. Session replay
will show you how users interact with your app, along with feature flags and UI
experiments like A/B testing. Most importantly, it’s easy to implement, thanks to
SDKs for web, mobile, and server-side applications, along with great documentation
for developers. Not only is it open-source and self-hostable, but it also offers a
fully managed, no-credit-card-required free plan. Give PostHog a try using the link
below. This has been The Code Report. Thanks for watching, and I’ll see you in the
next one!