Jensen Huang on "distillation"

In this interview with axios, he was asked this question: "Should open source model companies be allowed to distill closed models"

"Distillation, learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence.

We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something.

The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI.

As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI."

----

From "Axios" YouTube channel, (full video link in comment)