The Medium post goes over various flavors of distillation, including response-based distillation, feature-based distillation and relation-based distillation. It also covers two fundamentally different modes of distillation – off-line and online distillation.

OpenAI accuses Chinese AI firm DeepSeek of stealing its content through "knowledge distillation," sparking concerns over security, ethics, and national interests.

OpenAI thinks DeepSeek may have used its AI outputs inappropriately, highlighting ongoing disputes over copyright, fair use, and training data.