In today’s issue of Command Line, I reported that ByteDance has been violating the developer license of both Microsoft and OpenAI by using GPT-generated data to train its own, competing model in China. After my report was published, OpenAI spokesperson Niko Felix sent the following statement confirming that ByteDance’s account has been suspended: As I reported, most of ByteDance’s GPT usage has been done through Microsoft’s Azure platform, not through OpenAI directly. I’ve asked Microsoft if it will follow OpenAI and suspend ByteDance’s access as well.
Distilling has been around since forever. It’s a legitimate technique that can give you a better model depending on your needs.
OpenAI does it too to improve its models.