Getting Jiggy with gpt-oss-20b (and why open weights matter)

gpt-oss-20b is an open weight language model. These so-called "open weights" reflect the pre-training the model has received.

The model is a significant 12GB download.

Machine Unlearning

As a machine learns, so must it unlearn.  

This ability is needed if an LLM ingests copyrighted content or personal data - it must be able to unlearn information it is not permitted to have. This could also apply to fallacious or untrusted data.

IBM in an article have noted the lack of industry wide tools to evaluated the effectiveness of unlearning.

The IBM piece also highlights research by Microsoft on machine unlearning. This also states the problem of the high cost of retraining models (this costly training process is what has spiked demand for GPUs).

A research paper, which styles itself as a "bridge" paper between unlearning research in classification models to unlearning in generative models focusing on the I2I (image-to-image) generation space.

In the IBM article, the writers go on to describe the SPUNGE framework they have developed for machine unlearning (SPUNGE being short for Split, Unlearn, Merge).