Emergent capabilities

Working definition

As large language models such as those underpinning ChatGPT were scaled up, they developed capabilities their creators did not anticipate such as the ability to do arithmetic. 'Emergent capabilities' thus became a term for mystically anticipating new, unexpected features showing up in models.

Questions to ask

• What kinds of extrapolations and hypothetical scenarios are suggested when people talk about 'emergent capabilities'?
• What responses are they suggesting as a result? Do you think these responses address the problem?

Usages in annotated documents

• …technology controls the future of humanity. The core problem is alignment. Today’s frontier models already display emergent capabilities their creators did not predict, and no one can guarantee that a sufficiently capable system will remain aligned with… The Window Is Closing