Alignment

Working definition

Within AI safety, alignment is often used to describe technical means of making models abide by human values. Yet most alignment research does not address what those values are or how they are captured, instead treating them as interchangeable with "human preferences" or "human intentions." These preferences and intentions are themselves gleaned from, e.g., Reddit upvotes, polls, text analyses, or more recently, synthetic data. Solving 'the alignment problem' is often touted as the way to ensure AGI will usher in utopia, while the specter of 'misaligned AI' is held up as portending dire outcomes for human survival if left unchecked.

Questions to ask

• When you encounter advocacy around AI alignment, call to mind the one or two of the human values that matter to you most. Do you have examples of these values being successfully encoded in any technology you use?
• Can you envision how that implementation would extend to AI systems you interact with?
• Does the argument in favor of alignment being presented to you address these concerns?

Usages in annotated documents

• …the stakes are simple: whoever controls this technology controls the future of humanity. The core problem is alignment. Today’s frontier models already display emergent capabilities their creators did not predict, and no one can… The Window Is Closing
• …a sufficiently capable system will remain aligned with human values. Researchers warn of power-seeking behavior: a misaligned model that resists shutdown, deceives its overseers, and copies itself beyond our reach. Once loss of control occurs,… The Window Is Closing