Principle 10
Progressive Autonomy
The degree of independence granted to an agent should increase with demonstrated reliability and decrease with the stakes of the action.
AX proposalTrust & Control
Why it matters
Neither full autonomy nor constant confirmation works. The first is unsafe for consequential actions; the second makes agents more tedious than doing the task yourself.
Progressive autonomy treats autonomy as a dial set per action and per relationship, adjusted as trust is established.
Examples
Good exampleIllustrative
An agent may reorder household items up to 40 EUR without approval; anything new, or anything above the limit, requires confirmation.
const policy: AutonomyPolicy = {
default: "approve_each",
rules: [
{ action: "reorder_item", when: { previouslyPurchased: true, maxAmount: 40 }, mode: "autonomous" },
{ action: "cancel_order", mode: "notify_after" },
{ action: "change_payment_method", mode: "always_approve" },
],
};Bad exampleIllustrative
A single toggle switches an agent between "read-only" and "can do anything".
Implementation guidance
- 1Classify actions by reversibility and cost.
- 2Offer at least three modes: autonomous, notify after, approve before.
- 3Let people adjust the policy based on track record.
Evaluation criteria
- Autonomy can be configured per action type.
- High-stakes actions default to requiring approval.
- Policy changes are logged.