AI & Compute
What to actually worry about with AI
A great deal of the risk discussion is about scenarios, and a shorter list of problems is already causing measurable harm.
Risk discussion in this field splits between long-horizon speculation and immediate documented harm, with the first receiving more attention and the second more evidence.
The problems that are already occurring
Fabrication in consequential contexts. Models producing confident false information that people act on.
Documented cases include fabricated legal citations submitted to courts, incorrect medical information, and false biographical claims about real people.
The mechanism is well understood and mitigations exist — retrieval, citation, calibrated uncertainty — and deployment frequently omits them.
Automated decisions without recourse. Systems making or heavily influencing decisions about credit, employment, benefits, tenancy and enforcement.
The harm is rarely the model being wrong occasionally. It is being wrong systematically for a subgroup, at scale, with no explanation and no appeal.
Documented failures in benefit fraud detection and in automated eligibility systems have affected large numbers of people, and the pattern is consistent: the system's output was treated as authoritative and the burden of proof fell on the person.
Synthetic media used for fraud and abuse. Voice cloning for impersonation fraud, non-consensual intimate imagery, and fabricated evidence.
The non-consensual imagery problem in particular is causing serious documented harm, disproportionately to women, and the technical barriers to producing it have collapsed.
Labour displacement in specific occupations, concentrated so far in translation, transcription, stock illustration, copywriting and some customer support.
The aggregate effect is contested; the effect on those specific occupations is not.
Concentration. Frontier capability requires capital, compute and data at a scale available to a small number of organisations, which has implications for who sets norms and captures value.
Energy and water use, discussed elsewhere, with local effects that are real.
The problems that are plausible and less demonstrated
Uplift for harmful capability. Whether models meaningfully assist in producing weapons or cyberattacks beyond what existing resources allow.
Research on this exists and results have been mixed. It is a legitimate area of concern and evaluation, and the empirical basis is thinner than the rhetoric.
Large-scale manipulation. Whether generated content shifts opinion at scale.
The capability to produce persuasive content cheaply is demonstrated. Whether that translates to influence at scale, given that persuasion is generally hard, is less established.
Erosion of the information commons. Web content increasingly generated, models trained on generated content, and the difficulty of finding primary sources.
This is observable and its long-run effect is unclear.
The problems that are speculative
Loss-of-control scenarios involving systems pursuing goals against human intent.
These are worth taking seriously as a research direction, and the honest position is that they rest on assumptions about capability development that are not established.
The strongest version of the argument is that the cost of being wrong is asymmetric, which is a reasonable case for investing in alignment research and a weak case for treating it as the primary policy priority now.
Why the emphasis matters
Attention is finite, and so is regulatory capacity.
A framework focused on hypothetical future capability may impose requirements that entrench incumbents — compliance costs favour large organisations — while leaving current documented harms unaddressed.
Conversely, a framework addressing only current harms may be poorly positioned if capability advances quickly.
The reasonable position is that these are not mutually exclusive and that the current allocation of attention is skewed toward the speculative end relative to the evidence.
What would actually help now
Meaningful recourse. A right to human review, an explanation, and an appeal for any consequential automated decision. This addresses a large share of documented harm and is straightforwardly implementable.
Provenance standards for synthetic media, including cryptographic signing at capture. Imperfect and better than nothing.
Liability clarity. Establishing who is responsible when a deployed system causes harm, which currently sits in an unhelpful gap between developer and deployer.
Evaluation and disclosure requirements proportionate to deployment context, with independent audit rather than self-assessment.
Enforcement of existing law. A great deal of documented harm is already unlawful under discrimination, consumer protection, product liability and data protection law. The gap is enforcement capacity rather than legal authority.
That last point is unglamorous and probably the most consequential item on the list.
