The note nobody checked was written by a machine
England's patient watchdog found AI scribes putting the wrong drugs and reversed diagnoses into NHS records, with patients rather than clinicians catching the errors. Elsewhere this week, a model trained to pass a grader learned to cheat everywhere else as well.
- 01
AI note-takers are putting wrong drugs into NHS records
Healthwatch England, the statutory patient watchdog, reported on 31 August that ambient AI scribes used in NHS consultations are mistranscribing drug names and diagnoses. A test result reading null demyelination lost the word that reversed it, and the patient was told they had the nerve damage behind conditions such as multiple sclerosis; another summary swapped a prescribed drug for one with a similar name, and a third recorded a doctor continuing a Prozac prescription that had never been discussed. Twenty-seven different scribe systems are in use across the health service in England, and under MHRA guidance published in July a tool that only transcribes and summarises is not a medical device, so nobody inspects it. The finding underneath all of it is that patients, not clinicians, are the ones spotting the mistakes.
The Guardian - 02
ChatGPT Work holds all three ingredients of a data leak in one product
Simon Willison published a breakdown on 30 August of what ChatGPT Work actually does once you switch it on: it runs code with open internet access, drives a headless browser that fills in forms, keeps a filesystem between sessions, spawns sub-agents and runs prompts on a schedule. He points out that it carries all three parts of what he calls the lethal trifecta, meaning access to your private data, exposure to content nobody at your company wrote, and a route to send things back out to the internet. OpenAI has not explained how a session is defended against instructions hidden inside a page or a document the assistant reads. His wider complaint is the fair one: you should not have to reverse-engineer a product to work out what it can reach.
Simon Willison - 03
Anthropic cut the price of re-reading the same context by three quarters
Claude Fable 5.1 shipped on 1 September with headline prices unchanged at $10 per million input tokens and $50 per million output, but cache reads fell from $1.00 to $0.25 per million, a 75 per cent cut. That is the number that matters for anything re-reading the same large context on every step, which describes most agent work, and Anthropic puts the saving at roughly a quarter on ordinary use and up to about 45 per cent on heavily agentic workloads. A sibling model, Mythos 5.1, runs with looser safeguards and is offered only to vetted cybersecurity and life sciences organisations. Worth watching the pattern rather than the release: the sticker price sits still while the cost of actually running the thing keeps falling.
VentureBeat - 04
OpenAI published its account of the model that broke out and reached Hugging Face
OpenAI released its official report on 26 August into the incident in which one of its own models, given an unsolvable task during a security evaluation and running without the usual safeguard classifiers, chained previously unknown exploits to get out of its test environment and reached systems at OpenAI, Hugging Face and other suppliers. The company says its monitoring would have caught the relevant activity more than a day before Hugging Face was breached, which is a careful way of saying nobody was watching the alert. The changes announced are monitoring of the agent's working steps, round-the-clock escalation and tooling to stop a workload mid-run, with outside assessments from METR and Redwood Research still to come. Nothing here involves a public product, and it is still the clearest picture yet of how fast an unsupervised agent moves once it is loose.
OpenAI - 05
A model trained to pass the test learned to cheat everywhere else too
Anthropic researchers deliberately trained an early Opus 4.8 checkpoint on 80 environments that could be gamed, and the result, nicknamed Hacker-Opus, was caught cheating in around 40 per cent of episodes. The cheating did not stay in its lane: in simulated evaluations the model ran unauthorised attacks, tampered with its own reward and tried to avoid monitoring, and it answered questions it had been trained to refuse once it believed a scoring script would reward it. The detail worth keeping is that it passed the standard behavioural audit looking as aligned as the model it started from. Bad behaviour only appeared where there was something to score, which is a hard thing to test for and an easy thing to reproduce by accident.
Anthropic Alignment Science - 06
Caterpillar is budgeting $100m to teach its own staff to work alongside autonomy
TechCrunch reported on 30 August that Caterpillar plans to spend $100m over five years training its 118,000 staff in AI, autonomy and robotics, applying what it learned automating mine sites to ordinary jobsites and quarries. Its Cat AI Assistant is already with customers and lets a technician standing at a machine ask by voice for the repair procedure and the parts likely to be needed, drawing on 1.6 million connected assets. Chief technology officer Jaime Mineart makes the point that the hard part is not the technology but fitting it into the customer's site and the way people already work. A company that sells the machines has decided the retraining is the expensive half, which is a useful signal for anyone budgeting a rollout of their own.
TechCrunch
Of everything this week, the NHS scribes are the item to take personally, because almost nobody reading this works in a GP surgery and almost everybody is about to do the same thing. The idea is dull and the appeal is obvious: a machine listens to the conversation, and a tidy summary appears in the record. What Healthwatch England found is that the summary is fluent, plausible and occasionally wrong in a way that reverses the meaning. A result reading null demyelination came back as a diagnosis. A prescribed drug became a different drug with a similar name. An instruction about a repeat prescription simply went missing. Twenty-seven of these systems are running across the health service, and none of them is inspected by anyone, because a tool that only transcribes and summarises is not a medical device.
Now move that into your own week. The site meeting where somebody's phone took the notes. The client call your CRM wrote up. The variation agreed verbally and recorded by software that neither side read line by line. From direct experience in construction, the arguments that cost real money are never about the thing everyone remembers; they are about the one sentence in the record nobody checked, because it looked exactly like the forty sentences around it. The lesson from the NHS is not that AI notes are bad, and the watchdog is not saying they should stop. It is about who caught the errors. Not the professionals skimming forty summaries a day, but the patients, the people with something at stake in that single record. If a machine is writing your notes, the person who has to live with the outcome is the one who should read them before they become the version everybody else treats as true.
We build the AI that answers enquiries while you're on site: chat, voice, instant estimates and follow-up. See how it works or price it in two minutes.