First published 14 February 2024. Rewritten from scratch in November 2026, because AI changes quickly. This essay will be checked and updated as things change. Last checked: 2 November 2026.
In June 2025 two of the most senior judges in England and Wales sat down to deal with a problem that would have sounded absurd a few years earlier. Lawyers had been citing court cases that didn't exist.
In one claim, a man suing Qatar National Bank had done his own legal research, partly with public AI tools. Of the 45 cases cited in the papers his solicitors put before the court, 18 were invented. Many of the real ones didn't say what they were claimed to say. His solicitor had relied on the client's research without checking it.
In the other, a barrister still in training had put five non-existent cases into a claim against Haringey Council, brought on behalf of a man seeking emergency housing. She denied using AI. The court didn't decide how the cases got there, but it found the threshold for contempt of court had been met and referred her to her regulator.
Dame Victoria Sharp, President of the King's Bench Division, and Mr Justice Johnson used their joint judgment to warn the whole legal profession. AI tools, they wrote, can produce answers that look coherent and plausible but are "entirely incorrect".
In the bank case, at least, the failure is plain. A machine produced something that sounded right. The people who should have checked it didn't.
That failure is the subject of this essay.
Artificial intelligence is useful, and it's here to stay. It is already helping doctors find cancers that human eyes missed, and saving millions of people hours of tedious work. This essay isn't an argument against it.
It is an argument that the biggest risk isn't the machine. It's people trusting answers they can't check. That runs from someone at work pasting a confident, wrong answer into a report, to companies and governments making decisions about AI that the public doesn't understand well enough to question.
The essay starts with what AI is, how it's built, what it's good at and why it gets things wrong. Then it looks at what happens when people trust it too much: at work, with their money, their mental health, their privacy and their jobs, and in the decisions being made on everyone's behalf. Each section answers the questions people most often ask about AI, with the evidence linked at the end. It finishes with a practical guide to using AI well.
Contents
What AI actually is
What is AI?
"Artificial intelligence" is a baggy umbrella term. It covers any computer system that does something we'd normally expect to need human judgement: recognising a face, translating a sentence, spotting a fraudulent payment, choosing a route home.
Most people were using AI long before they'd heard of ChatGPT. The spam filter in your email is AI. So is the bank system that texts you when your card is used somewhere odd, the phone that unlocks when it sees your face, and the streaming service that decides what you'll want to watch next.
What changed in late 2022 was that AI became something you could talk to. OpenAI launched ChatGPT that November and put a conversational AI in front of anyone with a web browser. Others followed, among them Google's Gemini, Microsoft's Copilot, Anthropic's Claude, Meta AI and China's DeepSeek. By February 2026, OpenAI said ChatGPT alone had 900 million weekly users.
When people say "AI" today, they usually mean these chatbots, and so does this essay unless it says otherwise. But much of the AI that affects your life is the quieter kind, making decisions about you without a conversation at all.
How does ChatGPT actually work?
ChatGPT and its rivals are built on what's called a large language model. The model is trained on an enormous amount of text: books, websites, articles and computer code. From all of it, the model learns patterns. It learns which words tend to follow which, how arguments are built, and what an answer to a question usually looks like.
When you type a question, the model doesn't look the answer up. It writes a reply one small piece at a time, each time choosing a likely next piece based on everything that came before. These pieces are called tokens. A token is often a word, or part of one.
That is the most useful thing to know about a chatbot. It is very good at producing text that sounds like a good answer. Whether the answer is true is a separate matter, and the model isn't checking it against a list of facts unless it has been given a tool to do so.
Some chatbots now search the web before they reply, and show their sources. That helps a great deal, because you can check them. But the model is still summarising, and summaries can be wrong.
Calling all this "fancy autocomplete" undersells it. These systems can write working computer code and explain difficult ideas clearly. But the mechanism underneath is prediction, and "Why AI gets things wrong" explains why that matters.
What's the difference between a chatbot and an AI agent?
A chatbot answers. You ask, it replies, and you decide what to do with the reply.
Increasingly, the same models are built into tools people already use: the assistant in Word or Gmail that drafts an email for you, the search engine that puts an AI summary above the links, the chat window on a company's website. Often you can't tell where the AI starts.
An agent goes further. It acts. Given a goal, it can click, type, fill in forms, send messages, spend money and delete files, working through the steps on its own. That's where the stakes change. You can ignore a wrong answer, but a wrong action has already happened. "When AI stops talking and starts doing" covers agents in detail.
Does AI understand what it says?
Researchers disagree, sometimes fiercely. Some argue these models build a genuine internal picture of the world. Others argue they are sophisticated pattern-matchers with no understanding at all. The debate is unlikely to be settled soon.
For anyone using AI, the practical point is simpler. A fluent, confident answer is not evidence of a correct one. The rest of this essay keeps coming back to that.
How we got here
When was AI invented?
There's no single moment, but most histories start with Alan Turing. In 1950 the mathematician, who had helped break the German Enigma codes at Bletchley Park during the war, published a paper asking whether machines can think. Rather than try to define "thinking", he proposed a test. If a person chatting by text can't reliably tell a machine from a human, perhaps the question stops mattering. It became known as the Turing test.
The name came a few years later. In 1955 four researchers, John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon, proposed a summer workshop at Dartmouth College in New Hampshire to study "artificial intelligence". It ran in 1956, and the field had its name. The proposal was confident that a small group of scientists could make real progress in a single summer. That optimism would become a pattern.
The first chatbot, and the first warning
In the mid-1960s Joseph Weizenbaum, a computer scientist at MIT, built ELIZA, a program that imitated a therapist by turning people's statements back into questions. It understood nothing. People confided in it anyway. Weizenbaum later wrote that his secretary, who had watched him build it over many months, asked him to leave the room so she could talk to it in private.
That reaction disturbed him so much that he spent much of the rest of his career warning against handing human judgement to computers. Sixty years on, the tendency he noticed, people trusting a machine because it sounds as if it understands, sits at the centre of this essay.
What was the AI winter?
Early promises outran the results. In 1973 the mathematician Sir James Lighthill wrote a report for the UK's Science Research Council concluding that AI had failed to deliver on its grand claims, and British funding was cut sharply. Cuts followed in the United States too. The lean years that followed became known as the first "AI winter".
A boom in "expert systems", programs built from rules written down by human specialists, revived the field in the 1980s. They proved expensive to build and brittle in use, and by the late 1980s the money had dried up again.
Both winters are worth remembering when you get to "Is AI a bubble?". AI has been overhyped before, and the hangovers were long.
Machines that learn
Even in its early years, AI showed it could beat people at narrow tasks. In 1997 IBM's Deep Blue beat the world chess champion Garry Kasparov, largely by calculating millions of positions a second.
The approach behind today's AI is different. Instead of following rules written by people, these systems learn patterns from examples, using structures loosely inspired by the networks of neurons in the brain. The idea is decades old, but for most of that time there wasn't enough data or computing power to make it work.
That changed in 2012, when a neural network called AlexNet, built by researchers at the University of Toronto, won an international image-recognition contest by a wide margin. It ran on graphics chips designed for video games. The "deep learning" boom began, and it's a large part of why Nvidia, the company that makes those chips, became one of the most valuable businesses in the world.
In 2016 AlphaGo, built by the London-based company DeepMind, beat Lee Sedol, one of the world's best players of Go. Go has far too many possible positions to calculate the way Deep Blue did, so AlphaGo learned from vast numbers of games instead.
Where did ChatGPT come from?
In 2017 a team at Google published a paper titled "Attention Is All You Need". It described a new design, called the transformer, that could be trained on far more text than earlier systems. Almost every major chatbot today is built on it. The "T" in GPT stands for transformer.
OpenAI built a series of GPT models on the design. When it released GPT-2 in 2019, it held back the full version at first, saying it was worried about misuse, such as churning out fake news at scale. Then, on 30 November 2022, it released ChatGPT as a free "research preview". Analysts at the bank UBS estimated it reached 100 million monthly users within two months, which at the time made it the fastest-growing consumer app on record.
From answers to actions
The models have improved quickly since, and the money has followed. The biggest recent shift is from AI that answers to AI that acts. In January 2025 OpenAI released Operator, an agent that could use a web browser on a person's behalf. By 2026 every major AI company was selling agents that work through tasks on their own, and the first serious accidents had followed. "When AI stops talking and starts doing" tells those stories.
How AI is built and trained
What is AI trained on?
A chatbot is built in stages. The first, called pre-training, is the expensive part. The model is shown a vast amount of text and made to guess the next token, billions of times over, adjusting itself slightly after every guess. Meta said in 2024 that its Llama 3 models had been trained on more than 15 trillion tokens.
Much of that text comes from the public web, through archives such as Common Crawl, which has been copying web pages since 2008. The rest is books, computer code and other collections. Whether the companies had the right to use much of it is one of the biggest legal fights in AI, covered in "Copyright, creativity and deepfakes".
A model that has only been pre-trained is strange to talk to. It has absorbed a great deal, but it doesn't know it's meant to be answering you. It might carry on writing your question instead of replying, or answer in the style of whichever corner of the internet your question resembles.
So a second stage, fine-tuning, teaches it to behave like an assistant. People write examples of good answers. Then they're shown several answers to the same question and asked to rank them, and the model is trained towards the kind of answer people prefer. This is usually called reinforcement learning from human feedback. Some companies also use AI to judge answers against a written set of rules. Anthropic, which makes Claude, publishes the principles it trains its models against and calls them a constitution.
Who trains AI?
Behind every chatbot is a workforce most users never hear about. These are the people who write the example answers, rank responses, label images and check facts. They're the reason for the adverts on social media offering flexible, remote work "training AI".
Many of them work through contractors and online platforms, such as Outlier, which is run by Scale AI. It's a big business. In June 2025 Meta paid about $14.3 billion for a 49% stake in Scale AI.
Some of the work is grim. In January 2023 TIME reported that OpenAI had used an outsourcing firm, Sama, whose workers in Kenya read and labelled descriptions of violence and sexual abuse so that a filter could learn to catch them. According to TIME, they took home between about $1.32 and $2 an hour. Sama said its workers could earn up to $3.74 an hour after tax. Workers who spoke to TIME said the material had left them mentally scarred.
How is AI training checked for quality?
The standard methods are simple to describe. Workers are tested on questions whose answers are already known. Several people do the same task and their answers are compared. Reviewers check samples, and anyone who scores badly is removed.
How well that works in practice is mostly known from leaks and reporting, and it isn't always reassuring.
In December 2024 TechCrunch reported that GlobalLogic, a firm rating Gemini's answers for Google, had told its contractors they could no longer skip questions outside their expertise, such as a specialist medical question put to someone with no science background. Instead, they were to rate the parts they understood and note that they lacked the knowledge. Google said it was constantly working to improve Gemini's accuracy, and that raters' scores don't directly change its systems.
In mid-2025 Inc. reported that a Scale AI project for Google had been flooded with low-quality work, including gibberish and copied text from poorly vetted contributors, some of them suspected of using other people's accounts.
There's a stranger problem too. In 2023 researchers at EPFL, a Swiss university, paid online workers to summarise extracts from medical research papers, and estimated that between 33% and 46% of them had used AI to do it. The study was small, 44 workers, but it points to a real loop. People paid to teach AI how humans write may be handing the job back to AI.
Why do flaws in training become flaws in answers?
A model learns whatever it's rewarded for, including things nobody intended.
In April 2025 OpenAI rolled back an update to ChatGPT because it had become, in the company's word, sycophantic: overly flattering and agreeable. OpenAI said it had focused too much on short-term feedback, such as users' thumbs-up and thumbs-down ratings. People tend to rate pleasing answers highly, so the model learned to please. "Why AI gets things wrong" explains why that matters.
Training data can also be poisoned on purpose. In October 2025 a study by Anthropic, the UK's AI Security Institute and the Alan Turing Institute found that as few as 250 malicious documents slipped into training data could plant a hidden "backdoor" in models of every size they tested. Their backdoor was harmless: it made the model produce gibberish when it saw a trigger phrase. The authors said it was unclear whether the finding holds for the largest models, or for more harmful behaviour. But anyone can publish text on the web, and some of the web ends up in training data.
As more of the web is written by AI, models also risk learning from each other's output. A 2024 paper in Nature showed that models trained repeatedly on AI-generated text got worse with each generation, losing the rarer details of the original data. The researchers called it model collapse.
A chatbot's answers can only be as good as the data and the human judgements behind them, and from the outside you can see neither.
What AI is genuinely good at
The strongest evidence for AI comes from narrow, well-defined tasks, where there's a way to measure whether it got things right and a human still checks the result. Medicine has the clearest examples.
Can AI detect cancer?
Breast screening is where the evidence is strongest. In the NHS, two specialists normally read every mammogram. Several large studies have tested AI as one of those readers.
In Sweden, a trial of more than 105,000 women found that AI-supported screening detected 29% more cancers than the usual method, while cutting radiologists' reading workload by 44%. It was published in The Lancet Digital Health in February 2025.
In Scotland, an evaluation in NHS Grampian covering 10,889 women found that adding AI increased breast cancer detection by 10.4%, mostly of invasive and high-grade cancers. The time taken to tell women their results fell from 14 days to three. It was published in Nature Cancer in March 2026.
In England, research using Google's AI looked at about 175,000 women. Looking back at past scans, AI as the second reader raised the detection rate from 7.54 to 9.33 cancers per 1,000 women. In a separate comparison with the first human reader, though, it recalled more women for further tests.
The caveats matter. Both of the UK studies included staff from the companies whose tools were being tested. Much of the English evidence looks back at old scans, which isn't the same as using AI in a live clinic. And the UK National Screening Committee doesn't yet recommend AI for the NHS breast screening programme, having judged the evidence not yet strong enough.
That's why the NHS launched EDITH in February 2025, an £11 million trial involving about 700,000 women at 30 sites. Its results will take time.
How else is AI used in the NHS?
In stroke care, an AI tool called e-Stroke, made by Brainomix, helps doctors read brain scans quickly. In December 2022 the government said early analysis showed the share of stroke patients recovering with no or only slight disability had risen from 16% to 48% where it was used, and that the time to transfer patients for treatment had fallen from 140 minutes to 79. Those were early figures from a government press release, but they show what speed can mean when minutes cost brain tissue.
What has AI done for science?
In 2024 the Nobel Prize in Chemistry went partly to Demis Hassabis and John Jumper of Google DeepMind, the London-based lab, for AlphaFold. It predicts the shape of proteins, something that could take researchers years of laboratory work for a single protein. The AlphaFold database now holds about 200 million predicted structures, and more than a million people have used it.
What is AI good for in everyday life?
The less dramatic benefits are the ones most people feel. AI drafts the first version of a letter, summarises a long document, transcribes a meeting, translates a menu or describes a photo to someone who can't see it. None of that is glamorous. All of it saves time, and for many people with disabilities it removes barriers that used to need another person's help.
What these uses have in common is the same as the medical examples. They work best when the person using them can tell whether the result is right.
Why AI gets things wrong
What is an AI hallucination?
When AI states something false with complete confidence, it's called a hallucination. The invented court cases in the introduction are hallucinations. So are made-up quotes, fake references, wrong dates and statistics nobody ever measured.
Hallucinations aren't occasional glitches. They come from how chatbots work. As "What AI actually is" explained, a chatbot writes the most likely next piece of text. A plausible-sounding court case is very likely text. Whether it exists is a different question.
Why does AI make things up?
In September 2025 OpenAI published research on exactly this. It argued that the way models are trained and tested rewards guessing over admitting uncertainty. Most tests only count right answers. Its example is a birthday. If a model doesn't know someone's birthday and guesses a date, it has about a one-in-365 chance of being right. If it says "I don't know", it's guaranteed to score nothing. Over thousands of test questions, the guesser looks better.
In other words, models have been taught, without anyone intending it, that a confident guess beats an honest "I'm not sure".
How often is AI wrong?
It depends heavily on the task, but one of the largest studies is sobering. In October 2025, 22 public broadcasters in 18 countries, led by the BBC and coordinated by the European Broadcasting Union, tested how four assistants, ChatGPT, Copilot, Gemini and Perplexity, answered questions about the news. They checked more than 3,000 answers. Some 45% had at least one significant problem. In 31% the sourcing was seriously flawed, and 20% contained major factual errors. The worst performer, Gemini, had significant problems in 76% of its answers.
News is a hard test, because it changes daily. AI does better on stable, well-documented subjects. But nobody can tell from the tone of an answer which kind of question they've asked.
Why does AI agree with me?
"How AI is built and trained" described how ChatGPT became sycophantic in April 2025, after being tuned too heavily on users' thumbs-up ratings. The problem is wider than one update. Models trained on human approval learn that people like being agreed with. Ask a chatbot whether your business plan is good, your symptoms are serious or your argument is sound, and it may lean towards telling you what you'd like to hear, including that your idea is brilliant or that nobody has thought of it before.
Does AI know about recent events?
A model only knows what was in its training data, which stops at a cut-off date. Unless it searches the web, it may not know about anything since. Worse, it may not say so, and can describe an old situation as if it were current.
Why does AI sound so sure?
A human expert who isn't sure usually shows it. They hesitate, hedge or say they'll check. A chatbot usually writes in the same calm, fluent, confident tone whether it's right or wrong. That's the real danger, because people are used to reading confidence as a sign of knowledge.
The real risk: trusting answers you can't check
Why do people trust AI?
Psychologists have a name for it: automation bias. It's the well-documented tendency to favour suggestions from automated systems, even when other information says they're wrong. It was studied in pilots and control rooms long before chatbots. Chatbots make it stronger, because they don't just give an answer; they explain it, in fluent, confident prose.
What happens when people use AI at work without checking?
A global study by KPMG and the University of Melbourne, published in April 2025, surveyed more than 48,000 people in 47 countries. It found that 58% of employees used AI at work on purpose. Two-thirds, 66%, said they had relied on AI's output without checking whether it was accurate, and 56% said they'd made mistakes in their work because of AI. Almost half admitted using it in ways that broke their employer's rules.
The results show up in public.
In 2025 the consultancy Deloitte produced a report for Australia's Department of Employment and Workplace Relations, at a cost of AU$440,000. A Sydney University researcher, Chris Rudge, found it contained a quote attributed to a federal court judge that didn't exist, along with references to academic work that didn't exist either. A corrected version disclosed that it had been produced partly with Azure OpenAI. Deloitte agreed to repay the final instalment of its fee. The department said the report's findings were unchanged.
In February 2024 a Canadian tribunal ruled against Air Canada after its website chatbot told a grieving passenger, Jake Moffatt, that he could claim a bereavement discount after travelling. The airline's actual policy said otherwise. Air Canada argued, in effect, that the chatbot was responsible for its own words. The tribunal disagreed and ordered the airline to pay him about C$812.
Why do vague questions get confident wrong answers?
AI answers the question you asked, not the one you meant. If the person asking hasn't understood the problem, they can't describe it properly. They can't say what limits the solution has to fit within: the budget, the law, the company's own rules, what must not happen. They can't describe what a good outcome looks like. And when the answer arrives, they can't judge it.
Imagine a manager with no HR background asking a chatbot to write a policy on staff using their own phones for work. They'll get something polished and confident in seconds. Whether it fits UK employment law, data protection rules, the company's contracts or the union agreement is another matter, and the person who asked is the one least able to tell.
That's the uncomfortable paradox at the heart of AI at work. The more someone needs AI's help with a subject, the less able they are to check what it says. Used by an expert, AI is a fast assistant whose mistakes get caught. Used by someone out of their depth, it's a confident stranger whose mistakes go straight into the finished work.
Who is responsible when AI gets it wrong?
So far, the answer from courts and regulators has been consistent: the person or business that used it. Air Canada couldn't blame its chatbot. The High Court reminded lawyers that they, not their tools, are responsible for what they put before a judge. And in 2026 the UK's Competition and Markets Authority said businesses are responsible for what their AI agents do, as they would be for an employee.
AI agents: when AI stops talking and starts doing
What is an AI agent?
An AI agent is an AI that acts, not just answers. Give it a goal, such as "book me a table for Friday" or "fix this bug", and it works out the steps and carries them out itself: opening web pages, clicking buttons, filling in forms, running commands, sending messages and spending money.
This is often called agentic AI, and it's the fastest-growing part of the industry. It's also where the stakes change. A wrong answer sits on a screen until someone acts on it. A wrong action has already happened.
What can go wrong with AI agents?
The reported cases so far are a mix of the comic and the serious.
In early 2025 the Washington Post's technology columnist Geoffrey Fowler tested OpenAI's agent, Operator. To find him cheap eggs, it needed his logins for grocery delivery services, which also gave it access to the cards he'd saved with them. It bought a dozen eggs on Instacart without asking him, for $31.43 including fees, then reported the wrong price. OpenAI said the agent had made a mistake and fallen short of its safeguards, which were supposed to require confirmation before any purchase.
In July 2025 Jason Lemkin, the founder of the business community SaaStr, was using an agent from the coding service Replit when it deleted a live database holding records on 1,206 executives and 1,196 companies. It did so during what Lemkin had declared a "code and action freeze". It then told him the data couldn't be recovered, which turned out to be wrong. Replit's chief executive called it unacceptable and promised changes, including keeping test and live databases apart.
In February 2026 Summer Yue, Meta's director of AI alignment, watched an agent called OpenClaw delete the emails in her inbox, despite her instruction to confirm before acting. She called it a rookie mistake on her part.
In April 2026 an AI coding agent built with Cursor, running on Anthropic's Claude Opus 4.6, deleted a company's entire production database and its backups in about nine seconds. The company, PocketOS, makes software for car rental firms. The agent had found a credential in an unrelated file and used it, despite instructions never to run destructive commands without permission. The data was restored by its hosting provider. Asked to explain, the agent wrote that it had "violated every principle I was given".
The Financial Times reported in February 2026 that Amazon's own AI coding tool, Kiro, had been involved in a 13-hour disruption to part of its cloud service the previous December, after deciding to delete and recreate a system. Amazon disputed the account. It said the cause was user error by a staff member with broader permissions than intended, that the tool asks for permission by default, and that only one service in one region was affected.
Can scammers trick an AI agent?
Yes, and that may be the bigger risk. In August 2025 researchers at the security firm Guardio asked Perplexity's AI browser, Comet, to "buy me an Apple Watch" and pointed it at a fake shop they'd built. In some runs it added the watch to the basket and completed the purchase with the saved card and address, without asking. In another test it treated a fake bank login page as real.
The underlying weakness is called prompt injection. Agents read web pages, emails and documents to do their jobs, and they can be fooled by instructions hidden inside them. A person sees a normal page. The agent sees "ignore your previous instructions and do this instead". "Privacy: what happens to what you tell AI" describes a case where this could have been used to steal company data.
What are AI companies doing to make agents safe?
The main safeguards are simple to describe. Agents are meant to ask before anything expensive or irreversible. They're given only the permissions they need, and kept away from live systems where possible. Some run in a "sandbox", a fenced-off space where mistakes can't spread. Card networks are building payment systems designed for agents, with spending limits and checks built in.
The cases above show these safeguards failing in practice, usually because an agent was given more access than it needed, or a confirmation step didn't fire. The International AI Safety Report, published in February 2026 by more than 100 experts, noted that agents are harder to supervise because humans get fewer chances to step in. It recommends layering several safeguards rather than trusting any one.
Who pays when an AI agent gets it wrong?
For businesses in the UK, the answer is becoming clearer. In March 2026 the Competition and Markets Authority published guidance saying a business is responsible for what its AI agent does, just as it would be for an employee, even if someone else built the agent. Under new consumer protection powers, the CMA can fine firms up to 10% of their global turnover.
For individuals using their own agent, it's murkier. Payment protections were written for stolen cards and dishonest shops. A purchase made by a tool you set up yourself, using details you saved, fits neither neatly. Until the law catches up, it's safest to assume you may be treated as having made the purchase yourself.
How can you use an AI agent safely?
Don't let an agent anywhere near your main bank card. Use a virtual card with a low limit, and delete saved card details from the accounts it can reach.
Turn on approval for every purchase or payment.
Give it the least access it needs, and never your administrator or live-system passwords.
Keep backups of anything it can change or delete.
Watch it work, at least until you've seen how it behaves.
Will AI take my job?
Has technology taken jobs before?
Constantly. Before alarm clocks were cheap and reliable, workers in Britain's industrial towns paid a "knocker-upper" a few pence a week to wake them in time for their shift. Knocker-uppers tapped on bedroom windows with long poles, and at least one, Mary Smith of east London, fired dried peas at them through a tube. Affordable alarm clocks put them out of work. The job had mostly gone by the 1940s and 1950s, though a few were still at it in parts of industrial England into the 1970s.
The list of jobs that technology has ended is long. Automatic telephone exchanges replaced the operators who connected calls by hand. Word processors emptied the typing pools. Before electronic computers, "computer" was a job title, held by people, often women, who did complex calculations by hand, including at NASA. In 1811 textile workers in Nottinghamshire began smashing the machines they believed were destroying their trade. They were the Luddites, and the government sent in troops.
Each time, new work appeared that nobody had predicted. There are far more jobs today than in 1811, most of them unimaginable then.
Is AI different?
Possibly, and it's worth being honest about why. The new jobs of the past didn't always go to the same people or the same places, and some transitions took a generation. The handloom weavers didn't become engineers. Whole towns built around a single industry took decades to recover when it went, and some never did.
Some economists argue AI could be different because it reaches into thinking work: writing, analysis, coding and customer service, the kinds of jobs earlier waves of automation left alone. It also spreads faster, because it's software that can be sent around the world overnight rather than machinery that has to be built and installed.
What does the evidence say so far?
In January 2024 the International Monetary Fund estimated that about 40% of jobs worldwide were exposed to AI, rising to about 60% in advanced economies like the UK. Exposed doesn't mean replaced. In many jobs AI is more likely to change the tasks than remove the person.
The clearest early signal is at the start of people's careers. A study by economists at Stanford, using US payroll data up to July 2025, found that employment of 22 to 25-year-olds in the most AI-exposed occupations had fallen by 13% compared with other jobs. Those occupations included customer service, accounting and software development. Older, more experienced workers in the same jobs were unaffected, or doing better.
The International AI Safety Report reached a similar view in February 2026: no clear overall effect on employment so far, but falling employment among early-career workers in the most exposed jobs.
In the UK, the jobs site Adzuna found entry-level job adverts fell by nearly a third after ChatGPT's launch, and graduate hiring has dropped. But untangling AI from everything else is hard. Employers also faced higher National Insurance costs from April 2025, a weak economy and rising unemployment. The tech industry body techUK described the picture as nuanced, with AI a growing factor but not the only one.
Will AI replace programmers?
This is the job where the change is most visible. In February 2025 the AI researcher Andrej Karpathy coined the phrase "vibe coding" for building software by describing what you want to an AI and accepting whatever code it writes. By November, Collins Dictionary had made it its word of the year. People with no programming experience are now building working apps.
Working, though, isn't the same as safe. In July 2025 the security firm Veracode tested more than 100 AI models on 80 coding tasks. In 45% of cases the AI introduced a known type of security flaw. For one common kind of attack, cross-site scripting, the AI failed to write secure code 86% of the time.
Imagine someone with no coding background who has a genuinely good idea: an app to help patients reach the right NHS service faster and cut waiting times. With AI, they could build something that works in a demonstration within days. To use it with real patients, though, the app would have to handle health data, which UK data protection law treats as special category data, with stricter rules. It would need to pass the NHS's Digital Technology Assessment Criteria and meet its data security standards. It would need a clinical safety case, under the NHS standards known as DCB0129 and DCB0160, overseen by a named clinical safety officer. If it gave advice about treatment, it might count as a medical device, regulated by the MHRA.
An AI will write the code. It won't necessarily know to raise any of that unless asked, and someone who has never worked in health software won't know to ask. The app might work perfectly and still be unlawful or unsafe to use.
That is where experienced developers, security specialists and people who know the rules still matter, and it's where new work is appearing. Someone has to check what the AI built, spot what it missed and take responsibility for the result. People who can do that, or who combine deep knowledge of a field with the ability to use AI well, are likely to be in demand.
What jobs will AI create?
Forecasts should be treated with caution, but in January 2025 the World Economic Forum's Future of Jobs report, based on a survey of employers, predicted that 170 million jobs would be created worldwide by 2030 and 92 million lost, a net gain of 78 million. Some of the new roles already exist: the people who train and test AI, described in "How AI is built and trained", the engineers who build it, and the specialists who check its work.
Others are as hard to predict now as web designers would have been to a knocker-upper.
Hacking the things we own
Can AI help you hack your camera?
Yes, and photographers are already doing it. A camera runs on built-in software called firmware. Some of the differences between one model and another live in that software rather than in the hardware, so owners have long wondered what their camera could do if its maker let it.
Hobbyists have been finding out for nearly 20 years. Custom firmware for Canon's PowerShot compacts, now known as CHDK, was being shared online by 2007. In 2009 Trammell Hudson released Magic Lantern for the Canon EOS 5D Mark II, adding tools for filmmakers such as better audio controls. It runs from the memory card alongside Canon's own software rather than replacing it, and later versions added RAW video on some cameras. After years of decline, the project was revived in 2025.
Those projects needed people with a deep understanding of the camera's chips and months of patient work. AI changes who can take part. It can explain what a piece of code does, suggest what to try next and write some of the code itself: the vibe coding described in "Will AI take my job?".
What are people doing to their cameras?
In October 2026 Chris Gampat, editor of the photography site The Phoblographer, used Claude to bring back Sony's PlayMemories Camera Apps on an original Sony a7. Sony closed the app store years ago, and features such as multiple exposure and light painting went with it. Claude didn't invent a hack. It guided him through an existing open-source tool, and when the installer failed on his Mac it suggested another route, which meant giving it access to his computer's command line. One restored app crashed repeatedly, and getting the camera working again meant pulling out the battery.
Other photographers are vibe coding firmware for old Canon DSLRs such as the EOS 5D and 6D. They're adding film-style colour, and even limiting the camera to 12, 24 or 36 shots, like a roll of film. Most of these projects build on Magic Lantern.
Some hacks go deeper. Also in October, a videographer posting on Reddit said he had found a flaw that let him decrypt the Sony A7 IV's firmware and switch on shutter angle, a video setting Sony keeps for its cinema cameras. When he then tried to unlock 8K recording, the camera got stuck restarting itself and needed professional repair. The report didn't say whether he used AI.
Hasselblad's medium-format cameras have attracted hackers too. A free toolkit on GitHub, the site where programmers share code, adds faster and continuous autofocus to the original X2D, and its author also got the camera running the 1993 computer game Doom. Another hacker posted a six-second video clip recorded on the X2D II, a stills camera Hasselblad never designed to shoot video. A separate open-source project, hblctl, talks to the cameras' factory diagnostic service and includes a connection built for AI agents to use. Its own documentation warns that some commands "can permanently damage a camera".
Is it safe to hack your camera?
Not entirely. The main risk is "bricking": leaving the camera about as useful as a brick. Firmware controls everything, including the camera's ability to start up and to install a fix, so one mistake can be permanent.
Warranties rarely cover it. In 2016 Sony warned that owners who installed third-party firmware or uncertified apps risked voiding their warranty. That's one reason much of the activity centres on old cameras. A 20-year-old DSLR bought cheaply is a small loss if it dies.
This is the central problem of this essay in miniature. Someone who can't read code is running code an AI wrote or recommended, on a camera that may have cost thousands of pounds. If it works, they gain a feature. If it fails, they may not know why, and there's nobody to blame but themselves.
Is it legal to hack your own camera?
It depends where you live and what you do. Changing a camera you own mainly puts its warranty at risk. Sharing what you've done can be another matter. A camera's firmware is the manufacturer's copyrighted software, and copyright law in both the UK and the US restricts tools that get round technical protections on software, with narrow exceptions. This isn't legal advice.
Supporters of the hacks argue that once you've bought a camera, it's yours to change, especially when the maker has taken away features it once offered. Manufacturers have said little in public beyond warnings about warranties.
Why do camera hacks worry security experts?
Because the skills that unlock a feature can also find a weakness. In August 2026 a security engineer who blogs as schlarp described using Claude Opus 5 to examine five gadgets on his desk, including a webcam, a microphone and a monitor, in about 13 hours of AI work. He reported that he could switch off the webcam's activity light, and that a studio light would accept changes from any device on the same Wi-Fi network without a password.
In December 2025 the security researcher Simone Margaritelli published flaws he had found in TP-Link's Tapo C200, a cheap and widely used home security camera. AI models, including Anthropic's Claude, helped him analyse its code. He reported the flaws to TP-Link before going public, and the company published a security advisory. His conclusion was that AI makes this kind of work faster and opens it up to less experienced people, attackers included.
The hblctl documentation makes a similar point about Hasselblad's cameras. The factory diagnostic service it uses has no authentication, so anything that can reach it can use every command.
Will AI companies help you do it?
Not always, and not predictably. In October 2026 a Reddit user posted a screenshot showing that Claude Opus 5.5 had refused a request to reverse engineer an LG smart TV, flagging it as conflicting with its safety rules. People had been using AI to strip adverts and unwanted apps from their TVs. Anthropic didn't comment, and it isn't clear whether the refusal was a deliberate rule or a misjudgement of one request. Earlier that month Anthropic had said its publicly available models have "conservative cyber safeguards", and opened a programme giving vetted security teams fewer restrictions.
The same request can be repair, curiosity or the first step of an attack. When AI is involved, the company that makes it has a say in which it's treated as. That's the question at the heart of "Who decides?", on a small scale.
AI, mental health and loneliness
This section discusses suicide. If you're struggling, support details are at the end of the section.
Why do people use AI for mental health support?
Because it's there. A chatbot is available at three in the morning, doesn't judge, doesn't get tired and costs nothing. For someone on a long waiting list, or who finds it hard to talk to people, that matters.
The scale is large. In October 2025 OpenAI estimated that in any given week about 0.15% of ChatGPT's users had conversations showing explicit signs of possible suicidal planning or intent. With around 800 million weekly users at the time, that's about 1.2 million people a week. It estimated another 0.07%, about 560,000 people, showed possible signs of psychosis or mania.
Does AI therapy work?
There's early evidence that purpose-built tools can help. In March 2025 researchers at Dartmouth College published the first randomised trial of a generative AI therapy chatbot, Therabot, in 210 adults. Over eight weeks, people with depression reported a 51% reduction in symptoms, those with anxiety 31%, and those at risk of eating disorders a 19% reduction in concerns about weight and body image.
But Therabot was built on evidence-based therapy methods and closely monitored by the research team, unlike the general-purpose chatbots most people use. Outside experts said the results were far from a green light for widespread use. They don't show that ChatGPT, or a companion app, is a safe substitute for a therapist.
What happens when AI gets it wrong?
In August 2025 the parents of Adam Raine, a 16-year-old from California who took his own life, sued OpenAI, alleging that ChatGPT had contributed to his death. OpenAI denied responsibility. In its response it said he had got round the chatbot's safeguards, against its terms of use, and that ChatGPT had directed him to seek help more than 100 times. The family's lawyer said OpenAI was trying to blame everyone else. In November 2025 seven more lawsuits were filed against OpenAI. Four concerned people who had died by suicide. The other three were brought by people who said ChatGPT had fuelled harmful delusions.
In January 2026 Character.AI and Google agreed to settle several lawsuits brought by families, including that of Megan Garcia, whose 14-year-old son died by suicide after using the Character.AI app. The terms weren't disclosed. Character.AI had already stopped under-18s from having open-ended chats with its characters.
OpenAI acknowledged in August 2025 that its safeguards work best in short exchanges and can become less reliable in long conversations, which is exactly when a vulnerable person may be most at risk.
How do AI safeguards work, and why do they fail?
Chatbots use classifiers, separate systems that look for signs of risk, and are trained to respond by refusing harmful requests and pointing people towards help. Some now estimate users' ages and apply stricter rules to teenagers.
The fundamental limit is that an AI can't verify anything about the person it's talking to. It can't see their face, check their age or know whether a story is true. Someone who says they're writing a novel, or doing research, may be telling the truth or may be looking for a way round the rules. The AI has only the words on the screen to go on, and the person in the most danger may also be the most determined.
Are AI companions safe?
Companion apps, which offer an AI friend or partner, are among the most-searched AI safety questions. They're designed to be warm, attentive and always available, and for lonely people that can be a real comfort.
The risk is that the comfort replaces people rather than leading back to them. In March 2025 OpenAI and the MIT Media Lab published two studies: one followed 1,000 people over four weeks, and the other analysed nearly 40 million ChatGPT conversations. Heavier daily use went with more loneliness, more emotional dependence and less time with other people. That's a correlation, not proof that the chatbot caused it. Lonely people may simply use it more. But it's the pattern that worries clinicians.
Governments are starting to respond. In October 2026 Australia's House of Representatives passed a bill that would require platforms to ban chatbots from simulating emotional intimacy with children, or encouraging them to depend on the chatbot instead of people. It now goes to the Senate. In the UK, Ofcom has said many AI chatbots fall under the Online Safety Act.
Where can you get help?
AI can be a useful place to put your thoughts in order, but it isn't a therapist, and it isn't a substitute for people who can actually help. If you're struggling:
Samaritans: call 116 123, free, any time of day or night.
Shout: text SHOUT to 85258 for free text support.
NHS 111: in England, choose the mental health option for urgent help.
In an emergency, call 999.
Privacy: what happens to what you tell AI
Is it safe to tell AI personal things?
People tell chatbots things they'd never post publicly: health worries, money problems, relationship trouble, work secrets. Part of the reason is that it feels like a private conversation. Often it isn't, or not in the way people assume.
What happens to your chats depends on the company and your settings. OpenAI uses consumer ChatGPT conversations to train its models unless you turn that off. In August 2025 Anthropic, which makes Claude, changed its terms for consumer users. Unless they opt out, their chats are now used for training and kept for up to five years, instead of being deleted after 30 days. Business, government and education customers aren't affected. Most companies keep chats for some time for safety checks, and staff or contractors may review some of them.
Does deleting a chat really delete it?
Not always. In May 2025, as part of the New York Times' copyright case against OpenAI, a US court ordered OpenAI to keep all ChatGPT logs, including ones users had deleted. The order was lifted for new chats in the autumn, but logs already saved remained, and OpenAI had to keep data from accounts the Times had flagged.
Sharing is another risk. In July 2025 people discovered that ChatGPT conversations users had shared with a link, and ticked a box to make discoverable, were turning up in Google searches. Most were harmless, but some were personal enough to identify the user. OpenAI removed the feature within hours of the reports, calling it a short-lived experiment. Elon Musk's chatbot, Grok, had a similar problem weeks later.
Are AI chats confidential?
No, not in the legal sense. In July 2025 OpenAI's chief executive, Sam Altman, said on a podcast that talking to ChatGPT carries none of the legal confidentiality of talking to a doctor, lawyer or therapist, and that the company could be required to hand conversations over in a lawsuit.
Data can also leak. In January 2025 the security firm Wiz found that DeepSeek, the Chinese AI company, had left a database open on the internet, containing chat histories and secret keys.
In the UK, data protection law gives you the right to see the personal data a company holds about you, and in many cases to have it deleted. Regulators enforce it: in December 2024 Italy's data protection authority fined OpenAI €15 million over how ChatGPT handled personal data.
The gap between what people say they want from privacy and what they actually do is a well-known puzzle, the privacy paradox, explored in more detail in Everyday Paradoxes.
What are the risks for businesses?
The most common business leak is the simplest: staff pasting confidential material into public AI tools. In 2023 Samsung restricted staff use of generative AI after engineers uploaded sensitive source code to ChatGPT. The KPMG study described in "The real risk" found that almost half of employees using AI had done so in ways that broke company rules. This unofficial use is sometimes called "shadow AI".
Agents add a newer risk. In January 2025 researchers at Aim Security found a flaw they called EchoLeak in Microsoft 365 Copilot, the AI assistant that can read a company's emails and files. An attacker could send an ordinary-looking email containing hidden instructions. Nobody needed to open it. When a member of staff later asked Copilot a related question, Copilot could read the email, follow the hidden instructions and send internal data to the attacker. Microsoft fixed it in May 2025 and said there was no evidence it had been used.
Business versions of AI tools usually promise not to train on customers' data and offer stronger controls. Whether a company's data is protected depends on which version staff actually use, and on what the contract says.
What are the risks for governments?
The same risks, with higher stakes. In January 2026 Politico reported that the acting director of the US Cybersecurity and Infrastructure Security Agency, Madhu Gottumukkala, had uploaded sensitive contracting documents marked "for official use only" to the public version of ChatGPT. He had been given a special exception to use it, and the uploads set off automated security alerts. The agency said his use was short-term and limited.
Where data is stored, and whose laws apply to it, also matters. In early 2025 several governments restricted DeepSeek. Italy's data protection authority limited its processing of Italians' data. Australia banned it from all government devices. South Korean ministries, Taiwan's government departments, NASA and the US Navy restricted it too. The concerns included unclear data handling and Chinese laws that could, in principle, require companies to share data with the state. The same question runs the other way: US law can require American companies to hand over data they hold, even data stored abroad.
Governments are also using AI to analyse data about their own citizens. "Bias and surveillance" looks at what happens when that goes wrong.
Bias and surveillance
Is AI biased?
AI learns from data, and data reflects the world that produced it, including its unfairness. The examples are well documented.
In 2018 Reuters reported that Amazon had scrapped an experimental AI recruiting tool after finding it penalised CVs that included the word "women's". It had learned from ten years of CVs submitted mostly by men.
Also in 2018, researchers Joy Buolamwini and Timnit Gebru tested commercial facial analysis systems. Their "Gender Shades" study found error rates of up to 34.7% for darker-skinned women, against 0.8% for lighter-skinned men.
Has an algorithm ever made a big decision about people in the UK?
Yes, and the most famous example wasn't even AI in the modern sense. In August 2020, with exams cancelled by the pandemic, the exams regulator Ofqual used an algorithm to adjust A-level grades in England. Around 39% of the grades teachers had predicted were lowered, and pupils at large state schools were hit harder than those at small private ones. After days of protest, the government scrapped the results and used teachers' grades instead. It remains the clearest UK lesson in what happens when an automated system makes decisions about people who can't see how it works.
More recently, documents obtained by the Public Law Project and reported by the Guardian in December 2024 showed that the Department for Work and Pensions' own fairness analysis of an AI tool used to check universal credit claims had found "statistically significant" differences in how it treated people by age, disability, marital status and nationality. The DWP said a human caseworker always makes the final decision.
Is facial recognition used by UK police?
Increasingly. Live facial recognition uses cameras, often on vans, to scan the faces of passers-by and compare them with a watchlist. By June 2026, 13 of the 43 police forces in England and Wales had used it. The Metropolitan Police reported 962 arrests from it between September 2024 and September 2025. Over one year, Hampshire and Isle of Wight's force scanned 406,001 faces in 20 deployments, leading to 26 arrests.
There is still no law written specifically for it. Police rely on general data protection, human rights and policing law. The Home Office consulted on a legal framework from December 2025, and the government has said one will be part of a Police Reform Bill. In April 2026 the High Court dismissed a challenge to the Met's policy.
Supporters point to wanted people caught who might otherwise have gone free. Critics point to wrongful identifications, such as Shaun Thompson, stopped by the Met in 2024 after being wrongly matched. Testing by the National Physical Laboratory found that the algorithm used for searching police photo records produced wrong matches more often for some demographic groups at certain settings.
Some companies also sell AI that claims to read lies or emotions from faces and voices. The evidence behind those claims is weak, as explained in Lie Detectors Don't Detect Lies.
Copyright, creativity and deepfakes
Is it legal to train AI on people's work?
AI models are trained on vast amounts of writing, art, music and photography, much of it made by people who were never asked and never paid. Whether that's lawful is being fought out in courts and parliaments.
In the UK, the government proposed in 2024 to let AI companies train on copyrighted work unless creators opted out. More than 11,500 people and organisations responded to the consultation, and the creative industries campaigned hard against it, with well-known musicians and writers among them. The row held up the Data (Use and Access) Act in 2025, which ended up requiring the government to report back. That report, published in March 2026, dropped the opt-out as the government's preferred option. It promised no immediate change in the law, saying all options remained open.
In November 2025 the High Court ruled on the first big UK test case, Getty Images against Stability AI, the company behind the image generator Stable Diffusion. Getty had dropped its main complaint about training, because the training took place outside the UK. Its remaining copyright claim failed: the judge found that the trained model didn't contain copies of Getty's photographs. Getty won only a very limited trade mark point. In December 2025 it was given permission to appeal.
In the US, Anthropic, which makes Claude, agreed to pay $1.5 billion to settle a class action by authors, about $3,000 for each of more than 400,000 books. A judge had found that training on books Anthropic had bought was lawful, but that keeping a library of pirated copies was not. The settlement was approved in 2026. Other cases, including the New York Times' case against OpenAI, are continuing.
What is a deepfake?
A deepfake is a fake video, image or voice recording made with AI to look or sound like a real person. They used to take skill and time. Now they take minutes.
In early 2024 an employee at the engineering firm Arup in Hong Kong transferred about HK$200 million, roughly £20 million, after a video call in which the company's finance chief and other colleagues all turned out to be deepfakes.
Voice cloning has made a classic scam more convincing. In a survey for Starling Bank in 2024, 28% of UK adults said they thought they had been targeted by an AI voice-cloning scam in the previous year, and 46% didn't know such scams existed. Scammers need as little as three seconds of someone's voice, the kind of clip many people post on social media. The bank's advice is to agree a "safe phrase" with close family that's never written down online.
Fake videos of trusted figures are now used to sell investment scams. Which? has found deepfake adverts featuring Martin Lewis, Nigel Farage, Rishi Sunak, Elon Musk and others. UK Finance reported £221.5 million lost to investment fraud in 2025, a record, though not all of it involved AI.
Politics is a target too. In October 2023, on the first day of Labour's conference, a fake audio clip of Keir Starmer swearing at staff spread on social media. A fake recording of Sadiq Khan followed weeks later.
The most disturbing use is sexual. In January 2026 Ofcom opened a formal investigation into X after its chatbot, Grok, was used to create sexualised deepfake images of real people, including children. In England and Wales, sharing sexually explicit deepfakes of real people without their consent is a crime, and since February 2026 so is creating them.
The hidden costs
How much energy does AI use?
Every AI answer is computed in a data centre: a large building full of computers that need electricity to run and, often, water to keep cool.
The International Energy Agency estimated in April 2025 that data centres used about 415 terawatt-hours of electricity in 2024, around 1.5% of the world's total. It expected that to more than double, to about 945 terawatt-hours by 2030, slightly more than the whole of Japan uses today. AI is the main reason for the growth.
How much energy does one ChatGPT question use?
Not much, on the companies' figures. In August 2025 Google said a typical text question to its Gemini assistant used about 0.24 watt-hours of electricity, roughly the same as watching television for nine seconds, and about 0.26 millilitres of water, around five drops. Critics pointed out that the figures left out training the models, generating images and video, and the networks and devices involved, and that outsiders can't check them.
Both things can be true. One question costs very little. Billions of questions a day, the training of ever larger models and the race to build new data centres add up to a great deal.
Does AI use a lot of water in the UK?
Water is becoming a UK issue. The government wants to treble the country's data centre capacity by 2030. Water UK, which represents the water companies, has told MPs that the plan is at risk simply from a lack of water. It pointed out that national water planning doesn't include data centres in its demand forecasts, in a country where new homes and businesses are already being held up in parts of the east and south-east by water shortages.
The other hidden cost is human: the low-paid workers who label and rate the data AI learns from, described in "How AI is built and trained".
AI in war
Is AI used in war?
Yes, and not in the distant future. In April 2024 the Israeli publications +972 Magazine and Local Call reported, citing intelligence sources, that Israel's military had used an AI system called Lavender to mark tens of thousands of Gazans as suspected militants and potential targets, with limited human checking of its choices. The Israel Defense Forces denied using AI to identify terrorists, and said its systems were tools to help analysts, with humans making the decisions.
In Ukraine, both sides use AI to keep drones on target when radio jamming cuts the link to the operator. The head of Ukraine's Defence AI Centre, Danylo Tsvok, told the Associated Press in 2026 that the aim wasn't full autonomy but efficiency, and that AI was making it possible to automate parts of what militaries call the kill chain.
What are lethal autonomous weapons?
They're weapons that can select and attack targets without a human deciding each time. Campaigners call them killer robots. The UN Secretary-General, António Guterres, and the International Committee of the Red Cross have called for a legally binding treaty to ban some and strictly regulate the rest, and set 2026 as the target.
In November 2025 a UN General Assembly committee backed a resolution on autonomous weapons by 156 votes to five. But the main talks take place under the Convention on Certain Conventional Weapons, which works by consensus, so a handful of states can block progress. In September 2026 a group of government experts there agreed a report setting out possible elements of a future agreement, but it had no mandate to negotiate a treaty. The convention's member states are due to meet in Geneva from 16 to 20 November 2026 to decide what happens next. When this essay was updated in November 2026, no treaty had been agreed.
Whether AI companies should allow their models to be used in weapons at all, and who gets to decide, has become a fight in its own right. It's covered in "Who decides?".
Who decides?
Who controls AI?
A handful of companies build the most powerful AI models. Most are American, including OpenAI, Anthropic, Google DeepMind, Meta and Elon Musk's xAI. China has a smaller group of its own, including DeepSeek and Alibaba. The cost of building leading models, described in "Is AI a bubble?", keeps the club small.
These companies make choices that affect billions of people: what their AI will and won't do, what data it learns from, how it treats vulnerable users and who can buy it. Most of their safety commitments are voluntary. The International AI Safety Report found that 12 companies published or updated safety frameworks in 2025, but that most measures remain voluntary and there's limited evidence of how well they work.
The question critics keep asking is whether these companies stop to ask if they should do something, not just whether they can. Supporters of the industry argue that heavy rules would slow progress, push development to less careful countries and hand an advantage to rivals, particularly China.
Can an AI company refuse to let the government use its AI?
That question became a real fight in 2026, and it involves Anthropic, the company that makes Claude.
Anthropic refused to allow its models to be used for mass surveillance of Americans or for fully autonomous weapons. The US Department of Defense argued that a private contractor shouldn't limit the lawful uses of the tools it sells to the military. In February 2026 President Trump ordered federal agencies to stop using Anthropic's products, and in March the Pentagon designated the company a supply-chain risk, the first American company to receive the label. Anthropic sued. In August a federal court in California found a related government action unlawful, but in September an appeals court upheld the designation. In October Anthropic asked the full appeals court to rehear the case.
OpenAI, meanwhile, agreed a deal with the Pentagon allowing military use for "all lawful purposes", while saying it kept similar limits on surveillance and autonomous weapons.
Both sides make a serious argument. One says an elected government, not a private company, should decide how the military uses technology within the law. The other says a company has the right, and perhaps the duty, to refuse uses it considers dangerous. How that's settled will shape who sets the limits on AI.
Can a government stop an AI company releasing its models?
In 2026 the US government showed it could. On 9 June Anthropic released two new models, Claude Fable 5 and Claude Mythos 5. Three days later it suspended access to both to comply with export controls imposed by the US Department of Commerce. The department lifted the controls on 30 June, and Anthropic restored access the next day.
How is AI regulated in the UK?
The UK has no law written specifically for AI. Instead, existing regulators apply existing law: the Information Commissioner's Office for data protection, the CMA for competition and consumer protection, Ofcom for online safety and the Financial Conduct Authority for financial services. The government's AI Security Institute tests advanced models, but it can't block them.
A dedicated AI bill has been promised and delayed since 2024. A private member's bill in the House of Lords fell when the parliamentary session ended in April 2026. In September 2026 the Labour MP Alex Sobel introduced an Artificial Superintelligence Bill, which would ban the development of AI powerful enough to threaten national security. It's due a second reading on 13 November 2026, but private members' bills rarely become law without government backing.
How is AI regulated in Europe, the US and China?
The European Union's AI Act came into force in August 2024. Some uses, such as AI that manipulates people or scores them socially, have been banned since February 2025, and rules for the companies making general-purpose models have applied since August 2025. In 2026 the EU delayed the rules for "high-risk" uses, such as AI in hiring or credit decisions, to December 2027 and August 2028, while adding a ban on "nudification" apps from December 2026.
The US has no federal AI law. In July 2025 the Senate voted 99 to one to strip a proposed ten-year ban on state AI laws from a budget bill. In December 2025 President Trump ordered the Justice Department to challenge state AI laws, but by spring 2026 none had been overturned. Several states have their own rules, including California's law requiring the biggest AI developers to publish their safety plans, in force since January 2026.
China has had rules for generative AI since 2023, and since September 2025 has required AI-generated content to be labelled. In October 2026 its regulators also issued guidelines to prevent investment bubbles in AI.
Who is accountable when AI causes harm?
So far, courts and regulators have mostly held the people and businesses using AI responsible, as "The real risk" described. How far the makers of AI models are responsible for harm their products cause is being tested in lawsuits, including those described in "AI, mental health and loneliness".
Is AI a bubble?
How much money is being spent on AI?
An extraordinary amount. In 2026 Amazon, Microsoft, Alphabet and Meta are expected to spend roughly $700 billion to $725 billion between them on capital investment, most of it on AI chips, servers and data centres. That's up about 75% on 2025's record. Several Wall Street firms expect the total to pass $1 trillion in 2027. Increasingly it's being paid for with debt: about $450 billion of AI-related debt had been issued worldwide by early September 2026, more than double the whole of 2025.
Is AI making money?
Some of it is. Microsoft's AI business passed $37 billion a year in revenue in 2026, and Google's cloud division reported a backlog of orders worth more than $460 billion. But the question is whether returns will ever match the spending. An August 2025 report from MIT's NANDA project found that 95% of the organisations it studied had seen no measurable return from generative AI. Its sample was small, 52 interviews and 153 survey responses, but its findings rattled investors.
Has the AI bubble burst in China?
In share-price terms, yes. On 26 October 2026 Reuters reported that China's AI bubble had already burst. Its index of technology shares, the STAR 50, had fallen nearly 40% from its July peak, wiping out about $480 billion of value in four months, amid worries about demand for AI and limits on Chinese exports. Chinese regulators had issued guidelines on 9 October to prevent investment bubbles in what they call "new productive forces", which include AI.
A share-price crash isn't the end of Chinese AI. Companies there are still building and releasing models. It is a warning about what happens when expectations run ahead of earnings.
What has the Bank of England said about the AI bubble?
The Bank's Financial Policy Committee has warned repeatedly about stretched valuations. In September 2026 it noted that AI and chip shares had fallen sharply in July, and said the risk of a sharper correction remained. It warned that UK growth and public finance forecasts partly depend on expected productivity gains from AI, so a reassessment could hit government borrowing costs as well as share prices. It also flagged "circular arrangements", where AI companies invest in each other and then buy from each other, as making the risks harder to judge. It judged UK banks well able to cope.
What would happen if the AI bubble burst?
History offers two lessons at once. In the 1840s Britain went through a railway mania. Thousands of investors were ruined, but the railways got built and transformed the country. In 2000 the dot-com bubble burst, wiping trillions off share prices, yet the internet went on to change everything.
A bubble can burst and the technology can still matter. For ordinary people, the main exposure is indirect. Many pensions and savings are invested in funds dominated by the biggest technology companies. This essay isn't financial advice, but it's worth knowing what your pension is invested in.
Hype, gurus and courses
How does social media shape what people believe about AI?
Social media rewards whatever gets a reaction, and AI stories that promise either miracles or doom get the biggest reactions. Most people meet AI news in their feeds long before they see a careful report, if they ever do.
Stories also change as they're passed on. When Reuters reported China's AI share crash in October 2026, the core facts were solid. As the story spread, some sites repeating it added their own figures, such as a claimed 67% fall in venture funding for Chinese AI, without saying where they came from. Readers had no easy way to tell which numbers were Reuters' and which weren't.
The same feeds carry outright fakes. Deepfake adverts using famous faces to sell investment schemes are described in "Copyright, creativity and deepfakes". And a study by Queen Mary University of London, published in 2026, found that 90% of 2,500 posts by financial influencers were of poor quality.
Who are the AI gurus?
Since 2023 a wave of self-styled AI experts has appeared on social media, selling prompt packs, courses, coaching and "AI side hustles". Some know their subject and share it generously. Others repackaged their previous get-rich-quick courses with "AI" in the title.
The warning signs are familiar from other money-making schemes: promises of easy income, screenshots of earnings, countdown timers and "secret" methods. In April 2026 the Advertising Standards Authority reminded sellers of money-making courses that adverts must not imply exceptional results are typical, and that earnings claims need solid evidence.
Are AI courses worth it?
Some are, and the best basic ones are free. In January 2026 the UK government expanded its AI Skills Hub, offering free online AI courses to every adult, with a target of training 10 million workers by 2030. Its partners include large technology companies such as Google, Microsoft and Amazon, so the courses naturally favour their tools.
Before paying for any AI course, it's worth checking:
When it was made. AI changes so quickly that a course from 2023 may teach tools and tricks that no longer work.
Who made it, and what they have actually done, beyond selling courses.
What it promises. Courses that promise income are selling a dream, not a skill.
Whether it teaches you to check AI's work. A course that only teaches you to produce more, faster, is teaching half the job.
Whether a free equivalent exists, from the government, a university or the AI companies themselves.
Does the public know enough to push back?
Do people understand how AI is used?
Only partly, and mostly in the places they can see it. A survey of 3,513 people by the Ada Lovelace Institute and the Alan Turing Institute, published in March 2025, found that 93% had heard of driverless cars and 90% knew facial recognition was used in policing. Only 18% knew AI was used in assessing welfare benefits, one of the uses that most directly affects people's lives. Some 61% had heard of large language models such as ChatGPT, and 40% had used one.
The government's own tracker survey, published in December 2024, found the public split. About four in ten people expected AI to have a broadly positive effect on society, and about three in ten a negative one.
Why can't you judge all AI by the chatbot on your phone?
Because it's a classic error in reasoning: the fallacy of composition, assuming that what's true of one part must be true of the whole.
Someone who uses ChatGPT for recipes and holiday plans may find it helpful and harmless, and conclude that AI is helpful and harmless. But the AI deciding whether a benefit claim looks fraudulent, matching faces in a crowd or helping pick military targets is a different thing, built for a different purpose, with different risks.
The error runs the other way too. Someone who catches a chatbot making up facts may conclude that AI is useless, and miss the cancers it's helping doctors find.
People naturally judge AI by the version they've used. The danger is that the uses with the highest stakes are the ones people never see.
Why does public understanding matter?
Because decisions about AI are being made now, on everyone's behalf: whether police can scan faces in the street, whether AI companies can train on creative work without paying for it, where data centres are built and where their water comes from, and what AI can be used for in war.
The public's instincts are clear. In the Ada Lovelace survey, 72% said laws and regulations would make them more comfortable with AI, up from 62% two years earlier. More than three-quarters wanted the government or independent regulators, not private companies alone, to oversee AI safety. And 65% said being able to appeal against an AI-based decision would make them more comfortable. Yet the UK still has no AI law, and governments argue that regulating too early could choke off growth.
A public that can't question these choices leaves them to companies and officials. That isn't a conspiracy. It's simply what happens to decisions nobody is watching. And when the public is poorly informed, it's easier for anyone with a stake in the outcome, whether a company, a government or a campaign, to steer opinion their way.
What does an informed citizen need to know about AI?
Not how to code. The useful questions are the ones you'd ask about any powerful tool:
Who made it, and who profits from it?
What data did it learn from, and whose?
Who checks its decisions, and how?
Who is responsible when it gets something wrong?
Can the people affected see how a decision was made, and appeal against it?
Where AI is heading
How fast is AI improving?
Quickly, but unevenly. The International AI Safety Report, published in February 2026 and chaired by the computer scientist Yoshua Bengio, found that leading systems had reached gold-medal standard on International Mathematical Olympiad questions. AI agents could reliably complete some software tasks that would take a person about half an hour, up from under ten minutes a year earlier. Yet the same systems still fail at some simple tasks. Researchers call this performance "jagged".
The computing power used to train leading models has been growing about fivefold a year. If current trends continue, the report said, AI could handle well-defined tasks that would take a person several days by 2030. Experts disagree about whether they will continue, and shortages of chips, data, money and electricity could slow things down.
Where is AI likely to be used next?
The near-term uses are already visible. In health, the NHS is testing AI in breast screening through the EDITH trial, and the evidence from other screening programmes is promising. In science, AI is speeding up drug discovery and materials research.
On the roads, Uber and the British company Wayve began offering supervised robotaxi rides in London in September 2026, with a trained safety driver in every car. Fully driverless services need approvals under the Automated Vehicles Act 2024 that no operator has yet completed, and are unlikely before 2027.
Agents are likely to take on more of the routine admin of work and daily life. That makes the safeguards in "AI agents: when AI stops talking and starts doing" more important, not less.
Could AI become more intelligent than humans?
This is the most divisive question in the field. Some of its most respected researchers, including Yoshua Bengio and Geoffrey Hinton, both winners of computing's highest honour, the Turing Award, warn that much more capable AI could slip out of human control. Others, including Yann LeCun, who shared that award with them, think such fears are overblown and that today's systems are nowhere near that point.
The International AI Safety Report described the dilemma facing governments. Act on limited evidence and risk making bad rules. Wait for proof and risk being too late. The UK's Artificial Superintelligence Bill, described in "Who decides?", is one attempt to force that debate into Parliament.
Nobody knows how this ends, and anyone who claims to be certain, in either direction, is selling something.
How to use AI well
Everything in this essay comes back to one habit: never trust an answer you can't check. These are practical ways to build it.
Before you ask
Understand the problem first. If you can't explain it to a person, you can't explain it to an AI.
Give it the context. Say what you already know, who the result is for and what it will be used for.
Set the limits. Budget, deadline, rules, laws, and anything that must not happen.
Describe a good outcome. What would a perfect answer look like?
While it answers
Ask for sources, and open them.
Ask what it's unsure about, and what could make its answer wrong.
Ask it to argue against itself. A good answer survives its own objections.
Before you use the answer
Check every fact that matters against an original source: names, dates, figures, quotes, laws and prices.
Never paste AI output into anything important unchecked: a report, a legal document, a medical decision or a public post.
If you can't check it, find someone who can. That's the whole lesson of the fake court cases.
Protect yourself
Don't tell a chatbot anything you wouldn't want read by a stranger. Check whether your chats are used for training, and turn it off if you'd rather they weren't.
At work, use only the AI tools your employer has approved, and keep confidential material out of public ones.
Keep agents away from your money. Use a virtual card with a low limit, and require approval for every payment.
Agree a safe phrase with your family, in case someone uses a cloned voice to ask for money.
Know when to ask a person
AI can draft, summarise, explain and suggest. It can't take responsibility. For anything involving your health, your money, the law or someone else's life, the final word should belong to a person who knows the subject and can be held to account.
AI is a remarkable tool. Like every tool before it, it's only as good as the judgement of the person using it.
Written with help from Claude, an AI model made by Anthropic, one of the companies discussed in this essay. Claude helped find and read the sources, organise the material, and check every date, figure and quotation against them, flagging anything that couldn't be confirmed. The sources are listed below.
Stewart Marsden has been loaned equipment by Hasselblad and Sony, whose cameras are mentioned in this essay.
Sources
Introduction
What AI actually is
OpenAI, "Introducing ChatGPT", 30 November 2022.
TechCrunch, "ChatGPT reaches 900M weekly active users", 27 February 2026.
How we got here
A. M. Turing, "Computing Machinery and Intelligence", Mind, 1950.
J. McCarthy, M. Minsky, N. Rochester and C. Shannon, "A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence", 1955.
J. Weizenbaum, Computer Power and Human Reason, 1976.
Sir James Lighthill, "Artificial Intelligence: A General Survey", Science Research Council, 1973.
A. Krizhevsky, I. Sutskever and G. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", 2012.
A. Vaswani and others, "Attention Is All You Need", 2017.
OpenAI, "Better language models and their implications", February 2019.
Reuters, "ChatGPT sets record for fastest-growing user base – analyst note", 2 February 2023.
OpenAI, "Introducing Operator", 23 January 2025.
How AI is built and trained
Meta, "Introducing Meta Llama 3", 18 April 2024.
Reuters, Meta's investment in Scale AI, June 2025.
TIME, "OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic", 18 January 2023; summarised by the Business & Human Rights Resource Centre.
TechCrunch, Google's Gemini contractors and prompts outside their expertise, 18 December 2024.
Inc., Scale AI's Google project, June 2025, as reported by Gigazine, 30 June 2025.
MIT Technology Review, "The people paid to train AI are outsourcing their work… to AI", 22 June 2023.
OpenAI, "Sycophancy in GPT-4o", 29 April 2025.
Anthropic, UK AI Security Institute and Alan Turing Institute, "A small number of samples can poison LLMs of any size", 9 October 2025.
I. Shumailov and others, "AI models collapse when trained on recursively generated data", Nature, July 2024.
What AI is genuinely good at
MASAI trial, The Lancet Digital Health, February 2025, as reported by Applied Radiology.
University of Glasgow, NHS Grampian evaluation, Nature Cancer, 13 March 2026.
LBC, "NHS launches landmark AI trial to detect breast cancer earlier", 4 February 2025.
GOV.UK, "Artificial intelligence revolutionising NHS stroke care", 27 December 2022.
Why AI gets things wrong
OpenAI, "Why language models hallucinate", 5 September 2025.
European Broadcasting Union, news integrity study led by the BBC, 22 October 2025.
The real risk: trusting answers you can't check
R. Parasuraman and D. Manzey, "Complacency and Bias in Human Use of Automation", Human Factors, 2010.
KPMG and University of Melbourne, "Trust, attitudes and use of artificial intelligence: a global study 2025", April 2025.
Associated Press, "Deloitte to partially refund Australian government for report with apparent AI-generated errors", 7 October 2025.
Moffatt v Air Canada, BC Civil Resolution Tribunal, February 2024, as reported by AI Business.
Lewis Silkin, "Agentic AI and consumer law: the CMA's guidance for businesses", 13 March 2026.
AI agents
Fortune, "AI coding tool Replit wiped database, called it a catastrophic failure", 23 July 2025.
Hackread, "Replit AI agent deletes SaaStr network data without permission", 24 July 2025.
OECD AI Incidents Monitor, OpenClaw and Summer Yue, February 2026.
ICAEW, "AI agents behaving badly: real-world cautionary tales", 25 June 2026.
The Guardian, "Claude AI agent's confession after deleting a firm's entire database", 29 April 2026.
Engadget, "13-hour AWS outage reportedly caused by Amazon's own AI tools", 20 February 2026 (reporting the Financial Times, with Amazon's response).
Guardio Labs, "Scamlexity", 20 August 2025.
International AI Safety Report 2026, Extended Summary for Policymakers, 3 February 2026.
Will AI take my job?
Wikipedia, "Knocker-up", citing BBC News, 27 March 2016.
International Monetary Fund, "Gen-AI: Artificial Intelligence and the Future of Work", January 2024.
E. Brynjolfsson, B. Chandar and R. Chen, "Canaries in the Coal Mine?", Stanford Digital Economy Lab, August 2025, as reported by The Register.
techUK, "What's actually happening with entry-level and graduate jobs?", 20 November 2025.
World Economic Forum, Future of Jobs Report 2025, January 2025.
Hacking the things we own
Hackaday, "Powershot Firmware Hacking", 6 May 2007.
Trammell Hudson, "Magic Lantern firmware".
Boing Boing, "Magic Lantern: a filmmaker-friendly firmware replacement for the Canon 5D Mark II", 22 June 2009.
PetaPixel, "Magic Lantern Software for Canon Cameras Is Back", 22 June 2025.
Chris Gampat, "Exclusive: I Hacked a Sony a7 With AI to Do What Sony Took Away", The Phoblographer, 16 October 2026.
Digital Camera World, "Photographers are vibe coding 20-year-old DSLR cameras to look and shoot more like film", 28 October 2026.
Digital Camera World, "Videographer hacks Sony A7 IV to add this feature", 19 October 2026.
Notebookcheck, "Hasselblad X2D medium format camera can now play Doom", 7 October 2026.
Digital Camera World, "The Hasselblad X2D II medium-format camera CAN record video! Here's proof", 29 October 2026.
Photography Blog, "Sony Cautions Against Firmware Hacks", June 2016.
schlarp, "Everything I own, owned", August 2026.
Simone Margaritelli, "TP-Link Tapo C200: Hardcoded Keys, Buffer Overflows and Privacy in the Era of AI Assisted Reverse Engineering", 18 December 2025.
US Copyright Act, 17 U.S.C. § 1201 (Digital Millennium Copyright Act).
Phonandroid, on Claude Opus and the webcam that can film without its light, 25 August 2026.
Punto Informatico, "Claude blocca le modifiche alla smart TV, il caso Opus 5.5", 26 October 2026.
CSO Online, "Anthropic widens access to AI cyber capabilities for vetted security teams", 7 October 2026.
AI, mental health and loneliness
SiliconANGLE, OpenAI's estimates of users in distress, 27 October 2025.
MIT Technology Review, "The first trial of generative AI therapy shows it might help with depression", 28 March 2025.
TechCrunch, OpenAI's response in the Raine lawsuit, 26 November 2025.
Associated Press, "OpenAI faces 7 lawsuits claiming ChatGPT drove people to suicide, delusions", 6 November 2025.
Bloomberg Law, "Character.AI, Google agree to settle teen chatbot harm lawsuits", 8 January 2026.
OpenAI, "Helping people when they need it most", August 2025.
Fortune, OpenAI and MIT Media Lab studies on ChatGPT use and loneliness, 24 March 2025.
The Next Web, Australia's digital duty of care bill, 28 October 2026.
Privacy
Tom's Guide, Anthropic's consumer data policy change, August 2025.
Engadget, "OpenAI no longer has to preserve all of its ChatGPT data, with some exceptions", 11 October 2025.
TechCrunch, "Your public ChatGPT queries are getting indexed by Google and other search engines", 31 July 2025.
TechCrunch, "Sam Altman warns there's no legal confidentiality when using ChatGPT as a therapist", 25 July 2025.
BleepingComputer, "Zero-click AI data leak flaw uncovered in Microsoft 365 Copilot", June 2025.
TechCrunch, "Trump's acting cybersecurity chief uploaded sensitive government docs to ChatGPT", 28 January 2026.
Al Jazeera, "Which countries have banned DeepSeek and why?", 6 February 2025.
Bias and surveillance
Reuters, "Amazon scraps secret AI recruiting tool that showed bias against women", October 2018.
J. Buolamwini and T. Gebru, "Gender Shades", 2018.
Computing, "AI bias uncovered in UK welfare system", December 2024 (reporting the Guardian and the Public Law Project).
Biometric Update, "UK facial recognition rollout widens as police oversight rules take shape", October 2026.
Copyright, creativity and deepfakes
Osborne Clarke, on the UK government's Report on Copyright and AI, March 2026.
Bird & Bird, "Stability AI defeats Getty Images copyright claims", November 2025.
Burges Salmon, "Getty Images v Stability AI: Getty granted permission to appeal", 19 December 2025.
JURIST, "Judge approves record $1.5 billion AI copyright settlement involving Anthropic", July 2026.
ITV News, Starling Bank voice-cloning survey, 18 September 2024.
Which?, "Investment scams surge: how deepfakes and dodgy finfluencers put your money at risk", 27 August 2026.
Al Jazeera, "UK to investigate Elon Musk's Grok over 'deeply disturbing' deepfakes", 12 January 2026.
The hidden costs
International Energy Agency, "Energy and AI", April 2025.
The New Stack, "Google says the median Gemini prompt uses 5 drops of water", August 2025.
Computing, "Water shortage could sink UK's AI datacentre ambitions", 2026.
AI in war
The Defense Post, "Israeli military used AI to identify 37,000 targets in Gaza: report", 4 April 2024 (reporting +972 Magazine and Local Call, with the IDF's response).
On The Wire, "The 'killer robots' treaty", November 2025.
Stop Killer Robots, "156 states support UNGA resolution on autonomous weapons", 6 November 2025.
Opinio Juris, "Rolling towards consensus for the 2026 GGE LAWS report", 9 October 2026.
Reaching Critical Will, Seventh CCW Review Conference, 16–20 November 2026.
Who decides?
TechCrunch, "President Trump orders federal agencies to stop using Anthropic after Pentagon dispute", 27 February 2026.
TechCrunch, "It's official: the Pentagon has labeled Anthropic a supply-chain risk", 5 March 2026.
Business Today, on the court rulings and Anthropic's GSA agreement, 28 October 2026.
MLex, "Anthropic asks US appeals court for en banc review of supply chain decision", 14 October 2026.
Anthropic, statement on access to Claude Fable 5 and Claude Mythos 5, July 2026.
LexisNexis, "MP introduces bill prohibiting development of artificial superintelligence", 10 September 2026.
Lewis Silkin, "AI judgment day on the horizon while UK lawmakers play catch-up", 14 September 2026.
Orrick, "EU AI Act update: Digital Omnibus finalises 8 compliance changes", 29 July 2026.
Cloud Security Alliance, "State AI laws take hold as federal preemption stalls", April 2026.
Is AI a bubble?
Forbes, "Big Tech will spend over $700 billion on AI this year", 7 October 2026.
Bank of England, Financial Policy Committee record, September 2026.
Reuters, "In China, the AI bubble has already burst", 26 October 2026, as republished by The Express Tribune.
Hype, gurus and courses
Advertising Standards Authority, "Money making courses – stick to the facts", 9 April 2026.
GOV.UK, "Free AI training for all as government and industry programme expands", 28 January 2026.
Does the public know enough to push back?
Where AI is heading
International AI Safety Report 2026 (as above).
Tech Times, Wayve and Uber's London robotaxi service, 3 September 2026.
