Responding to the Risks of Open-Weight AI Models

Criminals are rapidly adopting AI. Google’s May 2026 threat report described criminals preparing a mass-exploitation campaign using a zero-day exploit assessed with high confidence to have been developed with AI assistance . Anthropic’s June 2026 analysis documented increasing model use for malware development, credential theft and data exfiltration. And in July 2026, OpenAI reported disrupting a Cambodia-based scam network that used ChatGPT to facilitate investment, romance and law-enforcement impersonation fraud. The UK’s National Cyber Security Centre reached a consistent judgement at the strategic level, assessing that AI will ‘almost certainly’ continue to make cyber intrusion operations more effective and efficient, and that the proliferation of open source and commercially available models will lower the barrier to entry for a widening range of actors.

So far, most documented activity has used closed-weight models – those served by the frontier AI companies that also develop them. In contrast, open-weight models introduce new and unique risks – they provide near-frontier capabilities which could be used for criminal purposes, are widely available to anyone who intends harm, and are easier to misuse. This piece explores those risks and how UK investigatory powers might apply to them, and argues that policymakers, law enforcement and researchers need to understand them now, so that relationships, guidance and capabilities can be put in place before the use of highly capable open-weight models becomes routine in serious criminal operations.

Closed-Weight versus Open-Weight

A model’s weights are the numerical values – typically billions – that a model learns during training. Given a model’s weights and enough compute to run them, anyone can use the model. They are also just a file – very expensive to produce, but trivial to copy, and impossible to recall once distributed. Major frontier AI companies such as OpenAI, Anthropic and Google do not release the weights of their frontier models. This means they alone can patch the model, add or tighten safeguards, and as a last resort remove the model from circulation entirely, as happened in June 2026 when the US government directed Anthropic to suspend access to Fable 5. An open-weight model, on the other hand, has no such restrictions – the developer publishes the weights and thereafter has no meaningful ability to recall, patch or supervise any copies.

Open-weight models are run in one of two ways. They can be served by inference providers – companies that host the model behind an API (application programming interface) and charge for access, often renting the underlying GPUs (graphics processing units) from a third party rather than owning the chips themselves. Or they can be self-hosted – run on infrastructure owned by an individual or group, which can be expensive. – R unning a frontier open-weight model released in July 2026 would cost roughly £220,000–260,000 to purchase or £31,000–34,000 per month to rent the required infrastructure (eight B300 GPUs or equivalent). Self-hosting has advantages – models can be run within local jurisdictions, giving governments more control over deployments and a greater degree of sovereignty. However, self-hosted deployments may be easier to use in the furtherance of criminal activity, as using the models requires limited interaction with or reliance on other parties.

No model developer wants their models to be used to cause harm. o S afeguards are implemented to prevent this, and can broadly be split into two categories. The first is refusals learned through alignment training, so the model itself declines certain requests – this can be considered ‘baked in’ to the model, and as such travels with the model weights. The second is safety classifiers outside the model itself that screen prompts and responses and block or flag those which might be dangerous. Safety classifiers can be used with any deployment, and indeed standalone safetyclassifiers are available. But there is nothing to compel an open-weight provider to run them, and given that there are many providers that supply access to the same models, a determined malicious actor can search for a deployment that does not run with those safeguards. A review of the published documentation for five prominent open-weight providers – Together AI, Fireworks AI, Groq, DeepInfra and Novita AI – found that none publicly documented a universal, mandatory external safety classifier applied to every prompt and generated output. Where classifiers were offered, they were generally configurable, separately invoked, or left to customers to integrate, should they wish to do so.

In-model safety training is also fragile. Open-weight models can be fine-tuned to strip out refusals cheaply. For example, researchers at Palisade Research removed the safety training from Meta’s Llama 3 8B in around five minutes on a single GPU for under a dollar, and from the 70B model in well under an hour. A technique known as ‘abliteration’goes further, the model from refusing unsafe requests while retaining most of its general capability. It is now relatively inexpensive to fine-tune an open-weight model to remove the in-model safety layer, meaning such models are much easier to use for criminal activities. Abliterated models are readily available, both through inference providers and to download for self-hosting.

Privacy and Jurisdiction

Alongside this, there is rising demand for more private forms of inference – AI deployments where the provider does not see the prompts and responses at all. Some of that assurance is contractual, such as zero-data-retention arrangements, under which a provider commits not to store inputs and outputs beyond what is needed to run the request or comply with the law. Some of it is technical – confidential computing allows workloads to run inside a hardware-based trusted execution environment so that prompts and responses stay encrypted in memory and remain inaccessible even to the host operator. Both are legitimate and desirable in many cases – privacy-preserving inference is something citizens and businesses have good reason to want. But such policies and technology make it harder for outside parties to lawfully surveil the use of both closed- and open-weight models.

There is a mitigation for the major closed-weight providers – they are all US companies, and Western intelligence and law-enforcement agencies have a long history of lawful cooperation with US firms on matters such as terrorism. On the other hand, open-weight inference providers are more varied in their jurisdictional reachability. A provider serving an open-weight model from a country with no cooperative data-sharing framework is, for practical purposes, out of reach.

The result is that criminals can access models that materially increase their capacity to cause harm, while law enforcement lacks a well-understood set of options for monitoring or intervening.

Monitoring Open-Weight Inference Usage

UK law enforcement typically use one or more of the following for surveilling suspected subjects of interest: communications data (the why, when, who, and how of a communication), the interception of content, or equipment interference, which targets a user’s device and can collect both content and metadata in the process. Importantly, a ‘communication’ under the Investigatory Powers Act 2016 (the IPA) does not have to pass between two people. Section 261(2) defines it to include “signals serving either for the impartation of anything between persons, between a person and a thing or between things’. At first estimation, a person querying a model, or one agent querying another, fits that language, and so would likely be covered.

Companies providing access to AI models are also likely to count as telecommunications operators for the purposes of the IPA. A telecommunications operator is defined as a person who offers or provides a telecommunications service to persons in the UK, or who controls or provides a telecommunication system, and a telecommunications service as one providing access to and facilities for using such a system. A hosted inference API, which accepts a request over a network and returns a response, seems to fit inside that description. In this case, the metadata surrounding an AI session, including IP addresses, connection times, and request volumes, would appear to meet the definitions of communications data, while the prompts and responses themselves are likely to be classed as content. For self-hosted models, there is no telecommunications operator and arguably no telecommunication system, so the prompt and response may not be a ‘communication’ in the statutory sense – they sit on the user’s equipment as private information reachable by equipment interference or ordinary seizure powers.

It is likely that individuals’ interactions with an AI system would merit a strengthened necessity and proportionality case. Individuals disclose extraordinarily sensitive data to chatbots: financial circumstances, medical symptoms, relationship difficulties, religious and spiritual questions. The IPA anticipates material of this kind – the general privacy duties in section 2 require the authorising authority to have regard to the particular sensitivity of the information being sought, and the codes of practice demand heightened consideration with regard to confidential personal information. A chatbot could aggregate all of the se categories in a single record, and as such the necessity and proportionality case for reaching into it would therefore need to be more compelling than for typical communications data.

Policy Priorities

The UK should begin preparing for this shift before private inference from highly capable open-weight models becomes a routine part of serious criminal operations. The following policy actions should be considered:

1. Develop specific guidance on AI inference under the IPA

The Home Office and the Investigatory Powers Commissioner’s Office should develop guidance covering hosted, enterprise, confidential and local inference. This should clarify the likely treatment of prompts and responses, as well as other surrounding data such as interaction metadata, tool calls, model identifiers, and infrastructure logs.

2. Map the inference supply chain

Relationships and lawful-access channels are easier to establish before they are urgently required. Law enforcement and intelligence agencies should develop a data availability map for major open-weight AI services, including the model developer, inference provider, and cloud or GPU operator; the jurisdiction of each entity; the information each retains; and the technical assistance each is able or willing to provide.

3. Take a balanced approach towards confidential computing

One tempting response would be to compel providers to retain readable prompts. However, a blanket requirement to do so would create databases at high risk of exploitation, move demand to less reachable jurisdictions, and undermine many of the security benefits the technology is intended to provide. The UK government should not respond to confidential computing by seeking to prevent its adoption – private inference can improve the protection of citizens, businesses and sensitive government work, and there are clear lessons here to be learnt from the crypto wars.

4. Strengthen endpoint and agentic forensics

As inference moves away from centralised providers, the endpoint becomes more important. Law enforcement and intelligence agencies will need the ability to identify and recover local model deployments, agent memory, orchestration histories, tool-call records, generated files, API credentials and links to external infrastructure, and importantly link this activity to investigative questions. This will require cooperation between equipment interference specialists, digital forensics teams and AI experts, potentially giving rise to a new discipline of ‘agentic forensics’.

Conclusion

The definitions in the Investigatory Powers Act are broad enough to encompass much AI activity that uses open-weight models, including communications between persons and automated systems. However, the operational assumptions behind those powers are changing. The original model developer may no longer control access, observe misuse or possess any relevant records. Hosted inference can involve several providers across different jurisdictions. Confidential computing can prevent an operator from accessing unencrypted prompts and responses. Local models and autonomous agents can shift the most useful evidence to the target’s equipment.

The result is that investigative visibility becomes fragmented – a familiar concept for those working in law enforcement and intelligence. Authorities will need to understand the complete inference architecture, identify which component holds which information and select the power appropriate to that component. Privacy protections should follow the meaning and sensitivity of the information rather than the architecture used to store and process it. Preserving that principle will allow the UK to adapt its investigatory capabilities without discarding the legitimate security and privacy benefits of open and confidential inference.

© George Balston, 2026, published by RUSI with permission of the author.

The views expressed in this Commentary are the author's, and do not represent those of RUSI or any other institution.

For terms of use, see Website Terms and Conditions of Use.

Have an idea for a Commentary you'd like to write for us? Send a short pitch to commentaries@rusi.org and we'll get back to you if it fits into our research interests. View full guidelines for contributors.