code looks different from legitimate instructions, and we use signatures, patterns, and statistical anomaly detection to detect it. But getting inside someone?s AI OODA loop uses the system?s native language. The attack is indistinguishable from normal operation because it is normal operation. The vulnerability isn?t a defect -- it?s the feature working correctly.

Where to Go Next?

The shift to an AI-saturated world has been dizzying. Seemingly overnight, we have AI in every technology product, with promises of even more -- and agents as well. So where does that leave us with respect to security?

Physical constraints protected Boyd?s fighter pilots. Radar returns couldn?t lie about physics; fooling them, through stealth or jamming, constituted some of the most successful attacks against such systems that are still in use today. Observations were authenticated by their presence. Tampering meant physical access. But semantic observations have no physics. When every AI observation is potentially corrupted, integrity violations span the stack. Text can claim anything, and images can show impossibilities. In training, we face poisoned datasets and backdoored models. In inference, we face adversarial inputs and prompt injection. During operation, we face a contaminated context and persistent compromise. We need semantic integrity: verifying not just data but interpretation, not just content but context, not just information but understanding. We can add checksums, signatures, and audit logs. But how do you checksum a thought? How do you sign semantics? How do you audit attention?

Computer security has evolved over the decades. We addressed availability despite failures through replication and decentralization. We addressed confidentiality despite breaches using authenticated encryption. Now we need to address integrity despite corruption.4

Trustworthy AI agents require integrity because we can?t build reliable systems on unreliable foundations. The question isn?t whether we can add integrity to AI but whether the architecture permits integrity at all.

AI OODA loops and integrity aren?t fundamentally opposed, but today?s AI agents observe the Internet, orient via statistics, decide probabilistically, and act without verification. We built a system that trusts everything, and now we hope for a semantic firewall to keep it safe. The adversary isn?t inside the loop by accident; it?s there by architecture. Web-scale AI means web-scale integrity failure. Every capability corrupts.

Integrity isn?t a feature you add; it?s an architecture you choose. So far, we have built AI systems where ?fast? and ?smart? preclude ?secure.? We optimized for capability over verification, for accessing web-scale data over ensuring trust. AI agents will be even more powerful -- and increasingly autonomous. And without integrity, they will also be dangerous.

References

1. S. Willison, Simon Willison?s Weblog, May 22, 2025. [Online]. Available: https://simonwillison.net/2025/May/22/tools-in-a-loop/

2. S. Willison, ?Prompt injection attacks against GPT-3,? Simon Willison?s Weblog, Sep. 12, 2022. [Online]. Available: https://simonwillison.net/2022/Sep/12/prompt-injection/

3. K. Thompson, ?Reflections on trusting trust,? Commun. ACM, vol. 27, no. 8, Aug. 1984. [Online]. Available: https://www.cs.cmu.edu/~rdriley/487/papers/Thom pson_1984_ReflectionsonTrustingTrust.pdf

4. B. Schneier, ?The age of integrity,? IEEE Security & Privacy, vol. 23, no. 3, p. 96, May/Jun. 2025. [Online]. Available: https://www.computer.org/csdl/magazine/sp/2025/03/11038984/27COaJtjDOM

This essay was written with Barath Raghavan, and originally appeared in IEEE Security & Privacy.

** *** ***** ******* *********** *************

A Cybersecurity Merit Badge

[2025.10.21] Scouting America (formerly known as Boy Scouts) has a new badge in cybersecurity. There?s an image in the article; it looks good.

I want one.

** *** ***** ******* *********** *************

Failures in Face Recognition

[2025.10.22] Interesting article on people with nonstandard faces and how facial recognition systems fail for them.

Some of those living with facial differences tell WIRED they have undergone multiple surgeries and experienced stigma for their entire lives, which is now being echoed by the technology they are forced to interact with. They say they haven?t been able to access public services due to facial verification services failing, while others have struggled to access financial services. Social media filters and face-unlocking systems on phones often won?t work, they say.

It?s easy to blame the tech, but the real issue are the engineers who only considered a narrow spectrum of potential faces. That needs to change. But also, we need easy-to-access backup systems when the primary ones fail.

** *** ***** ******* *********** *************

Serious F5 Breach

[2025.10.23] This is bad:

F5, a Seattle-based maker of networking software, disclosed the breach on Wednesday. F5 said a ?sophisticated? threat group working for an undisclosed nation-state government had surreptitiously and persistently dwelled in its network over a ?long-term.? Security researchers who have responded to similar intrusions in the past took the language to mean the hackers were inside the F5 network for years.

During that time, F5 said, the hackers took control of the network segment the company uses to create and distribute updates for BIG IP, a line of server appliances that F5 says is used by 48 of the world?s top 50 corporations. Wednesday?s disclosure went on to say the threat group downloaded proprietary BIG-IP source code information about vulnerabilities that had been privately
discovered but not yet patched. The hackers also obtained configuration settings that some customers used inside their networks.

Control of the build system and access to the source code, customer configurations, and documentation of unpatched vulnerabilities has the potential to give the hackers unprecedented knowledge of weaknesses and the ability to exploit them in supply-chain attacks on thousands of networks, many of which are sensitive. The theft of customer configurations and other data further raises the risk that sensitive credentials can be abused, F5 and outside security experts said.

F5 announcement.

** *** ***** ******* *********** *************

Part Four of The Kryptos Sculpture

[2025.10.24] Two people found the solution. They used the power of research, not cryptanalysis, finding clues amongst the Sanborn papers at the Smithsonian?s Archives of American Art.

This comes as an awkward time, as Sanborn is auctioning off the solution. There were legal threats -- I don?t understand their basis -- and the solvers are not publishing their solution.

** *** ***** ******* *********** *************

First Wap: A Surveillance Computer You?ve Never Heard Of

[2025.10.27] Mother Jones has a long article on surveillance arms manufacturers, their wares, and how they avoid export control laws:

Operating from their base in Jakarta, where permissive export l

--- BBBS/LiR v4.10 Toy-7
* Origin: TCOB1: https/binkd/telnet binkd.rima.ie (21:1/229)