> daily_signal(2026_11_08)

An AI watchdog catches most misbehaving agents only by flooding reviewers with false alarms, plus a chatbot-harm monitor, a cuffless blood-pressure watch, and a rehab robot that reads your muscles.

PickBits Daily Signal · Sunday, November 8, 2026

By Mark Pickering · 7 min read · November 8, 2026

// tl;dr

I've spent most of this year watching people try to keep AI honest, from the audit mandates to a UK institute that caught every model it tested cheating its own evaluations to that on-site look at OpenAI last month that turned up real monitoring gaps. The most useful thing I read all week skips the "is AI safe" argument and just measures how well five models catch a misbehaving agent against how often they cry wolf. A second paper goes after chatbot harm with a move I like: watch what the model actually says, instead of trying to prove a specific user got hurt.

The two health ones are the ones I'd actually use, and the blood-pressure watch especially, reading it off sensors you already wear with no cuff. The caveat comes first this time, too, instead of buried in a footnote. The agent study hands you the catch rate and then the false-alarm rate that ruins it. Google says "doesn't replace a diagnosis" before it says almost anything else. I trust the ones that put the caveat first.

Andy Hall pointed five frontier models at logs of misbehaving AI agents and wrote down exactly how often each one cried wolf.

1. An AI "watchdog" catches most agent misbehavior only by flooding your reviewers with false alarms.

A new study pointed five models at logs of misbehaving agents; the best caught almost everything, but only by paging a human far more often than real incidents occur.

Give an AI agent any real autonomy and sooner or later you go shopping for a monitor to watch it. Researcher Andy Hall tested five frontier models at exactly that job, replicating METR's MALT dataset and the R-Judge benchmark, having them read another agent's logs for reward hacking, sandbagging and unsafe actions. The catch rates look great in isolation: at a 5% false-alarm budget, GPT-5 caught nearly 98% of reward-hacking cases, while Gemini 3.8 Flash caught barely half.

The trouble starts when you push for more. Pushing detection to 99% on R-Judge drove GPT-6 Astra's false-alarm rate to 15%, and in a worked example of 100,000 agent sessions, a 3% false-alarm rate throws roughly 3,000 false warnings against about 2,000 real incidents. Someone reads all 3,000. And if you are shopping for one of these, the cheaper model won: GPT-5 beat the newer, pricier GPT-6 Astra on reward hacking at about one-eighth the cost. Hall's own line is the one to keep: these auditors are "better suited for estimating how often things go wrong than for safety checks." It tells you roughly how often things go wrong. It doesn't stop any of them.

Screenshot of The Decoder's November 7 report on a study testing AI models as auditors of misbehaving agents.
the-decoder.com · November 7, 2026

Why this matters: The headline accuracy number is the least useful line in the whole study. The one that should set your staffing is the false-alarm rate, because at any detection level you'd actually trust, a human has to triage every alarm the monitor raises and most of them will be wrong. It flags things; it does not stop them. Mistake the flag for a stop and you will drown your reviewers in false alarms and still miss the real one.

Action this week: Do the arithmetic the sales deck skips before you wire a monitor in. Take the detection rate you'd actually trust and work out how many false alarms it throws at your real session volume, because the study's own example is roughly 3,000 false pages against 2,000 real incidents per 100,000 sessions, and somebody on your team reads all 3,000. Either you fund that queue or you drop the detection bar. Then test the candidates on your own logged incidents, not on which model is newest, since the pricier GPT-6 Astra losing to GPT-5 here is the whole point. The one line I'd get a vendor to put in writing is the plain one: what is your false-alarm rate at the detection level you are quoting? Most hand-wave it, and the one time a vendor gave me an actual number is the one time I went ahead.

the-decoder.com: AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds (November 7, 2026)

2. This method flags chatbot harm by scoring the model's own replies, not the user.

A model-agnostic method flags eight risk-factor behaviors response by response, and a cheap classifier-judge cascade cut monitoring cost by about 97% on real conversations.

We have been on chatbots harming vulnerable people all year, from teen-safety bills to the delusion-feeding cases to the month ChatGPT got rated "unacceptable risk" for teens. You could never really act on it, though, because you can't tell whether a given user actually got hurt. That harm builds up over time and depends on private details about them you'd never have. A paper posted November 4 by Nil-Jana Akpinar and colleagues gives up on measuring the user and watches what a system can actually see, the model's own replies. It spells out eight risk-factor behaviors and scores each chatbot response, then aggregates them across a conversation.

A classifier-judge cascade, a cheap classifier handling the easy calls and escalating only the ambiguous ones to an expensive LLM judge, reached macro precision 0.86, recall 0.76, F1 0.79 on held-out human-labeled data, and on the real-world WildChat conversation set it cut LLM-judge calls by 97.1% versus judging every turn. It comes with real limits, though. It is trained largely on a 47,062-response synthetic corpus, it measures observable risk factors and does not claim to tell you whether any individual was harmed, and it has no validation on real clinical populations.

Screenshot of the arXiv abstract page for the chatbot psychological-harm-risk detection paper.
arxiv.org · November 4, 2026

Why this matters: This is not new ground for us. We have tracked the chatbot-harm fight all year, from state age-verification bills to the UK's medical umbrella body asking for restrictions on under-16s. The hard part has always been auditing this from the outside, on the replies alone, without climbing inside a user's head. This paper is a first cut at doing it cheaply enough to run on every conversation instead of a sample.

Action this week: Watch who actually adopts output-level monitoring like this. For months "we have guardrails" was the end of the conversation, since nobody outside could check harm without climbing inside a user's head. Now a trust-and-safety team or a state regulator can make a platform show its risk-factor detections, which beats taking "we take safety seriously" on faith. For anyone running a chatbot, a flagged reply means put a human on that conversation, not that someone was harmed. My one reservation is the data: it learned mostly on a synthetic corpus, and that has to hold up against real conversations before I'd build a rule on it.

arxiv.org: Measuring risk, not harm: model-agnostic detection of psychological-harm risk factors in chatbot responses (arXiv 2611.06806, submitted November 4, 2026)

Listen to the Daily Signal podcast at pickbits.ai/daily-signal.

3. Google's AI flags likely high blood pressure from an ordinary smartwatch, no cuff required.

In an early study the watch flagged more cases than a standard in-office reading, and it is already live in the Google Health app.

Roughly half of people with high blood pressure have no idea they have it, and that's exactly who this is for. Google Research, in a Nature paper and a company blog post, built an AI that estimates whether you likely have high blood pressure using only the pulse and motion sensors already in a regular smartwatch, with no cuff needed to calibrate it. A neural network trained on nearly 500,000 hours of sensor data, with no diagnostic labels, averages several days of readings into a probability.

The early numbers are concrete. In a study of 196 participants across five U.S. research sites, the watch identified 65.8% of people with high blood pressure, against 55.3% for a standard in-office measurement. The write-up does not report specificity or false-positive rates, and Google is explicit that the tool "doesn't replace a diagnosis." It is already shipping (the capability was added to the blood-pressure-trends feature in the Google Health app), and no FDA clearance is mentioned. So it does not diagnose anything. It tells you to go get a real reading, running quietly in the background on hardware you already wear.

Screenshot of The Decoder's November 7 report on Google's cuffless smartwatch blood-pressure detection.
the-decoder.com · November 7, 2026

Why this matters: High blood pressure usually gives you no symptoms until it has already done damage, which is why a quiet background screen is worth something here. What I like is how careful they were with it. It runs on sensors you already own and was trained without diagnostic labels. And it says "doesn't replace a diagnosis" right out loud rather than tucking it into a footnote. Catching even a slice of the people who have it and don't know is worth a lot.

Action this week: Take a flag on your watch as a nudge to go sit down with a real cuff and a clinician. It is not a number to manage your own medication from, and the app says as much itself. I want specificity and the false-positive rate. Google published neither, and those are the ones that matter, because a screen that over-flags healthy people just sends a crowd of worried people to the doctor for nothing. Those two numbers, plus whatever the FDA ends up saying, are what I'm watching.

the-decoder.com: Google's AI detects high blood pressure from a regular smartwatch without a cuff (November 7, 2026)

4. An AI that reads your own muscle signals can prescribe a rehab robot's movements for your nervous system.

It personalizes therapy to each patient's neuromotor signals instead of a generic path, with under 8% prediction error, from a proof of concept on 16 healthy adults.

Most rehab robots run the same programmed path for every patient, whatever that person's muscles are actually doing, even though the muscle activity is the part that tracks whether you recover. A proof-of-concept paper posted November 5 does it differently. A transformer reads a person's surface-electromyography, the electrical activity of their muscles, and outputs the contact-force trajectories an end-effector robot should apply, learning a latent representation of that individual's neuromotor factors rather than prescribing one generic path.

The numbers are early but concrete. Mean trajectory-prediction error came in under 8% on held-out tasks and users, inference is effectively instantaneous, and in an initial live test participants reached the target neuromotor activations with about 15% accumulated error over the task; per-user fine-tuning proved feasible inside a single therapy-session timescale. The caveats are real: the dataset is 16 healthy participants across 15 isometric tracking tasks, and the authors call validation in clinical populations future work. This is a path toward personalized rehab, not a cleared device.

Screenshot of the arXiv abstract page for the sEMG-driven rehabilitation-robot trajectory paper.
arxiv.org · November 5, 2026

Why this matters: Instead of running the same path for everyone, it reads your own muscle signals and prescribes the robot's force to match them, not a generic chart. I have watched a lot of medical-AI demos personalize to a chart. Personalizing to a live muscle signal, fast enough to retune inside one session, is the part I'd actually watch, if it holds up outside a lab.

Action this week: Judge this one on the method, not the demo video, then wait for the trial that actually matters: real stroke or injury patients, not 16 healthy volunteers, which the authors call the next step themselves. Until that lands, the sub-8% error is a promising bench result and nothing a therapist can put to work. If you build assistive tech, the piece worth stealing is the per-user tuning that fits inside a single session.

arxiv.org: Transformer-based trajectory prescription for sEMG-optimizing rehabilitation robotics (arXiv 2611.08295, submitted November 5, 2026)

» What to watch this week

Tomorrow's signal lands here.