Insights
Research notes and small experiments from our AI work
Short write-ups of experiments on AI agents, local models, AI in public services and open data on real-world problems. Each one links to the code and data, so the results can be checked and reproduced.
I tried to predict next week's price shocks in 17 Pakistani cities. Past prices could not do it
On 35 weeks of PBS prices for 51 essential items, a model could not beat "next week's price stays the same". Tomatoes, chicken and onions jumped 5% or more in a fifth to a third of weeks.
- Pakistan
- Food prices
- Forecasting
In Pakistan Punjab, about a third of long-record wells show water levels moving deeper
38,450 readings from 3,311 Pakistan wells in a BGS dataset. Among wells with 10 or more readings, 37% moved deeper and 3% shallower; short records made the picture look better than it is.
- Open data
- Groundwater
- Pakistan
Following the cursor still let an agent claim a complete list in 41% of cases
A synthetic benchmark of how agents decide they have every page from an API, and why only a count check stopped false "complete" claims.
- Agentic AI
- Tool use
- Synthetic benchmark
An evidence gate cut unsupported agent answers to 6%, but also dropped over half the good ones
A synthetic benchmark of when a tool-using agent should say "I don't know", with every result checked against a hand-worked formula.
- Agentic AI
- Abstention
- Synthetic benchmark
A medical similarity metric scores "left knee" closer to "right knee" than "lumbar spine" to "lower back"
RaTEScore-style meaning-based matching, with the same medical encoder, flagged only 11 of 25 notes where left and right were swapped. A plain code check flagged all 25.
- Clinical AI
- Evaluation
- Paper follow-up
Checking the injured side with code, not the AI, caught all 43 wrong sides in Roman Urdu notes
Comparing the side words in each consultation with 200 notes from two local models caught every wrong or unsupported side and flagged 4 of 123 safe notes. Asking "is a side written?" caught none.
- Clinical AI
- Roman Urdu
- Evaluation
Agent Retry Amplification: when a failed tool response repeats a successful side effect
ARAM compares four retry policies across 6,000 synthetic scenarios and 24,000 policy evaluations, measuring task success, retry amplification and duplicate side effects.
- Agentic AI
- Tool reliability
- Synthetic benchmark
Agent Cancellation Propagation: what happens after an AI agent is told to stop?
A synthetic benchmark evaluates four cancellation policies across 6,000 workflows and 24,000 policy episodes. Cooperative stopping prevented new dispatches, but in-flight work could still cause side effects.
- Agentic AI
- Cancellation
- Synthetic benchmark
How secure is an AI agent's persistent memory?
AMIB v0.2.0 compares five memory-governance policies across 150,000 synthetic evaluations, exposing trade-offs between poisoning resistance, memory retention and task completion.
- Agentic AI
- Agent memory
- Security benchmark
My jersey-number detector learned nothing. Its rebuild showed that pretraining was not the gap
A detector trained from scratch reached 0.241 AP where the 2025 model scored zero. Against YOLOv8n, COCO pretraining was worth 0.03 AP; the detector design and recipe were worth 0.11.
- Computer vision
- Object detection
- Training from scratch
A 30-line TextRank beat every model I fine-tuned on BBC News Summary. The reference summaries explain why
TextRank scored 82.9 ROUGE-Lsum against 72.7 for PEGASUS-large, the best of seven fine-tuned models. 95% of reference sentences are copied from the article, so the benchmark rewards picking sentences.
- NLP
- Summarisation
- Evaluation
A small local AI checked clinic notes for missing facts. It caught lost urgency, and passed two missing red flags
qwen2.5:3b and llama3.1:8b checked 80 physiotherapy notes for missing facts. Kappa against keyword checks was 0.28 to 0.48; the judge caught a missing "today" for a possible clot but passed two missing red flags.
- Clinical AI
- Evaluation
- Local LLMs
Telling a local AI to write the injured side made it write the wrong side
A side rule, a glossary and one retry all brought left and right back into Roman Urdu notes, and all raised wrong sides too: qwen2.5:3b with the glossary named the other side in 9 of 14 notes.
- Clinical AI
- Roman Urdu
- Local LLMs
My 2022 heart disease model scored 92%. Duplicate rows and one hospital explain why that number meant little
In the widely shared 1,025-row file, 97% of test rows are also in training. At an unseen hospital all six models, TabPFN included, fell to about 0.72 AUROC and their risks stopped matching reality.
- Clinical prediction
- Data leakage
- External validation
Three training runs, one bug and a sicker test set: the second rebuild of my chest X-ray model
Three DenseNet121 seeds scored 0.801 to 0.803 and their average 0.814. A validation-fitted correction cut calibration error to 0.011, but pneumothorax stayed under-predicted on the sicker test set.
- Medical imaging
- Calibration
- Deep learning
In 20 clinic consultations, local AI lost red flags and the injured side when patients spoke Roman Urdu
llama3.1:8b lost 16 of 168 facts from Roman Urdu against 1 from English (p = 0.0003), and critical facts only in Roman Urdu. Both models almost never wrote left or right.
- Clinical AI
- Roman Urdu
- Local LLMs
ProvenanceFlow: can tracking data origins secure AI agent tool chains?
A synthetic benchmark of 1,500 scenarios compares five policies. Field-level provenance preserved task completion while blocking simulated attacks, but only when metadata labels were correct.
- Agentic AI
- Tool security
- Data provenance
I rebuilt my chest X-ray model: 0.80 AUROC on a patient-wise split, and probabilities up to 11 times too high
DenseNet121 reached a test mean AUROC of 0.801 on NIH ChestX-ray14 with the official patient-wise split. Its raw probabilities were 2.5 to 10.9 times the real prevalence until a per-class correction.
- Medical imaging
- Calibration
- Deep learning
Translating Roman Urdu first did not make a local AI's clinic notes safer
llama3.1:8b and qwen2.5:3b drafted physio notes from Roman Urdu, directly and after translating. Translating first raised missed critical facts from 3 to 5 and swapped left and right; the 3B model did not translate at all.
- Clinical AI
- Roman Urdu
- Local LLMs
RevocationLag: delayed and lost permission-revocation notifications
Across 1,800 synthetic workflows, notification delays of 1, 3 and 5 action positions raised unauthorized-action rates to 16.53%, 47.26% and 71.58%; lost notifications reached 100%. A separate section reports earlier local-model behaviour tests.
- Agentic AI
- Authorization
- Local LLMs
AI memory filters break when the labels are wrong, and no model said CONFLICT
With 20% of labels wrong, unsafe memory got past a gate in about 21% of questions. Given notes that disagree, neither qwen2.5:3b nor llama3.1:8b ever answered CONFLICT.
- Agentic AI
- AI memory
- Local LLMs
AgentRetry: when a tool times out, should an AI agent retry or verify first?
A reproducible synthetic benchmark compares four recovery policies and measures task completion, safe completion, duplicate-side-effect risk and escalation.
- Agentic AI
- Tool recovery
- Reliability
Small local AI agents followed hidden instructions in clinic notes. Checking the tool's arguments stopped them.
qwen2.5:3b and llama3.1:8b planned tool calls for 10 invented clinic tasks in English and Roman Urdu. A hidden instruction was followed in 6 to 8 of 10; only a rule that checks the patient and recipient stopped every one.
- Agentic AI
- Prompt injection
- Roman Urdu
When AI agents' logs went missing together, every audit rule either missed gaps or raised false alarms
Three simulated agents lose their logs one field at a time or two at once. Judging only complete logs missed about half the skipped duties, and counting missing logs as failures flagged more than half of the clean episodes when outages were frequent.
- AI accountability
- Multi-agent systems
- Synthetic benchmark
Three local AI models mixed up left and right in Roman Urdu, and were sure they were right
qwen2.5:3b and llama3.1:8b read the same physio notes in English and Roman Urdu. In Roman Urdu all three agents said yes to the wrong side on 3 of 4 questions, almost always with 100% confidence.
- LLM agents
- Roman Urdu
- Clinical AI
When AI agents fail together, more votes do not help
Three simulated agents that can fail for the same reason: majority vote accuracy fell from 79.9% to 70.0% as shared failures became more likely, and never beat the best single agent.
- Agentic AI
- Multi-agent systems
- Synthetic benchmark
A one-number fix for overconfident agents worked, but barely
Subtracting the measured overconfidence improved the Brier score of three simulated agents at every level, but by less than 0.001, and calibration error did not improve.
- Agentic AI
- Calibration
- Synthetic benchmark
Averaging agents' confidence made the team less trustworthy
Five ways to combine three simulated agents: averaging their confidence gave the worst calibrated team, and adding evidence in log-odds the best, once each agent was corrected.
- Agentic AI
- Uncertainty
- Synthetic benchmark
A planted memory fooled a simple AI agent 74% of the time. A trust filter cut it to 9%.
A synthetic simulator of agent memory compares three retrieval rules. A trust filter removed most planted memories at no cost; a corroboration rule scored 0%, but only because of how the test was built.
- Agentic AI
- Agent memory
- Security
In Roman Urdu, a small AI wrote notes about the wrong body part
A 3B local model drafted physiotherapy notes from the same consultations in two languages. Roman Urdu notes left out 13 of 42 key facts, against 2 in English, and three of four described the wrong body part.
- Clinical AI
- Hallucination
- Roman Urdu
ToolGuard: testing tool guards and retries in an AI agent
A 120-task synthetic benchmark tests action validation and bounded retries; a separate eight-task Qwen 2.5 3B pilot is reported as preliminary evidence.
- Agentic AI
- Tool use
- Reproducible research
The validator said yes. The refund was still wrong.
16 clinic tool calls from a 3B local model in English and Roman Urdu: all passed the validator first time, yet four were wrong, including a refund amount the model made up.
- LLM agents
- Tool use
- Roman Urdu
Whisper halved its errors on Urdu clinic speech, but not on the medical words
Three self-recorded Urdu physiotherapy conversations on a MacBook: word errors fell from 44% to 19% with a bigger model, but only about half of the clinical key terms survived.
- Speech-to-text
- Clinical AI
- Urdu
AI road cameras: how many flags are wrong?
Using public trial figures and Scotland's own estimates: when fewer drivers offend, a larger share of AI flags is wrong, so human review matters more over time.
- AI accountability
- Base rates
- UK
Six Pakistani cities, one year of air: Faisalabad, not Lahore, ranked worst
The same method across Lahore, Karachi, Islamabad, Faisalabad, Peshawar and Quetta: no city met the WHO guideline, and Peshawar quietly ranked third.
- Air quality
- Open data
- Pakistan
Lahore did not have a single clean-air day in the past year
One year of open data from 77 PM2.5 sensors: all 364 days were above the WHO 24-hour guideline, and the yearly mean was about 21 times the annual guideline.
- Air quality
- Open data
- Pakistan
Can a small local AI agent understand Roman Urdu?
ReAct with a 3B model running locally on a MacBook, tested in English and Roman Urdu. Changing only the names dropped Roman Urdu accuracy from 83% to 58%.
- LLM agents
- Robustness
- Roman Urdu