Post
154
A 4B model that decides whether an AI agent’s payment should go through, with every prediction published so you can check the score yourself
AI agents are starting to pay for things on their own: x402 API calls, stablecoin transfers, token approvals. One bad approval can’t be reversed. So I fine-tuned a small model to sit in front of the payment tool and answer one question: approve, hold or reject?
Try it:
Free Colab notebook (T4 GPU), linked from the model card
GGUF + Ollama: sriram1983007/SRA-RiskGate-4B-GGUF
pip install sra-riskgate: a guard for x402 + Coinbase AgentKit
pip install sra-riskgate-mcp: an MCP server
pip install langchain-sra-riskgate: LangChain middleware
🔗 Model: sriram1983007/SRA-RiskGate-4B
🔗 Benchmark: data
🔗 Benchmark: sriram1983007/sra-stablecoin-risk-bench
🔗 Published predictions (re-score them yourself): sriram1983007/sra-bench-results
🔗 SDK (x402 + AgentKit): https://github.com/sriram1983007-dev/sra-riskgate
🔗 MCP server: https://github.com/sriram1983007-dev/sra-riskgate-mcp
🔗 LangChain middleware: https://github.com/sriram1983007-dev/langchain-sra-riskgate
🔗 Live demo: sriram1983007/sra-riskgate-demo
🔗 Try it free in Colab: use the “Open in Colab” badge link
SRA-RiskGate-4B reads the payment, the policy and the verification results, and returns a JSON verdict with flags and reasons.
On 1,175 held-out payment cases:
99.49% of decisions correct
Unsafe approvals: 4 of 847 (0.47%). The base model approved about 35%.
Legitimate payments wrongly blocked: 0 of 328
Also: 825 dispute cases at 77.8%, with impossible on-chain remedies proposed only 0.38% of the time
Also be aware: the benchmark is synthetic. How the model handles real-world attacks it has never seen is still an open question, and that’s what the next version of the benchmark will test.
Check the numbers yourself: every prediction from two full runs is public, and the benchmark has an eval.yaml for Community Evals.
AI agents are starting to pay for things on their own: x402 API calls, stablecoin transfers, token approvals. One bad approval can’t be reversed. So I fine-tuned a small model to sit in front of the payment tool and answer one question: approve, hold or reject?
Try it:
Free Colab notebook (T4 GPU), linked from the model card
GGUF + Ollama: sriram1983007/SRA-RiskGate-4B-GGUF
pip install sra-riskgate: a guard for x402 + Coinbase AgentKit
pip install sra-riskgate-mcp: an MCP server
pip install langchain-sra-riskgate: LangChain middleware
🔗 Model: sriram1983007/SRA-RiskGate-4B
🔗 Benchmark: data
🔗 Benchmark: sriram1983007/sra-stablecoin-risk-bench
🔗 Published predictions (re-score them yourself): sriram1983007/sra-bench-results
🔗 SDK (x402 + AgentKit): https://github.com/sriram1983007-dev/sra-riskgate
🔗 MCP server: https://github.com/sriram1983007-dev/sra-riskgate-mcp
🔗 LangChain middleware: https://github.com/sriram1983007-dev/langchain-sra-riskgate
🔗 Live demo: sriram1983007/sra-riskgate-demo
🔗 Try it free in Colab: use the “Open in Colab” badge link
SRA-RiskGate-4B reads the payment, the policy and the verification results, and returns a JSON verdict with flags and reasons.
On 1,175 held-out payment cases:
99.49% of decisions correct
Unsafe approvals: 4 of 847 (0.47%). The base model approved about 35%.
Legitimate payments wrongly blocked: 0 of 328
Also: 825 dispute cases at 77.8%, with impossible on-chain remedies proposed only 0.38% of the time
Also be aware: the benchmark is synthetic. How the model handles real-world attacks it has never seen is still an open question, and that’s what the next version of the benchmark will test.
Check the numbers yourself: every prediction from two full runs is public, and the benchmark has an eval.yaml for Community Evals.