Causal decision and audit tool for AI agents. A/B testing and Difference-in-Differences analysis.
curl -fsSL https://cookapps.net/api/skills/install?skills=β’β’β’β’β’β’β’ | nodeSource: https://github.com/ZhuMorris/agent-causal-decision-tool
Agent Causal Decision Tool helps you and your AI agents answer one question from experiment data: "should we ship this change, keep running the test, or roll it back?" It takes in simple A/B or rollout summaries and returns a structured JSON decision, key statistics, and an audit record you can store or review later.
Rather than being a full experimentation platform, it is a decision engine. You bring the data (from your logs, BI tool, or CSV); it handles the stats, decision logic, and audit trail.
In many teams, experiment decisions happen in ad hoc spreadsheets or dashboards. People glance at lift, argue about whether the sample size is enough, and sometimes ship features based on noisy or biased results. Agents make this worse if they are wired to react to any small uplift they see.
This tool wraps a few standard methods into one consistent, agentβfriendly interface:
decide)** β no need to know which statistical method to use. Paste your numbers and it auto-selects A/B, Bayesian, DiD, or planning from your input fields.The goal is not to replace your analytics stack, but to give agents a small, reliable decision block they can call inside workflows.
Use this tool whenever you or your agents have experiment or rollout results and need a decision you can defend:
decide auto-detect from your numbers.For AI agent integrations, Agent Causal exposes a JSON-RPC 2.0 API over both stdio and HTTP.
python -m src.api stdio
python -m src.api http --port 8000 # Or with uvicorn directly: uvicorn src.api:app --port 8000
| Action | Description |
|---|---|
decide | Easy-mode dispatcher β auto-selects A/B, Bayesian, DiD, or planning from your input fields |
decide_ab | Frequentist A/B test (mode: frequentist) or Bayesian A/B (mode: bayesian) |
decide_rollout | Difference-in-differences for staged rollouts |
plan_test | Experiment planning (sample size, MDE, feasibility) |
audit_result | Full audit of a stored result |
save_result | Persist a decision result to SQLite history |
get_result | Retrieve a stored result by ID |
compare_results | Compare multiple stored experiments |
connect | Fetch experiment data from external connectors (e.g. PostHog) |
run_workflow | Orchestrator β fetch + decide + audit + save + notify + compare in one call |
{
"jsonrpc": "2.0",
"method": "decide_ab",
"params": {
"input": {
"control_conversions": 100,
"control_total": 5000,
"variant_conversions": 130,
"variant_total": 5000
}
},
"id": 1
}{
"jsonrpc": "2.0",
"result": {
"decision": "ship",
"recommended_next_action": "Deploy variant β statistical significance achieved with positive lift.",
"selected_method": "ab_test",
"selection_reason": "User requested frequentist A/B test via decide_ab action",
"confidence": "medium",
"effect_summary": "Estimated lift: +30.00% (positive)",
"warnings": [],
"limitations": ["Binary conversion outcome only", "No multiple testing correction applied"],
"audit_summary": "ab_test: Decision",
"source_metadata": null,
"internal_result": { ... } # Full ABTestOutput / BayesOutput / DIDOutput / PlanningOutput
},
"id": 1
}{
"jsonrpc": "2.0",
"error": {
"code": "VALIDATION_ERROR",
"message": "Invalid A/B test inputs",
"data": {
"details": [{"field": "control_total", "issue": "Input should be greater than or equal to 1"}],
"request_id": null
}
},
"id": 1
}from src.actions import run_action
resp = run_action("decide_ab", {
"input": {
"control_conversions": 100,
"control_total": 5000,
"variant_conversions": 130,
"variant_total": 5000
}
})
if "error" in resp:
print(f"Error: {resp['error']['message']}")
else:
print(f"Decision: {resp['result']['decision']}")# Via pip pip install agent-causal-decision-tool # Via GitHub pip install git+https://github.com/ZhuMorris/agent-causal-decision-tool.git # Via OpenClaw (clawhub) clawhub install agent-causal
Unlock application details to view installation, configuration, and workflow guides.
decide)Don't know which method you need? decide auto-detects from your input fields:
# A/B test (auto-detected from --control/--variant)
PYTHONPATH=. python3 -m src.cli decide --control 100/5000 --variant 130/5000
PYTHONPATH=. python3 -m src.cli decide --control 100/5000 --variant 130/5000 --format text
# Bayesian A/B (--bayesian flag)
PYTHONPATH=. python3 -m src.cli decide --control 100/5000 --variant 130/5000 --bayesian
# DiD / staged rollout (auto-detected from pre/post treated fields)
PYTHONPATH=. python3 -m src.cli decide --pre-control 1000 --post-control 1200 --pre-treated 200 --post-treated 280
# Experiment planning (auto-detected from --baseline + --mde)
PYTHONPATH=. python3 -m src.cli decide --baseline 0.05 --mde 10 --traffic 10000
# JSON-RPC API β same auto-detection
{"jsonrpc":"2.0","method":"decide","params":{"control_conversions":100,"control_total":5000,"variant_conversions":130,"variant_total":5000},"id":"1"}Auto-detection:
| You provide... | It runs... |
|---|---|
--control + --variant | Frequentist A/B |
--control + --variant + --bayesian | Bayesian A/B |
--pre-control + --post-control + --pre-treated + --post-treated | DiD (Difference-in-Differences) |
--baseline + --mde | Experiment planning |
plan)Estimate required sample size, duration, and feasibility before running an experiment:
cd ~/clawd/agent-causal-decision-tool PYTHONPATH=. python3 -m src.cli plan --baseline 0.02 --mde 10 --traffic 5000 PYTHONPATH=. python3 -m src.cli plan --baseline 0.02 --mde 5 --traffic 500 --format text # Custom traffic allocation PYTHONPATH=. python3 -m src.cli plan --baseline 0.02 --mde 10 --traffic 5000 --allocation custom --allocation-ratio 0.3/0.7
Parameters: --baseline, --mde, --traffic, --confidence (default 0.95), --power (default 0.8), --allocation, --allocation-ratio
Feasibility: feasible β€14 days | slow 15β60 days | not_recommended >60 days
ab)PYTHONPATH=. python3 -m src.cli ab --control 100/5000 --variant 130/5000 PYTHONPATH=. python3 -m src.cli ab --control 100/5000 --variant 70/5000 --format text # Auto-save to history PYTHONPATH=. python3 -m src.cli ab --control 100/5000 --variant 130/5000 --save
bayes)Beta-Binomial conjugate model with Jeffreys prior β no p-value, returns P(variant wins):
PYTHONPATH=. python3 -m src.cli bayes --control 100/5000 --variant 130/5000 PYTHONPATH=. python3 -m src.cli bayes --control 80/5000 --variant 85/5000 --format text # Adjust Monte Carlo samples PYTHONPATH=. python3 -m src.cli bayes --control 100/5000 --variant 130/5000 --samples 50000 --save
Decision thresholds: P(variant wins) β₯ 0.95 β ship | β€ 0.05 β reject
did)For non-randomized experiments where parallel groups exist:
PYTHONPATH=. python3 -m src.cli did --pre-control 1000 --post-control 1100 --pre-treated 900 --post-treated 1150 PYTHONPATH=. python3 -m src.cli did --pre-control 1000 --post-control 1100 --pre-treated 900 --post-treated 1150 --save
# List recent experiments PYTHONPATH=. python3 -m src.cli history PYTHONPATH=. python3 -m src.cli history --mode ab_test --limit 10 # Compare multiple experiments by ID PYTHONPATH=. python3 -m src.cli compare 1 2 3 # Save a prior JSON result to history PYTHONPATH=. python3 -m src.cli save /tmp/result.json --name "checkout-v3-test"
# Save a result first PYTHONPATH=. python3 -m src.cli ab --control 100/5000 --variant 130/5000 > /tmp/result.json # Human-readable audit PYTHONPATH=. python3 -m src.cli audit /tmp/result.json --format text # Audit with experiment maturity score (0β100) PYTHONPATH=. python3 -m src.cli audit /tmp/result.json --maturity # JSON audit with maturity PYTHONPATH=. python3 -m src.cli audit /tmp/result.json --maturity --format json
Maturity labels: mature β₯90 | adequate β₯70 | immature β₯50 | inadequate = 8.1.0
Copyright 2026 ZHU YUMING. Apache License 2.0.
One MCP for reverse engineering across binaries, applications, and runtime behavior. Point your AI coding agent at any compiled app to understand its mechanics down to the binary level.
The definitive suite of 39 production grade agent skills by Matt Pocock for real software engineering, domain modeling, rigorous alignment grilling, TDD, and deep module architecture.
Get instant access to complete setup guides, repository source code, and integration assets.