AgentTrust-8004

Unsupervised trustworthiness assessment for AI agents registered under ERC-8004 (Trustless Agents).

To the best of our knowledge, this is the first machine learning model trained on features derived from the ERC-8004 Identity, Reputation and Validation registries. Prior empirical work on the standard (arXiv:2604.22652, arXiv:2606.12128, arXiv:2606.26028) relies on descriptive statistics and heuristics only.

Developed as part of the monograph "Verifiable identity and trustworthiness assessment of AI agents: classification of trustworthy and malicious agents from on-chain reputation data (ERC-8004)" — Master's in Data Science & AI, UAGRM School of Engineering (Bolivia, 2026). Author: Ricardo Esteban Pari Cahuna.

Why unsupervised

ERC-8004 registries on Base mainnet expose no trust/malicious label, so a supervised model cannot be trained on real chain data. An anomaly detector needs no label: it learns the distribution of the "normal agent" and flags outliers. A real mainnet census (54,802 agents, June 2026 — see the census dataset) confirms the ecosystem is real but young (median identity age 91 days, median 1 feedback client per agent) and label-free, which motivates this design.

Models included

File	Feature set	AUC-ROC	AUC-PR	Notes
anomaly_model_mainnet.joblib	16 features (no val_*)	0.869	0.794	Deployable today — excludes ValidationRegistry features, which are not on Base mainnet
anomaly_model_full.joblib	20 features	0.849	0.763	Includes simulated ValidationRegistry features

Isolation Forest (300 trees, contamination 0.1) outperformed Elliptic Envelope (0.818), One-Class SVM (0.738) and Local Outlier Factor (0.442) on the deployable feature set. A supervised XGBoost baseline reaches AUC-ROC 0.959 / F1 0.940 with labels — the ~0.09 gap quantifies the cost of operating without them. Full comparison in results_anomaly.json and results_supervised.json.

Evaluation data — read this before citing numbers

The evaluation labels are synthetic. Models were fit (label-free) and scored on a simulated dataset of 6,000 agents (30% malicious: sybil, collusion rings, reputation laundering, plus sophisticated adversaries and atypical-legitimate "cold start" agents for realistic class overlap) — see the simulated dataset. The label is used only for a-posteriori evaluation, never for fitting. Validation against real labeled outcomes is future work; the real-mainnet artifact in this project is the census, not the metrics.

Usage

Each .joblib is a dict {model, scaler, features}. Trained with scikit-learn 1.5.1 (use 1.5.x to avoid unpickling warnings).

import joblib, pandas as pd

art = joblib.load("anomaly_model_mainnet.joblib")
model, scaler, features = art["model"], art["scaler"], art["features"]

df = pd.read_csv("your_agents.csv")          # one row per agent
X = scaler.transform(df[features])
scores = -model.score_samples(X)             # higher = more anomalous
flagged = model.predict(X) == -1             # True = anomaly

The 16 deployable features (order matters):

identity_age_days, agent_card_completeness, linked_dids, identity_validated, interaction_count, rep_positive, rep_negative, issuer_diversity, rep_recency_days, rep_velocity, interaction_value_avg, temporal_regularity, graph_degree, graph_clustering_coef, reciprocity, rep_ratio

Retrain on your own export with python train_anomaly.py --input your_agents.csv (label column optional; evaluation is skipped if absent).

Intended use & limitations

• Intended: research on agent-economy trust; screening/ranking signal for platforms that consume ERC-8004 data; baseline for future work (GNNs, continuous trust scores).
• Not intended: as a sole automated gate for denying service. Scores are anomaly signals, not verdicts; false positives concentrate on atypical-but-legitimate agents (cold start). Pair with explainability and an appeal path (see AI Act / NIST AI RMF discussion in the monograph).
• Rich graph/temporal features require indexing off-chain feedback events; the census currently exposes identity_age_days and client_count directly.

Citation

@mastersthesis{pari2026agenttrust,
  author = {Pari Cahuna, Ricardo Esteban},
  title  = {Identidad verificable y evaluaci{\'o}n de la confiabilidad de agentes de IA: clasificaci{\'o}n de agentes confiables y maliciosos a partir de datos de reputaci{\'o}n on-chain (ERC-8004)},
  school = {UAGRM School of Engineering},
  year   = {2026},
  type   = {Diploma monograph, Master's in Data Science \& AI}
}