NEERAJ VENNACV
← Project Alice
Prototype · synthetic evaluation

Alice Trace

Alice Trace investigates Solana transactions and keeps track of the evidence behind each finding. I built it to make transaction histories easier to examine and to test how agents handle missing or conflicting information.

  • Go
  • PostgreSQL
  • Python
  • FastAPI
  • LangGraph
  • React
  • TypeScript

Implementation

A Go service retrieves transaction signatures, transactions, balances, and token accounts through Solana RPC. PostgreSQL keeps the original observations and their sources so a finding can be traced back to the report that supports it.

The parser reconstructs supported SOL and standard SPL Token transfers with exact integer amounts. Fees are recorded separately, and failed transactions are not treated as successful transfers. Missing metadata remains unavailable rather than becoming a zero balance or proof that nothing happened.

A Python/FastAPI service runs bounded investigations with deterministic tools or an optional LangGraph model/tool loop. The React interface shows cases, evidence, missing coverage, and reports. The Alice memory layer keeps competing claims and their dependencies, so revising one claim need not discard unrelated history.

Results

The experiments introduce gaps and contradictions into synthetic transaction histories. Policies choose which evidence to request and must produce a supported account of the balance, transaction status, fees, and transfers.

  • In the primary closed-loop comparison (32 episodes per policy), Alice and the strong deterministic policy both completed 62.5% of cases with supported conclusions. Alice used more evidence: 2.25 queries on average, compared with 1.75.
  • A read-only live RPC smoke test checked transaction status and fees. Model-driven investigation has not been evaluated live; LangGraph and SDK tests use test doubles.
Evaluation details

Three CPU studies use the application’s Go normalizer and memory code: 256 main policy episodes, 128 planner-diagnostic episodes, and 320 ablation episodes.

All 704 episodes reuse two synthetic trace seeds, one for development and one for validation. There is no held-out test trace. Replaying a fixed set of evidence and letting a policy request evidence are separate comparisons.

Limitations

Two synthetic SOL trace seeds cannot establish performance on new histories or live providers. The parser does not cover arbitrary program behavior or Token-2022 economic extensions, and transaction activity cannot establish wallet identity or a complete ownership history.

Evidence scores are not calibrated probabilities. The reliability-shift tests check how sources are grouped and queried; they do not establish adaptation to changing source reliability.

Memory revisionReconProject Alice