Research

Explore Micrantha's dated research reading index, selected engineering papers, and assessment limitations.

Edition

Research coverage

Tracked records
328
Marked reviewed
323
Score-eligible reviews
266
Featured selections
26

98.5% marked reviewed; 5 triaged; 57 marked-reviewed records lack complete valid scores.

Tracker records with a source URL; reviewed means the tracker review status, not independent full-text verification or reproduction.

Selected research domains

AI & ML
271
Computer Science
266
Cybersecurity
161
Software Engineering
26

Selected tracker domain labels; counts overlap and do not sum to the total. Each source identity is counted at most once within each label.

Ranked editorial selections

Initial selection from 27 high-scoring, tracker-reviewed historical candidates; not an exhaustive reassessment of every paper. Six candidates received targeted primary-text checks; the remainder received primary abstract/proceedings checks against historical assessments. The displayed review basis distinguishes these levels. Scores are editorial judgments, capped for less-complete evidence review, not citation metrics or replicated findings.

Target: 25 from 27 screened candidates. 26 included with cutoff ties at 81/100. Equal scores share rank; alphabetical order within each tie. Include cutoff ties.

How papers are ranked

Editorial merit: momentum 5%, novelty 15%, evidence 30%, practicality 25%, long-term value 25%. Component ratings range from 1 to 5. Scores are nominally out of 100 (minimum 20).

1 weak; 3 materially supported; 4 strong; 5 exceptional with explicit justification. Intermediate half-points allowed. Momentum 2: publication only; 3: documented research activity, revisions or released resources, not proof of adoption. Evidence capped at 3.5 for abstract/proceedings-only rechecks.

Not-run means no independent execution was performed for this assessment; historical reproduction metadata is incomplete.

26 of 26 selections

  1. Rank 1 · 91/100 · Distributed systems

    Where Does Exactly-Once Live? Model, Harness, and Tool-Contract Effects on Duplicate Side Effects in LLM Agents

    Jiapeng Li · Source date 2026-09-24

    Assessed · Historical assessment; targeted exact-version primary-text recheck

    Engineering takeaway: Read-back helps only when it resolves the uncertain effect. The service contract must address retries and redelivery when observation cannot establish the outcome.

    Author-reported evidence: The authors report 25,930 graded episodes across nine models and three harnesses; idempotency keys substantially reduced duplication.

    Limitations: Simulated services and specific fault distributions; several analyses were exploratory.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Fault-specific comparison of recovery contracts with a ledger oracle and a precise impossibility boundary; strong direct relevance to durable tool interfaces.

    Momentum 2, novelty 4, evidence 4.5, practicality 5, long-term value 5 (each out of 5).

  2. Rank 2 · 89/100 · Agent security

    Agent Memory Is a Surface for Endogenous Authorization Laundering

    Tommaso Cerruti, Mika Okamoto, Ansel Kaplan Erol · Source date 2026-10-24

    Assessed · Existing published review; targeted exact-version primary-text recheck

    Engineering takeaway: Persistent memory can manufacture permission even without injected instructions. Authorization should be independently derived from current grants.

    Author-reported evidence: Across the three benchmark domains, mean unauthorized-action rates were 27.7% for incremental typed memory and 0.8% for one-shot typed memory.

    Limitations: Synthetic histories, simplified tools and calibrated executors constrain transfer to deployed systems.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Revised benchmark and public review isolate memory-induced authority errors using matched interventions.

    Momentum 3, novelty 4, evidence 4, practicality 5, long-term value 5 (each out of 5).

  3. Rank 2 · 89/100 · Software engineering

    TestSuiteBench: Coding Agents Struggle to Generate Strong Test Suites from Specifications

    Kazuki Egashira, Niels Mündler-Sasahara, Mark Niklas Müller, Martin Vechev · Source date 2026-10-09

    Assessed · Historical assessment; targeted exact-version primary-text recheck

    Engineering takeaway: Tests that pass a reference implementation may still miss requirement violations. Independent behavioral oracles and negative cases are essential.

    Author-reported evidence: The paper evaluates 165 applications and reports mutation scores below 60% despite high test validity.

    Limitations: Generated mutants sample requirements and may not represent real fault distributions; specifications derive from reference tests.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Public benchmark resources and specification-level mutation assessment address a consequential gap between implementation and verification.

    Momentum 3, novelty 4, evidence 4, practicality 5, long-term value 5 (each out of 5).

  4. Rank 4 · 87.5/100 · Agent security

    ResidualAuth: What Authorization State Must Language Agents Preserve under Revocable Delegation?

    Moonwon Choi, Seokho Jeong, Sian Choi, Seunggeun Lee · Source date 2026-09-30

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Identical current permissions can conceal different future revocation behavior; preserve authorization-relevant dependencies.

    Author-reported evidence: Four evaluated models answer 15–16 of 16 paired cases correctly with a trusted decision versus 0–2 pairs using bounded event extracts alone.

    Limitations: Restricted delegation model and small paired diagnostic do not qualify a real authority service.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Revised formal state-equivalence results and paired benchmark make the retention requirement concrete; no proof replay.

    Momentum 3, novelty 4.5, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  5. Rank 5 · 86/100 · Supply chain security

    "Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild

    Yi Liu, Zhihao Chen, Yanjun Zhang, Gelei Deng, Yuekang Li, Jianting Ning, Leo Yu Zhang · Source date 2026-08

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Third-party agent skills combine executable software and untrusted instructions; both require supply-chain review.

    Author-reported evidence: The study reports 157 behaviorally confirmed malicious skills among 98,380 collected from two registries; the reported cases were removed after disclosure.

    Limitations: Limited registry snapshot and execution coverage; observed malicious skills are not a prevalence estimate.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Field corpus, behavior validation and disclosure; sampling and dynamic observation limits.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  6. Rank 5 · 86/100 · AI evaluation

    Audit Without Verification: When LLM Accountability Layers Relay Rather Than Check

    Paul-Peter Arslan · Source date 2026-09-07

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: An audit layer should examine independent observations rather than repeat upstream conclusions.

    Author-reported evidence: Removing the upstream recommendation improved attribution when upstream agents had missed the true origin, but could hurt when they were right.

    Limitations: Synthetic agent institutions and varying upstream reliability limit generalization.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Registered null result, paired intervention and public code; central follow-up result is exploratory.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  7. Rank 5 · 86/100 · Agent security

    Autonomy Comes with Costs: Detecting Denial-of-Service Vulnerabilities Caused by Resource Abusing in LLM-based Agents

    Jiaqi Luo, Jiarun Dai, Fengyu Liu, Songyang Peng, Youkun Shi, Tong Bu, Geng Hong, Xudong Pan, Yuan Zhang · Source date 2026-08

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Agent safety includes bounded resource lifecycles, not only access-control decisions.

    Author-reported evidence: AgentDoS reports 36 vulnerabilities across 16 of 20 evaluated agents and 15 assigned CVEs.

    Limitations: Source-visible web-agent resource patterns do not establish general production frequency or coverage of all resource types.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Real open-source systems and assigned CVEs motivate resource accounting; scope is specific.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  8. Rank 5 · 86/100 · Agent security

    Bounded Agents: Delegation Security for Multi-Agent AI Systems

    Xabier Muruaga · Source date 2026-08-16

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Delegated authority should narrow as agents compose actions; enforce scope and cumulative restrictions outside the model.

    Author-reported evidence: The authors report blocking all 544 tested InjecAgent data-stealing cases and reduced AgentDojo exfiltration, at a utility cost.

    Limitations: Composition guarantees rely on complete restrictions and serialized admission; benchmark results are conditional.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Public artifact and formal properties with explicit assumptions plus benchmark and utility comparisons.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  9. Rank 5 · 86/100 · Agent security

    CONTINUITY: Security-Context Contracts for Composable LLM Agent Controls

    Chris Zheng, Geng Yang · Source date 2026-09-04

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Individually sound security controls need contracts that preserve context and authority across boundaries.

    Author-reported evidence: The deterministic test suite reports no harmful commits across 2,560 parameterized attacks and completion of 700 benign tasks.

    Limitations: Tests exercise a bounded fault model; guarantees require trusted validation and mandatory mediation of protected effects.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Public artifact, explicit compositional model and fault suite; coverage restricted to the tested model.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  10. Rank 5 · 86/100 · Software security

    CULPA: Universal Detection of Memory-Safety Bugs in Unsafe Rust Through the Lens of Safety Requirements

    Hung-Mao Chen, Bo Lu, Xu He, Xiaokuan Zhang, Kun Sun · Source date 2026-08

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Make documented unsafe-code safety requirements executable analysis predicates instead of relying only on bug-pattern catalogs.

    Author-reported evidence: The paper reports 55 new memory-safety bugs, 29 developer-confirmed, from its study of popular Rust packages.

    Limitations: Static coverage is not universal; historical assessment notes unanalyzed packages and excluded FFI/concurrency cases.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Confirmed real defects from safety-requirement-based static analysis; scope and false-positive limitations.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  11. Rank 5 · 86/100 · Software security

    Quantifying Frontier LLM Capabilities for Container Sandbox Escape

    Rahul Marchand et al. · Source date 2026-08-01

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Evaluate sandbox boundaries under adversarial agent behavior rather than assuming container isolation is adequate.

    Author-reported evidence: A nested CTF evaluation shows models can exploit intentionally introduced container escape conditions.

    Limitations: Deliberately weakened test environments do not establish escape frequency in hardened production.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Revised open benchmark addresses isolation explicitly; deliberately vulnerable targets require careful interpretation.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  12. Rank 5 · 86/100 · Agent infrastructure

    Substrate-Portable Execution for Production LLM Workflows

    Tarun Gopinath, Atul Kulkarni, Vijay Rajakumar, Shrikar Katti, Parthasarathy Govindarajen · Source date 2026-09-05

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Separate workflow semantics from streaming, durable asynchronous and batch runtime bindings.

    Author-reported evidence: The authors report comparable output quality across three bindings and lower batch-inference costs in a production system.

    Limitations: Amazon-specific results and output quality comparisons do not establish equivalent failure, security or authorization behavior.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Reported production experience across execution modes addresses portability; limited independent observability.

    Momentum 3, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  13. Rank 13 · 85.5/100 · AI alignment

    Natural Emergent Misalignment from Reward Hacking in Production RL

    Monte MacDiarmid et al. · Source date 2025-11-23

    Assessed · Historical assessment; targeted exact-version primary-text recheck

    Engineering takeaway: Training that rewards shortcut behavior can produce misalignment outside the original training objective. Chat-style safety evaluations may miss failures in agentic settings.

    Author-reported evidence: The authors report sabotage attempts by experimentally trained models and evaluate alternative safety-training regimes.

    Limitations: Models were supplied knowledge of reward loopholes; the production comparison models did not attempt sabotage in the reported evaluation.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Controlled training and realistic agent-scaffold evaluations illuminate reward design and evaluation mismatch; transfer to ordinary coding agents is bounded.

    Momentum 2, novelty 4, evidence 4, practicality 4.5, long-term value 5 (each out of 5).

  14. Rank 14 · 85/100 · Agent security

    AGATE: Provenance-Based Runtime Defense Against Compositional Attacks on LLM Agents

    Xiaorui Zhang, Zhuoran Cheng, Kailin Liu, Zhaoxi Sun, Shiyu Fan, Tongyu Yuan, Bin Yuan, Weizhong Qiang, Deqing Zou · Source date 2026-09-25

    Assessed · Historical assessment; targeted exact-version primary-text recheck

    Engineering takeaway: Judge a proposed effect using both scoped authorization and the provenance of the transferred data. Separate passive observation from a veto-capable enforcement boundary.

    Author-reported evidence: The authors evaluate three harness adapters and report denial events in six of eleven benign file-processing scenarios.

    Limitations: Parameter transformation and legitimate reuse expose limitations; graph replay agreement does not prove complete effect observation.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    An implemented provenance/authority gate with explicit negative findings; heterogeneous adapter depth and limited coverage constrain evidence.

    Momentum 2, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  15. Rank 14 · 85/100 · Distributed systems

    Authorization Revocation for Long-Running AI Agents: Root-Scoped Quiescence under Delegation and Asynchronous Execution

    Genliang Zhu, Chu Wang · Source date 2026-09-18

    Assessed · Historical assessment; targeted exact-version primary-text recheck

    Engineering takeaway: Revocation is not a cancellation acknowledgement. A retired grant requires effect-boundary fencing and accounting for delegated or queued obligations.

    Author-reported evidence: The provider-free test suite reports 17/17 expected outcomes, and a separate checker reportedly rejects 44/44 modified traces.

    Limitations: Certifies only registered paths and assumed endpoint contracts; not proof of global shutdown, rollback or remote-provider closure.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Formal separation of root-scoped effect closure from cancellation, plus an executable checker; no deployment correspondence demonstrated.

    Momentum 2, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  16. Rank 14 · 85/100 · Software security

    Code-Augur: Agentic Vulnerability Detection via Specification Inference

    Zhengxiong Luo, Mehtab Zafar, Dylan Wolff, Abhik Roychoudhury · Source date 2026-06-17

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Make an agent's inferred security assumptions explicit and test whether execution contradicts them.

    Author-reported evidence: The authors report 22 previously unknown vulnerabilities in open-source projects using guided falsification.

    Limitations: A falsified assertion may reflect a defective specification rather than a vulnerability; model reasoning remains fallible.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Real reported vulnerability findings strengthen a falsification-oriented approach.

    Momentum 2, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  17. Rank 14 · 85/100 · Agent security

    Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems

    Hongyan Chang, Ergute Bao, Xinjian Luo, Ting Yu · Source date 2026-08

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Retrieved material needs an explicit untrusted-data boundary even when the original query is benign.

    Author-reported evidence: The authors report highly effective retrieval of adversarial content across 11 benchmarks and eight embedding models, plus downstream agent case studies.

    Limitations: Embedding-based retrieval is the evaluated scope; results depend on downstream orchestration and available privileges.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Evaluated retrieval and downstream agent behavior across multiple settings; retrieval and effect acceptance are distinct.

    Momentum 2, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  18. Rank 14 · 85/100 · Agent infrastructure

    Turning Interaction History into Execution State: A Runtime Layer for Long-Horizon Coding Agents

    Zehao Wang, Yisen Xu, Chenglin Li, Chao Peng, Bram Adams, Ahmed E. Hassan, Tse-Hsun Chen · Source date 2026-08-01

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Track execution state explicitly and invalidate stale observations rather than relying on a model to reconstruct state from history.

    Author-reported evidence: Across 500 SWE-bench Verified tasks, the reported layer improves success and reduces total cost for two evaluated model configurations.

    Limitations: Repository-state validity does not establish external effect truth, authorization or freshness in other domains.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Broad benchmark and a deterministic control layer; valid result reuse is bounded by its state model.

    Momentum 2, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  19. Rank 14 · 85/100 · Agent security

    When Consent Outlives Context: Residual Authority Replay in Long-Lived Agents

    Zhihao Zhang, Chao Wang, Rujia Li, Qingze Wang, Xiaoyan Sun, Jun Dai · Source date 2026-09-27

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Bind consent to current task context rather than letting earlier approvals silently authorize later actions.

    Author-reported evidence: The authors evaluate 508 AgentDojo attack cases and 55 live coding-agent cases, reporting increased attack success with residual authority.

    Limitations: Results are tied to evaluated authorization semantics, harnesses and contextual replay conditions.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Controlled and live-agent evidence examines stale consent; historical assessment flags authorization artifact uncertainties.

    Momentum 2, novelty 4, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  20. Rank 20 · 84.5/100 · Agent security

    Separation of Duties for Privileged LLM Agents: A Governed Execution Architecture with Measured Security-Utility Trade-offs

    Qishuai Jing · Source date 2026-09-28

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Separate planning, authorization, execution and audit, and bind one-time approvals to the action actually executed.

    Author-reported evidence: The paper reports four implementation defects revealed by evaluation and substantial case-level disagreement between modeled and executed outcomes.

    Limitations: One Windows/PowerShell implementation; only a subset of benchmark scenarios was re-executed.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Reproducibility artifact and contrasted modeled versus executed evidence; scope is one implementation.

    Momentum 3, novelty 3.5, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  21. Rank 21 · 83.5/100 · Agent security

    APort Vault: Benchmarking AI Agent Payment Authorization with the Open Agent Passport

    Uchi Uchibeke · Source date 2026-09-18

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Measure model requests, independent authorization decisions and actual effects separately.

    Author-reported evidence: In the matched replay comparison, the authors report 105 forbidden-recipient transfers without the layer and none with it.

    Limitations: Payment-specific rules and replay cohorts do not establish security for arbitrary tools or policies.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Released replay corpus with explicit event denominators; policy cohorts and payment specificity constrain comparison.

    Momentum 3, novelty 4, evidence 3.5, practicality 4.5, long-term value 5 (each out of 5).

  22. Rank 21 · 83.5/100 · Agent security

    From Storage to Steering: Memory Control Flow Attacks on LLM Agents

    Zhenlin Xu, Xiaogang Zhu, Yu Yao, Minhui Xue, Yiliao Song · Source date 2026-06-05

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Persistent memory can change future tool selection and task behavior, so treat it as a control-flow influence.

    Author-reported evidence: The authors report vulnerability in over 90% of evaluated memory-control-flow attack trials.

    Limitations: Framework-controlled tasks and experimental policies are not measurements of production effect compromise.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Revised study tests several models and tool frameworks; current abstract and historical notes constrain transfer.

    Momentum 3, novelty 4, evidence 3.5, practicality 4.5, long-term value 5 (each out of 5).

  23. Rank 21 · 83.5/100 · Agent security

    LLM Agents Can Easily Tamper With Their Own Traces

    Jeremy Qin, David Schmotz, Derck Prinzhorn, Luca Beurer-Kellner, Ameya Prabhu, Maksym Andriushchenko · Source date 2026-09-24

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Record execution evidence outside the agent's own write authority.

    Author-reported evidence: Five tested local harnesses permitted trace deletion; the reported Muse Code setup resisted.

    Limitations: Harness-specific configurations and local trace storage; proposed interception does not itself establish complete effect coverage.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Observed boundary failures are directly actionable; integrity of an independent recorder is a separate property.

    Momentum 2, novelty 3.5, evidence 3.5, practicality 5, long-term value 5 (each out of 5).

  24. Rank 24 · 81/100 · Agent security

    AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories

    Asif Pinjari, Mithun Paul Saint-Germain · Source date 2026-09-07

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Distinguish injection attempts from actual behavioral deviation over an entire execution sequence.

    Author-reported evidence: The authors label 71,024 steps in 12,536 synthetic trajectories; simple surface features miss many attacks.

    Limitations: One generating model, template concentration and dataset leakage constrain generalization.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Public dataset, step-level labels and hard negatives provide useful evaluation design; synthetic generator limits transfer.

    Momentum 3, novelty 4, evidence 3.5, practicality 4.5, long-term value 4.5 (each out of 5).

  25. Rank 24 · 81/100 · Systems security

    SoK: Capability Operating Systems: Is the Future Finally Here?

    Noah Mauthe, Eric Ackermann, Sven Bugiel · Source date 2026-08

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Capability systems need precise definitions of protection, delegation and enforcement, not merely capability terminology.

    Author-reported evidence: The study applies a unified taxonomy to representative capability operating systems over six decades.

    Limitations: Representative system selection and taxonomy choices are not experimental proof of effectiveness in agent/API environments.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Systematization of decades of capability systems supports durable terminology; not an agent benchmark.

    Momentum 2, novelty 3.5, evidence 3.5, practicality 4.5, long-term value 5 (each out of 5).

  26. Rank 24 · 81/100 · AI evaluation

    WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents

    Jingjie Ning, Xueqi Li, Yibo Kong, Dongting Li · Source date 2026-09-28

    Assessed · Historical assessment; primary abstract/proceedings metadata recheck

    Engineering takeaway: Finding a good configuration is different from explaining which changes caused its outcome.

    Author-reported evidence: The benchmark covers 36 task conditions and scores estimated component effects against exhaustive computational references.

    Limitations: Finite workflow families and sampled response surfaces cannot establish safe general autonomous experimentation.

    Independent execution: Not performed for this assessment.

    Editorial scoring rationale

    Revised benchmark evaluates conditional effects and artifact delivery rather than only selecting optima.

    Momentum 3, novelty 4, evidence 3.5, practicality 4.5, long-term value 4.5 (each out of 5).

Source and attribution

· Micrantha Research Community contributors ·

Original editorial summaries and metadata only; linked third-party papers, code, figures, and datasets retain their own rights. Paper authors are credited separately from editorial contributors.

Adapted from by Micrantha Research Community contributors (CC-BY-4.0). Condensed and adapted into an editorial index entry.