Articles | Open Access | https://doi.org/10.55640/ijdsml-06-02-16

Faogf: Auditable Evidence Provenance For Autonomous Financial Ai Agents On Opentelemetry

Sowmith Javvaji , Independent Researcher, New York, NY, USA

Abstract

Autonomous AI agents now read regulatory filings, compute figures and distribute reports inside financial firms, but their telemetry cannot tell an auditor which filing versions a decision rested on, under which authority and policy version an action ran, or whether untrusted content was in the model’s context. We present FAOGF (Financial Agent Observability and Governance Framework), an OpenTelemetry-compatible audit layer. Its core architectural contribution is context-bound evidence provenance recorded on two paths: a gateway binds every model call and action to the version-exact, content-hashed evidence actually in the model’s context, and writes that binding synchronously to a hash-chained evidence store before exporting standard spans, so audit completeness no longer depends on what the tracing pipeline keeps. We evaluate nine audit queries. In 15 controlled experiments with a scripted agent (200 workflows × 5 seeds, and a real OpenTelemetry Collector on two machines), all nine queries were evaluated: FAOGF answered the seven answerability queries (Q1–Q7) exactly in every configuration, detected 88–97% of spans lost to export-queue overflow (Q8) and 100% of audit attributes stripped by a Collector processor (Q9). Standard OpenTelemetry tracing recovered only 9–12% of protected actions at 10% head sampling and was silently stripped by a common Collector configuration; full prompt logging could not answer authority or policy questions. In a live study on SEC EDGAR filings with gpt-4o-mini and gpt-4o on public Finance Agent Benchmark questions, we scored the two queries the live workload exercises, supporting filings (Q3) and untrusted-context exposure (Q7). FAOGF was exact on both, standard tracing misattributed 4–18% of supporting filings, and full prompt logging used 7–10× more storage. Neither live model followed any of the 78 injected instructions, so enforcement was evaluated on a simulated susceptible agent. There, escalating only the sends requested with untrusted content in context blocked every injected send with 37% fewer human approvals than requiring approval for every send. FAOGF adds about 0.2 ms median overhead per call.

 

Keywords

AI agents, observability, OpenTelemetry, audit logging

References

OpenTelemetry, “Semantic conventions for generative AI systems,” OpenTelemetry Specification, 2025. [Online]. Available: https://opentelemetry.io/docs/specs/semconv/gen-ai/

B. H. Sigelman et al., “Dapper, a large-scale distributed systems tracing infrastructure,” Google Technical Report dapper-2010-1, 2010.

OpenTelemetry, “OpenTelemetry Specification: Tracing API and SDK,” 2025. [Online]. Available: https://opentelemetry.io/docs/specs/otel/

P. Las-Casas, G. Papakerashvili, V. Anand, and J. Mace, “Sifter: Scalable sampling for distributed traces, without feature engineering,” in Proc. ACM Symp. Cloud Computing (SoCC), 2019.

Mishra and Sharad, arXiv preprint arXiv:2606.09692, 2026.

W3C, “Trace Context,” W3C Recommendation, 2021.

OpenTelemetry, “Weaver: OpenTelemetry semantic convention registry tooling,” v0.28.0. [Online]. Available: https://github.com/open-telemetry/weaver

Langfuse, LangSmith, Arize Phoenix and OpenLLMetry product documentation, accessed Oct. 2026.

L. Dong, Q. Lu, and L. Zhu, “AgentOps: Enabling observability of LLM agents,” arXiv:2411.05285, 2024.

L. Moreau and P. Missier, Eds., “PROV-DM: The PROV data model,” W3C Recommendation, 2013.

A. Bigeard, L. Nashold, R. Krishnan, and S. Wu, “Finance Agent Benchmark: Benchmarking LLMs on real-world financial research tasks,” arXiv:2508.00828, 2025. Public validation split (50 questions, CC BY 4.0), Vals AI. [Online]. Available: https://github.com/vals-ai/finance-agent/blob/main/data/public.csv. The exact file used is redistributed in the FAOGF artifact as data/fab_public.csv (SHA-256 901e8cff562819007d43ff0c10b43b2e386481eb40f2157ba18da8fbbe05bff5).

U.S. Securities and Exchange Commission, Rule 17a-4, “Records to be preserved by certain exchange members, brokers and dealers,” 17 CFR 240.17a-4.

FINRA, Rule 3110, “Supervision.”

Board of Governors of the Federal Reserve System and OCC, “Supervisory guidance on model risk management,” SR 11-7, 2011.

FINRA, Regulatory Notice 24-09, “FINRA reminds members of regulatory obligations when using generative artificial intelligence and large language models,” 2024.

Regulation (EU) 2024/1689 (Artificial Intelligence Act), Art. 12, “Record-keeping,” Official Journal of the European Union, 2024.

B. Schneier and J. Kelsey, “Secure audit logs to support computer forensics,” ACM Trans. Information and System Security, vol. 2, no. 2, 1999.

[18] S. A. Crosby and D. S. Wallach, “Efficient data structures for tamper-evident logging,” in Proc. USENIX Security Symp., 2009.

R. C. Merkle, “A digital signature based on a conventional encryption function,” in Advances in Cryptology — CRYPTO ’87, 1988.

B. Laurie, A. Langley, and E. Kasper, “Certificate Transparency,” RFC 6962, IETF, 2013.

K. Greshake et al., “Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection,” in Proc. ACM Workshop on Artificial Intelligence and Security (AISec), 2023.

E. Debenedetti et al., “Defeating prompt injections by design,” arXiv:2503.18813, 2025.

M. Costa et al., “Securing AI agents with information-flow control,” arXiv:2505.23643, 2025.

Z. Chen et al., “FinQA: A dataset of numerical reasoning over financial data,” in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2021.

F. Zhu et al., “TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance,” in Proc. Annu. Meeting Assoc. Computational Linguistics (ACL), 2021.

Z. Chen et al., “ConvFinQA: Exploring the chain of numerical reasoning in conversational finance question answering,” in Proc. EMNLP, 2022.

P. Islam et al., “FinanceBench: A new benchmark for financial question answering,” arXiv:2311.11944, 2023.

V. Reddy et al., “DocFinQA: A long-context financial reasoning dataset,” in Proc. ACL, 2024.

S. Wu et al., “BloombergGPT: A large language model for finance,” arXiv:2303.17564, 2023.

H. Yang, X.-Y. Liu, and C. D. Wang, “FinGPT: Open-source financial large language models,” arXiv:2306.06031, 2023.

P. Lewis et al., “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.

A. Singh et al., “Agentic retrieval-augmented generation: A survey on agentic RAG,” arXiv:2501.09136, 2025.

B. Sarmah et al., “HybridRAG: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction,” arXiv:2408.04948, 2024.

D. Zhou et al., “Least-to-most prompting enables complex reasoning in large language models,” in Proc. Int. Conf. Learning Representations (ICLR), 2023.

V. Gupta et al., “TempTabQA: Temporal question answering for semi-structured tables,” in Proc. EMNLP, 2023.

Article Statistics

Downloads

Download data is not yet available.

Copyright License

Download Citations

How to Cite

Sowmith Javvaji. (2026). Faogf: Auditable Evidence Provenance For Autonomous Financial Ai Agents On Opentelemetry. International Journal of Data Science and Machine Learning, 6(02), 261-281. https://doi.org/10.55640/ijdsml-06-02-16