FinAgent
ONLINE08 / 2026
An end-to-end platform for building, evaluating, and observing LLM agents in financial workflows.
01
PARTS LIST
PythonLangGraphClaude APIRayKubernetesAirflowPrometheusPyTorch
02
BUILD LOG
- Building: two LangGraph agents — a router/synthesizer and a mixture-of-experts with parallel fan-out — run behind one env-swappable runner protocol, so the harness is provably reusable rather than wired to a single design. The same tools are served over MCP, so external clients reuse them without duplicated logic.
- Evaluating: an LLM-as-judge harness scores a golden dataset with Ray fanning out the runs, an optimizer proposes and adopts better prompts against that score, and PyTorch fine-tunes on the judge-graded transcripts — so quality is measured rather than asserted.
- Observing: LangFuse traces every run, Prometheus scrapes a /metrics endpoint into Grafana, and an Airflow DAG runs canaries against a rolling baseline with Alertmanager firing on drift — so a degrading agent pages someone before a user reports it.