FinQuery
An OpenEnv-compatible reinforcement-learning environment that simulates a financial data terminal for training agents on multi-step analytical reasoning. Each reset procedurally generates a verifiable finance question; the agent decides which data to fetch and how to combine it, earning dense per-step rewards. Covers 25 companies across 7 sectors over 9 years (2017-2025).
Problem
Most RL environments for agents are single-shot QA. Real analytical work is multi-step: decide what data to fetch, combine sources, and defend a conclusion, and the reward signal for that behavior is hard to generate at scale without humans grading answers.
Approach
- Simulated a financial data terminal as an OpenEnv-compatible RL environment.
- Made every reset procedurally generate a verifiable finance question. No two episodes identical, no hand-labeled data.
- Let the agent choose which deterministic tools to call (statements, ratios, price history) with dense per-step rewards.
- Shipped easy/medium/hard task tiers plus composite curriculum mixing, with gpt-4o-mini baselines included.
Outcome
Tens of thousands of unique verifiable training episodes covering 25 companies across 7 sectors over 9 years (2017-2025), deployed as a public Hugging Face Space.
Architecture
env = FinQuery(difficulty="hard")
obs = env.reset() # new episode
while not done:
action = agent(obs) # fetch? compute? answer?
obs, reward, done = env.step(action)
# reward is per-step & verifiable:
# ground truth derived from the same generatorHighlights
Procedurally generated, verifiable finance tasks, no two episodes identical.
25 companies, 7 sectors, 9 years; tens of thousands of unique training episodes.
Deterministic data tools (statements, ratios, price history) with dense per-step rewards.
Easy / medium / hard tasks plus composite curriculum mixing; gpt-4o-mini baselines included.
OpenEnv-compatible; deployed as a Hugging Face Space.
Stack
- Python
- FastAPI
- Reinforcement Learning
- Large Language Models
- Docker