Skip to content
Pratim/Dasude
04/Multi-agent3 min read

AI Builder: multi-agent workers

Outcome

Three FastAPI agent workers that turn CSV data into structured decision memos.

Multi-agentFastAPIStreamingForecasting

FastAPI, Gradio, XGBoost, SQLite, OpenAI

View on GitHub
The AI Builder system overview diagram: a Gradio frontend launching three FastAPI workers for finance, logistics and growtharchitecture
The system overview from the project README: one control panel, three workers, one shared layer shape.

Draft write-up

AI Builder is three self-contained agent workers behind a single Gradio control panel. Each worker is a FastAPI service that wraps a multi-agent pipeline over a domain CSV bundle and returns a structured memo plus the raw analytical data. A finance worker acts as a CFO copilot, a growth worker as a growth strategist, and a logistics worker as a dispatch planner.

Source on GitHub ↗

01Problem

Operations teams sit on ledgers, orders and campaign data and need a clear recommendation, not a spreadsheet. A model alone should not be trusted with the numbers, and three separate agent apps would drift apart into three codebases nobody can reason about.

02Approach

  1. 01

    One shape for every worker

    The workers share no code but share the same layered shape: API, supervisor, planner or orchestrator, agents, tools, services, CSV connectors, memory and a response formatter with streaming. Understand one and you understand all three.

  2. 02

    Numbers from analyzers, words from the model

    Deterministic services compute the analytics, for example reconciliation, margin and revenue forecasting. The language model writes the memo, while risk flags in the finance worker stay deterministic.

  3. 03

    Specialist agents that run together

    A supervisor and planner hand work to specialist agents, which run concurrently with asyncio, and each worker keeps its own SQLite memory.

  4. 04

    Stream the result

    Each worker exposes chat endpoints including a streaming one (server-sent events), and the control panel launches the workers and health-checks them.

03Architecture

A Gradio panel starts three FastAPI workers. Inside each worker the same layers repeat, from the API down to its own CSV data and memory.

01Control panelGRADIO02Worker APIFASTAPI03Supervisor and plannerAGENTS04Agents, tools and servicesASYNCIO05Connectors and MLCSV, XGBOOST06Memory and outputSQLITE, MEMO FORMATTER

  1. 01Control panel

    Gradio

    Launches each worker as a subprocess and checks its health.

  2. 02Worker API

    FastAPI

    Chat, raw-data and streaming endpoints, one service per domain.

  3. 03Supervisor and planner

    Agents

    Plans the task and delegates to specialist agents.

  4. 04Agents, tools and services

    asyncio

    Specialists call a tool registry and deterministic analyzers, running concurrently.

  5. 05Connectors and ML

    CSV, XGBoost

    Domain CSV data, and a persisted XGBoost model for the 7-day revenue forecast.

  6. 06Memory and output

    SQLite, memo formatter

    Per-worker memory, then a structured memo with streaming.

  • Finance worker: revenue forecast, invoice reconciliation, leakage, margin and risk flags, as a CFO memo
  • Growth worker: promotion scoring, posting time, segments and six content variants, as a Growth Action Memo
  • Logistics worker: warehouse assignment, clustering, routing and risk, as a Markdown dispatch memo
  • Falls back gracefully when no API key is set

04Challenges

  • Three agent apps can drift apart over time.

    FixThey follow one fixed layered shape, so each worker is structured the same way and can be read the same way.

  • Model-written memos should not invent numbers.

    FixThe figures come from deterministic analyzers and models, and the model only writes up the result.

05Learnings

Outcomes

  • Three working agent workers, each producing a structured memo from its own data
  • One control panel that starts and monitors all of them

What I learned

  • A shared structure matters more than shared code when several agents must stay maintainable.
  • Keep calculations deterministic and let the model explain them.

06Next

  • 01Add an evaluation set that checks memo claims against the analyzer output
  • 02Move worker memory from SQLite to a shared store