Architecture model (C4)

These diagrams are generated from the canonical Structurizr model in architecture/workspace.dsl, so they stay in sync with it. For the interactive, drill-down viewer and the other renderers, see architecture/README.md.

See also the hand-drawn cross-repo overview in Architecture diagrams.

L1 - System landscape

graph LR

  subgraph diagram ["System Landscape View"]

    1["Implementer / Analyst<br/>[Person]<br/>Configures models and reviews forecasts inside the DHIS2 Modelling App."]
    2["Model developer<br/>[Person]<br/>Develops models locally (CHAP CLI or chapkit), then publishes them one of two ways: an MLproject repo or a chapkit service."]
    20["chapkit model services [0..*]<br/>[Software System]<br/>Self-contained model services - one per model, the now-preferred path. Each exposes the standard CHAP train/predict contract over HTTP and registers itself with CHAP Core."]
    3["DHIS2<br/>[Software System]<br/>Health information system. Source of case, climate and org-unit data; destination for forecast data values."]
    4["CHAP Modelling App (chap-frontend)<br/>[Software System]<br/>Embedded DHIS2 app. The primary client of CHAP Core and the component that writes forecasts back into DHIS2."]
    5["Direct API client<br/>[Software System]<br/>Scripts, integrations and the CHAP CLI driving the REST API directly (OpenAPI client) rather than through the Modelling App."]
    6["Model source repos<br/>[Software System]<br/>Git repositories / MLproject definitions - one per model - run in-process by CHAP Core."]
    7["CHAP Core<br/>[Software System]<br/>Climate-and-health modelling backend: ingests data, runs evaluations and predictions, and serves results."]

    1-. "Configures models, reviews forecasts" .->4
    2-. "Develops & evaluates models locally" .->7
    2-. "Publishes a model -- Option A: MLproject repo" .->6
    2-. "Develops & tests a model -- Option B: chapkit service" .->20
    4-. "Reads case/climate/org-unit data; after review, optionally writes approved forecast data values<br/>[DHIS2 Web API]" .->3
    4-. "Submits data, runs evaluations/predictions, polls jobs, pulls forecasts<br/>[HTTPS/JSON (OpenAPI client)]" .->7
    5-. "Imports reusable datasets, runs evaluations/predictions<br/>[HTTPS/JSON (OpenAPI client)]" .->7
    7-. "Clones & runs model code<br/>[git / Docker / MLflow / UV]" .->6
    7-. "Trains & predicts<br/>[HTTP $train / $predict]" .->20
    20-. "Registers & sends heartbeats<br/>[HTTP $register / $ping]" .->7

  end

Hold "Alt" / "Option" to enable pan & zoom

L2 - Containers: CHAP Core

graph LR

  subgraph diagram ["Container View: CHAP Core"]

    2["Model developer<br/>[Person]<br/>Develops models locally (CHAP CLI or chapkit), then publishes them one of two ways: an MLproject repo or a chapkit service."]
    4["CHAP Modelling App (chap-frontend)<br/>[Software System]<br/>Embedded DHIS2 app. The primary client of CHAP Core and the component that writes forecasts back into DHIS2."]
    5["Direct API client<br/>[Software System]<br/>Scripts, integrations and the CHAP CLI driving the REST API directly (OpenAPI client) rather than through the Modelling App."]
    6["Model source repos<br/>[Software System]<br/>Git repositories / MLproject definitions - one per model - run in-process by CHAP Core."]
    20["chapkit model services [0..*]<br/>[Software System]<br/>Self-contained model services - one per model, the now-preferred path. Each exposes the standard CHAP train/predict contract over HTTP and registers itself with CHAP Core."]

    subgraph 7 ["CHAP Core"]

      13["Celery worker<br/>[Container: Celery (Python)]<br/>Consumes queued jobs and runs dataset harmonisation, backtests and predictions."]
      17["CHAP CLI<br/>[Container: Cyclopts (Python)]<br/>Local entry point for running and evaluating models without the API."]
      18[("Redis / Valkey<br/>[Container: Valkey 8]<br/>Celery broker and result backend, job metadata (job_meta) and chapkit service registry.")]
      19[("PostgreSQL<br/>[Container: PostgreSQL 17]<br/>Datasets, observations, model templates/configs, backtests, predictions.")]
      8["REST API<br/>[Container: FastAPI / Uvicorn (Python)]<br/>Serves the v1/v2 HTTP API, validates input, enqueues long-running jobs and serves results."]
    end

    2-. "Develops & evaluates models locally" .->17
    2-. "Publishes a model -- Option A: MLproject repo" .->6
    2-. "Develops & tests a model -- Option B: chapkit service" .->20
    4-. "Submits data, runs evaluations/predictions, polls jobs, pulls forecasts<br/>[HTTPS/JSON (OpenAPI client)]" .->8
    5-. "Imports reusable datasets, runs evaluations/predictions<br/>[HTTPS/JSON (OpenAPI client)]" .->8
    8-. "Readiness / deep probe: connectivity check" .->19
    8-. "Readiness / deep probe: broker ping and Celery round-trip" .->18
    13-. "Job lifecycle: fetch job, write job_meta (via Celery task wrapper)" .->18
    13-. "Reads datasets/models; writes forecasts & metrics" .->19
    13-. "Clones & runs model code<br/>[git / Docker / MLflow / UV]" .->6
    17-. "Runs model code locally" .->6
    13-. "Trains & predicts<br/>[HTTP $train / $predict]" .->20
    20-. "Registers & sends heartbeats<br/>[HTTP $register / $ping]" .->8
    8-. "Read-only proxy to live service (artifacts/configs/jobs)<br/>[HTTP GET/HEAD]" .->20

  end

Hold "Alt" / "Option" to enable pan & zoom

L2 - Containers: chapkit model service

graph LR

  subgraph diagram ["Container View: chapkit model services [0..*]"]

    2["Model developer<br/>[Person]<br/>Develops models locally (CHAP CLI or chapkit), then publishes them one of two ways: an MLproject repo or a chapkit service."]
    7["CHAP Core<br/>[Software System]<br/>Climate-and-health modelling backend: ingests data, runs evaluations and predictions, and serves results."]

    subgraph 20 ["chapkit model services [0..*]"]

      21["Service API<br/>[Container: FastAPI (Python)]<br/>FastAPI app assembled by chapkit's MLServiceBuilder; implements the standard train/predict/config/artifact/job REST contract."]
      31["Web console<br/>[Container: React SPA]<br/>Built-in SPA to browse configs/artifacts/jobs and trigger train/predict against the service."]
    end

    2-. "Develops & evaluates models locally" .->7
    2-. "Develops & tests a model -- Option B: chapkit service" .->31
    2-. "Publishes (deploys the self-registering service)" .->21
    7-. "Trains & predicts<br/>[HTTP $train / $predict]" .->21
    21-. "Registers & sends heartbeats<br/>[HTTP $register / $ping]" .->7
    31-. "Browses configs/artifacts/jobs; triggers train/predict<br/>[REST]" .->21

  end

Hold "Alt" / "Option" to enable pan & zoom

L3 - Components: CHAP Core REST API

graph LR

  subgraph diagram ["Component View: CHAP Core - REST API"]

    20["chapkit model services [0..*]<br/>[Software System]<br/>Self-contained model services - one per model, the now-preferred path. Each exposes the standard CHAP train/predict contract over HTTP and registers itself with CHAP Core."]
    4["CHAP Modelling App (chap-frontend)<br/>[Software System]<br/>Embedded DHIS2 app. The primary client of CHAP Core and the component that writes forecasts back into DHIS2."]
    5["Direct API client<br/>[Software System]<br/>Scripts, integrations and the CHAP CLI driving the REST API directly (OpenAPI client) rather than through the Modelling App."]

    subgraph 7 ["CHAP Core"]

      subgraph 8 ["REST API"]

        10["v1 routers<br/>[Component]<br/>crud, analytics, jobs, visualization. Datasets, backtests, predictions, job polling."]
        11["v2 routers<br/>[Component]<br/>Service registry (chapkit self-registration) plus a read-only reverse proxy to live chapkit services."]
        12["Orchestrator<br/>[Component]<br/>Redis-backed TTL registry of live chapkit services. Shared module (chap_core.rest_api.services.orchestrator): the API components and the Celery worker both resolve live services through it."]
        9["Common routes<br/>[Component]<br/>Health, readiness and system-info endpoints."]
      end

      18[("Redis / Valkey<br/>[Container: Valkey 8]<br/>Celery broker and result backend, job metadata (job_meta) and chapkit service registry.")]
      19[("PostgreSQL<br/>[Container: PostgreSQL 17]<br/>Datasets, observations, model templates/configs, backtests, predictions.")]
    end

    4-. "Submits data, runs evaluations/predictions, polls jobs, pulls forecasts<br/>[HTTPS/JSON (OpenAPI client)]" .->10
    5-. "Imports reusable datasets, runs evaluations/predictions<br/>[HTTPS/JSON (OpenAPI client)]" .->10
    9-. "Readiness / deep probe: connectivity check" .->19
    9-. "Readiness / deep probe: broker ping and Celery round-trip" .->18
    11-. "Registers / lists services" .->12
    10-. "Queues jobs; reads job status" .->18
    10-. "Reads/writes datasets, models, results" .->19
    12-. "Live service registry (TTL keys)" .->18
    10-. "Reads live services to sync model templates" .->12
    20-. "Registers & sends heartbeats<br/>[HTTP $register / $ping]" .->11
    11-. "Read-only proxy to live service (artifacts/configs/jobs)<br/>[HTTP GET/HEAD]" .->20

  end

Hold "Alt" / "Option" to enable pan & zoom

L3 - Components: CHAP Core worker

graph LR

  subgraph diagram ["Component View: CHAP Core - Celery worker"]

    20["chapkit model services [0..*]<br/>[Software System]<br/>Self-contained model services - one per model, the now-preferred path. Each exposes the standard CHAP train/predict contract over HTTP and registers itself with CHAP Core."]
    6["Model source repos<br/>[Software System]<br/>Git repositories / MLproject definitions - one per model - run in-process by CHAP Core."]

    subgraph 7 ["CHAP Core"]

      subgraph 13 ["Celery worker"]

        14["Worker functions<br/>[Component]<br/>db_worker_functions: harmonise datasets, run backtests, run predictions; read/write the database."]
        15["TrainPredict runners<br/>[Component]<br/>Run MLproject models in-process: Docker / UV / Conda / Renv / MLflow / CLI."]
        16["Chapkit REST client<br/>[Component]<br/>CHAPKitRestAPIWrapper: httpx client calling remote chapkit model services."]
      end

      18[("Redis / Valkey<br/>[Container: Valkey 8]<br/>Celery broker and result backend, job metadata (job_meta) and chapkit service registry.")]
      19[("PostgreSQL<br/>[Container: PostgreSQL 17]<br/>Datasets, observations, model templates/configs, backtests, predictions.")]
    end

    14-. "Job lifecycle: fetch job, write job_meta (via Celery task wrapper)" .->18
    14-. "Reads datasets/models; writes forecasts & metrics" .->19
    14-. "Resolves live service URL from registry (TTL keys, via the shared Orchestrator module)" .->18
    14-. "Runs in-process models" .->15
    14-. "Calls remote models" .->16
    15-. "Clones & runs model code<br/>[git / Docker / MLflow / UV]" .->6
    16-. "Trains & predicts<br/>[HTTP $train / $predict]" .->20

  end

Hold "Alt" / "Option" to enable pan & zoom

L3 - Components: chapkit Service API

graph LR

  subgraph diagram ["Component View: chapkit model services [0..*] - Service API"]

    7["CHAP Core<br/>[Software System]<br/>Climate-and-health modelling backend: ingests data, runs evaluations and predictions, and serves results."]

    subgraph 20 ["chapkit model services [0..*]"]

      subgraph 21 ["Service API"]

        22["Registration & health<br/>[Component]<br/>Self-registers with CHAP Core and sends heartbeats; serves /health and /api/v1/info (service identity CHAP Core reads)."]
        23["ML router<br/>[Component]<br/>/api/v1/ml: $train, $predict, $validate, $generate-sample-data."]
        24["Config router<br/>[Component]<br/>/api/v1/configs: typed, Pydantic-validated model configuration CRUD."]
        25["Artifact router<br/>[Component]<br/>/api/v1/artifacts: artifact CRUD - tree, expand, metadata, linked config, download (trained models, predictions)."]
        26["Jobs router<br/>[Component]<br/>/api/v1/jobs: async job status and cancellation."]
        27["ML manager<br/>[Component]<br/>Train/predict pipelines; turns runner output into typed, versioned artifacts."]
        28["Job scheduler<br/>[Component]<br/>In-memory async scheduler; runs train/predict as ULID-tracked background jobs."]
        29["Model runner<br/>[Component]<br/>Pluggable train/predict implementation: functional, class-based, or shell (Python / R)."]
        30[("Artifact & config store<br/>[Component: SQLite]<br/>Trained-model artifacts, predictions and configs; tree-structured and Alembic-migrated. Embedded in-process (same service).")]
      end

    end

    7-. "Trains & predicts<br/>[HTTP $train / $predict]" .->23
    22-. "Registers & sends heartbeats<br/>[HTTP $register / $ping]" .->7
    23-. "Submits train/predict requests" .->27
    27-. "Schedules background job" .->28
    28-. "Runs train / predict" .->29
    27-. "Reads/writes artifacts & configs" .->30
    24-. "Reads/writes configs" .->30
    25-. "Reads artifact tree" .->30
    26-. "Reads job status" .->28

  end

Hold "Alt" / "Option" to enable pan & zoom

Flow - Run a backtest

graph LR

  subgraph diagram ["Dynamic View: CHAP Core"]

    subgraph 7 ["CHAP Core"]

      13["Celery worker<br/>[Container: Celery (Python)]<br/>Consumes queued jobs and runs dataset harmonisation, backtests and predictions."]
      18[("Redis / Valkey<br/>[Container: Valkey 8]<br/>Celery broker and result backend, job metadata (job_meta) and chapkit service registry.")]
      19[("PostgreSQL<br/>[Container: PostgreSQL 17]<br/>Datasets, observations, model templates/configs, backtests, predictions.")]
      8["REST API<br/>[Container: FastAPI / Uvicorn (Python)]<br/>Serves the v1/v2 HTTP API, validates input, enqueues long-running jobs and serves results."]
    end

    4["CHAP Modelling App (chap-frontend)<br/>[Software System]<br/>Embedded DHIS2 app. The primary client of CHAP Core and the component that writes forecasts back into DHIS2."]
    20["chapkit model services [0..*]<br/>[Software System]<br/>Self-contained model services - one per model, the now-preferred path. Each exposes the standard CHAP train/predict contract over HTTP and registers itself with CHAP Core."]

    4-. "1. POST /v1/analytics/create-backtest-with-data<br/>[HTTPS/JSON (OpenAPI client)]" .->8
    8-. "2. Validate input, then queue backtest job (existing model_id)" .->18
    13-. "3. Fetch job" .->18
    13-. "4. Save DataSet; load ConfiguredModel by id" .->19
    13-. "5. Train + predict per fold (chapkit service, or an in-process MLproject runner)<br/>[HTTP $train / $predict]" .->20
    13-. "6. Write Backtest + forecasts + metrics" .->19
    4-. "7. GET evaluation results<br/>[HTTPS/JSON (OpenAPI client)]" .->8
    8-. "8. Read forecasts + actuals" .->19

  end

Hold "Alt" / "Option" to enable pan & zoom

Flow - Run a prediction

graph LR

  subgraph diagram ["Dynamic View: CHAP Core"]

    subgraph 7 ["CHAP Core"]

      13["Celery worker<br/>[Container: Celery (Python)]<br/>Consumes queued jobs and runs dataset harmonisation, backtests and predictions."]
      18[("Redis / Valkey<br/>[Container: Valkey 8]<br/>Celery broker and result backend, job metadata (job_meta) and chapkit service registry.")]
      19[("PostgreSQL<br/>[Container: PostgreSQL 17]<br/>Datasets, observations, model templates/configs, backtests, predictions.")]
      8["REST API<br/>[Container: FastAPI / Uvicorn (Python)]<br/>Serves the v1/v2 HTTP API, validates input, enqueues long-running jobs and serves results."]
    end

    1["Implementer / Analyst<br/>[Person]<br/>Configures models and reviews forecasts inside the DHIS2 Modelling App."]
    3["DHIS2<br/>[Software System]<br/>Health information system. Source of case, climate and org-unit data; destination for forecast data values."]
    4["CHAP Modelling App (chap-frontend)<br/>[Software System]<br/>Embedded DHIS2 app. The primary client of CHAP Core and the component that writes forecasts back into DHIS2."]
    20["chapkit model services [0..*]<br/>[Software System]<br/>Self-contained model services - one per model, the now-preferred path. Each exposes the standard CHAP train/predict contract over HTTP and registers itself with CHAP Core."]

    4-. "1. POST /v1/analytics/make-prediction (observations + geojson; a stored PredictionSetup fires the same job on a schedule)<br/>[HTTPS/JSON (OpenAPI client)]" .->8
    8-. "2. Queue prediction job" .->18
    13-. "3. Fetch job" .->18
    13-. "4. Predict quantile forecasts (chapkit service, or an in-process MLproject runner)<br/>[HTTP $train / $predict]" .->20
    13-. "5. Store Prediction samples (collected in CHAP)" .->19
    4-. "6. Pull prediction quantiles (/v1/analytics/prediction-entry)<br/>[HTTPS/JSON (OpenAPI client)]" .->8
    1-. "7. Review forecasts" .->4
    4-. "8. If approved, write forecast data values (dataValueSets)<br/>[DHIS2 Web API]" .->3

  end

Hold "Alt" / "Option" to enable pan & zoom

Flow - Import a reusable dataset (not the Modelling App path)

graph LR

  subgraph diagram ["Dynamic View: CHAP Core"]

    subgraph 7 ["CHAP Core"]

      13["Celery worker<br/>[Container: Celery (Python)]<br/>Consumes queued jobs and runs dataset harmonisation, backtests and predictions."]
      18[("Redis / Valkey<br/>[Container: Valkey 8]<br/>Celery broker and result backend, job metadata (job_meta) and chapkit service registry.")]
      19[("PostgreSQL<br/>[Container: PostgreSQL 17]<br/>Datasets, observations, model templates/configs, backtests, predictions.")]
      8["REST API<br/>[Container: FastAPI / Uvicorn (Python)]<br/>Serves the v1/v2 HTTP API, validates input, enqueues long-running jobs and serves results."]
    end

    5["Direct API client<br/>[Software System]<br/>Scripts, integrations and the CHAP CLI driving the REST API directly (OpenAPI client) rather than through the Modelling App."]

    5-. "1. POST /v1/analytics/make-dataset (observations + geojson)<br/>[HTTPS/JSON (OpenAPI client)]" .->8
    8-. "2. Validate input, then queue harmonise-dataset job" .->18
    13-. "3. Fetch job" .->18
    13-. "4. Harmonise & save DataSet + Observations" .->19

  end

Hold "Alt" / "Option" to enable pan & zoom