Phase 5 — Controlling + Optimisation

What this phase delivers: the single cross-team overview that ties the whole system together — one dashboard spanning outreach → connect → closing — plus the two optimisation loops: smart budget allocation optimization across campaigns/sources, and A/B testing of the benchmark weighting.

Everything in this phase derives from data the system already produces in Phases 1–4, and everything renders inside our own UI.

1. What we build

This phase is entirely custom. There is nothing external to fetch — it all reads from our own data model. We build:

• the controlling overview (the cross-team dashboards),
• the budget allocation optimization,
• the benchmark weighting A/B testing,
• the reporting and exports,
• and the dashboards themselves, rendered in our own interface (Next.js App Router + React 19 + shadcn/ui, with a React charting library).

2. Dashboard / controlling presentation

The controlling views are built directly into our own Next.js UI, rendering from pre-computed aggregate tables. This keeps the whole platform in a single interface, keeps all data inside our own infrastructure (the safer GDPR posture), and stays tightly coupled to the same benchmark and data model used across every stage.

For analytics access, this repo prepares read-only bi_* views exposed to a scoped, read-only database user (see technical-requirements §13) — reporting reads those, not the raw operational tables, and avoids soft-deletes on reporting tables so figures stay consistent.

Two external tools can be layered on later if a specific need arises, but neither is part of the core build:

• Metabase (self-hosted) — for internal ad-hoc data exploration without writing SQL, if analysts ever want to slice the data outside the built dashboards (pointed at the bi_* views). Docs: https://www.metabase.com/docs/latest
• Grafana — for monitoring the pipeline's operational health (BullMQ queue health, API budget burn), which is a separate concern from business reporting. Docs: https://grafana.com/docs

3. What we build — the substance of Phase 5

Controlling overview One dashboard aggregating the three team views — outreach, connect, closing — into a single picture: from businesses discovered, through emails sent and replies, to calls, appointments, and closed contracts. Built as pre-computed aggregates (refreshed nightly and on-demand) rather than querying raw logs live, so it stays fast as data grows. Everything it needs was already logged in Phases 1–4 (audit_logs, bandit_events, research_api_calls, PipelineEvents).

Budget allocation optimization The optimisation loop that reallocates spend. Instead of splitting the API budget evenly, the system watches which campaigns and which data sources actually produce the best results per euro (highest-benchmark or highest-converting targets) and shifts more of the remaining budget toward the winners. It plugs into the campaign budget guard built in Phase 1, turning a hard spend cap into smart spend allocation.

Not the outreach Bandit. This budget allocation optimization is a separate, later concern from the Phase 2 Bandit (the Thompson-Sampling picker that selects the best outreach StepVersion at send time). One tunes how spend is distributed across campaigns/sources; the other tunes which email content variant is sent. Keep them distinct in code and in reporting.

Benchmark weighting A/B Because every score is stored with its benchmark version (from Phase 1), we can run two different weightings side by side and compare what actually happened downstream — which weighting's "high scores" really did reply, pick up the phone, and close. The winner's weights become the new default. This closes the loop: the benchmark stops being a guess and starts being tuned by real outcomes.

Reporting / exports Scheduled and on-demand reports (per campaign, per agent, per team) with CSV/spreadsheet export for sharing outside the system, read from the bi_* views.

4. How it works

1. Phases 1–4 have already logged every step (discovery, score, email, call, deal) via the immutable audit_logs and the append-only event tables.
2. Nightly (and on demand, via BullMQ scheduled jobs), the system rolls those logs into aggregate tables / bi_* views.
3. The controlling dashboard reads the aggregates — fast, whole-funnel visibility in one place.
4. The budget allocation optimization reviews yield per campaign/source and rebalances the remaining budget for the next runs.
5. The A/B engine compares benchmark versions against real downstream outcomes and proposes better weights.
6. Reports are generated from the same aggregates and exported as needed.

5. Legal & compliance notes for this phase

The full compliance layer is out of scope for v1 in this repo; treat these as design considerations:

• Aggregation over raw data — controlling views work off aggregates and the bi_* views, which reduces how much personal data is exposed in reporting.
• GDPR retention — the underlying logs are still personal data; retention and erasure handling should cover the aggregates and exports too (an export shouldn't resurrect deleted data). Note the deliberate exception: reporting tables avoid soft-deletes so figures stay consistent, with the bi_* views filtering out records kept only for audit/forensics.
• Data residency — building the dashboards in-app keeps reporting data in our own infrastructure, in-region.

6. Decisions needed for Phase 5

1. Controlling KPIs: which cross-team metrics belong on the single overview, and at what granularity (campaign / agent / team)? ⚠️ Needs client sign-off.
2. Optimization scope: should budget reallocation run automatically, or only propose changes for a human to approve?
3. A/B cadence: how often do we re-evaluate benchmark weightings, and who signs off on a new default? ⚠️ Needs client sign-off.
4. Reporting: which reports are scheduled vs. on-demand, and who receives them?
5. Data residency: any requirement that reporting data stay in the EU?

Appendix — Optional add-on documentation

These are not part of the core build, only potential later additions:

Part of the project docs — Phase 5 of 5 (final phase).