Phase 1 — Target Discovery + Benchmark

What this phase delivers: the ability to define a campaign (postal codes + industries + budget), have the system automatically find local businesses, enrich them with signals, and produce a benchmark-scored, sortable Target list.

This is the front-end differentiator and we build it first — it's the hardest custom part, and on its own it's already something we can demo.

1. What we build vs. what we integrate (this phase)

We build the campaign setup, the discovery orchestration (finding, enriching, scoring in a controlled, budget-aware way), the benchmark scoring, and the target list. That's our IP.

We integrate the data itself — nobody should be re-inventing a Google Maps crawler or an email-finding database. Those are mature, paid services we plug in. The rest of this doc is about which services, their trade-offs, how each one works, and where the docs are.

Every integration is its own NestJS module behind an interface (mockable), with retry + exponential backoff, a configurable timeout (default 30s), secret-redacted logging, a recorded cost per call, and it is mocked in tests — never called for real in CI. The discovery orchestration runs on the research queue (BullMQ). Every stored fact carries provenance in research_data_points (source, retrieved_at, confidence), and every external HTTP/LLM call is appended to research_api_calls (provider, endpoint, tokens, cost, duration — never updated or deleted).

The integrations fall into three layers.

2. Layer 1 — Business discovery (Google Maps by postal code + industry)

The entry point: given a postal code and an industry (business type), return the list of local businesses. This repo's primary identification source is Serper.dev (Google Maps/Places), called in parallel as a free-to-cheap, fast source. The options below are recorded for context and trade-offs.

How each one works + docs

• Serper.dev — A real-time Google Search/Maps API. You query its Maps/Places endpoint with a search term + location and get back structured JSON place records (name, address, phone, website, rating) almost instantly, without managing proxies. This is the primary ID source, run in parallel across queries. Docs: https://serper.dev/
• Google Places API — You call HTTP endpoints (Text Search or Nearby Search) with a search term + location, and get back structured place records (name, address, phone, website, rating). A "field mask" controls which fields you get, which also controls cost. Docs: https://developers.google.com/maps/documentation/places/web-service/overview
• Outscraper — A hosted scraping service with a REST API. You send a search term + location; it returns Google Maps business records, and can enrich each one with emails/socials pulled from the business's own website. Pay per record. Docs: https://app.outscraper.com/api-docs
• Apify — A platform of pre-built scrapers called "actors". You trigger the Google Maps actor through the Apify API (a "run and get results" endpoint), pass a search + location, and receive a dataset back (JSON/CSV). Supports webhooks and scheduling. Docs: https://apify.com/compass/crawler-google-places/api

Decision (this repo): Serper.dev is the primary discovery/ID source. If a specific campaign needs the official Places guarantee or richer fields, the scraping/official alternatives can be added behind the same dialer-style interface. ⚠️ Needs client sign-off on whether any scraping route (with its ToS/GDPR exposure) is acceptable at all.

3. Layer 2 — Contact enrichment & website signals

Maps gives us the business; we still need a named person, verified contact channels, and website/register signals. In this repo the enrichment layer is built from the platform's own integration list rather than a third-party contact-database waterfall.

Because this targets the German / EU market, GDPR-compliant European coverage matters more than raw database size.

How each one works + docs

• Firecrawl.dev — A crawl/scrape API: you point it at the company's website and it returns clean, structured page content (and contact emails) we store as provenance-tagged facts. Called on demand and budget-pre-checked because it is a paid source. Docs: https://docs.firecrawl.dev
• handelsregister.ai — A company-register API for German commercial-register data (legal form, registered officers), queried only when a register entry is likely so we don't spend on businesses that won't have one. Docs: https://handelsregister.ai
• TheirStack — Identifies hiring managers / decision-makers; used only as a fallback when a named contact isn't found from cheaper sources. Docs: https://theirstack.com
• INWX — Domain registrar API, used on demand for sender-domain purchases that support the Outreach phase. Docs: https://www.inwx.com/en/help/apidoc
• Anthropic Claude — Given the collected raw facts, Claude produces a structured company profile and de-duplicates hiring managers. Responses are schema-validated before use and user/research content is XML-tag-isolated in the prompt (prompt-injection defense). Docs: https://docs.claude.com/en/api/overview

Note on the contact-database market: third-party finders such as Dropcontact, Datagma, Prospeo, or aggregators like FullEnrich exist and could be slotted behind the enrichment interface later if match rates from the website crawl + register lookups disappoint. They are not part of the v1 integration list. ⚠️ Needs client sign-off before adding a contact-database vendor (GDPR compliance then spans the sub-providers).

4. Layer 3 — Benchmark signals (what feeds the score)

These are the extra checks that make each business scoreable — they don't find contacts, they tell us how promising a Target is. All three are essentially free, run in parallel, so there's no real vendor decision here.

How each one works + docs

• Meta Ad Library API — A single Graph API endpoint (GET /ads_archive). You query by advertiser page or search term plus a target country, and get back the ads a business is running or has run — so we can tell whether they're actively advertising. Free developer tier (one-time identity verification), ~200 calls/hour. Docs: https://www.facebook.com/ads/library/api
• Wayback / Internet Archive API — A simple HTTP call (archive.org/wayback/available?url=…) returns the closest archived snapshot of a site with a timestamp; the CDX server API returns first/last capture dates. We use these to gauge how old or stale a site is. Free, no key. Docs: https://archive.org/help/wayback_api.php
• jobsuche.api.bund.dev — The German Federal Employment Agency's job-postings API. We query it (free, in parallel) to check hiring activity; if it returns nothing for a business, SerpAPI Google Jobs is queried as a paid fallback. Docs: https://jobsuche.api.bund.dev

5. The benchmark score

The score is computed once per target here, in Phase 1, and then flows through every later stage — it sorts the outreach emails, prioritises the call queues, and drives shift assignment. It combines the signals above into a single 0–100 number with a stored breakdown, so we can always show why a Target scored the way it did. There is no automatic refresh; the only re-research is when a stale target (>60 days old data) is assigned to a campaign.

This repo's benchmark is versioned across 5 weighted dimensions:

1. Segment fit — industry, legal form.
2. Size & hiring pressure — headcount, hiring intensity, hiring trend.
3. Growth & dynamics — location expansion, org growth, activity.
4. Marketing / sales intent — paid-ads activity, employer-brand signal.
5. Contactability & data quality — contact coverage, channel quality, confidence.

Re-scoring never rewrites history — each score is stored with its benchmark version, which is exactly what lets Phase 5 run benchmark-weighting A/B tests later.

The important point for the meeting: domain experts should define what matters most. Their 15+ years of experience is exactly what should set the initial weighting — which signals count double, which are minor. The weighting is adjustable so it can be retuned over time without a rebuild. ⚠️ Needs client sign-off on the initial dimension weights.

6. Legal & compliance notes for this phase

Phase 1 is where the data-collection questions live. Note that a full GDPR / spam / telemarketing compliance layer is explicitly out of scope for v1 in this repo — treat the points below as design considerations to get a written legal opinion on before Outreach, not as build items for this phase:

• Scraping vs. official API — Google's and Meta's terms restrict scraping; the safest route is official APIs or licensed providers (this is the Layer 1 consideration). This repo defaults to Serper.dev + free official signal sources.
• GDPR — we're storing named contact persons at named businesses. There should be a lawful basis, a retention limit, and a way to delete a company/contact on request. Designing clean deletion from the start (rather than retrofitting) is a sensible posture even though the compliance layer itself is deferred.

7. Decisions needed for Phase 1

1. Discovery route: Serper.dev is the default primary source — ⚠️ needs client sign-off on whether any scraping alternative (Outscraper/Apify) is ever acceptable given ToS/GDPR risk, or whether to add the official Places API for zero-risk campaigns.
2. Enrichment depth: confirm the v1 layer (Firecrawl website crawl + handelsregister.ai + TheirStack) is sufficient, or whether a contact-database vendor should be added behind the interface. ⚠️ Needs client sign-off.
3. Budget: is the campaign budget measured in currency spend or provider credits, and what triggers the warning — a fixed amount or a percentage? (This repo pre-checks every paid call against the campaign budget and warns at 80% / auto-pauses at 100%.)
4. Job-portal access: confirm jobsuche.api.bund.dev suffices as primary, with SerpAPI Google Jobs as the paid fallback.
5. Benchmark weighting: domain experts to rank which of the 5 dimensions matter most. ⚠️ Needs client sign-off.
6. Procurement: who sets up and pays for the Serper.dev + enrichment + Anthropic accounts? (Third-party secrets are stored encrypted in the settings table, set via UI — the Anthropic key is the one infrastructure credential.)

Appendix — Documentation links

Prices and rate limits are approximate and change — confirm on each provider's site. Part of the project docs — Phase 1 of 5.