Build a Daily AI SEO Analyst

The emphasis is decision support, not autonomous optimization. The workflow you build here reads data and writes a brief; a person decides what to change.

Every factual product capability in this guide is backed by the vendor's own documentation. Any suggested workflow, threshold, or implementation pattern is presented as a recommendation, not a platform requirement — where we recommend something, we say so explicitly.

Unlike a dashboard, which primarily reports metrics, an AI analyst investigates relationships across multiple data sources before presenting prioritized findings. The output is a short daily brief: what moved, whether the movement is material, the most likely explanation, and the checks a human should run next.

This guide deliberately does not describe an autonomous agent. The analyst does not publish content, edit metadata, deploy code, change redirects, submit URLs, modify analytics configuration, or make any production decision without review.

Everything in this chapter is Connector Scout editorial guidance. It is not a vendor requirement, and no platform documents an "investigation loop", an "evidence layer" or a confidence band. What the vendors document is what their APIs return; how you reason across those returns is a house style, and this is ours. Sections that cite official documentation are labeled as official documentation throughout the guide, so the difference is always visible on the page.

The mindset has four moving parts: a repeatable investigation loop, a layered view of what each evidence source can and cannot prove, an explicit separation of facts from hypotheses with a confidence score attached, and a prioritization framework that decides what the reviewer should look at first.

The investigation loop

Every flagged change runs through the same six steps, in the same order. The order matters more than any single step: skipping validation produces false alarms, and skipping corroboration produces confident hypotheses dressed up as findings.

Five layers of SEO evidence

Each layer answers a different question, and none of them answers all four daily questions alone. Naming the layer a number came from is what stops the analyst from arguing that a session drop proves a ranking loss.

Evidence chains

An evidence chain is how a finding is presented: each link is labeled as a verified fact, a calculated metric, a hypothesis or a recommended check, so a reviewer can see exactly where observation ends and inference begins. Two worked examples follow.

Confidence scoring

Every hypothesis carries a score from 0 to 100 and the band that score falls in. The bands are our editorial convention — they exist so “probably” means the same thing in every brief.

Prioritization framework

Five inputs decide what reaches the top of the brief. They are deliberately not collapsed into one score: reviewers need to see why something was ranked first, and a hidden weighting is impossible to argue with.

Three principles keep the output trustworthy. Evidence before explanation: every recommendation should be traceable to one or more verified data sources. Facts before hypotheses: the brief must separate verified facts, calculated metrics, hypotheses and recommended validation steps, and never blur them. Multiple sources beat one: a finding supported by two independent sources deserves higher confidence than a finding from a single API.

Useful pairings in practice are Search Console with GA4, Search Console with Ahrefs, GA4 with Clarity, and Search Console with Clarity and Semrush. Corroboration is what makes the difference between an analyst and a chart.

Option A — Interactive AI workspace suits independent consultants, small teams, experimentation and ad hoc analysis. You work in an AI chat interface, use official MCP servers where the vendor provides one, and add custom read-only tools for platforms that do not. Investigations are human-driven. It is fast to set up, flexible and needs almost no infrastructure, but it is less repeatable, depends on manual prompting, and handles historical processing poorly.

Option B — Scheduled AI analyst suits agencies, enterprise SEO, large content libraries and recurring executive reporting. Data sources feed a scheduled extraction step, then a normalized data store, then validation rules, then the LLM investigation, then the daily brief, then human review. It is repeatable, easier to audit, better at historical analysis, cheaper in API cost over time, and easier to govern.

The labels matter for governance as much as for setup. An official MCP server is maintained by the vendor and documented publicly. A custom pipeline over an official API is your code, your credentials and your responsibility — which is fine, provided the distinction is recorded rather than assumed.

A minimum viable analyst does not need all six sources. Search Console, GA4 and Core Web Vitals data are enough to answer the four daily questions for most sites; the rest add off-site and behavioral context.

If the analyst only reads analytics data, it cannot tell a demand problem from a ranking problem, a CTR problem or a tracking problem. Search Console fills that gap by showing what happened in Google Search before the visitor ever reached the site, which makes it the primary evidence source whenever organic visibility changes.

Google documents three API surfaces for this work: Search Analytics for performance reporting, the URL Inspection API for the indexing and rich-results state of a single URL, and the Sitemaps API for sitemap submission and status. Treat those references as the authoritative description of what the platform provides.

An analyst that reads only search data can tell you that clicks fell. It cannot tell you whether the visitors who did arrive found what they came for, whether a template change broke a form, or whether a traffic gain reached anything the business cares about. GA4 supplies that half of the story, and only that half: it has no view of impressions, queries or ranking.

GA4 is event-based rather than session-based. Everything it records — a page view, a scroll, a click, a purchase — is an event with parameters, and sessions are derived from those events rather than the other way round. That single fact changes how the analyst should query it: you are asking which interactions occurred and how often, not how long someone stayed on a site.

Everything stated about GA4's behavior, metric definitions, reports and limits in this section comes from Google's own Analytics Help and Data API documentation, linked at the end of the section. The investigation workflows, diagnostic patterns and checklists are Connector Scout editorial guidance and are labeled as such.

What GA4 measures: the event model

GA4 records interactions as events with parameters, and derives sessions from those events. Google documents four event categories: automatically collected events such as page_view, session_start, first_visit and user_engagement; enhanced measurement events such as scroll, outbound click, site search, video engagement and form interactions, which are toggled in the interface rather than coded; recommended events such as purchase, which have documented names and parameters but are not sent automatically; and custom events you define yourself.

Above that sits the concept the brief should actually lead with. Google documents a key event as an event that measures an action particularly important to the success of the business, and states that any event you collect can become a key event: you create or identify the event, then mark it as key. Google also documents that historical conversion data remains available as key-event data, so year-over-year comparisons still work for events that were already being collected. What does not exist is data for an event that was never collected — an event first configured on Tuesday has nothing before Tuesday — and Google Ads conversions shared with Analytics have no historical data associated with them.

GA4 in the evidence layer

GA4 is layer two of the five evidence layers described earlier in this guide: the behavioral layer. It proves what visitors did. It cannot prove why they did it, and it has no view of search visibility at all.

Primary questions the analyst answers with GA4

• Did the visitors who arrived engage, or did they leave immediately?
• Which landing pages absorb traffic well, and which lose it?
• Which pages carry disproportionate business value?
• Did the acquisition mix change, and did quality change with it?
• Where in the journey do users drop off before a key event?
• Did the outcome the business cares about move, or only the traffic?
• Is this a behavior change at all, or a measurement change dressed up as one?

Core reports and the questions they answer

The distinction that matters most here is traffic acquisition versus user acquisition. Google documents the traffic acquisition report as showing where new and returning users come from, and the user acquisition report as showing how new users found the site for the first time. They disagree by design, and an analyst that treats the difference as an error will keep filing false discrepancies.

Google also documents a Search Console integration for GA4 that surfaces two reports — Google Organic Search Queries and Google Organic Search Traffic — once a property is linked. It is a convenience for human review, not a substitute for reading the Search Console API directly, which is where Part 2 of this guide gets its evidence.

Important metrics, and when each one lies

Metric definitions come from Google’s documentation, and each badge shows the Data API metric name from the official GA4 schema — which does not always match the interface wording (the UI’s Average engagement time is derived from the API’s userEngagementDuration, and the API’s current outcome metric is keyEvents). The failure modes are Connector Scout editorial guidance — a metric without a stated failure mode is how a brief ends up claiming a number proves something it cannot.

Documented data-quality constraints

Four documented behaviors produce most of the false alarms we see in automated GA4 reporting. They are separate mechanisms and should not be conflated: processing latency, sampling in explorations and ad-hoc queries, cardinality and the (other) row, and system-defined thresholding. Encode each in the pipeline rather than hoping the model notices.

Data freshness

Google documents that data processing can take 24–48 hours and that data in reports may change during that time. Realtime data arrives within minutes, standard intraday within 2–6 hours, and Analytics 360 has shorter intervals. A daily analyst that reads yesterday and treats it as final will manufacture declines.

Sampling in explorations and ad-hoc queries

Sampling is documented as an explorations and ad-hoc query behavior: Google's configuration limits state 10M events per query for explore sampling and that ad-hoc queries processing beyond 10M events will be sampled, while standard queries are not affected by this limit. Do not describe every Data API report as sampled past 10 million events; the Data API has its own separate quota system.

Cardinality and the (other) row

Google defines high-cardinality dimensions as those with more than 500 unique values in a day and documents that they make a report more likely to hit its row limit, condensing the remainder into a row labeled (other). Page path on a large site is the usual trigger, and stricter cardinality limits can apply while only intraday data is available.

Data thresholds

Google documents system-defined thresholds that withhold data from a report, exploration or API call — typically where demographic, audience or search-query data is involved and there are not enough users. Withheld rows are not zeros, and a narrow date range makes thresholding more likely. The data-quality indicator in the interface reports when it has been applied.

Investigation workflows

Two worked chains, following the same evidence-labeling convention as the mindset chapter: each link is a verified fact, a calculated metric, a hypothesis or a recommended check.

Common diagnostic patterns

Recurring shapes in the data, the explanation each usually supports, and the check that settles it. This is Connector Scout editorial guidance built on the documented GA4 behaviors above.

Combining GA4 with other evidence

The landing page is the join key. Search Console reports on the page that appeared in search results; GA4 reports on the page a visit started on. Store both against the same URL and a single question becomes answerable in one pass: did fewer people arrive, or did the people who arrived behave differently?

Common mistakes

Best practices

Key takeaway

Ahrefs earns its place in a daily analyst for one reason: some declines have no first-party explanation. Impressions fall, engagement is unchanged, nothing shipped — and the only remaining explanation lives outside the site, in links that disappeared or competitors that moved. Ahrefs is where that evidence is found, and it is the only layer in this guide that is not measuring your own users.

That also makes it the layer most easily over-read. Ahrefs backlink records come from Ahrefs' own crawl, and its traffic and traffic-value figures are estimates that Ahrefs models from keyword rankings — neither is Google data and neither can override Search Console. This chapter treats counts of crawled links as observations, and everything traffic-shaped as an estimate that must be labeled as one.

Every product, endpoint, plan and pricing claim below comes from Ahrefs' own documentation, linked at the end of the section. The daily-versus-weekly split, the interpretation rules, the diagnostic patterns and the confidence caps are Connector Scout editorial guidance and are labeled as such.

What Ahrefs contributes to the evidence model

Ahrefs sits in the off-site layer: third-party crawl records of links and modelled estimates of organic performance. It corroborates first-party findings and explains declines that have no on-site cause. It never measures your search performance or your users, so it cannot overrule Search Console or GA4.

MCP for investigation, API v3 for schedules

Connecting, plans and API units

Connecting a supported AI client to the hosted MCP server presents an authorization consent screen and creates a dedicated API key tagged with MCP scope; workspace admins can set a monthly API-unit limit for each key. The recommended Streamable HTTP endpoint is https://api.ahrefs.com/mcp/mcp, and the SSE endpoint https://api.ahrefs.com/mcp/mcpSse is legacy.

Endpoints worth wiring

Reading Ahrefs metrics

Data quality and interpretation limits

Worked evidence chains

Common diagnostic patterns

Semrush is the second third-party layer in this guide, and the temptation to treat it as a first-party one is stronger here than anywhere else, because Position Tracking looks like your rankings and Traffic & Market looks like traffic. Neither is. Position Tracking is a measurement of the keywords, search engine, location and device you configured in a Semrush campaign, and Traffic & Market is modelled market data. Search Console and GA4 remain the arbiters of your own performance.

What Semrush genuinely adds to a daily analyst is scoped, repeatable observation of a keyword set you chose, a documented backlink dataset with explicit first- and last-seen fields, keyword demand context in a specific regional database, and crawl-based Site Audit observations from your own campaign. Every one of those is corroboration or context. None of them proves a Google ranking, actual traffic, user intent, content quality or causation.

Every product, endpoint, version, plan, unit and field claim below comes from Semrush's own developer documentation, linked at the end of the section, and where two current official pages disagree this chapter says so rather than choosing silently. The cadence split, the interpretation rules, the diagnostic patterns, the confidence caps and the sequencing are Connector Scout editorial guidance and are labeled as such.

What Semrush contributes beyond Search Console, GA4 and Ahrefs

Semrush earns its place for three things the other layers cannot supply: a repeatable observation of a keyword set you configured in a Position Tracking campaign, including SERP-feature flags for AI features; a documented link dataset with explicit first-seen and last-seen fields; and, for subscribers, modelled Traffic & Market data about a market rather than a site. Everything else it returns overlaps Ahrefs or duplicates first-party data at lower authority.

What changed from older Semrush guidance

MCP versus direct API access

Setup is short. The endpoint is https://mcp.semrush.com/v2/mcp and it accepts streamable HTTP only. OAuth is the default and needs no headers — the client registers itself and redirects you to the Semrush login. Where a client cannot do OAuth, Semrush documents an API-key fallback sent as Authorization: Apikey YOUR_API_KEY. Keys are version-specific: v4 keys can be created with their own permissions and TTL, but they only reach Backlinks v4, Keyword Reports v4, Projects v4 and Listing Management. Domain, Overview, Subdomain, Subfolder, URL, Position Tracking and Site Audit pulls still need the single account-wide v3 key, which cannot be scoped or revoked — so separate credentials per version where you can, and keep the v3 key in managed secret storage with tight access and a re-issue plan for the integrations that depend on it.

Datasets worth wiring

API units, cost control and failure handling

Reading Semrush datasets

Data quality and interpretation limits

Worked evidence chains

Common diagnostic patterns

Clarity is the last layer to open and the easiest to over-read. It can show collected interactions, aggregate behavior patterns, heatmaps and retained recordings. It cannot prove search ranking, user intent, content quality, user motivation or causation, and nothing it returns can override Search Console or GA4 for your own performance.

That sequencing is deliberate. Clarity should usually follow Search Console and GA4 in an investigation rather than originate an SEO claim: it observes behavior after arrival, so it explains what happened to visitors who already got there, and its most valuable output is often a negative one — behavior did not change, so look elsewhere.

Every product capability, endpoint, limit, retention period and field name below comes from Microsoft's own Clarity documentation and the official Microsoft MCP repository, linked at the end of the section, and where two current official pages disagree this chapter says so rather than choosing silently. The diagnostic patterns, confidence scores, recommended cadence, dimension choices, evidence chains, token-security advice and the MCP-versus-API split are Connector Scout editorial guidance and are labeled as such. Nothing here is legal advice: consent and privacy implementation must be validated with your own privacy and legal owners.

What Clarity contributes — and what it cannot tell you

Clarity is the behavioral-observation layer for pages that first-party search and analytics evidence has already flagged. Microsoft documents it as collecting interactions, aggregating them into dashboard metrics, rendering heatmaps of clicks and scroll reach, and retaining a limited set of session recordings reconstructed from DOM and interaction data. Those four things are what it contributes.

What it cannot do matters just as much. Clarity has no view of queries, impressions or ranking, no measure of intent or content quality, and no mechanism for establishing causation. A behavioral signal is an indicator that something is worth reproducing — never a finding on its own.

What changed from older Clarity guidance

MCP versus direct API access

Setup for the MCP server is local. Microsoft publishes it as the npm package @microsoft/clarity-mcp-server, run with npx @microsoft/clarity-mcp-server --clarity_api_token=your-token-here or from a global install, with Node.js 16 or later and a Clarity project Data Export API token as the prerequisites. The token can also be supplied as a tool parameter. There is no Microsoft-hosted Clarity MCP endpoint and no OAuth flow. The direct route is a single documented request: GET https://www.clarity.ms/export-data/api/v1/project-live-insights, authenticated with Authorization: Bearer <token>, where only project admins can manage tokens.

Data surfaces and cadence

Limits, timezones, row caps and failure handling

Privacy, masking and consent

Microsoft’s client API documentation classifies input-box content, numbers and email addresses as sensitive and masked by default, and masked content is not uploaded. Masking can be expanded, and unmasking is an explicit choice. That is a strong default, but it is not a claim that privacy is automatic or complete — what is collected still depends on how your implementation configures masking.

Consent is enforced. Since 31 October 2025 Microsoft enforces consent signals for page visits from the EEA, the UK and Switzerland before full Clarity functionality is available. consentv2 is the current recommended method and the older consent API is planned for deprecation. Without granted consent, Clarity runs in no-consent mode with no first- or third-party cookies: the documentation says it assigns a unique ID per pageview and cannot persist the cross-page journey, which fragments sessions and affects funnels and recordings.

Reading Clarity metrics and signals

Data quality and interpretation limits

Worked evidence chains

Common diagnostic patterns

The current Core Web Vitals are Largest Contentful Paint for loading, Interaction to Next Paint for responsiveness and Cumulative Layout Shift for visual stability. web.dev documents the good thresholds as 2.5 seconds or less for LCP, 200 milliseconds or less for INP and 0.1 or less for CLS, evaluated at the 75th percentile of page loads and segmented across mobile and desktop devices.

What this layer cannot do is as important as what it can. Core Web Vitals are real-user performance measurements: they do not measure search intent, content quality, business impact, or the cause of a regression, and they cannot establish that a performance change caused a ranking change. Google states that Core Web Vitals are used by its ranking systems and recommends good scores, while also saying that good tool results do not guarantee top rankings, that relevance remains primary, and that there is no single page-experience signal.

Field and lab evidence stay in separate namespaces throughout this chapter. Field data is aggregated over a rolling collection period across many real users; a lab run is one synthetic load in one configured environment. Lighthouse cannot measure INP at all without real user input, and Total Blocking Time is a lab diagnostic for responsiveness rather than a substitute for INP.

Every capability, endpoint, field name, threshold, aggregation rule, update cadence and limitation below comes from Google's and Chrome's own documentation, linked at the end of the section. The cadence, the multiple-run median rule, the prioritization advice, the diagnostic patterns, the evidence chains and the confidence caps are Connector Scout editorial guidance and are labeled as such.

What Core Web Vitals contribute — and what they cannot tell you

Core Web Vitals are the performance-evidence layer. web.dev documents three metrics: Largest Contentful Paint for loading, Interaction to Next Paint for responsiveness and Cumulative Layout Shift for visual stability, with good defined as 2.5 seconds or less, 200 milliseconds or less and 0.1 or less respectively, assessed at the 75th percentile of page loads and segmented across mobile and desktop.

They measure real-user performance and nothing else. They do not measure intent, content quality or business impact, they cannot identify a cause on their own, and they cannot establish that a performance change moved rankings. Google states that Core Web Vitals are used by its ranking systems and recommends good scores, while also saying that good tool results do not guarantee top rankings, that relevance remains primary, and that there is no single page-experience signal.

What changed from older performance guidance

Field versus lab: two different questions

Storing that separation is what keeps it true. The record below is an illustrative Connector Scout design, not a Google schema: field and lab are sibling namespaces that are never merged or averaged, each field value carries its exact collection window, form factor and granularity, each lab value carries its Lighthouse version and environment, and unavailable data is stored as its own state rather than as a zero.

1{2  "recordId": "cwv:example.com/pricing:2026-07-28",3  "note": "Illustrative Connector Scout record design, not a Google schema. FIELD and LAB are sibling namespaces and are never merged or averaged.",4  "page": {5    "requestedUrl": "https://example.com/pricing",6    "template": "pricing"7  },8  "field": {9    "source": "crux",10    "api": "chromeuxreport.googleapis.com/v1/records:queryRecord",11    "availability": "available",12    "scope": "url",13    "requestedUrl": "https://example.com/pricing",14    "normalizedUrl": "https://example.com/pricing",15    "urlNormalizationDetails": {16      "originalUrl": "https://example.com/pricing?ref=nav",

Documented access routes

The daily field request is a single documented call: POST https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=YOUR_API_KEY with either a url or an origin in the JSON body and an optional form factor. The trend request uses the same body against the History API plus an optional collectionPeriodCount between 1 and 40, which defaults to 25.

Percentiles, collection periods, eligibility and limits

Reading the metrics

Worked evidence chains

Common diagnostic patterns

Search Console organises rows around search dimensions such as date, query, page, country and device. GA4 reports combinations of dimensions and metrics. Clarity contributes aggregated behavioral signals. Ahrefs and Semrush provide vendor-defined search and link estimates. CrUX reports field performance over a rolling collection period.

If raw responses go straight to an AI model, two failures follow almost immediately: identical concepts arrive under different names, and incompatible periods or grains get compared as though they were equivalent. Normalization removes both failure modes by giving every record the same envelope, the same identifiers and the same explicit statement of what it covers.

The instinct to force all providers into one flat table is the mistake to avoid. Instead, normalize in four layers: a shared record envelope carrying provenance, identity, period and quality; source-specific dimensions and metrics kept in their own namespaces; a canonical page-identity table used for page joins; and comparison records calculated after ingestion rather than inferred at read time. Records must not be joined until their grains are compatible.

The schema, naming rules, baselines and validation steps in this chapter are Connector Scout implementation recommendations, not provider requirements. The behavior of each source — its row shapes, time models and documented limits — comes from the primary documentation linked at the end of the chapter.

Preserve each record’s grain

A record’s grain is the exact level one row represents. Write it down before anything else, because every join, comparison and aggregation downstream depends on it.

• Search Console: property × date × page × query × country × device.
• Google Analytics 4: property × date × landing page × device category.
• Microsoft Clarity: project × requested date range × URL.
• Ahrefs and Semrush domain metrics: domain × observation date.
• CrUX: URL or origin × form factor × collection period.

A Search Console page-and-query row must be aggregated to page level before it is joined to a page-level GA4 row. Skip that step and GA4 metrics are duplicated once per query. Storing the grain on the record makes the requirement explicit: "grain": ["date", "page", "query", "country", "device"].

Use a shared record envelope

The envelope answers the provenance questions an analyst has to be able to ask of any number: where did this come from, what does it identify, what period does it describe, how complete is it, and when was it retrieved? Those answers live in the same place for every source. Only the dimensions, metrics, entity_key and quality contents vary by source.

1{2  "schema_version": "1.0.0",3  "record_id": "gsc:sc-domain:example.com:2026-07-27:sha256…",4  "source": "google_search_console",5  "source_report": "search_analytics",6  "property_id": "sc-domain:example.com",7  "entity_type": "page_query",8  "entity_key": {9    "page": "https://example.com/guides/ai-seo/",10    "query": "ai seo workflow"11  },12  "grain": ["date", "page", "query", "country", "device"],13  "period": {14    "start": "2026-07-27",15    "end": "2026-07-27",16    "timezone": "America/Chicago",

The example above is an illustrative Connector Scout design, not a provider schema. Note the quality block: Google’s Search Analytics documentation states that the API does not guarantee every matching row, that responses may be limited to top rows, and that dates with no data are omitted when results are grouped by date. Your records therefore have to distinguish a reported zero from an absent row.

Standardize identifiers and names

Pair the naming rules with a metric dictionary, so every normalized metric records the provider field it came from, its type and unit, and how it must be aggregated.

1{2  "engagement_rate": {3    "source": "google_analytics_4",4    "source_field": "engagementRate",5    "type": "ratio",6    "unit": "decimal",7    "aggregation": "weighted_recalculation",8    "definition_owner": "provider",9    "notes": "Recalculate from engaged_sessions / sessions when aggregating; never average across rows."10  }11}

Build a canonical page-identity table

Page-level joins should reference a page identifier, never a raw URL string. One row per page holds the canonical URL, the variants observed in the wild, and the grouping attributes your analysis needs.

1{2  "page_id": "pg_7f3c1a",3  "canonical_url": "https://example.com/guides/ai-seo/",4  "observed_urls": [5    "https://example.com/guides/ai-seo/",6    "https://example.com/guides/ai-seo",7    "https://www.example.com/guides/ai-seo/?utm_source=newsletter"8  ],9  "path": "/guides/ai-seo/",10  "host": "example.com",11  "page_group": "guides",12  "locale": "en-US",13  "status": "active"14}

1. Parse the URL rather than editing it as a string.
2. Lowercase the scheme and hostname only.
3. Remove the fragment.
4. Remove only the tracking parameters on your documented list.
5. Apply your own documented host and trailing-slash rules consistently.
6. Resolve redirects and canonical relationships through a separate mapping step, not inside the parser.
7. Store both the observed URL and the normalized URL on every record.

Separate event dates, observation dates and retrieval times

Chrome documents the CrUX API as returning a rolling 28-day collection period, so a CrUX value normally represents that window rather than a measurement taken on the retrieval date, and a new page may have fewer underlying days inside it. A rolling 28-day CrUX value and a single-day GA4 value are not equivalent quantities and must not be compared as though they were.

Retain raw data and provenance

Keep three distinct stages, each traceable to the one before it: the raw response exactly as received, the normalized record derived from it, and the calculated finding derived from the normalized record. Retain the following with every record.

• Source and endpoint or report name.
• Request parameters, with credentials and authorization headers excluded.
• Retrieval timestamp in UTC.
• A hash of the raw payload, so a stored record can be tied to the exact response it came from.
• Schema version and transformation version.
• Pagination state, including whether all pages were retrieved.
• Filters, segments and sampling or thresholding notes applied to the request.
• Any API warnings and the quality flags derived from them.

Normalize values without changing their meaning

• Parse numeric strings into numbers; never store a formatted string as a metric.
• Store ratios as decimals between 0 and 1.
• Store durations in the unit you declare in the metric dictionary, consistently.
• Store timestamps in UTC while retaining the report time zone on the record.
• Store unavailable values as null, never as zero.
• Preserve an explicit zero the source actually reported as a zero.
• Keep rankings and positions numeric, and record that lower is better.
• Label provider estimates as estimates, attributed to the provider that produced them.
• Reject NaN and infinity at validation rather than storing them.
• Store currency values with their ISO currency code.
• Retain the original value whenever a conversion would be lossy.
• Never sum CTR, engagement rate or average position; recalculate CTR as total clicks divided by total impressions, and engagement rate as engaged sessions divided by sessions.

Create explicit comparison windows

Comparisons are records the pipeline calculates after ingestion, not something the model works out while writing the brief. Use complete periods only.

1{2  "metric": "clicks",3  "current_value": 842,4  "comparison_value": 1018,5  "absolute_change": -176,6  "percent_change": -0.1729,7  "baseline_method": "same_weekday_median",8  "baseline_observations": 89}

When the comparison value is zero or missing, store percent_change as null with a stated reason rather than emitting an infinite or misleading percentage.

Record data-quality conditions

1{2  "status": "partial",3  "is_partial": true,4  "is_sampled": false,5  "is_thresholded": false,6  "is_top_rows_limited": true,7  "issues": [8    {9      "code": "SOURCE_ROW_LIMIT",10      "severity": "warning",11      "message": "The source may not have returned every matching row."12    }13  ]14}

Two source-specific conditions are worth encoding directly. Microsoft Clarity’s Data Export API returns aggregated dashboard data, so those records are not universal event rows and must not be treated as GA4-style events. Semrush reports can be monthly or daily depending on the report, so retain the database, device and display-date semantics on every Semrush record.

Validate before analysis

Quarantine invalid records and log every validation failure. Malformed values must never be silently coerced to zero.

Minimum viable normalized layer

1. Immutable raw responses, stored exactly as received.
2. A shared record envelope for provenance, identity, period and quality.
3. Source-specific transformation mappings, one per report you consume.
4. A canonical page-identity table used for every page-level join.
5. A validation report produced on every run.

The schemas below are Connector Scout implementation recommendations, not provider requirements. What each source can supply — its grain, fields, limits and freshness — comes from the primary documentation consolidated at the end of the guide.

1{2  "finding_id": "2026-08-03-product-mobile-ctr",3  "status": "verified_change",4  "title": "Mobile CTR declined on the primary product page",5  "scope": {6    "page_id": "product-main",7    "canonical_url": "https://www.example.com/product/",8    "country": "USA",9    "device": "MOBILE",10    "search_type": "web"11  },12  "comparison": {

1{2  "run_id": "daily-seo-2026-08-03T12:00:00Z",3  "site": "example.com",4  "timezone": "America/Chicago",5  "started_at": "2026-08-03T12:00:00Z",6  "completed_at": "2026-08-03T12:07:42Z",7  "latest_complete_date": "2026-07-31",8  "status": "completed_with_warning",9  "sources": [10    {"name": "google_search_console", "status": "complete", "rows": 18240},11    {"name": "google_analytics_4", "status": "complete", "rows": 824},12    {"name": "ahrefs", "status": "complete", "rows": 250},

Use an absolute rule with every relative rule. A rise from one click to two clicks is 100%, but it is not a material business event.

Do not use the incomplete current day in primary alert logic. Google says Search Console performance data is typically available after two to three days and exposes metadata for incomplete recent data. Select the latest complete date from the source rather than assuming yesterday is complete.

1# Connector Scout starting recommendations — calibrate before automating.2thresholds:3  minimum_volume:4    gsc_impressions: 2005    gsc_clicks: 256    ga4_organic_sessions: 507    key_events_long_baseline: 108    tracked_keywords: 59    clarity_sessions: 301011  material_change:12    gsc_clicks_relative: 0.2513    gsc_impressions_relative: 0.3014    gsc_ctr_relative: 0.2515    gsc_ctr_absolute_points: 1.016    average_position_absolute: 2.017    ga4_organic_sessions_relative: 0.2518    organic_key_events_relative: 0.301920  rules:21    require_complete_periods: true22    require_absolute_and_relative_tests: true23    require_second_source_for_high_confidence: true24    suppress_incomplete_current_day: true25    maximum_daily_findings: 5

priority_score =  magnitude  × volume_confidence  × business_priority  × corroboration  × urgency