Docs / Analytics

First-Party Analytics (AI-Signal-First)

AI OS ships a built-in, first-party analytics panel with a deliberate bias: the AI signals answer engines generate come first, traditional web metrics second. The core insight is structural — AI crawlers do not execute JavaScript, so tag-based tools like Google Analytics cannot see them. AI OS reads your origin server's access logs instead, where that traffic actually exists. No tags, no cookies, no third-party services.

What it tracks

AI crawl signals — the layer tag-based tools miss

  • Live activity feed — "PerplexityBot fetched /pricing · 12s ago", streamed to the dashboard as it happens
  • Engine leaderboard — hits per engine, classified by purpose: training crawls (GPTBot, ClaudeBot, CCBot, Bytespider, Amazonbot), search-index crawls (OAI-SearchBot, PerplexityBot, Bingbot, DuckAssistBot), and live user-triggered fetches (ChatGPT-User, Claude-User, Perplexity-User — a human is asking an AI about your site right now)
  • Crawl heat — which pages each engine fetches most: the engines telling you which content they consider answer-worthy
  • Bot hygiene — robots.txt and llms.txt fetches, plus 404s served to AI crawlers (each one a missed citation opportunity)

Bot identity is self-reported by user agent, and the panel labels it accordingly ("reported identity"); IP-range verification against the engines' published address ranges is on the roadmap.

Answer-engine referrals

Human visitors arriving from ChatGPT, Perplexity, Gemini, and Copilot are classified separately from classic search, social, and direct traffic — the conversion side of AEO. Crawl signals show engines reading you; referral signals show engines sending you buyers.

The traditional baseline

Pageviews, unique-visitor estimates, top pages, and referrer classes — enough to sanity-check any other tool's numbers. Human counts are log-derived and strictly filtered: assets, health checks, vulnerability scanners, and non-AI crawlers are excluded, so a "pageview" here means a browser fetched a page.

Per-site attribution

A vhost-aware log format records which domain served every request, so each hosted site's traffic lands in its own bucket. The dashboard's Analytics view has a site selector — operators see the platform plus every site; a managed client sees only their own sites, enforced server-side by the same ownership checks that scope their sites, audits, and leads. Each site's Site Manager tab shows the same numbers next to its lead inbox.

Privacy posture

  • No cookies, no localStorage, no fingerprinting scripts — there is no client-side tag at all
  • Unlinkable visitor estimates — uniques use a hash of IP + user agent salted with a value that rotates every 24 hours; identities cannot be joined across days by construction
  • First-party only — data never leaves your server; there is no vendor to phone home to
  • Bounded retention — raw events are pruned after 30 days; long-term charts read from small daily rollups

Setup

StepDetail
Log accessThe platform process needs read access to the nginx access log — typically usermod -aG adm <app-user>, then restart the PM2 daemon (group changes apply at daemon start)
Log formatInstall deploy/aios-logformat.conf to /etc/nginx/conf.d/ and use access_log ... aios_vhost; in each server block — this adds the serving hostname for per-site attribution. Plain combined-format lines keep working (they land in the platform bucket)
Env overrideANALYTICS_ACCESS_LOG points the ingester at a non-default log path; when the file is absent the ingester idles harmlessly

Ingestion is incremental (byte-offset tracked, restart-safe, logrotate-aware) and the first run backfills whatever history the current log file holds.

API Endpoints

GET /api/analytics/summary?days=7&site=<siteId>

Operator summary: bot hits, pageviews, series, referrer classes. site defaults to the platform itself.

GET /api/analytics/ai-crawlers?days=7&site=<siteId>

Engine leaderboard (by purpose) + the recent AI-bot event feed.

GET /api/web-studio/sites/:id/analytics?days=30

Owner-scoped combined payload for one hosted site (summary + crawlers + crawl heat) — available to the site's managed client.

Frequently asked questions

Why can't Google Analytics see this traffic?

GA measures by running JavaScript in the visitor's browser. AI crawlers don't execute JavaScript, so they never appear — the only record of their visits is the origin server's access log, which is what AI OS reads.

Do I need a cookie consent banner for this?

There are no cookies and no third-party data flows; visitor estimates are daily-rotating hashes that can't be linked across days. It's designed for consent-free deployment — a design goal, not legal advice.

No AI crawlers showing up at all?

Check robots.txt (the AEO audit includes an AI-crawler allowlist check) and your CDN's bot settings — Cloudflare's "block AI scrapers" toggle stops these crawlers before they reach your origin.