# [](#log-export--overview)Log Export — Overview

## [](#what-it-is)What it is

A periodic, source-side filtered export of LLM bot traffic from your origin server logs to Maya.

## [](#why-it-exists)Why it exists

LLM bots interact with your site through standard HTTP requests. Every request leaves a trace in your access logs. By extracting **only the bot traffic** and **only the allowlisted fields**, you can give Maya enough signal to produce dashboards and insights without sending any user data, any PII, or any business content.

## [](#core-principle)Core principle

> The brand applies the filter. Maya never receives raw logs.

Maya provides reference filter implementations. The brand reviews, may modify, deploys, and audits. Maya's ingestion endpoint validates the schema and rejects records containing fields outside the allowlist.

## [](#what-gets-transmitted)What gets transmitted

Only:

*   Timestamp
*   User-Agent (matched against the LLM bot allowlist)
*   Request path (with query-string keys on the denylist stripped)
*   Request method
*   Status code
*   Response bytes
*   Referrer (if present, server-side)
*   Host
*   Optional: hashed IP (per-tenant secret)

That is the complete list. See [Data Minimization](../../security/data-minimization.md) for the authoritative schema.

## [](#integration-modes)Integration modes

There are two ways to get bot traffic to Maya, depending on where your logs live. Both enforce the same allowlist and the same core principle — Maya never receives raw logs.

### [](#-connect-ready--edge--cdn-providers)🟢 Connect Ready — edge / CDN providers

For these platforms Maya connects to the provider's **native log export**. There is no server-side script to deploy: you enable the export, scope it to the allowlisted fields, and point it at your Maya tenant destination. Field filtering happens inside the provider's own log pipeline, at the edge.

Provider

Native export

Status

**Cloudflare**

Logpush (filtered job)

🟢 Connect Ready

**AWS (CloudFront / ALB)**

Access logs → S3

🟢 Connect Ready

**Akamai**

DataStream 2

🟢 Connect Ready

**How to connect** — all three follow the same shape:

1.  Create a log export / stream that emits **only** the [allowlisted fields](../../security/data-minimization.md).
2.  Add a User-Agent filter for the LLM bot allowlist so non-bot traffic is dropped at the edge.
3.  Strip denylisted query-string keys in the export template.
4.  Deliver to your Maya tenant destination — an S3 / R2 bucket (Cloudflare Logpush, AWS access logs, Akamai DataStream all support object-store delivery) or the Logpush/DataStream HTTPS endpoint.
5.  Run 24h in dry-run, spot-check 100 rows, then enable.

Because the fields are identical across providers, dashboards and downstream analysis are the same regardless of which edge you connect.

### [](#️-self-managed-filter--origin-servers--custom-stacks)🛠️ Self-managed filter — origin servers & custom stacks

Origin web servers have no native bot-log export, so you deploy Maya's **reference filter** next to your log rotation. The brand owns, reviews, and audits the script; Maya only ever receives the filtered output.

Stack

Guide

Status

**IIS / Windows Server**

[`iis.md`](./iis.md)

✅ Available

**Nginx**

[`nginx.md`](./nginx.md)

✅ Available

**Apache**

[`apache.md`](./apache.md)

✅ Available

**Custom BFF endpoint**

[`bff-endpoint.md`](./bff-endpoint.md)

✅ Available

**How it works** — the same pattern for every stack, differing only in log paths and formats:

1.  A scheduled job (cron / Task Scheduler) reads the rotated access log.
2.  It keeps only requests whose User-Agent matches the LLM bot allowlist.
3.  It projects each row to the allowlisted fields and strips denylisted query keys.
4.  The filtered output is uploaded daily/weekly to Maya, or pulled over mTLS (see [delivery patterns](#choosing-a-delivery-pattern)).

[`iis.md`](./iis.md) is a complete worked example; Nginx, Apache, and custom BFF stacks follow the same four steps with their own log format. Ask Maya for a starter filter for any stack not yet listed.

## [](#choosing-a-delivery-pattern)Choosing a delivery pattern

Pattern

When to use

**Daily batch upload**

Default. Easiest. Smallest operational burden.

**Weekly batch upload**

When the brand prefers larger, less frequent windows for review/audit.

**Endpoint pull (mTLS)**

When the brand wants Maya to fetch from a brand-controlled endpoint over a mutually authenticated TLS tunnel. Recommended in BFF topologies.

Switching patterns is straightforward; the schema is identical, so dashboards and downstream analysis are unaffected.

## [](#validation-expectations)Validation expectations

Before any filter goes to production:

1.  Run the filter in `--dry-run` mode for 24 hours.
2.  Compare output schema to the [allowlist](../../security/data-minimization.md). No extra fields.
3.  Spot-check 100 random rows for PII or denylisted content.
4.  Sign off in the brand's change management system.
5.  Schedule and enable.

Maya assists at every step but does not require access to raw logs to do so.

[PreviousLooker Studio Connector](/docs/integrations/looker-studio)[Next IIS / Windows Server](/docs/integrations/log-export/iis)