Security — Data Minimization (Field Allowlist)
Copy the prompt and paste it into Cursor, Claude Code, Codex, Copilot… — it does the install for you.
This is the explicit, authoritative list of every field Maya ingests. Anything not on this list is not ingested. The list is designed to be small, machine-checkable, and copy-pasteable into a filter query.
Allowlisted fields
| Field | Type | Required | Example | Why it's needed |
|---|---|---|---|---|
timestamp | ISO 8601 | Yes | 2026-04-22T13:15:42Z | Time-series analysis |
user_agent | string | Yes | Mozilla/5.0 (compatible; GPTBot/1.2; +https://openai.com/gptbot) | Bot classification |
request_path | string | Yes | /products/super-widget | Which page was crawled |
request_method | string | Yes | GET | Validate it was a read |
status_code | integer | Yes | 200 | Crawl error detection |
response_bytes | integer | Yes | 48329 | Bandwidth and content delivery |
referrer | string | No | https://chatgpt.com/ | Attribution (only when present, server-side) |
host | string | Yes | www.example.com | Multi-domain support |
protocol | string | No | HTTP/2 | Request shape |
client_ip_hash | string | No | a4c6... | Hashed at source for unique-bot counting |
All other fields are dropped at source. The filter query produces only the fields above. Anything else in the raw log is never transmitted.
Verified LLM bot allowlist
The user_agent field is filtered against this allowlist. Patterns are case-insensitive substring matches.
GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-Web
anthropic-ai
PerplexityBot
Perplexity-User
Google-Extended
Googlebot
Bingbot
Applebot-Extended
Bytespider
CCBot
Meta-ExternalAgent
FacebookBot
DuckAssistBot
YouBot
Amazonbot
DiffbotThis list is versioned. Maya publishes updates monthly. The brand controls when to adopt updates.
Note:
GooglebotandBingbotare included because Gemini and Copilot use grounded search through these crawlers. The brand may choose to exclude them if non-LLM search traffic should be scoped out.
Forbidden fields (never produced by the filter)
The filter query MUST NOT include any of the following:
- Raw IP address (only
client_ip_hashmay be transmitted, and only if the brand opts in) Cookieheader (any cookies)AuthorizationheaderSet-Cookieresponse header- POST/PUT request body
- Query string parameters that may contain PII (the filter strips
email,tckn,phone,customer_id,account_id,session,token,key,secretand any parameter the brand adds to its denylist) - Form data
- Any header containing
csrf,session,token,auth - TCKN, customer numbers, account numbers (in any field)
Minimization in practice
The brand-side filter applies, in order:
- Pre-filter — drop traffic outside the user-agent allowlist (most rows discarded here).
- Field projection — produce only the allowlisted fields above.
- Sensitive-key strip — for
request_path, strip query-string keys on the denylist. - Hashing — if
client_ip_hashis enabled, hash with HMAC-SHA256 using a per-tenant secret held by the brand.
Reference implementations:
- IIS / PowerShell
- Nginx (coming soon)
- Apache (coming soon)
- Custom BFF endpoint (coming soon)
Validating the filter
Before deploying the filter to production, the brand should:
- Run it in
--dry-runmode for a 24h sample. - Diff the output schema against the allowlist above. There must be no extra fields.
- Spot-check 100 random rows to confirm no PII or denylisted content slipped through.
- Sign off the query in your change management system.
Maya can assist with the validation but does not need access to the raw log to do so.