API — Log Ingestion

Audience: Engineering, AI agents performing integrationUpdated 2026-04-28

API — Log Ingestion

POST https://ingest.withmaya.ai/v1/logs

Receives a batch of filtered LLM bot log records from a brand's source-side filter.

Authentication

Bearer token in the Authorization header. Tokens are issued per tenant. Rotate every 90 days; Maya supports two active tokens during rotation.

Plaintext

Authorization: Bearer mya_pk_live_<random>

Required headers

Header	Value
`Authorization`	`Bearer <token>`
`Content-Type`	`application/x-ndjson`
`X-Maya-Tenant`	Tenant slug (e.g., `acme-prod`)
`X-Maya-Schema`	Schema version (current: `1`)

Optional headers

Header	Value	Purpose
`X-Maya-Mode`	`test`	Validate without ingesting. Useful for filter validation.
`X-Maya-Batch-Id`	Caller-supplied UUID	For idempotency; duplicate batch IDs are deduplicated.
`Content-Encoding`	`gzip`	Compressed bodies accepted.

Request body

NDJSON. One record per line. Each record must conform to the allowlist schema:

JSON

{"timestamp":"2026-04-22T13:15:42Z","user_agent":"Mozilla/5.0 (compatible; GPTBot/1.2; +https://openai.com/gptbot)","request_method":"GET","request_path":"/products/super-widget","status_code":200,"response_bytes":48329,"referrer":"","host":"www.example.com","client_ip_hash":"a4c6f1b2..."}
{"timestamp":"2026-04-22T13:16:01Z","user_agent":"PerplexityBot/1.0","request_method":"GET","request_path":"/pricing","status_code":200,"response_bytes":62104,"referrer":"https://www.perplexity.ai/","host":"www.example.com","client_ip_hash":"7d9e22aa..."}

Try the API with a real sample: download maya-bot-logs.sample.ndjson (15 rows, ~5 KB) and POST it with X-Maya-Mode: test to validate the schema end-to-end before integrating your filter.
Bash
curl -X POST https://ingest.withmaya.ai/v1/logs \
  -H "Authorization: Bearer $MAYA_API_KEY" \
  -H "Content-Type: application/x-ndjson" \
  -H "X-Maya-Tenant: acme-prod" \
  -H "X-Maya-Schema: 1" \
  -H "X-Maya-Mode: test" \
  --data-binary @maya-bot-logs.sample.ndjson

Limits

Limit	Value
Max body size	50 MB compressed (≈250 MB uncompressed)
Max records per batch	500,000
Rate limit	60 batches/hour per tenant
Burst	10 requests in any 10-second window

For larger volumes, split into multiple batches or contact Maya for an increased limit.

Responses

202 Accepted

JSON

{
  "batch_id": "01J7M3Q9V8X2KH4F5E7G3M1B2Y",
  "received": 12483,
  "accepted": 12483,
  "rejected": 0
}

207 Multi-Status

Some records were rejected (e.g., contained fields outside the allowlist). The body lists each rejection with line number and reason.

JSON

{
  "batch_id": "01J7M3Q9V8X2KH4F5E7G3M1B2Y",
  "received": 12483,
  "accepted": 12480,
  "rejected": 3,
  "rejections": [
    {"line": 47,  "reason": "field_not_allowed", "field": "cookie"},
    {"line": 102, "reason": "user_agent_not_in_allowlist"},
    {"line": 1209, "reason": "missing_required_field", "field": "timestamp"}
  ]
}

400 Bad Request

Malformed payload. The body identifies the first parse failure.

401 Unauthorized

Missing or invalid token.

413 Payload Too Large

Body exceeds limits. Split the batch.

429 Too Many Requests

Rate limit exceeded. The Retry-After header provides a backoff hint.

5xx Server Error

Transient. Retry with exponential backoff. Maya's ingestion endpoint is idempotent on X-Maya-Batch-Id — replaying the same batch ID will not double-count.

Validation rules (server-side)

The endpoint validates each record. A record is rejected (and counted in rejections) if any of the following are true:

A field outside the allowlist is present.
A required field is missing.
user_agent does not match the LLM bot allowlist.
timestamp is older than 90 days or in the future.
request_path exceeds 8 KB.
request_path contains a query-string parameter on the denylist.

Rejection of individual records does not fail the batch. The 207 response identifies which records were dropped, so you can correct your filter.

Idempotency

If X-Maya-Batch-Id is supplied, the endpoint deduplicates. A duplicate ID returns the original 202/207 response without ingesting again. This makes safe retries trivial: the same batch can be replayed indefinitely without skewing counts.

Compression

Content-Encoding: gzip is supported and recommended. Typical compression ratio for filtered NDJSON is 8:1 to 12:1.

Example — PowerShell

PowerShell

$apiKey = $env:MAYA_API_KEY
$batchId = [guid]::NewGuid().ToString('N')
$body = Get-Content 'D:\maya\export-2026-04-22.ndjson' -Raw
 
$response = Invoke-WebRequest `
    -Uri 'https://ingest.withmaya.ai/v1/logs' `
    -Method Post `
    -Body $body `
    -ContentType 'application/x-ndjson' `
    -Headers @{
        'Authorization' = "Bearer $apiKey"
        'X-Maya-Tenant' = 'acme-prod'
        'X-Maya-Schema' = '1'
        'X-Maya-Batch-Id' = $batchId
    }
 
$response.Content | ConvertFrom-Json

Example — curl

Bash

curl -X POST https://ingest.withmaya.ai/v1/logs \
  -H "Authorization: Bearer $MAYA_API_KEY" \
  -H "Content-Type: application/x-ndjson" \
  -H "X-Maya-Tenant: acme-prod" \
  -H "X-Maya-Schema: 1" \
  -H "X-Maya-Batch-Id: $(uuidgen)" \
  --data-binary @export-2026-04-22.ndjson

Example — Node.js

TypeScript

import { readFile } from "node:fs/promises";
import { randomUUID } from "node:crypto";
 
const body = await readFile("./export-2026-04-22.ndjson");
 
const res = await fetch("https://ingest.withmaya.ai/v1/logs", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.MAYA_API_KEY}`,
    "Content-Type": "application/x-ndjson",
    "X-Maya-Tenant": "acme-prod",
    "X-Maya-Schema": "1",
    "X-Maya-Batch-Id": randomUUID(),
  },
  body,
});
 
if (!res.ok) {
  throw new Error(`Maya ingestion ${res.status}: ${await res.text()}`);
}
 
console.log(await res.json());

Example — Python

Python

import os
import uuid
import requests
 
with open("export-2026-04-22.ndjson", "rb") as f:
    body = f.read()
 
res = requests.post(
    "https://ingest.withmaya.ai/v1/logs",
    headers={
        "Authorization": f"Bearer {os.environ['MAYA_API_KEY']}",
        "Content-Type": "application/x-ndjson",
        "X-Maya-Tenant": "acme-prod",
        "X-Maya-Schema": "1",
        "X-Maya-Batch-Id": str(uuid.uuid4()),
    },
    data=body,
    timeout=60,
)
res.raise_for_status()
print(res.json())

Test mode

Set X-Maya-Mode: test to validate without ingesting. The endpoint runs the full validation pipeline and returns the same 202/207 response, but does not persist any record. This is the recommended way to validate a new filter before going to production.

PreviousSharePoint