# [](#apache-log-export)Apache Log Export

This page documents how to extract LLM bot traffic from Apache HTTP Server access logs and ship it to Maya, with all data minimization rules applied at source.

> **When to use this guide:** your production stack is Apache-hosted (origin or reverse proxy) with access logs on the host or a log-aggregation box. If Apache sits behind a CDN you can connect natively, prefer the [Connect Ready](./overview.md#-connect-ready--edge--cdn-providers) path. If a BFF / API gateway already builds request records, see [`bff-endpoint.md`](./bff-endpoint.md).

The filtering, output schema, transmission, and validation are **identical to the Nginx guide** — only the log-format definition differs. This page gives the Apache-specific config; the filter script is shared.

## [](#prerequisites)Prerequisites

Requirement

Notes

Apache 2.4+

For the `%{Host}i` and modern `LogFormat` directives

A dedicated JSON access log

So the filter parses one record per line

Python 3.8+

For the shared filter script. No third-party packages.

Outbound HTTPS to `ingest.withmaya.ai` on port 443

For transmission

Maya API key + tenant secret

Provided by Maya; stored as env vars / secret manager, never in source

## [](#recommended-a-json-access-log)Recommended: a JSON access log

Define a `LogFormat` that emits exactly the fields Maya needs, matching the JSON keys the filter expects. Add to your vhost or `httpd.conf`:

Apache

```
LogFormat "{ \"time\":\"%{%Y-%m-%dT%H:%M:%S%z}t\", \"method\":\"%m\", \"uri\":\"%U\", \"query\":\"%q\", \"status\":%>s, \"bytes\":%B, \"referer\":\"%{Referer}i\", \"host\":\"%{Host}i\", \"ua\":\"%{User-Agent}i\", \"ip\":\"%a\" }" maya_json
 
# Separate log — leaves your existing logging untouched.
CustomLog "/var/log/apache2/maya_access.log" maya_json
```

Field notes:

*   `%U` is the URL path **without** the query string; `%q` is the query string **with** a leading `?`. The shared filter tolerates the leading `?`, so no change is needed.
*   `%B` is bytes sent excluding headers, and emits `0` (not `-`) for empty responses — safe to parse as an integer.
*   `%a` is the client IP. If Apache is behind a trusted proxy and you want the real client, use `%{c}a` with `mod_remoteip` configured; the IP is hashed either way.
*   The `%{...}t` time token emits an offset like `+0000`; the filter normalizes it to UTC.

Reload Apache (`apachectl configtest && systemctl reload apache2`). As with Nginx, this log still contains all traffic and raw IPs — the filter is what removes non-bot rows and hashes IPs before anything leaves your network.

## [](#filter-script)Filter script

Use the same **`maya_bot_filter.py`** from the [Nginx guide](./nginx.md#filter-script) — it is format-agnostic because both stacks emit the same JSON keys. Run it against the Apache log:

Bash

```
MAYA_TENANT_SECRET=… python3 maya_bot_filter.py \
  --log  /var/log/apache2/maya_access.log \
  --since 2026-07-28T00:00:00Z \
  --out  /var/maya/export-2026-07-28.ndjson \
  --dry-run
```

The script keeps only allowlisted LLM bots, strips denylisted query keys, HMAC-hashes the client IP with your tenant secret (Maya never learns it), and writes NDJSON. See the [Nginx guide's script notes](./nginx.md#notes-on-the-script) for the full behavior.

## [](#expected-output-sample)Expected output (sample)

Identical schema to every other stack — one bot request per line:

JSON

```
{"timestamp":"2026-07-28T03:14:07Z","user_agent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)","request_method":"GET","request_path":"/products/super-widget","status_code":200,"response_bytes":48329,"referrer":"","host":"www.example.com","client_ip_hash":"a4c6f1b2e7d8c1bb"}
{"timestamp":"2026-07-28T03:15:11Z","user_agent":"Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)","request_method":"GET","request_path":"/pricing","status_code":200,"response_bytes":62104,"referrer":"https://www.perplexity.ai/","host":"www.example.com","client_ip_hash":"3f8b22cc09111ab7"}
```

> **Download a full sample (15 rows, ~5 KB):** [`maya-bot-logs.sample.ndjson`](/docs/samples/maya-bot-logs.sample.ndjson)

### [](#sanity-checks)Sanity checks

The same three checks as the [Nginx guide](./nginx.md#sanity-checks-on-your-output): valid JSON per line, no disallowed content, every `user_agent` on the allowlist.

## [](#transmission)Transmission

Bash

```
curl -sS -X POST 'https://ingest.withmaya.ai/v1/logs' \
  -H "Authorization: Bearer $MAYA_API_KEY" \
  -H 'X-Maya-Tenant: acme-prod' \
  -H 'X-Maya-Schema: 1' \
  -H 'Content-Type: application/x-ndjson' \
  --data-binary @/var/maya/export-2026-07-28.ndjson \
  -w '%{http_code}\n' -o /dev/null
```

Returns `202 Accepted` on success with a batch ID.

## [](#scheduling)Scheduling

Cron

```
# /etc/cron.d/maya-log-export  — daily at 03:00, prior day's window
0 3 * * * maya MAYA_TENANT_SECRET=@secret MAYA_API_KEY=@secret \
  /usr/bin/python3 /opt/maya/maya_bot_filter.py \
    --log /var/log/apache2/maya_access.log \
    --since "$(date -u -d 'yesterday 00:00' +\%Y-\%m-\%dT00:00:00Z)" \
    --out /var/maya/export-$(date -u +\%Y-\%m-\%d).ndjson \
  && /opt/maya/upload.sh /var/maya/export-$(date -u +\%Y-\%m-\%d).ndjson
```

## [](#validation-checklist)Validation checklist

*    `apachectl configtest` passes after adding the `LogFormat` / `CustomLog`.
*    Run the filter with `--dry-run` on 24 hours of logs; retained count is 0.1%–5% of total rows.
*    Inspect 100 random rows: no raw IPs; no cookies/auth/session; no PII in `request_path`; every `user_agent` on the allowlist.
*    POST a sample batch with `X-Maya-Mode: test` to verify schema acceptance without ingesting.
*    Sign off the config + script in change management.
*    Schedule and enable.

## [](#troubleshooting)Troubleshooting

Symptom

Cause

Fix

Lines are not valid JSON

A logged value contained an unescaped `"`

Rare with these fields; if it happens, switch the `LogFormat` to pipe through a JSON logger, or escape with `%{User-Agent}i` already quoted as shown.

`request_path` missing query

`%q` omitted from `LogFormat`

Add `\"query\":\"%q\"` back; the filter strips the leading `?`.

Client IP is the proxy, not the visitor

Apache behind a load balancer

Configure `mod_remoteip` and use `%{c}a`; the value is hashed regardless.

`bytes` is `-` in old logs

Legacy `%b` instead of `%B`

Use `%B` (emits `0` for empty responses).

`400 Bad Request` from Maya

Extra/missing field

Inspect output NDJSON; keys must match the schema exactly.

`401` / `429`

Auth or rate limit

Re-check API key; add backoff / split the batch.

[PreviousNginx](/docs/integrations/log-export/nginx)[Next Custom BFF Endpoint](/docs/integrations/log-export/bff-endpoint)