Documentation

Monitoring S3 Exposure Ingests

S3 snapshot sources provide an ingest history for uploaded exposure files. This is different from the hour-by-hour pixel Delivery Report: pixel reporting measures live event delivery, while S3 history follows each batch file through validation and import.

Opening S3 ingest history

Open the S3 exposure source from Account → Resources → Exposure Sources. The source shows its S3 upload destination, latest ingest, and previous ingest runs. Use Refresh to retrieve the newest pipeline state.

The latest-ingest section includes the start and finish times, current pipeline stage, uploaded object path, aggregate row counts, and bounded error messages:

Latest S3 exposure ingest completed with errors, showing total, retained, imported, and rejected row counts

Act on the ingest status
  • Idle — no uploaded file has been processed for this source yet. Upload a small validation file before production delivery.
  • Processing — the file is being validated or imported. Wait for a terminal status, then select Refresh if the view has not updated.
  • Succeeded — processing finished without reported validation errors. Reconcile the counts before treating the delivery as complete.
  • Completed with errors — usable rows were imported, but one or more rows were rejected during validation. Inspect the error and correct any required rows.
  • Failed — the file could not be processed or imported. Correct the reported problem and upload a new file.
  • Timed out — processing did not reach a terminal state within the allowed ingest window. Check the error details before uploading a new file.

The history retains the latest projection and individual run details, so a later successful upload does not erase the diagnostic record for an earlier failure.

Reading the row counts
  • Total — all data rows read from the uploaded CSV.
  • Retained — rows that passed validation and were eligible for import.
  • Imported — records written into the source's current exposure snapshot after the import pipeline completed.
  • Rejected — rows excluded during validation. Rejected rows are not processed as exposure records and do not participate in matching or reporting.

Imported can be lower than Retained when the snapshot pipeline consolidates records into its import representation. Use Rejected, rather than the difference between Retained and Imported, to measure validation loss.

Troubleshooting rejected rows

Start with the error message on the run, then check the source mappings and the uploaded file. Common causes include blank or invalid IP values, an unusable hashed IP, a missing UID required for import, an over-length dimension value, or the reserved null value in a configured dimension.

A Completed with errors run is partially successful: correct the rejected rows and upload a new snapshot if those records are required. Do not assume the rejected rows will be retried automatically.

A Failed or Timed out run did not produce a usable replacement snapshot. Correct the reported problem and upload a new file with a distinct name so the new attempt is easy to reconcile in history.

Unsupported uploads are deleted

Only lowercase .csv.gz objects under the source's assigned upload prefix are supported. An object with another suffix, including .CSV.GZ, .csv, or a temporary suffix, is rejected before the ingestion job starts and deleted from the upload bucket. It is not retained for later processing and does not contribute exposure records.

Because the rejected object is deleted, keep the source file in your own controlled storage until its run reaches Succeeded or Completed with errors. Correct the format locally and upload a new .csv.gz object; renaming an unsupported object after delivery is not possible in MX8 Labs.

What to reconcile

For every production delivery:

  1. Confirm the run appears in history and reaches a terminal status.
  2. Compare Total with the source system's row count.
  3. Investigate every nonzero Rejected count.
  4. Confirm Imported is nonzero when the file was expected to contain usable exposures.
  5. Keep the run ID, timestamps, object path, and counts with your delivery logs.

For preparation and upload rules, see Server-Side File Delivery & Format Requirements.