{
  "schema_version": 2,
  "id": "integrate/redis-data-integration/data-pipelines/rejected-records",
  "title": "Rejected records",
  "url": "https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/rejected-records/",
  "summary": "Learn how RDI stores records that cannot be processed.",
  "tags": [
    "docs",
    "integrate",
    "rs",
    "rdi"
  ],
  "last_updated": "2026-09-18T16:39:51+01:00",
  "page_type": "content",
  "content_hash": "cbd705239a2e53d572fd65d167d1ff7b2f0dd0e94a0c67f9dbe2b13cc6b7e89d",
  "sections": [
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "Redis Data Integration (RDI) sends records that it cannot process to a dead letter\nqueue (DLQ). In the Redis Cloud UI, these records are called **rejected records**.\n\nRejected records help you understand which source tables are affected and why\nspecific records could not continue through the pipeline. They are intended for\ntroubleshooting and support, not for normal pipeline operation."
    },
    {
      "id": "when-records-are-rejected",
      "title": "When records are rejected",
      "role": "content",
      "text": "RDI can reject a record when it cannot safely transform or write the change event.\nCommon causes include:\n\n- The incoming change event is malformed or missing required metadata.\n- A transformation job fails while processing the record.\n- A target write fails, for example because the target key already has an incompatible data type.\n\nBy default, RDI stores rejected records instead of silently dropping them. You can\nchange this behavior with the `processors.error_handling` setting. See the\n[pipeline configuration file](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/pipeline-config)\nfor more information."
    },
    {
      "id": "how-rdi-stores-rejected-records",
      "title": "How RDI stores rejected records",
      "role": "content",
      "text": "RDI stores rejected records in the RDI database as capped Redis streams. Each DLQ\nstream corresponds to a source table and tracks the records rejected for that\ntable.\n\nA DLQ stream is named after the source data stream it corresponds to, with a `dlq:` prefix:\n\n[code example]\n\nAbove, `<qualified_table_name>` is the qualified table name: `<database>.<table>`\nfor MySQL and MariaDB, `<database>.<collection>` for MongoDB, `<schema>.<table>` for Oracle,\nPostgreSQL, Snowflake, and Spanner, and `<database>.<schema>.<table>` for SQL Server.\n\nFor example, rejected records for the `public.users` table of a source named `postgresql`\nare stored in `dlq:data:{rdi}:postgresql.public.users`, and those for the `dbo.users` table of\nthe `inventory` database of a source named `sqlserver` are stored in\n`dlq:data:{rdi}:sqlserver.inventory.dbo.users`.\n\nThe dead-letter queue endpoints and the CLI report table names are in the same form,\nso you can easily attribute a rejected record to the source it came from. Sources that existed\nbefore the upgrade to RDI v2.0.0 or above do not have the source segment\nin the key because these earlier RDI versions did not support multiple sources.\n\nSome RDI versions or configurations can use a hash-tagged variant such as\n`dlq:{data:rdi:<qualified_table_name>}`. To find all DLQ streams in the\nRDI database, scan for stream keys that start with `dlq:`.\n\nThe maximum number of records stored per DLQ stream is controlled by\n`processors.dlq_max_messages`. When the stream reaches the configured limit,\nolder entries are evicted as newer entries are added."
    },
    {
      "id": "what-to-inspect",
      "title": "What to inspect",
      "role": "content",
      "text": "Start with the rejected count for each affected table, then inspect a sample\nrecord from the table with the highest count or the table that matters most to\nyour application.\n\nUseful fields include:\n\n- The affected table.\n- The rejection time.\n- The rejected operation. RDI stores this as an `opcode` value such as `c`,\n  `u`, `d`, `r`, `t`, or `m`. See [Using the operation code](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/transform-examples/redis-opcode-example)\n  for the operation labels.\n- The rejection reason.\n- The transformation job or operation, when the failure happened during transformation.\n\n> [!NOTE]\n> Rejected records can contain source data when you inspect them directly in the\n> RDI database. Treat DLQ contents as sensitive customer data. The Redis Cloud UI\n> uses the RDI DLQ API and shows a sanitized set of troubleshooting metadata. It\n> does not show the original record payload or every field stored in the\n> corresponding DLQ stream."
    },
    {
      "id": "resolve-rejected-records",
      "title": "Resolve rejected records",
      "role": "content",
      "text": "Use the rejection reason to identify the likely fix:\n\n- If a transformation job failed, update the job configuration and deploy the pipeline change.\n- If target writes failed because of incompatible existing keys, update the target data or key mapping.\n- If records are malformed, inspect the source connector and source database change data capture configuration.\n\nRDI does not automatically replay records from the DLQ after you fix the cause.\nIf you need existing source data to be processed again, reset the pipeline after\napplying the fix. See [Reset data pipeline](https://redis.io/docs/latest/operate/rc/rdi/view-edit#reset-data-pipeline)\nfor Redis Cloud, or use the appropriate self-managed RDI reset workflow."
    },
    {
      "id": "cli-and-api-access",
      "title": "CLI and API access",
      "role": "content",
      "text": "For self-managed RDI, use the [`redis-di list-dlqs`](https://redis.io/docs/latest/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs)\ncommand to see the dead-letter queues and the\n[`redis-di list-dlq-records`](https://redis.io/docs/latest/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records)\ncommand (also available as `redis-di get-rejected`) to inspect the rejected records of a queue.\n\nFor Redis Cloud RDI, connect to the RDI database and inspect the corresponding\nDLQ streams directly when you need details that are not shown in the Redis Cloud\nUI.\n\nRDI API v2 also includes DLQ inspection endpoints. See the\n[API reference](https://redis.io/docs/latest/integrate/redis-data-integration/reference/api-reference)\nfor endpoint details."
    }
  ],
  "examples": [
    {
      "id": "how-rdi-stores-rejected-records-ex0",
      "language": "text",
      "code": "dlq:data:{rdi}:<source>.<qualified_table_name>",
      "section_id": "how-rdi-stores-rejected-records"
    }
  ]
}