{
  "schema_version": 2,
  "id": "integrate/redis-data-integration/data-pipelines/transform-examples/redis-lookup-example",
  "title": "Reading Redis data with redis.lookup",
  "url": "https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/transform-examples/redis-lookup-example/",
  "summary": "",
  "tags": [
    "docs",
    "integrate",
    "rs",
    "rdi"
  ],
  "last_updated": "2026-09-18T16:39:51+01:00",
  "page_type": "content",
  "content_hash": "9a7ae171277a7f45564be965b6b46208d86fa2cf156b1bb1d9ec7bbc5e9325bc",
  "sections": [
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "You can use the\n[`redis.lookup`](https://redis.io/docs/latest/integrate/redis-data-integration/reference/data-transformation/lookup)\ntransformation to read existing data from Redis during the `transform` stage of a\njob. This lets you enrich an incoming record with values that are already present\nin the target database.\n\nFor example, a pipeline for the Chinook database might read an `artist` record\nthat is already stored in Redis and use `redis.lookup` in an `album` table job to\nadd selected artist details to each album record before writing it to the target\ndatabase.\n\n> [!WARNING]\n> Do not rely on `redis.lookup` to *denormalize* data that RDI writes from another\n> table in the **same pipeline**. RDI can't guarantee that the looked-up data will be\n> present or up to date when the lookup runs, for the following reasons:\n>\n> - **Snapshot order isn't guaranteed.** During the initial snapshot, RDI can't\n>   guarantee that the table you look up is ingested before the table that depends\n>   on it. If a dependent job runs before the referenced key has been written, the\n>   lookup misses.\n> - **Change (CDC) order isn't guaranteed.** If a parent and child record are\n>   inserted or updated at around the same time, RDI has no way to order these\n>   events, so the lookup can still miss.\n> - **Parent updates don't refresh existing keys.** Even if the lookup succeeds,\n>   updating the source record later does *not* update the keys that already copied\n>   its values. The denormalized data becomes stale.\n>\n> The only case where `redis.lookup` is safe for enrichment is when you can guarantee\n> that the looked-up data is present in the target database *independently* of the\n> RDI pipeline (for example, a reference table that is loaded and maintained\n> separately).\n>\n> To denormalize data that RDI ingests, use a supported technique instead. See\n> [Data denormalization](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/data-denormalization)\n> for one-to-one joins (using `merge`) and one-to-many joins (using nesting)."
    },
    {
      "id": "reading-a-hash-field",
      "title": "Reading a hash field",
      "role": "content",
      "text": "The `redis.lookup` transformation works by executing a Redis command and adding the\nresult to the record. You specify the command and its arguments in the\n`transform` configuration with the `cmd` and `args` properties. For example, the\nfollowing transformation job uses the\n[`HGET`](https://redis.io/docs/latest/commands/hget) command to read the `name` field from an\nartist [hash](https://redis.io/docs/latest/develop/data-types/hashes) and adds it to the\nalbum record under the `artist` field. A particularly important thing to note\nhere is that the `args` elements are all interpreted as [JMESPath](https://jmespath.org/)\nexpressions, but YAML syntax allows for each element to be a quoted string. This means that\nyou must *double quote* any string arguments that you want to be treated as\nliteral strings (as with `name` below), otherwise JMESPath will try to interpret\nthem as field names, which will generally give the wrong result. Specifically, use\na different quote character for the outer quotes and the inner quotes.\n\n[code example]\n\nBefore the lookup runs, the album hash object contains only the `artistid` field to\nreference the artist:\n\n[code example]\n\nAfter running the job specified above, querying one of the album hash objects shows the\nextra `artist` field obtained by looking up the artist with the `artistid`:\n\n[code example]"
    },
    {
      "id": "embedding-a-json-document",
      "title": "Embedding a JSON document",
      "role": "content",
      "text": "If you are using [JSON](https://redis.io/docs/latest/develop/data-types/json) objects,\nyou can read the whole of one object and embed it\nas a field of another. The following example shows how to do this using a temporary field\nto hold the result of the `redis.lookup` command. It then uses\n[`add_field`](https://redis.io/docs/latest/integrate/redis-data-integration/reference/data-transformation/add_field)\nto insert the new field and\n[`remove_field`](https://redis.io/docs/latest/integrate/redis-data-integration/reference/data-transformation/remove_field)\nto remove the temporary field and the now-redundant `artistid` field before writing the album object.\n\n[code example]\n\nAfter running this job, the album JSON object includes the artist object\nin a new `artist` field:\n\n[code example]"
    }
  ],
  "examples": [
    {
      "id": "reading-a-hash-field-ex0",
      "language": "yaml",
      "code": "source:\n  table: album\ntransform:\n  - uses: redis.lookup\n    with:\n      connection: target\n      cmd: HGET\n      args:\n        - concat(['artist:artistid:', artistid])\n        - '`name`'\n      language: jmespath\n      field: artist\noutput:\n  - uses: redis.write\n    with:\n      connection: target\n      data_type: hash\n      key:\n        expression: concat(['album:albumid:', albumid])\n        language: jmespath",
      "section_id": "reading-a-hash-field"
    },
    {
      "id": "reading-a-hash-field-ex1",
      "language": "bash",
      "code": "> hgetall album:albumid:1\n1) \"albumid\"\n2) \"1\"\n3) \"title\"\n4) \"For Those About To Rock We Salute You\"\n5) \"artistid\"\n6) \"1\"",
      "section_id": "reading-a-hash-field"
    },
    {
      "id": "reading-a-hash-field-ex2",
      "language": "bash",
      "code": "> hgetall album:albumid:1\n1) \"albumid\"\n2) \"1\"\n3) \"title\"\n4) \"For Those About To Rock We Salute You\"\n5) \"artistid\"\n6) \"1\"\n7) \"artist\"\n8) \"AC/DC\"",
      "section_id": "reading-a-hash-field"
    },
    {
      "id": "embedding-a-json-document-ex0",
      "language": "yaml",
      "code": "source:\n  table: album\ntransform:\n  - uses: redis.lookup\n    with:\n      connection: target\n      cmd: JSON.GET\n      args:\n        - concat(['artist:artistid:', artistid])\n      language: jmespath\n      field: artiststring\n  - uses: add_field\n    with:\n      field: artist\n      language: jmespath\n      expression: json_parse(artiststring)\n  - uses: remove_field\n    with:\n      fields:\n        - field: artistid\n        - field: artiststring\noutput:\n  - uses: redis.write\n    with:\n      connection: target\n      data_type: json\n      key:\n        expression: concat(['album:albumid:', albumid])\n        language: jmespath",
      "section_id": "embedding-a-json-document"
    },
    {
      "id": "embedding-a-json-document-ex1",
      "language": "json",
      "code": "{\n  \"albumid\": 239,\n  \"title\": \"War\",\n  \"artist\": {\n    \"artistid\": 150,\n    \"name\": \"U2\"\n  }\n}",
      "section_id": "embedding-a-json-document"
    }
  ]
}