{
  "schema_version": 2,
  "id": "operate/rc/rdi",
  "title": "Data Integration",
  "url": "https://redis.io/docs/latest/operate/rc/rdi/",
  "summary": "Use Redis Data Integration with Redis Cloud.",
  "aliases": [
    "/operate/rc/databases/rdi/",
    "/operate/rc/databases/rdi"
  ],
  "tags": [
    "docs",
    "operate",
    "rc"
  ],
  "last_updated": "2026-10-12T16:35:02+03:00",
  "children": [
    {
      "id": "operate/rc/rdi/quick-start",
      "summary": "Learn how to create a data pipeline between a PostgreSQL source database created with Terraform and a Redis Cloud target database.",
      "title": "RDI on Redis Cloud quick start",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/quick-start/"
    },
    {
      "id": "operate/rc/rdi/create-workspace",
      "summary": "Create and manage the infrastructure for your Data Integration pipelines.",
      "title": "Create and manage Data Integration workspace",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/create-workspace/"
    },
    {
      "id": "operate/rc/rdi/setup",
      "summary": "Prepare your source database, network setup, and database credentials for Data integration.",
      "title": "Prepare source database",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/setup/"
    },
    {
      "id": "operate/rc/rdi/define",
      "summary": "Connect one or more source databases to a Redis Cloud target and define your data pipeline.",
      "title": "Create data pipeline",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/define/"
    },
    {
      "id": "operate/rc/rdi/networking",
      "summary": "Network reference for connecting a Data Integration pipeline to your source database.",
      "title": "Networking",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/networking/"
    },
    {
      "id": "operate/rc/rdi/view-edit",
      "summary": "Observe and change your data pipeline.",
      "title": "View and edit data pipeline",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/view-edit/"
    },
    {
      "id": "operate/rc/rdi/scale-pipeline",
      "summary": "Increase collector, RDI database, and processor capacity for a Cloud RDI pipeline.",
      "title": "Scale a Cloud RDI pipeline",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/scale-pipeline/"
    },
    {
      "id": "operate/rc/rdi/faq",
      "summary": "Answers to common questions about RDI on Redis Cloud.",
      "title": "RDI Cloud FAQ",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/faq/"
    },
    {
      "id": "operate/rc/rdi/use-cases",
      "summary": "Reference setups for Data Integration pipelines that run in production.",
      "title": "Production use cases",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/use-cases/"
    },
    {
      "id": "operate/rc/rdi/rds-proxy",
      "summary": "Set up RDS Proxy for Redis Data Integration (not recommended).",
      "title": "RDS Proxy setup for RDI",
      "url": "https://redis.io/docs/latest/operate/rc/rdi/rds-proxy/"
    }
  ],
  "page_type": "content",
  "content_hash": "26d1480de000724846f38541821163df450b4ca5a69a461e9649f37cf38f518b",
  "sections": [
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "Redis Cloud now supports [Redis Data Integration (RDI)](https://redis.io/docs/latest/integrate/redis-data-integration), a fast and simple way to bring your data into Redis from other types of primary databases.\n\n![The Data Integration page in the Redis Cloud console, with the Create workspace button and supported source databases.](https://redis.io/docs/latest/images/rc/rdi-get-started.png)\n\nA relational database usually handles queries much more slowly than a Redis database. If your application uses a relational database and makes many more reads than writes (which is the typical case) then you can improve performance by using Redis as a cache to handle the read queries quickly. Redis Cloud uses [ingest](https://redis.io/docs/latest/integrate/redis-data-integration) to help you offload all read queries from the application database to Redis automatically.\n\nUsing a data pipeline lets you have a cache that is always ready for queries. RDI Data pipelines ensure that any changes made to your primary database are captured in your Redis cache within a few seconds, preventing cache misses and stale data within the cache. \n\nRDI helps Redis customers sync Redis Cloud with live data from their primary databases to:\n- Meet the required speed and scale of read queries and provide an excellent and predictable user experience.\n- Save resources and time when building pipelines and coding data transformations.\n- Reduce the total cost of ownership by saving money on expensive database read replicas.\n\nUsing RDI with Redis Cloud simplifies managing your data integration pipeline. No need to worry about hardware or underlying infrastructure, as Redis Cloud manages that for you. Creating the data flow from source to target is much easier, and there are validations in place to reduce errors."
    },
    {
      "id": "is-rdi-a-good-fit-for-my-architecture",
      "title": "Is RDI a good fit for my architecture?",
      "role": "content",
      "text": "RDI is designed to support apps that must use a disk-based database as the system of record\nbut must also be fast and scalable. This is a common requirement for mobile and web\napps with a rapidly-growing number of users; the performance of the main database is fine at first\nbut it will soon struggle to handle the increasing demand without a cache.\n\nUse the information in the sections below to determine whether RDI is a good fit for your architecture. See also the\n[decision tree for using RDI](https://redis.io/docs/latest/integrate/redis-data-integration/when-to-use#decision-tree-for-using-rdi)\nwhich presents the considerations in a straightforward question-and-answer format.\n\n"
    },
    {
      "id": "when-to-use-rdi",
      "title": "When to use RDI",
      "role": "content",
      "text": "RDI is a good fit when:\n\n- You want your app/micro-services to read from Redis to scale reads at speed.\n- You want to transfer data to Redis from one or more source databases.\n- You must use a slow database as the system of record for the app.\n- The app must always *write* its data to the slow database.\n- Your app can tolerate *eventual* consistency of data in the Redis cache.\n- You want a self-managed solution or AWS based solution.\n- The source data changes frequently in small increments.\n- The source database has no more than 20K changes per second.\n- RDI throughput during [full sync](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines#pipeline-lifecycle)\n  stays below 60K records per second, assuming an average record size of 1KB and a pipeline without transformations.\n- RDI throughput during [CDC](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines#pipeline-lifecycle)\n  stays below 20K records per second, assuming an average record size of 1KB and a pipeline without transformations.\n- The total data size is no larger than 200GB, so a full sync completes in under an hour without exceeding the throughput\n  limits above. RDI can ingest larger datasets, but it will take longer than an hour.\n- You don’t need to perform join operations on the data from several tables\n  into a [nested Redis JSON object](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/data-denormalization#joining-one-to-many-relationships).\n- RDI supports the [data transformations](https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/transform-examples) you need for your app.\n- Your data caching needs are too complex or demanding to implement and maintain yourself.\n- Your database administrator has reviewed RDI's requirements for the source database and\n  confirmed that they are acceptable."
    },
    {
      "id": "when-not-to-use-rdi",
      "title": "When not to use RDI",
      "role": "content",
      "text": "RDI is not a good fit when:\n\n- You are migrating an existing data set into Redis only once.\n- Your app needs *immediate* cache consistency (or a hard limit on latency) rather\n  than *eventual* consistency.\n- You need *transactional* consistency between the source and target databases.\n- The app must *write* data to the Redis cache, which then updates the source database\n  (write-behind/write-through patterns).\n- Your data set will only ever be small.\n- Your data is updated by some batch or ETL process with long and large transactions - RDI will fail\n  processing these changes.\n- You need complex stream processing of data (aggregations, sliding window processing, complex \n  custom logic).\n- You need to write data to multiple targets from the same pipeline (Redis supports other\n  ways to replicate data across Redis databases such as replicaOf and Active-Active).\n- Your database administrator has rejected RDI's requirements for the source database."
    },
    {
      "id": "data-pipeline-architecture",
      "title": "Data pipeline architecture",
      "role": "content",
      "text": "An RDI data pipeline connects one or more source databases to one target Redis database. Sources can use the same or different supported database types. Each source has its own collector, connectivity, credentials, and selection of tables and columns. All sources share the pipeline's processor and target.\n\nEach source first imports its selected data during the *initial sync* phase, then captures changes during the *streaming* phase. The pipeline transforms the captured records and writes them to Redis. You can monitor and manage each source from the pipeline dashboard.\n\nRDI Cloud uses the Flink processor for all pipelines.\n\nFor more info on how RDI works, see [RDI Architecture](https://redis.io/docs/latest/integrate/redis-data-integration/architecture)."
    },
    {
      "id": "pipeline-security",
      "title": "Pipeline security",
      "role": "content",
      "text": "Data pipelines are set up to ensure a high level of data security. Source database credentials and TLS secrets are stored in AWS secret manager and shared using the Kubernetes CSI driver for secrets. See [Share source database credentials](https://redis.io/docs/latest/operate/rc/rdi/setup#share-source-database-credentials) to learn how to share your source database credentials and TLS certificates with Redis Cloud.\n\nConfigure connectivity separately for each source. A source can use a public endpoint or [AWS PrivateLink](https://aws.amazon.com/privatelink/), subject to the source-specific requirements in [Prerequisites](#prerequisites). See [Set up connectivity](https://redis.io/docs/latest/operate/rc/rdi/setup#set-up-connectivity) to learn how to connect your PrivateLink to the Redis Cloud VPC.\n\nRDI encrypts all network connections with TLS. The pipeline will process data from the source database in-memory and write it to the target database using a TLS connection. There are no external connections to your data pipeline except from Redis Cloud management services."
    },
    {
      "id": "prerequisites",
      "title": "Prerequisites",
      "role": "content",
      "text": "Before you can create a data pipeline, you must have:\n\n- A [Redis Cloud Pro database](https://redis.io/docs/latest/operate/rc/databases/create-database/create-pro-database-new) hosted on Amazon Web Services (AWS). This will be the target database.\n- One or more supported source databases that are publicly accessible or hosted on an AWS EC2 instance, AWS RDS, or AWS Aurora:\n\n| Database | Versions | AWS RDS  Versions |\n|:---|:---|:---|\n| Oracle | 19c, 21c | 19c, 21c |\n| MariaDB | 10.5, 11.4.3 | 10.4 to 10.11, 11.4.3 |\n| MySQL | 5.7, 8.0.x, 8.2 | 8.0.x |\n| PostgreSQL | 10, 11, 12, 13, 14, 15, 16, 17, 18 | 11, 12, 13, 14, 15, 16, 17, 18 |\n| Supabase (uses PostgreSQL) | 10, 11, 12, 13, 14, 15, 16, 17 | - |\n| AWS Aurora PostgreSQL | 15 | 15 |\n| SQL Server | 2017, 2019, 2022 | 2016, 2017, 2019, 2022 |\n| MongoDB | 6.0, 7.0, 8.0 | - |\n| MongoDB Atlas | 6.0, 7.0, 8.0 | - |\n| Snowflake | - | - |\n\n\n> [!NOTE]\n> Please be aware of the following limitations:\n>\n> - The target database must be a Redis Cloud Pro database hosted on Amazon Web Services (AWS). Redis Cloud Essentials databases and databases hosted on Google Cloud do not support Data Integration.\n> - The target database must use [high availability](https://redis.io/docs/latest/operate/rc/databases/configuration/high-availability). It can use either single-zone or multi-zone high availability.\n> - The target database can use TLS, but can not use mutual TLS.\n> - The target database can't use Active-Active topology.\n> - If your source database is not publicly accessible, or if it is a MongoDB Atlas or Snowflake database, it must be hosted on AWS.\n> - You must use a [custom encryption key on AWS](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html) to create the instance hosting the database.\n> - Each pipeline has one target database shared by all of its sources.\n> - If the source database is not publicly accessible, you must be able to set up AWS PrivateLink to connect your source database to your target database. RDI only works with AWS PrivateLink and not VPC Peering or other private connectivity options.\n> - Mutual TLS is not supported for AWS RDS and AWS Aurora source databases."
    },
    {
      "id": "get-started",
      "title": "Get started",
      "role": "content",
      "text": "To get started fast with RDI on Redis Cloud, see the [RDI Cloud quick start](https://redis.io/docs/latest/operate/rc/rdi/quick-start) to create a data pipeline between a PostgreSQL source database and a Redis Cloud target database.\n\nTo create a new data pipeline, you need to:\n\n1. [Create a Data Integration workspace](https://redis.io/docs/latest/operate/rc/rdi/create-workspace) for your Pro subscription.\n1. [Prepare each source database](https://redis.io/docs/latest/operate/rc/rdi/setup) and any associated credentials.\n1. [Define the source connection and data pipeline](https://redis.io/docs/latest/operate/rc/rdi/define) by selecting which tables to sync.\n\nOnce your data pipeline is defined, you can [view and edit](https://redis.io/docs/latest/operate/rc/rdi/view-edit) it.\n\nFor complete production setups, including SQL Server failover handling, see [Production use cases](https://redis.io/docs/latest/operate/rc/rdi/use-cases).\n\nTo plan workspace capacity and scale a pipeline, see [Scale a Cloud RDI pipeline](https://redis.io/docs/latest/operate/rc/rdi/scale-pipeline)."
    },
    {
      "id": "billing-and-common-questions",
      "title": "Billing and common questions",
      "role": "content",
      "text": "See the [RDI Cloud FAQ](https://redis.io/docs/latest/operate/rc/rdi/faq) for billing examples, reset and flush behavior, and working with multiple sources."
    },
    {
      "id": "maintenance-windows",
      "title": "Maintenance windows",
      "role": "content",
      "text": "RDI Cloud maintenance follows the same subscription-wide maintenance window as your Redis Cloud Pro subscription. During a maintenance window, your data pipeline may experience brief interruptions as Redis applies updates.\n\nTo control when maintenance occurs, [set a manual maintenance window](https://redis.io/docs/latest/operate/rc/subscriptions/maintenance/set-maintenance-windows) for your Redis Cloud Pro subscription. Any maintenance window you configure applies to both your databases and your RDI data pipeline."
    }
  ],
  "examples": []
}