{
  "schema_version": 2,
  "id": "operate/rc/rdi/quick-start",
  "title": "RDI on Redis Cloud quick start",
  "url": "https://redis.io/docs/latest/operate/rc/rdi/quick-start/",
  "summary": "Learn how to create a data pipeline between a PostgreSQL source database created with Terraform and a Redis Cloud target database.",
  "aliases": [
    "/operate/rc/databases/rdi/quick-start/",
    "/operate/rc/databases/rdi/quick-start"
  ],
  "tags": [
    "docs",
    "operate",
    "rc"
  ],
  "last_updated": "2026-10-18T03:56:52+03:00",
  "page_type": "content",
  "content_hash": "223fbdb0dfea1d35ac1630f431c9290144f936a5f9d78924af2d214e1fb29401",
  "sections": [
    {
      "id": "overview",
      "title": "Overview",
      "role": "overview",
      "text": "The [`rdi-cloud-automation` GitHub repository](https://github.com/redis/rdi-cloud-automation) contains a Terraform script that quickly sets up a PostgreSQL source database on an EC2 instance and all required permissions and network setup to connect it to a Redis Cloud target database.\n\n> [!NOTE]\n> This guide is for demonstration purposes only. It is not recommended for production use."
    },
    {
      "id": "prerequisites",
      "title": "Prerequisites",
      "role": "content",
      "text": "To follow this guide, you need to:\n\n1. Create a [Redis Cloud Pro database](https://redis.io/docs/latest/operate/rc/databases/create-database/create-pro-database-new) hosted on Amazon Web Services (AWS).\n\n    Turn on Multi-AZ replication and [manually select the availability zones](https://redis.io/docs/latest/operate/rc/databases/configuration/high-availability#availability-zones) when creating the database.\n\n1. Install the [AWS CLI](https://aws.amazon.com/cli/) and set up [credentials for the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html).\n\n1. Install [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)."
    },
    {
      "id": "create-a-data-integration-workspace",
      "title": "Create a data integration workspace",
      "role": "content",
      "text": "Before you can create your first Data Integration pipeline for a Redis Cloud subscription, you must first deploy the cloud infrastructure needed to host the pipeline and run the workers associated with the pipeline. In Redis Cloud, this is called a **Workspace**. See [Create and manage Data Integration workspace](https://redis.io/docs/latest/operate/rc/rdi/create-workspace) for more information.\n\nTo create a Data Integration workspace for an existing [Pro subscription](https://redis.io/docs/latest/operate/rc/databases/create-database/create-pro-database-new):\n\n1. In the Redis Cloud console, select **Data Integration**.\n\n2. Select **Create workspace** if you have no workspaces yet.\n\n    ![The create workspace button.](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-button.png)\n\n    If you already have a workspace, select **New workspace**.\n\n    ![The new workspace button.](https://redis.io/docs/latest/images/rc/rdi/rdi-new-workspace-button.png)\n\n    You can also open **Data Integration** from a subscription or database page.\n    Select **Create workspace** to use that subscription.\n\n    ![The create workspace button.](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-button.png)\n\n3. Select your Pro subscription if it is not already selected.\n\n    ![The select pro subscription drop down.](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-select-subscription.png)\n\n4. Review **Data Integration subnet (CIDR)**. The console suggests a dedicated\n   `/22` Classless Inter-Domain Routing (CIDR) range for the workspace.\n\n    [Plan network capacity for all pipelines](https://redis.io/docs/latest/operate/rc/rdi/scale-pipeline#plan-workspace-network-capacity)\n    before you create the workspace. Choose a `/21` or larger range if you plan\n    more than 5 sources or more than 5 processor replicas in total.\n    These counts are a conservative planning trigger, not a capacity limit.\n    You cannot enlarge the workspace CIDR after creation.\n\n    For Amazon Web Services (AWS), the RDI workspace CIDR must:\n\n    - Be in the same [RFC 1918 private address range](https://datatracker.ietf.org/doc/html/rfc1918#section-3) as the subscription virtual private cloud (VPC)'s primary CIDR: `10.0.0.0/8`, `172.16.0.0/12`, or `192.168.0.0/16`.\n    - Not overlap with existing subscription, peering, transit gateway (TGW), application, database, or RDI workspace CIDR ranges.\n\n    For example, a subscription VPC has the primary CIDR `10.238.252.0/24`.\n    The range `192.168.0.0/22` is invalid because it is in a different\n    RFC 1918 range. An unused range such as `10.239.0.0/22` is valid.\n\n    If the suggestion is missing or unsuitable, select another unused range.\n    It must meet your capacity needs and use the same private address range.\n    See [VPC CIDR block association restrictions](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-cidr-blocks.html#vpc-resize).\n\n    ![The select pro subscription drop down.](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-cidr.png)\n\n5. Select **Create workspace**.\n\n    ![The create workspace button.](https://redis.io/docs/latest/images/rc/rdi/rdi-create-workspace-button.png)\n\nRedis Cloud creates the workspace in the background. Select **Create pipeline**\nto [create your pipeline](https://redis.io/docs/latest/operate/rc/rdi/define) while it provisions.\nSelect **Create pipeline later** to return to the Redis Cloud console."
    },
    {
      "id": "get-required-arns",
      "title": "Get required ARNs",
      "role": "content",
      "text": "This example creates one PostgreSQL source. You can [add more sources](https://redis.io/docs/latest/operate/rc/rdi/view-edit#add-source) after the pipeline is running.\n\n1. On the [Redis Cloud console](https://cloud.redis.io/), open your target database's **Data Integration** tab and select **Add pipeline**.\n\n    ![The Add pipeline control is available while the workspace is being created.](https://redis.io/docs/latest/images/rc/rdi/rdi-workspace-add-pipeline.png)\n\n1. In **Settings**, select your target database and choose **Hash** or **JSON** as the default data structure, then select **Continue**.\n1. In **Add sources**, select **PostgreSQL**.\n1. Enter a **Source name** of your choice, for example `inventory-postgres`.\n1. Select **Continue** to open **Configure source**.\n1. Under **Source connectivity**, copy the **Role ARN**. Use it as `redis_privatelink_arn` in the Terraform configuration.\n\n    ![The source connectivity Role ARN and availability zones.](https://redis.io/docs/latest/images/rc/rdi/rdi-setup-connectivity-arn.png)\n\n1. Under **Secrets**, copy the **Role ARN**. Use it as `redis_secrets_arn` in the Terraform configuration.\n\n    ![The Role ARN in the Secrets section.](https://redis.io/docs/latest/images/rc/rdi/rdi-credentials-arn.png)\n\n1. Select **Save & exit** while you create the source resources."
    },
    {
      "id": "create-the-source-database-and-network-resources",
      "title": "Create the source database and network resources",
      "role": "content",
      "text": "1. Clone or download the [`rdi-cloud-automation` GitHub repository](https://github.com/redis/rdi-cloud-automation).\n\n1. In a terminal window, go to the `examples/aws-ec2-privatelink` directory.\n\n1. Run `terraform init` to initialize the Terraform working directory.\n\n1. Open the `example.tfvars` file and edit the following variables:\n\n    - `region`: The AWS region where your Redis Cloud database is deployed.\n    - `azs`: The availability zone IDs where your Redis Cloud database is deployed.\n    - `port`: The port number for the new PostgreSQL source database.\n    - `name`: A prefix for all of the created AWS resources.\n    - `redis_secrets_arn`: The role ARN from **Secrets** in the Redis Cloud console.\n    - `redis_privatelink_arn`: The role ARN from **Source connectivity** in the Redis Cloud console.\n\n1. To view the configuration, run:\n\n    [code example]\n\n1. To create the AWS resources, run:\n\n    [code example]\n\n    This example creates the following resources on your AWS account:\n\n    - An AWS KMS key with the required permissions for RDI\n    - A VPC with a public and private subnet and all necessary route tables\n    - An EC2 instance running a PostgreSQL database with a security group that allows access from Redis Cloud\n    - An AWS Secrets Manager secret for the PostgreSQL database credentials\n    - A Network Load Balancer (NLB), a listener, and target group to route traffic to the EC2 instance with AWS PrivateLink\n    - An AWS PrivateLink endpoint service for the PostgreSQL database\n\nCreating the AWS resources will take some time. After the resources are created, you'll be able to view them in the AWS management console.\n\nSave the following outputs:\n\n- `database`: The name of the PostgreSQL database.\n- `port`: The port number for the PostgreSQL database.\n- `secret_arn`: The ARN of the AWS Secrets Manager secret for the PostgreSQL database credentials.\n- `vpc_endpoint_service_name`: The name of the AWS PrivateLink endpoint service for the PostgreSQL database.\n\nIf you lose any outputs, run `terraform output` to view them again."
    },
    {
      "id": "resume-pipeline-setup",
      "title": "Resume pipeline setup",
      "role": "content",
      "text": "1. Return to your database's **Data Integration** tab in the [Redis Cloud console](https://cloud.redis.io/).\n1. Open the draft pipeline's actions menu and select **Resume pipeline setup**.\n1. Open **Configure source** and select your PostgreSQL source in the **Sources** list.\n1. Under **Source connectivity**, enter the Terraform `vpc_endpoint_service_name` output as the **Private Link service name**.\n1. Select **Connect to Private Link** and wait for connectivity to complete.\n\n    ![AWS Private Link connectivity with the service name and Connect to Private Link control.](https://redis.io/docs/latest/images/rc/rdi/rdi-source-configuration-source-connectivity-privatelink.png)\n\n1. Under **Secrets**, enter the `secret_arn` output as **Credentials Secret ARN**.\n1. Select **Validate** to check access to the secret.\n\n    ![The Credentials Secret ARN field, transit security options, and Validate control.](https://redis.io/docs/latest/images/rc/rdi/rdi-source-configuration-secrets.png)\n\n1. Under **Source configuration**, enter the `database` and `port` Terraform outputs in the matching fields.\n1. Under **Collector properties**, set `publication.autocreate.mode` to `filtered` so RDI\n   creates a publication for the selected tables. This example uses the `postgres` user,\n   which has the required permissions. For an administrator-managed publication or an\n   existing all-tables publication, follow [PostgreSQL publications](https://redis.io/docs/latest/operate/rc/rdi/setup#postgresql-publications).\n1. Select **Test source** and correct any validation errors, then select **Continue**.\n1. In **Select data**, select the schemas, tables, and columns to ingest. Review the selected key for each table.\n\n    ![Selecting a schema shows its tables for ingestion.](https://redis.io/docs/latest/images/rc/rdi/rdi-dataset-schema-selected.png)\n\n    ![Selecting a table shows its columns and the columns selected for ingestion.](https://redis.io/docs/latest/images/rc/rdi/rdi-select-columns.png)\n\n1. Select **Continue** to open **Add transformations**. For this example, you can keep the default mapping without adding jobs.\n1. Select **Continue to review & deploy**.\n1. Review the source and target, then select **Deploy pipeline**.\n\n    ![The Deploy pipeline button.](https://redis.io/docs/latest/images/rc/rdi/rdi-confirm-deploy.png)\n\nThe source first imports its selected data, then captures ongoing changes. Open the pipeline's **Dashboard** or **Metrics** tab to follow its progress.\n\nThe following example shows the metrics for one selected source in a pipeline with multiple sources.\n\n![Metrics for one selected source in a pipeline, including snapshot progress and per-table record counts.](https://redis.io/docs/latest/images/rc/rdi/rdi-2-metrics.png)\n\nSee [View and edit data pipeline](https://redis.io/docs/latest/operate/rc/rdi/view-edit) for source actions, dataset changes, and monitoring."
    },
    {
      "id": "delete-sample-resources",
      "title": "Delete sample resources",
      "role": "content",
      "text": "> [!WARNING]\n> Make sure to [delete your data pipeline](https://redis.io/docs/latest/operate/rc/rdi/view-edit#delete-pipeline) before deleting the sample resources.\n\nTo delete the sample resources created by Terraform, run:\n\n[code example]"
    }
  ],
  "examples": [
    {
      "id": "create-the-source-database-and-network-resources-ex0",
      "language": "sh",
      "code": "terraform plan -var-file=example.tfvars",
      "section_id": "create-the-source-database-and-network-resources"
    },
    {
      "id": "create-the-source-database-and-network-resources-ex1",
      "language": "sh",
      "code": "terraform apply -var-file=example.tfvars",
      "section_id": "create-the-source-database-and-network-resources"
    },
    {
      "id": "delete-sample-resources-ex0",
      "language": "sh",
      "code": "terraform destroy -var-file=example.tfvars",
      "section_id": "delete-sample-resources"
    }
  ]
}