# Pipelines and transactions ```json metadata { "schema_version": 2, "title": "Pipelines and transactions", "description": "Learn how to use Redis pipelines and transactions", "categories": ["docs","develop","stack","oss","rs","rc","oss","kubernetes","clients"], "tableOfContents": {"sections":[{"id":"execute-a-pipeline","title":"Execute a pipeline"},{"id":"execute-a-transaction","title":"Execute a transaction"},{"id":"watch-keys-for-changes","title":"Watch keys for changes"}]} , "codeExamples": [{"codetabsId":"pipe_trans_tutorial-stepbasic_pipe","description":"Foundational: Use pipelines to batch multiple commands together and reduce network round trips","difficulty":"beginner","id":"basic_pipe","languages":[{"clientId":"redis-py","clientName":"redis-py","id":"Python","langId":"python","panelId":"panel_Python_pipe_trans_tutorial-stepbasic_pipe"}]},{"codetabsId":"pipe_trans_tutorial-steptrans_watch","description":"Optimistic locking: Use WATCH to monitor keys for changes and retry transactions when conflicts occur","difficulty":"intermediate","id":"trans_watch","languages":[{"clientId":"redis-py","clientName":"redis-py","id":"Python","langId":"python","panelId":"panel_Python_pipe_trans_tutorial-steptrans_watch"}]},{"codetabsId":"pipe_trans_tutorial-stepwatch_conv_method","description":"Optimistic locking with convenience method: Use the transaction() helper to simplify watched transaction patterns","difficulty":"intermediate","id":"watch_conv_method","languages":[{"clientId":"redis-py","clientName":"redis-py","id":"Python","langId":"python","panelId":"panel_Python_pipe_trans_tutorial-stepwatch_conv_method"}]}] } ``` ## Code Examples Legend The code examples below show how to perform the same operations in different programming languages and client libraries: - **Redis CLI**: Command-line interface for Redis - **C# (Synchronous)**: StackExchange.Redis synchronous client - **C# (Asynchronous)**: StackExchange.Redis asynchronous client - **Go**: go-redis client - **Java (Synchronous - Jedis)**: Jedis synchronous client - **Java (Asynchronous - Lettuce)**: Lettuce asynchronous client - **Java (Reactive - Lettuce)**: Lettuce reactive/streaming client - **JavaScript (Node.js)**: node-redis client - **PHP**: Predis client - **Python**: redis-py client - **Rust (Synchronous)**: redis-rs synchronous client - **Rust (Asynchronous)**: redis-rs asynchronous client Each code example demonstrates the same basic operation across different languages. The specific syntax and patterns vary based on the language and client library, but the underlying Redis commands and behavior remain consistent. --- Redis lets you send a sequence of commands to the server together in a batch. There are two types of batch that you can use: - **Pipelines** avoid network and processing overhead by sending several commands to the server together in a single communication. The server then sends back a single communication with all the responses. See the [Pipelining](https://redis.io/docs/latest/develop/using-commands/pipelining) page for more information. - **Transactions** guarantee that all the included commands will execute to completion without being interrupted by commands from other clients. See the [Transactions](https://redis.io/docs/latest/develop/using-commands/transactions) page for more information. ## Execute a pipeline To execute commands in a pipeline, you first create a pipeline object and then add commands to it using methods that resemble the standard command methods (for example, `set()` and `get()`). The commands are buffered in the pipeline and only execute when you call the `execute()` method on the pipeline object. This method returns a list that contains the results from all the commands in order. Note that the command methods for a pipeline always return the original pipeline object, so you can "chain" several commands together, as the example below shows: Foundational: Use pipelines to batch multiple commands together and reduce network round trips **Difficulty:** Beginner **Available in:** Python ##### Python ```python r = redis.Redis(decode_responses=True) pipe = r.pipeline() for i in range(5): pipe.set(f"seat:{i}", f"#{i}") set_5_result = pipe.execute() print(set_5_result) # >>> [True, True, True, True, True] pipe = r.pipeline() # "Chain" pipeline commands together. get_3_result = pipe.get("seat:0").get("seat:3").get("seat:4").execute() print(get_3_result) # >>> ['#0', '#3', '#4'] ``` ## Execute a transaction A pipeline actually executes as a transaction by default (that is to say, all commands are executed in an uninterrupted sequence). However, if you need to switch this behavior off, you can set the `transaction` parameter to `False` when you create the pipeline: ```python pipe = r.pipeline(transaction=False) ``` ## Watch keys for changes Redis supports *optimistic locking* to avoid inconsistent updates to different keys. The basic idea is to watch for changes to any keys that you use in a transaction while you are processing the updates. If the watched keys do change, you must restart the updates with the latest data from the keys. See [Transactions](https://redis.io/docs/latest/develop/using-commands/transactions) for more information about optimistic locking. The example below shows how to repeatedly attempt a transaction with a watched key until it succeeds. The code reads a string that represents a `PATH` variable for a command shell, then appends a new command path to the string before attempting to write it back. If the watched key is modified by another client before writing, the transaction aborts with a `WatchError` exception, and the loop executes again for another attempt. Otherwise, the loop terminates successfully. Optimistic locking: Use WATCH to monitor keys for changes and retry transactions when conflicts occur **Difficulty:** Intermediate **Available in:** Python ##### Python ```python r.set("shellpath", "/usr/syscmds/") with r.pipeline() as pipe: # Repeat until successful. while True: try: # Watch the key we are about to change. pipe.watch("shellpath") # The pipeline executes commands directly (instead of # buffering them) from immediately after the `watch()` # call until we begin the transaction. current_path = pipe.get("shellpath") new_path = current_path + ":/usr/mycmds/" # Start the transaction, which will enable buffering # again for the remaining commands. pipe.multi() pipe.set("shellpath", new_path) pipe.execute() # The transaction succeeded, so break out of the loop. break except redis.WatchError: # The transaction failed, so continue with the next attempt. continue get_path_result = r.get("shellpath") print(get_path_result) # >>> '/usr/syscmds/:/usr/mycmds/' ``` Because this is a common pattern, the library includes a convenience method called `transaction()` that handles the code to watch keys, execute the transaction, and retry if necessary. Pass `transaction()` a function that implements your main transaction code, and also pass the keys you want to watch. The example below implements the same basic transaction as the previous example but this time using `transaction()`. Note that `transaction()` can't add the `multi()` call automatically, so you must still place this correctly in your transaction function. Optimistic locking with convenience method: Use the transaction() helper to simplify watched transaction patterns **Difficulty:** Intermediate **Available in:** Python ##### Python ```python r.set("shellpath", "/usr/syscmds/") def watched_sequence(pipe): current_path = pipe.get("shellpath") new_path = current_path + ":/usr/mycmds/" pipe.multi() pipe.set("shellpath", new_path) trans_result = r.transaction(watched_sequence, "shellpath") print(trans_result) # True get_path_result = r.get("shellpath") print(get_path_result) # >>> '/usr/syscmds/:/usr/mycmds/' ```