# Index and query documents ```json metadata { "schema_version": 2, "title": "Index and query documents", "description": "Learn how to use Redis Search with JSON and hash documents.", "categories": ["docs","develop","stack","oss","rs","rc","oss","kubernetes","clients"], "topics": ["Redis Search","JSON","hash"], "relatedPages": ["/develop/clients/redis-py/vecsearch","/develop/ai/search-and-query"], "scope": "example", "tableOfContents": {"sections":[{"id":"initialize","title":"Initialize"},{"id":"create-data","title":"Create data"},{"id":"add-the-index","title":"Add the index"},{"id":"add-the-data","title":"Add the data"},{"id":"query-the-data","title":"Query the data"},{"children":[{"id":"bulk-import-with-himport","title":"Bulk import with HIMPORT"}],"id":"differences-with-hash-documents","title":"Differences with hash documents"},{"id":"more-information","title":"More information"}]} , "codeExamples": [{"buildsUpon":["add_hash_data"],"codetabsId":"py_home_json-stepadd_hash_himport","description":"Bulk ingestion: Import hash documents as compact hashes using HIMPORT PREPARE and a pipeline of HIMPORT SET calls","difficulty":"intermediate","id":"add_hash_himport","languages":[{"clientId":"redis-py","clientName":"redis-py","id":"Python","langId":"python","panelId":"panel_Python_py_home_json-stepadd_hash_himport"}]}] } ``` ## Code Examples Legend The code examples below show how to perform the same operations in different programming languages and client libraries: - **Redis CLI**: Command-line interface for Redis - **C# (Synchronous)**: StackExchange.Redis synchronous client - **C# (Asynchronous)**: StackExchange.Redis asynchronous client - **Go**: go-redis client - **Java (Synchronous - Jedis)**: Jedis synchronous client - **Java (Asynchronous - Lettuce)**: Lettuce asynchronous client - **Java (Reactive - Lettuce)**: Lettuce reactive/streaming client - **JavaScript (Node.js)**: node-redis client - **PHP**: Predis client - **Python**: redis-py client - **Rust (Synchronous)**: redis-rs synchronous client - **Rust (Asynchronous)**: redis-rs asynchronous client Each code example demonstrates the same basic operation across different languages. The specific syntax and patterns vary based on the language and client library, but the underlying Redis commands and behavior remain consistent. --- This example shows how to create a [search index](https://redis.io/docs/latest/develop/ai/search-and-query/indexing) for [JSON](https://redis.io/docs/latest/develop/data-types/json) documents and run queries against the index. It then goes on to show the slight differences in the equivalent code for [hash](https://redis.io/docs/latest/develop/data-types/hashes) documents. > [!NOTE] > From [v6.0.0](https://github.com/redis/redis-py/releases/tag/v6.0.0) onwards, > `redis-py` uses query dialect 2 by default. > Redis Search methods such as [`ft().search()`](https://redis.io/docs/latest/commands/ft.search) > will explicitly request this dialect, overriding the default set for the server. > See > [Query dialects](https://redis.io/docs/latest/develop/ai/search-and-query/advanced-concepts/dialects) > for more information. ## Initialize Make sure that you have [Redis Open Source](https://redis.io/docs/latest/operate/oss_and_stack) or another Redis server available. Also install the [`redis-py`](https://redis.io/docs/latest/develop/clients/redis-py) client library if you haven't already done so. Add the following dependencies. All of them are applicable to both JSON and hash, except for the `Path` class, which is specific to JSON (see [Path](https://redis.io/docs/latest/develop/data-types/json/path) for a description of the JSON path syntax). Foundational: Import required libraries for Redis Search, JSON operations, and search functionality ```python import redis from redis.commands.json.path import Path import redis.commands.search.aggregation as aggregations import redis.commands.search.reducers as reducers from redis.commands.search.field import TextField, NumericField, TagField from redis.commands.search.index_definition import IndexDefinition, IndexType from redis.commands.search.query import Query import redis.exceptions ``` ## Create data Create some test data to add to your database. The example data shown below is compatible with both JSON and hash objects. Foundational: Define sample user data structures for indexing and querying ```python user1 = { "name": "Paul John", "email": "paul.john@example.com", "age": 42, "city": "London" } user2 = { "name": "Eden Zamir", "email": "eden.zamir@example.com", "age": 29, "city": "Tel Aviv" } user3 = { "name": "Paul Zamir", "email": "paul.zamir@example.com", "age": 35, "city": "Tel Aviv" } ``` ## Add the index Connect to your Redis database. The code below shows the most basic connection but see [Connect to the server](https://redis.io/docs/latest/develop/clients/redis-py/connect) to learn more about the available connection options. Foundational: Establish a connection to a Redis server for query operations ```python r = redis.Redis(decode_responses=True) ``` The example uses an index called `idx:users` for JSON documents and adds some JSON documents with the `user:` key prefix. To avoid errors, first delete any existing index or documents whose names that might conflict with the example: Foundational: Clean up existing indexes and documents to prepare for fresh example data ```python try: r.ft("idx:users").dropindex(True) except redis.exceptions.ResponseError: pass r.delete("user:1", "user:2", "user:3") ``` Create an index for the JSON data. The code below specifies that only JSON documents with the key prefix `user:` are indexed. For more information, see [Query syntax](https://redis.io/docs/latest/develop/ai/search-and-query/query). Foundational: Create a search index for JSON documents with field definitions and key prefix filtering ```python schema = ( TextField("$.name", as_name="name"), TagField("$.city", as_name="city"), NumericField("$.age", as_name="age") ) indexCreated = r.ft("idx:users").create_index( schema, definition=IndexDefinition( prefix=["user:"], index_type=IndexType.JSON ) ) ``` ## Add the data Add the three sets of user data to the database as [JSON](https://redis.io/docs/latest/develop/data-types/json) objects. If you use keys with the `user:` prefix then Redis will index the objects automatically as you add them: Foundational: Store JSON documents in Redis with automatic indexing based on key prefix ```python user1Set = r.json().set("user:1", Path.root_path(), user1) user2Set = r.json().set("user:2", Path.root_path(), user2) user3Set = r.json().set("user:3", Path.root_path(), user3) ``` ## Query the data You can now use the index to search the JSON objects. The [query](https://redis.io/docs/latest/develop/ai/search-and-query/query) below searches for objects that have the text "Paul" in any field and have an `age` value in the range 30 to 40: Query with filters: Search JSON documents using text matching and numeric range filters to find specific records ```python findPaulResult = r.ft("idx:users").search( Query("Paul @age:[30 40]") ) print(findPaulResult) # >>> Result{1 total, docs: [Document {'id': 'user:3', ... ``` Specify query options to return only the `city` field: Query with field projection: Retrieve only specific fields from search results to reduce data transfer ```python citiesResult = r.ft("idx:users").search( Query("Paul").return_field("$.city", as_field="city") ).docs print(citiesResult) # >>> [Document {'id': 'user:1', 'payload': None, ... ``` Use an [aggregation query](https://redis.io/docs/latest/develop/ai/search-and-query/query/aggregation) to count all users in each city. Aggregation queries: Use GROUP BY and COUNT operations to summarize and analyze indexed data ```python req = aggregations.AggregateRequest("*").group_by( '@city', reducers.count().alias('count') ) aggResult = r.ft("idx:users").aggregate(req).rows print(aggResult) # >>> [['city', 'London', 'count', '1'], ['city', 'Tel Aviv', 'count', '2']] ``` ## Differences with hash documents Indexing for hash documents is very similar to JSON indexing but you need to specify some slightly different options. When you create the schema for a hash index, you don't need to add aliases for the fields, since you use the basic names to access the fields anyway. Also, you must use `HASH` for the `IndexType` when you create the index. First delete any existing index or documents whose names might conflict with the hash example: Foundational: Clean up existing hash indexes and documents to prepare for fresh example data ```python try: r.ft("hash-idx:users").dropindex(True) except redis.exceptions.ResponseError: pass r.delete("huser:1", "huser:2", "huser:3") ``` Create a new index called `hash-idx:users`, which is otherwise the same as the `idx:users` index used for JSON documents in the previous examples: Foundational: Create a search index for hash documents with HASH index type and field definitions ```python hashSchema = ( TextField("name"), TagField("city"), NumericField("age") ) hashIndexCreated = r.ft("hash-idx:users").create_index( hashSchema, definition=IndexDefinition( prefix=["huser:"], index_type=IndexType.HASH ) ) ``` You use [`hset()`](https://redis.io/docs/latest/commands/hset) to add the hash documents instead of [`json().set()`](https://redis.io/docs/latest/commands/json.set), but the same flat `userX` dictionaries work equally well with either hash or JSON: Foundational: Store hash documents in Redis with automatic indexing based on key prefix ```python huser1Set = r.hset("huser:1", mapping=user1) huser2Set = r.hset("huser:2", mapping=user2) huser3Set = r.hset("huser:3", mapping=user3) ``` The query commands work the same here for hash as they do for JSON (but the name of the hash index is different). The format of the result is almost the same except that the fields are returned directly in the result `Document` object instead of in an enclosing `json` dictionary: Query with filters: Search hash documents using text matching and numeric range filters (same as JSON queries) ```python findPaulHashResult = r.ft("hash-idx:users").search( Query("Paul @age:[30 40]") ) print(findPaulHashResult) # >>> Result{1 total, docs: [Document {'id': 'huser:3', # >>> 'payload': None, 'name': 'Paul Zamir', ... ``` ### Bulk import with HIMPORT From Redis Open Source 8.10, you can also add the hash documents with the [`HIMPORT`](https://redis.io/docs/latest/commands/himport) command family. This creates the same hashes as `hset()`, but lets Redis store them as [compact hashes](https://redis.io/docs/latest/develop/data-types/hashes#compact-hashes), which use less memory when many keys share the same field names. Redis Search indexes these hashes in the same way. Declare the field names once with [`himport_prepare()`](https://redis.io/docs/latest/commands/himport-prepare), then create each document with [`himport_set()`](https://redis.io/docs/latest/commands/himport-set), passing only its values, in the same order as the field names. Use a [pipeline](https://redis.io/docs/latest/develop/clients/redis-py/transpipe) to send all the imports in one round trip: Bulk ingestion: Import hash documents as compact hashes using HIMPORT PREPARE and a pipeline of HIMPORT SET calls **Difficulty:** Intermediate **Builds upon:** add_hash_data **Available in:** Python ##### Python ```python fields = ["name", "email", "age", "city"] r.himport_prepare("huser", fields) with r.pipeline(transaction=False) as pipe: for key, user in [("huser:1", user1), ("huser:2", user2), ("huser:3", user3)]: pipe.himport_set(key, "huser", [user[f] for f in fields]) himportRes = pipe.execute() ``` `himport_set()` replaces any existing key, so running this code after the `hset()` calls leaves you with the same three documents. The query returns the same result, although the fields can appear in a different order. > [!NOTE] > `HIMPORT` requires `redis-py` v8.1.0 or later, and `redis-py` marks its > support as experimental, so the API might change. Redis Cloud and > Redis Software don't support `HIMPORT`. ## More information See the [Redis Search](https://redis.io/docs/latest/develop/ai/search-and-query) docs for a full description of all query features with examples.