{ "schema_version": 2, "id": "develop/ai/search-and-query/administration/extensions", "title": "Extend existing search and query features", "url": "https://redis.io/docs/latest/develop/ai/search-and-query/administration/extensions/", "summary": "Details about extensions for query expanders and scoring functions", "aliases": [ "/develop/interact/search-and-query/administration/extensions", "/interact/search-and-query/administration/extensions/" ], "tags": [ "docs", "develop", "stack", "oss", "rs", "rc", "oss", "kubernetes", "clients" ], "last_updated": "2026-04-07T16:24:18+01:00", "page_type": "content", "content_hash": "956322f18e40a8fdd648932df2cb1a49aef311c1357c17b44dd1697dd9696299", "sections": [ { "id": "overview", "title": "Overview", "role": "overview", "text": "RediSearch supports an extension mechanism, much like Redis supports modules. The API is very minimal at the moment, and it does not yet support dynamic loading of extensions on a running server. Instead, extensions must be written in C (or a language that has an interface with C) and compiled into dynamic libraries that can be loaded at start up.\n\nThere are two kinds of extension APIs at the moment: \n\n1. **Query expanders**, whose role is to expand query tokens (i.e., stemmers).\n2. **Scoring functions**, whose role is to rank search results at query time." }, { "id": "registering-and-loading-extensions", "title": "Registering and loading extensions", "role": "content", "text": "Extensions should be compiled into dynamic library files (e.g., `.so` files), and loaded into the RediSearch module during initialization." }, { "id": "compiling", "title": "Compiling", "role": "content", "text": "Extensions should be compiled and linked as dynamic libraries. An example Makefile for an extension [can be found here](https://github.com/RediSearch/RediSearch/blob/master/tests/ctests/ext-example/Makefile). \n\n That folder also contains an example extension that is used for testing and can be taken as a skeleton for implementing your own extension." }, { "id": "loading", "title": "Loading", "role": "content", "text": "Loading an extension is done by appending `EXTLOAD {path/to/ext.so}` after the `loadmodule` configuration directive when loading the RediSearch module. For example:\n\n [code example]\n\n This causes the RediSearch module to automatically load the extension and register its expanders and scorers." }, { "id": "initializing-an-extension", "title": "Initializing an extension", "role": "content", "text": "The entry point of an extension is a function with the signature:\n\n[code example]\n\nWhen loading an extension, RediSearch looks for this function and calls it. This function is responsible for registering and initializing the expanders and scorers. \n\nIt should return REDISEARCH_ERR on error or REDISEARCH_OK on success." }, { "id": "example-init-function", "title": "Example init function", "role": "example", "text": "[code example]" }, { "id": "calling-your-custom-functions", "title": "Calling your custom functions", "role": "content", "text": "When performing a query, you can use your scorers or expanders by specifying the SCORER or EXPANDER arguments with the given alias. For example:\n\n[code example]\n\n**NOTE**: Expander and scorer aliases are **case sensitive**." }, { "id": "the-query-expander-api", "title": "The query expander API", "role": "content", "text": "Only basic query expansion is supported, one token at a time. An expander can decide to expand any given token with as many tokens it wishes, which will be union-merged in query time.\n\nThe API for an expander is the following:\n\n[code example]" }, { "id": "rsqueryexpanderctx", "title": "RSQueryExpanderCtx", "role": "content", "text": "`RSQueryExpanderCtx` is a context that contains private data of the extension, and a callback method to expand the query. It is defined as:\n\n[code example]" }, { "id": "rstoken", "title": "RSToken", "role": "content", "text": "`RSToken` represents a single query token to be expanded, and is defined as:\n\n[code example]" }, { "id": "the-scoring-function-api", "title": "The scoring function API", "role": "content", "text": "For the final ranking, the scoring function analyzes each document retrieved by the query, taking into account not only the terms that triggered the document's retrieval but also metadata like its prior score, length, and so on.\n\nSince the scoring function is evaluated for each document, potentially millions of times, and since\nredis is single threaded, it is important that it works as fast as possible and be heavily optimized. \n\nA scoring function is applied to each potential result for each document and is implemented with the following signature:\n\n[code example]\n\n`RSScoringFunctionCtx` is a context that implements some helper methods. \n\n`RSIndexResult` is the result information containing the document id, frequency, terms, and offsets. \n\n`RSDocumentMetadata` is an object holding global information about the document, such as its presumptive score. \n\n`minScore` is the minimal score that will yield a result that is relevant to the search. It can be used to stop processing midway or before or even before it starts.\n\nThe return value of the function is a `double` representing the final score of the result. \nReturning 0 causes the result to be counted, but if there are results with a score greater than 0, they will appear above it. \nTo completely filter out a result and not count it in the totals, the scorer should return the special value `RS_SCORE_FILTEROUT`, which is internally set to negative infinity, or -1/0." }, { "id": "rsscoringfunctionctx", "title": "RSScoringFunctionCtx", "role": "content", "text": "This is an object containing the following members:\n\n* `void *privdata`: a pointer to an object set by the extension on initialization time.\n* `RSPayload payload*`: A Payload object set either by the query expander or the client.\n* `int GetSlop(RSIndexResult *res)*`: A callback method that yields the total minimal distance between the query terms. This can be used to prefer results where the slop is smaller and the terms are nearer to each other." }, { "id": "rsindexresult", "title": "RSIndexResult", "role": "content", "text": "This is an object holding the information about the current result in the index, which is an aggregate of all the terms that resulted in the current document being considered a valid result. See `redisearch.h` for details." }, { "id": "rsdocumentmetadata", "title": "RSDocumentMetadata", "role": "content", "text": "This is an object describing global information, unrelated to the current query, about the document being evaluated by the scoring function." }, { "id": "example-query-expander", "title": "Example query expander", "role": "example", "text": "This example query expander expands each token with the term foo:\n\n[code example]" }, { "id": "example-scoring-function", "title": "Example scoring function", "role": "example", "text": "This is an actual scoring function, which calculates TF-IDF for the document, multiplies it by the document score, and divides it by the slop:\n\n[code example]" } ], "examples": [ { "id": "loading-ex0", "language": "sh", "code": "$ redis-server --loadmodule ./redisearch.so EXTLOAD ./ext/my_extension.so", "section_id": "loading" }, { "id": "initializing-an-extension-ex0", "language": "c", "code": "int RS_ExtensionInit(RSExtensionCtx *ctx);", "section_id": "initializing-an-extension" }, { "id": "example-init-function-ex0", "language": "c", "code": "#include //must be in the include path\n\nint RS_ExtensionInit(RSExtensionCtx *ctx) {\n\n /* Register a scoring function with an alias my_scorer and no special private data and free function */\n if (ctx->RegisterScoringFunction(\"my_scorer\", MyCustomScorer, NULL, NULL) == REDISEARCH_ERR) {\n return REDISEARCH_ERR;\n }\n\n /* Register a query expander */\n if (ctx->RegisterQueryExpander(\"my_expander\", MyExpander, NULL, NULL) ==\n REDISEARCH_ERR) {\n return REDISEARCH_ERR;\n }\n\n return REDISEARCH_OK;\n}", "section_id": "example-init-function" }, { "id": "calling-your-custom-functions-ex0", "language": "plaintext", "code": "FT.SEARCH my_index \"foo bar\" EXPANDER my_expander SCORER my_scorer", "section_id": "calling-your-custom-functions" }, { "id": "the-query-expander-api-ex0", "language": "c", "code": "#include //must be in the include path\n\nvoid MyQueryExpander(RSQueryExpanderCtx *ctx, RSToken *token) {\n ...\n}", "section_id": "the-query-expander-api" }, { "id": "rsqueryexpanderctx-ex0", "language": "c", "code": "typedef struct RSQueryExpanderCtx {\n\n /* Opaque query object used internally by the engine, and should not be accessed */\n struct RSQuery *query;\n\n /* Opaque query node object used internally by the engine, and should not be accessed */\n struct RSQueryNode **currentNode;\n\n /* Private data of the extension, set on extension initialization */\n void *privdata;\n\n /* The language of the query, defaults to \"english\" */\n const char *language;\n\n /* ExpandToken allows the user to add an expansion of the token in the query, that will be\n * union-merged with the given token in query time. str is the expanded string, len is its length,\n * and flags is a 32 bit flag mask that can be used by the extension to set private information on\n * the token */\n void (*ExpandToken)(struct RSQueryExpanderCtx *ctx, const char *str, size_t len,\n RSTokenFlags flags);\n\n /* SetPayload allows the query expander to set GLOBAL payload on the query (not unique per token)\n */\n void (*SetPayload)(struct RSQueryExpanderCtx *ctx, RSPayload payload);\n\n} RSQueryExpanderCtx;", "section_id": "rsqueryexpanderctx" }, { "id": "rstoken-ex0", "language": "c", "code": "/* A token in the query. The expanders receive query tokens and can expand the query with more query\n * tokens */\ntypedef struct {\n /* The token string - which may or may not be NULL terminated */\n const char *str;\n /* The token length */\n size_t len;\n \n /* 1 if the token is the result of query expansion */\n uint8_t expanded:1;\n\n /* Extension specific token flags that can be examined later by the scoring function */\n RSTokenFlags flags;\n} RSToken;", "section_id": "rstoken" }, { "id": "the-scoring-function-api-ex0", "language": "c", "code": "double MyScoringFunction(RSScoringFunctionCtx *ctx, RSIndexResult *res,\n RSDocumentMetadata *dmd, double minScore);", "section_id": "the-scoring-function-api" }, { "id": "example-query-expander-ex0", "language": "c", "code": "#include //must be in the include path\n\nvoid DummyExpander(RSQueryExpanderCtx *ctx, RSToken *token) {\n ctx->ExpandToken(ctx, strdup(\"foo\"), strlen(\"foo\"), 0x1337); \n}", "section_id": "example-query-expander" }, { "id": "example-scoring-function-ex0", "language": "c", "code": "#include //must be in the include path\n\ndouble TFIDFScorer(RSScoringFunctionCtx *ctx, RSIndexResult *h, RSDocumentMetadata *dmd,\n double minScore) {\n // no need to evaluate documents with score 0 \n if (dmd->score == 0) return 0;\n\n // calculate sum(tf-idf) for each term in the result\n double tfidf = 0;\n for (int i = 0; i < h->numRecords; i++) {\n // take the term frequency and multiply by the term IDF, add that to the total\n tfidf += (float)h->records[i].freq * (h->records[i].term ? h->records[i].term->idf : 0);\n }\n // normalize by the maximal frequency of any term in the document \n tfidf /= (double)dmd->maxFreq;\n\n // multiply by the document score (between 0 and 1)\n tfidf *= dmd->score;\n\n // no need to factor the slop if tfidf is already below minimal score\n if (tfidf < minScore) {\n return 0;\n }\n\n // get the slop and divide the result by it, making sure we prefer results with closer terms\n tfidf /= (double)ctx->GetSlop(h);\n \n return tfidf;\n}", "section_id": "example-scoring-function" } ] }