{ "schema_version": 2, "id": "operate/oss_and_stack/management/optimization/memory-optimization", "title": "Memory optimization", "url": "https://redis.io/docs/latest/operate/oss_and_stack/management/optimization/memory-optimization/", "summary": "Strategies for optimizing memory usage in Redis", "aliases": [ "/manual/optimization/memory-optimization/", "/management/optimization/memory-optimization/" ], "tags": [ "docs", "operate", "stack", "oss" ], "last_updated": "2026-09-21T15:59:03+01:00", "page_type": "content", "content_hash": "12df8219c72ae670fdae12119c9dba744e8c497e25c4929c441e7ccaeb4313cd", "sections": [ { "id": "special-encoding-of-small-aggregate-data-types", "title": "Special encoding of small aggregate data types", "role": "overview", "text": "Since Redis 2.2 many data types are optimized to use less space up to a certain size.\nHashes, Lists, Sets composed of just integers, and Sorted Sets, when smaller than a given number of elements, and up to a maximum element size, are encoded in a very memory-efficient way that uses *up to 10 times less memory* (with 5 times less memory used being the average saving).\n\nThis is completely transparent from the point of view of the user and API.\nSince this is a CPU / memory tradeoff it is possible to tune the maximum \nnumber of elements and maximum element size for special encoded types \nusing the following redis.conf directives (defaults are shown):" }, { "id": "redis--62", "title": "Redis <= 6.2", "role": "content", "text": "[code example]" }, { "id": "redis--70", "title": "Redis >= 7.0", "role": "content", "text": "[code example]" }, { "id": "redis--72", "title": "Redis >= 7.2", "role": "content", "text": "The following directives are also available:\n\n[code example]\n\nIf a specially encoded value overflows the configured max size,\nRedis will automatically convert it into normal encoding.\nThis operation is very fast for small values,\nbut if you change the setting in order to use specially encoded values\nfor much larger aggregate types the suggestion is to run some \nbenchmarks and tests to check the conversion time." }, { "id": "using-32-bit-instances", "title": "Using 32-bit instances", "role": "content", "text": "When Redis is compiled as a 32-bit target, it uses a lot less memory per key, since pointers are small,\nbut such an instance will be limited to 4 GB of maximum memory usage.\nTo compile Redis as 32-bit binary use *make 32bit*.\nRDB and AOF files are compatible between 32-bit and 64-bit instances\n(and between little and big endian of course) so you can switch from 32 to 64-bit, or the contrary, without problems." }, { "id": "bit-and-byte-level-operations", "title": "Bit and byte level operations", "role": "content", "text": "Redis 2.2 introduced new bit and byte level operations: [`GETRANGE`](https://redis.io/docs/latest/commands/getrange), [`SETRANGE`](https://redis.io/docs/latest/commands/setrange), [`GETBIT`](https://redis.io/docs/latest/commands/getbit) and [`SETBIT`](https://redis.io/docs/latest/commands/setbit).\nUsing these commands you can treat the Redis string type as a random access array.\nFor instance, if you have an application where users are identified by a unique progressive integer number,\nyou can use a bitmap to save information about the subscription of users in a mailing list,\nsetting the bit for subscribed and clearing it for unsubscribed, or the other way around.\nWith 100 million users this data will take just 12 megabytes of RAM in a Redis instance.\nYou can do the same using [`GETRANGE`](https://redis.io/docs/latest/commands/getrange) and [`SETRANGE`](https://redis.io/docs/latest/commands/setrange) to store one byte of information for each user.\nThis is just an example but it is possible to model several problems in very little space with these new primitives." }, { "id": "use-hashes-when-possible", "title": "Use hashes when possible", "role": "content", "text": "Small hashes are encoded in a very small space, so you should try representing your data using hashes whenever possible.\nFor instance, if you have objects representing users in a web application, \ninstead of using different keys for name, surname, email, password, use a single hash with all the required fields.\n\nIf you want to know more about this, read the next section." }, { "id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis", "title": "Using hashes to abstract a very memory-efficient plain key-value store on top of Redis", "role": "content", "text": "I understand the title of this section is a bit scary, but I'm going to explain in detail what this is about.\n\nBasically it is possible to model a plain key-value store using Redis\nwhere values can just be just strings, which is not just more memory efficient\nthan Redis plain keys but also much more memory efficient than memcached.\n\nLet's start with some facts: a few keys use a lot more memory than a single key\ncontaining a hash with a few fields. How is this possible? We use a trick.\nIn theory to guarantee that we perform lookups in constant time\n(also known as O(1) in big O notation) there is the need to use a data structure\nwith a constant time complexity in the average case, like a hash table.\n\nBut many times hashes contain just a few fields. When hashes are small we can\ninstead just encode them in an O(N) data structure, like a linear\narray with length-prefixed key-value pairs. Since we do this only when N\nis small, the amortized time for [`HGET`](https://redis.io/docs/latest/commands/hget) and [`HSET`](https://redis.io/docs/latest/commands/hset) commands is still O(1): the\nhash will be converted into a real hash table as soon as the number of elements\nit contains grows too large (you can configure the limit in redis.conf).\n\nThis does not only work well from the point of view of time complexity, but\nalso from the point of view of constant times since a linear array of key-value pairs happens to play very well with the CPU cache (it has a better\ncache locality than a hash table).\n\nHowever since hash fields and values are not (always) represented as full-featured Redis objects, hash fields can't have an associated time to live\n(expire) like a real key, and can only contain a string. But we are okay with\nthis, this was the intention anyway when the hash data type API was\ndesigned (we trust simplicity more than features, so nested data structures\nare not allowed, as expires of single fields are not allowed).\n\nSo hashes are memory efficient. This is useful when using hashes\nto represent objects or to model other problems when there are group of\nrelated fields. But what about if we have a plain key value business?\n\nImagine we want to use Redis as a cache for many small objects, which can be JSON encoded objects, small HTML fragments, simple key -> boolean values\nand so forth. Basically, anything is a string -> string map with small keys\nand values.\n\nNow let's assume the objects we want to cache are numbered, like:\n\n * object:102393\n * object:1234\n * object:5\n\nThis is what we can do. Every time we perform a\nSET operation to set a new value, we actually split the key into two parts,\none part used as a key, and the other part used as the field name for the hash. For instance, the\nobject named \"object:1234\" is actually split into:\n\n* a Key named object:12\n* a Field named 34\n\nSo we use all the characters but the last two for the key, and the final\ntwo characters for the hash field name. To set our key we use the following\ncommand:\n\n[code example]\n\nAs you can see every hash will end up containing 100 fields, which is an optimal compromise between CPU and memory saved.\n\nThere is another important thing to note, with this schema\nevery hash will have more or\nless 100 fields regardless of the number of objects we cached. This is because our objects will always end with a number and not a random string. In some way, the final number can be considered as a form of implicit pre-sharding.\n\nWhat about small numbers? Like object:2? We handle this case using just\n\"object:\" as a key name, and the whole number as the hash field name.\nSo object:2 and object:10 will both end inside the key \"object:\", but one\nas field name \"2\" and one as \"10\".\n\nHow much memory do we save this way?\n\nI used the following Ruby program to test how this works:\n\n[code example]\n\nThis is the result against a 64 bit instance of Redis 2.2:\n\n * USE_OPTIMIZATION set to true: 1.7 MB of used memory\n * USE_OPTIMIZATION set to false; 11 MB of used memory\n\nThis is an order of magnitude, I think this makes Redis more or less the most\nmemory efficient plain key value store out there.\n\n*WARNING*: for this to work, make sure that in your redis.conf you have\nsomething like this:\n\n[code example]\n\nAlso remember to set the following field accordingly to the maximum size\nof your keys and values:\n\n[code example]\n\nEvery time a hash exceeds the number of elements or element size specified\nit will be converted into a real hash table, and the memory saving will be lost.\n\nYou may ask, why don't you do this implicitly in the normal key space so that\nI don't have to care? There are two reasons: one is that we tend to make\ntradeoffs explicit, and this is a clear tradeoff between many things: CPU,\nmemory, and max element size. The second is that the top-level key space must\nsupport a lot of interesting things like expires, LRU data, and so\nforth so it is not practical to do this in a general way.\n\nBut the Redis Way is that the user must understand how things work so that he can pick the best compromise and to understand how the system will\nbehave exactly." }, { "id": "memory-allocation", "title": "Memory allocation", "role": "content", "text": "To store user keys, Redis allocates at most as much memory as the `maxmemory`\nsetting enables (however there are small extra allocations possible).\n\nThe exact value can be set in the configuration file or set later via\n[`CONFIG SET`](https://redis.io/docs/latest/commands/config-set) (for more info, see [Using memory as an LRU cache](https://redis.io/docs/latest/develop/reference/eviction)).\nThere are a few things that should be noted about how Redis manages memory:\n\n* Redis will not always free up (return) memory to the OS when keys are removed.\nThis is not something special about Redis, but it is how most malloc() implementations work.\nFor example, if you fill an instance with 5GB worth of data, and then\nremove the equivalent of 2GB of data, the Resident Set Size (also known as\nthe RSS, which is the number of memory pages consumed by the process)\nwill probably still be around 5GB, even if Redis will claim that the user\nmemory is around 3GB. This happens because the underlying allocator can't easily release the memory.\nFor example, often most of the removed keys were allocated on the same pages as the other keys that still exist.\n* The previous point means that you need to provision memory based on your\n**peak memory usage**. If your workload from time to time requires 10GB, even if\nmost of the time 5GB could do, you need to provision for 10GB.\n* However allocators are smart and are able to reuse free chunks of memory,\nso after you free 2GB of your 5GB data set, when you start adding more keys\nagain, you'll see the RSS (Resident Set Size) stay steady and not grow\nmore, as you add up to 2GB of additional keys. The allocator is basically\ntrying to reuse the 2GB of memory previously (logically) freed.\n* Because of all this, the fragmentation ratio is not reliable when you\nhad a memory usage that at the peak is much larger than the currently used memory.\nThe fragmentation is calculated as the physical memory actually used (the RSS\nvalue) divided by the amount of memory currently in use (as the sum of all\nthe allocations performed by Redis). Because the RSS reflects the peak memory,\nwhen the (virtually) used memory is low since a lot of keys/values were freed, but the RSS is high, the ratio `RSS / mem_used` will be very high.\n\nIf `maxmemory` is not set Redis will keep allocating memory as it sees\nfit and thus it can (gradually) eat up all your free memory.\nTherefore it is generally advisable to configure some limits. You may also\nwant to set `maxmemory-policy` to `noeviction` (which is *not* the default\nvalue in some older versions of Redis).\n\nIt makes Redis return an out-of-memory error for write commands if and when it reaches the \nlimit - which in turn may result in errors in the application but will not render the \nwhole machine dead because of memory starvation." } ], "examples": [ { "id": "redis--62-ex0", "language": "plaintext", "code": "hash-max-ziplist-entries 512\nhash-max-ziplist-value 64\nzset-max-ziplist-entries 128 \nzset-max-ziplist-value 64\nset-max-intset-entries 512", "section_id": "redis--62" }, { "id": "redis--70-ex0", "language": "plaintext", "code": "hash-max-listpack-entries 512\nhash-max-listpack-value 64\nzset-max-listpack-entries 128\nzset-max-listpack-value 64\nset-max-intset-entries 512", "section_id": "redis--70" }, { "id": "redis--72-ex0", "language": "plaintext", "code": "set-max-listpack-entries 128\nset-max-listpack-value 64", "section_id": "redis--72" }, { "id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis-ex0", "language": "plaintext", "code": "HSET object:12 34 somevalue", "section_id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis" }, { "id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis-ex1", "language": "ruby", "code": "require 'rubygems'\nrequire 'redis'\n\nUSE_OPTIMIZATION = true\n\ndef hash_get_key_field(key)\n s = key.split(':')\n if s[1].length > 2\n { key: s[0] + ':' + s[1][0..-3], field: s[1][-2..-1] }\n else\n { key: s[0] + ':', field: s[1] }\n end\nend\n\ndef hash_set(r, key, value)\n kf = hash_get_key_field(key)\n r.hset(kf[:key], kf[:field], value)\nend\n\ndef hash_get(r, key, value)\n kf = hash_get_key_field(key)\n r.hget(kf[:key], kf[:field], value)\nend\n\nr = Redis.new\n(0..100_000).each do |id|\n key = \"object:#{id}\"\n if USE_OPTIMIZATION\n hash_set(r, key, 'val')\n else\n r.set(key, 'val')\n end\nend", "section_id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis" }, { "id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis-ex2", "language": "plaintext", "code": "hash-max-zipmap-entries 256", "section_id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis" }, { "id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis-ex3", "language": "plaintext", "code": "hash-max-zipmap-value 1024", "section_id": "using-hashes-to-abstract-a-very-memory-efficient-plain-key-value-store-on-top-of-redis" } ] }