# Developer notes

```json metadata
{
  "schema_version": 2,
  "title": "Developer notes",
  "description": "Notes on RediSearch debugging, testing, and documentation",
  "categories": ["docs","develop","stack","oss","rs","rc","oss","kubernetes","clients"],
  "tableOfContents": {"sections":[{"id":"cloning-the-git-repository","title":"Cloning the git repository"},{"id":"working-in-an-isolated-environment","title":"Working in an isolated environment"},{"id":"installing-prerequisites","title":"Installing prerequisites"},{"id":"installing-redis","title":"Installing Redis"},{"id":"getting-help","title":"Getting help"},{"children":[{"id":"diagnosing-the-build-process","title":"Diagnosing the build process"}],"id":"building-from-source","title":"Building from source"},{"id":"running-redis-with-redisearch","title":"Running Redis with RediSearch"},{"id":"running-tests","title":"Running tests"},{"id":"debugging","title":"Debugging"}]}

,
  "codeExamples": []
}
```


Developing RediSearch features involves setting up a development environment (which can be either Linux-based or macOS-based), building the module, running tests and benchmarks, and debugging both the module and its tests.

## Cloning the git repository

Run the following command to clone the RediSearch module and its submodules:

```sh
git clone --recursive https://github.com/RediSearch/RediSearch.git
```

## Working in an isolated environment

There are several reasons to develop in an isolated environment, like keeping your workstation clean, and developing for a different Linux distribution.
The most general option for an isolated environment is a virtual machine. It's very easy to set one up using [Vagrant](https://www.vagrantup.com)).
Docker is even more agile, as it offers an almost instant solution:

```
search=$(docker run -d -it -v $PWD:/build debian:bullseye bash)
docker exec -it $search bash
```

Then, from within the container, `cd /build` and go on as usual.

In this mode, all installations remain in the scope of the Docker container.
Upon exiting the container, you can either re-invoke it with the above `docker exec` or commit the state of the container to an image and re-invoke it at a later stage:

```
docker commit $search redisearch1
docker stop $search
search=$(docker run -d -it -v $PWD:/build rediseatch1 bash)
docker exec -it $search bash
```

You can replace `debian:bullseye` with your choice of OS, with the host OS being the best choice allowing you to run the RediSearch binary on your host after it is built.

## Installing prerequisites

To build and test RediSearch you need to install several packages, depending on the underlying OS. The following OSes are supported:
- Ubuntu 18.04
- Ubuntu 20.04
- Ubuntu 22.04
- Debian linux 11
- Rocky linux 8
- Rocky linux 9
- Amazon linux 2
- Mariner 2.0
- MacOS

To install the prerequisites on your system using a setup script, first enter the `RediSearch` directory and then run:

```
cd ./install
./install_script.sh sudo
./install_boost.sh 1.83.0
```

Note that this will install various packages on your system using the native package manager (`sudo` is not required in a Docker environment).

If you prefer to avoid that, you can:

* Review the relevant setup scripts under the `./install` directory and install packages manually.
* Use an isolated environment as explained above.


## Installing Redis
As a rule of thumb, you should run the latest Redis version.

If your OS has a Redis 7.x package, you can install it using the OS package manager.

Otherwise, you can build it from source and install it as described in [redis GitHub page](https://github.com/redis/redis).

## Getting help

```make help``` provides a quick summary of the development features. Following is a partial list that contains  the most common and relevant ones:

```
make fetch         # download and prepare dependant modules

make build          # compile and link
  COORD=1             # build coordinator
  DEBUG=1             # build for debugging
  NO_TESTS=1          # disable unit tests
  WHY=1               # explain CMake decisions (in /tmp/cmake-why)
  FORCE=1             # Force CMake rerun (default)
  CMAKE_ARGS=...      # extra arguments to CMake
  VG=1                # build for Valgrind
  SAN=type            # build with LLVM sanitizer (type=address|memory|leak|thread) 
  SLOW=1              # do not parallelize build (for diagnostics)
  GCC=1               # build with GCC (default unless Sanitizer)
  CLANG=1             # build with CLang
  STATIC_LIBSTDCXX=0  # link libstdc++ dynamically (default: 1)
make parsers       # build parsers code (required after chaging files under query_parser dir)
make clean         # remove build artifacts
  ALL=1              # remove entire artifacts directory

make run           # run redis with RediSearch
  COORD=1            # run three local shards with coordinator (assuming the module was built with coordinator support)
  GDB=1              # invoke using gdb

make test          # run all tests
  COORD=1            # test coordinator
  TEST=name          # run specified test
make pytest        # run python tests (tests/pytests)
  COORD=1            # test coordinator 
  TEST=name          # e.g. TEST=test:testSearch
  RLTEST_ARGS=...    # pass args to RLTest
  REJSON=1|0|get     # also load JSON module (default: 1)
  REJSON_PATH=path   # use JSON module at `path`
  EXT=1              # External (existing) environment
  GDB=1              # RLTest interactive debugging
  VG=1               # use Valgrind
  VG_LEAKS=0         # do not search leaks with Valgrind
  SAN=type           # use LLVM sanitizer (type=address|memory|leak|thread) 
make unit-tests    # run unit tests (C and C++)
  TEST=name          # e.g. TEST=FGCTest.testRemoveLastBlock
make c_tests       # run C tests (from tests/ctests)
make cpp_tests     # run C++ tests (from tests/cpptests)

make callgrind     # produce a call graph
  REDIS_ARGS="args"

make sanbox        # create container with CLang Sanitizer
```

## Building from source

Run the following from the project root dir:

```make build``` will build RediSearch.

`make build COORD=1` will build Redis Open Source RediSearch Coordinator.

`make build STATIC=1` will build as a static library.

Notes:

* Binary files are placed under `bin`, according to platform and build variant.
* RediSearch uses [CMake](https://cmake.org) as its build system. ```make build``` will invoke both CMake and the subsequent make command that's required to complete the build.

Use ```make clean``` to remove build artifacts. ```make clean ALL=1``` will remove the entire `bin` subdirectory.

### Diagnosing the build process

`make build` will build in parallel by default.

For the purposes of build diagnosis, `make build SLOW=1 VERBOSE=1` can be used to examine compilation commands.

## Running Redis with RediSearch

The following will run ```redis``` and load the RediSearch module.

```
make run
```
You can open ```redis-cli``` in another terminal to interact with it.

## Running tests

There are several sets of unit tests:
* C tests, located in ```tests/ctests```, run by ```make c-tests```.
* C++ tests (enabled by GTest), located in ```tests/cpptests```, run by ```make cpp-tests```.
* Python tests (enabled by RLTest), located in ```tests/pytests```, run by ```make pytest```.

You can run all tests by invoking ```make test```.

A single test can be run using the ```TEST``` parameter, e.g., ```make test TEST=regex```.

## Debugging

To build for debugging (enabling symbolic information and disabling optimization), run ```make DEBUG=1```.
You can then use ```make run DEBUG=1``` to invoke ```gdb```.
In addition to the usual way to set breakpoints in ```gdb```, it is possible to use the ```BB``` macro to set a breakpoint inside the RediSearch code. It will only have an effect when running under ```gdb```.

Similarly, Python tests in a single-test mode, you can set a breakpoint by using the ```BB()``` function inside a test.