Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#65] Migrate adapter to Nebulex V3 #66

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .formatter.exs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ locals_without_parens = [
]

[
import_deps: [:nebulex],
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"],
line_length: 100,
locals_without_parens: locals_without_parens,
Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
jobs:
nebulex_test:
name: >-
NebulexRedisAdapter Test (Elixir ${{ matrix.elixir }} / OTP ${{ matrix.otp }} /
Nebulex.Adapters.Redis Test (Elixir ${{ matrix.elixir }} / OTP ${{ matrix.otp }} /
OS ${{ matrix.os }})
runs-on: ${{ matrix.os }}

Expand All @@ -33,9 +33,6 @@ jobs:
- elixir: 1.15.x
otp: 25.x
os: 'ubuntu-latest'
- elixir: 1.14.x
otp: 23.x
os: 'ubuntu-20.04'

env:
GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}'
Expand Down Expand Up @@ -94,13 +91,13 @@ jobs:
- name: Run tests
run: |
epmd -daemon
mix test --trace
mix test --exclude nebulex_test
if: ${{ !matrix.coverage }}

- name: Run tests with coverage
run: |
epmd -daemon
mix coveralls.github
mix coveralls.github --exclude nebulex_test
if: ${{ matrix.coverage }}

- name: Restore PLT Cache
Expand Down
10 changes: 0 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,6 @@ All notable changes to this project will be documented in this file.

This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v2.4.2](https://github.com/cabol/nebulex_redis_adapter/tree/v2.4.2) (2024-11-01)

[Full Changelog](https://github.com/cabol/nebulex_redis_adapter/compare/v2.4.1...v2.4.2)

**Closed issues:**

- `NebulexRedisAdapter.RedisCluster.Keyslot` incorrectly computes slot for
hash tags.
[#64](https://github.com/cabol/nebulex_redis_adapter/issues/64)

## [v2.4.1](https://github.com/cabol/nebulex_redis_adapter/tree/v2.4.1) (2024-09-01)

[Full Changelog](https://github.com/cabol/nebulex_redis_adapter/compare/v2.4.0...v2.4.1)
Expand Down
128 changes: 34 additions & 94 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# NebulexRedisAdapter
# Nebulex.Adapters.Redis
> Nebulex adapter for Redis (including [Redis Cluster][redis_cluster] support).

![CI](https://github.com/cabol/nebulex_redis_adapter/workflows/CI/badge.svg)
Expand All @@ -15,7 +15,7 @@ next sections.
See also [online documentation][nbx_redis_adapter]
and [Redis cache example][nbx_redis_example].

[nbx_redis_adapter]: http://hexdocs.pm/nebulex_redis_adapter/NebulexRedisAdapter.html
[nbx_redis_adapter]: http://hexdocs.pm/nebulex_redis_adapter/Nebulex.Adapters.Redis.html
[nbx_redis_example]: https://github.com/cabol/nebulex_examples/tree/master/redis_cache
[redis_cluster]: https://redis.io/topics/cluster-tutorial

Expand All @@ -26,9 +26,9 @@ Add `:nebulex_redis_adapter` to your list of dependencies in `mix.exs`:
```elixir
defp deps do
[
{:nebulex_redis_adapter, "~> 2.3"},
{:crc, "~> 0.10"}, #=> Needed when using Redis Cluster
{:jchash, "~> 0.1.4"} #=> Needed when using consistent-hashing
{:nebulex_redis_adapter, "~> 3.0"},
{:crc, "~> 0.10"}, #=> Needed when using `:redis_cluster` mode
{:ex_hash_ring, "~> 6.0"} #=> Needed when using `:client_side_cluster` mode
]
end
```
Expand All @@ -38,8 +38,8 @@ needed ones. For example:

* `:crc` - Required when using the adapter in mode `:redis_cluster`.
See [Redis Cluster][redis_cluster].
* `:jchash` - Required if you want to use consistent-hashing when using the
adapter in mode `:client_side_cluster`.
* `:ex_hash_ring` - Required when using the adapter in mode
`:client_side_cluster`.

Then run `mix deps.get` to fetch the dependencies.

Expand All @@ -51,7 +51,7 @@ After installing, we can define our cache to use Redis adapter as follows:
defmodule MyApp.RedisCache do
use Nebulex.Cache,
otp_app: :my_app,
adapter: NebulexRedisAdapter
adapter: Nebulex.Adapters.Redis
end
```

Expand All @@ -69,15 +69,15 @@ config :my_app, MyApp.RedisCache,

Since this adapter is implemented by means of `Redix`, it inherits the same
options, including regular Redis options and connection options as well. For
more information about the options, please check out `NebulexRedisAdapter`
more information about the options, please check out `Nebulex.Adapters.Redis`
module and also [Redix](https://github.com/whatyouhide/redix).

See also [Redis cache example][nbx_redis_example].

## Distributed Caching

There are different ways to support distributed caching when using
**NebulexRedisAdapter**.
**Nebulex.Adapters.Redis**.

### Redis Cluster

Expand All @@ -95,7 +95,7 @@ Then we can define our cache which will use **Redis Cluster**:
defmodule MyApp.RedisClusterCache do
use Nebulex.Cache,
otp_app: :my_app,
adapter: NebulexRedisAdapter
adapter: Nebulex.Adapters.Redis
end
```

Expand Down Expand Up @@ -127,20 +127,20 @@ The pool of connections to the different master nodes is automatically
configured by the adapter once it gets the cluster slots info.

> This one could be the easiest and recommended way for distributed caching
using Redis and **NebulexRedisAdapter**.
using Redis and **Nebulex.Adapters.Redis**.

### Client-side Cluster based on Sharding
### Client-side Cluster

**NebulexRedisAdapter** also brings with a simple client-side cluster
implementation based on Sharding distribution model.
**Nebulex.Adapters.Redis** also brings with a simple client-side cluster
implementation based on sharding distribution model.

We define our cache normally:

```elixir
defmodule MyApp.ClusteredCache do
use Nebulex.Cache,
otp_app: :my_app,
adapter: NebulexRedisAdapter
adapter: Nebulex.Adapters.Redis
end
```

Expand Down Expand Up @@ -182,101 +182,41 @@ config :my_app, MyApp.ClusteredCache,
]
```

By default, the adapter uses `NebulexRedisAdapter.ClientCluster.Keyslot` for the
keyslot. Besides, if `:jchash` is defined as dependency, the adapter will use
consistent-hashing automatically.

> **NOTE:** It is highly recommended to define the `:jchash` dependency
when using the adapter in `:client_side_cluster` mode.

However, you can also provide your own implementation by implementing the
`Nebulex.Adapter.Keyslot` and set it into the `:keyslot` option. For example:

```elixir
defmodule MyApp.ClusteredCache.Keyslot do
use Nebulex.Adapter.Keyslot

@impl true
def hash_slot(key, range) do
# your implementation goes here
end
end
```

And the config:

```elixir
config :my_app, MyApp.ClusteredCache,
# Enable client-side cluster mode
mode: :client_side_cluster,

client_side_cluster: [
# Provided Keyslot implementation
keyslot: MyApp.ClusteredCache.Keyslot,

# Nodes config (each node has its own options)
nodes: [
...
]
]
```

### Using `Nebulex.Adapters.Partitioned`

Another simple option is to use the `Nebulex.Adapters.Partitioned` and set as
local cache the `NebulexRedisAdapter`. The idea here is each Elixir node running
the distributed cache (`Nebulex.Adapters.Partitioned`) will have as local
backend or cache a Redis instance (handled by `NebulexRedisAdapter`).


This example shows how the setup a distributed cache using
`Nebulex.Adapters.Partitioned` and `NebulexRedisAdapter`:

```elixir
defmodule MyApp.DistributedCache do
use Nebulex.Cache,
otp_app: :my_app,
adapter: Nebulex.Adapters.Partitioned,
primary_storage_adapter: NebulexRedisAdapter
end
```

### Using a Redis Proxy

The other option is to use a proxy, like [Envoy proxy][envoy] or
[Twemproxy][twemproxy] on top of Redis. In this case, the proxy does the
distribution work, and from the adparter's side (**NebulexRedisAdapter**),
distribution work, and from the adparter's side (**Nebulex.Adapters.Redis**),
it would be only configuration. Instead of connect the adapter against the
Redis nodes, we connect it against the proxy nodes, this means, in the config,
we setup the pool with the host and port pointing to the proxy.

[envoy]: https://www.envoyproxy.io/
[twemproxy]: https://github.com/twitter/twemproxy

## Running Redis commands and/or pipelines
## Using the adapter as a Redis client

Since `NebulexRedisAdapter` works on top of `Redix` and provides features like
connection pools and "Redis Cluster" support, it may be seen also as a sort of
Redis client, but it is meant to be used mainly with the Nebulex cache API.
However, Redis API is quite extensive and there are a lot of useful commands
we may want to run taking advantage of the `NebulexRedisAdapter` features.
Therefore, the adapter injects two additional/extended functions to the
defined cache: `command!/2` and `pipeline!/2`.
Since the Redis adapter works on top of `Redix` and provides features like
connection pools, "Redis Cluster", etc., it may also work as a Redis client.
The Redis API is quite extensive, and there are many useful commands we may
want to run, leveraging the Redis adapter features. Therefore, the adapter
provides additional functions to do so.

```elixir
iex> MyCache.command!(["LPUSH", "mylist", "world"], key: "mylist")
iex> conn = MyCache.fetch_conn!()
iex> Redix.command!(conn, ["LPUSH", "mylist", "world"])
1
iex> MyCache.command!(["LPUSH", "mylist", "hello"], key: "mylist")
iex> Redix.command!(conn, ["LPUSH", "mylist", "hello"])
2
iex> MyCache.command!(["LRANGE", "mylist", "0", "-1"], key: "mylist")
iex> Redix.command!(conn, ["LRANGE", "mylist", "0", "-1"])
["hello", "world"]

iex> [
iex> conn = MyCache.fetch_conn!(key: "mylist")
iex> Redix.pipeline!(conn, [
...> ["LPUSH", "mylist", "world"],
...> ["LPUSH", "mylist", "hello"],
...> ["LRANGE", "mylist", "0", "-1"]
...> ]
...> |> cache.pipeline!(key: "mylist")
...> ])
[1, 2, ["hello", "world"]]
```

Expand All @@ -287,8 +227,8 @@ you have to pass the cache name explicitly.

## Testing

To run the **NebulexRedisAdapter** tests you will have to have Redis running
locally. **NebulexRedisAdapter** requires a complex setup for running tests
To run the **Nebulex.Adapters.Redis** tests you will have to have Redis running
locally. **Nebulex.Adapters.Redis** requires a complex setup for running tests
(since it needs a few instances running, for standalone, cluster and Redis
Cluster). For this reason, there is a [docker-compose.yml](docker-compose.yml)
file in the repo so that you can use [Docker][docker] and
Expand All @@ -302,7 +242,7 @@ $ docker-compose up
[docker]: https://www.docker.com/
[docker_compose]: https://docs.docker.com/compose/

Since `NebulexRedisAdapter` uses the support modules and shared tests
Since `Nebulex.Adapters.Redis` uses the support modules and shared tests
from `Nebulex` and by default its test folder is not included in the Hex
dependency, the following steps are required for running the tests.

Expand Down Expand Up @@ -373,4 +313,4 @@ all checks run successfully.

Copyright (c) 2018, Carlos Bolaños.

NebulexRedisAdapter source code is licensed under the [MIT License](LICENSE).
Nebulex.Adapters.Redis source code is licensed under the [MIT License](LICENSE).
10 changes: 5 additions & 5 deletions config/test.exs
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
import Config

# Standalone mode
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.Standalone,
config :nebulex_redis_adapter, Nebulex.Adapters.Redis.TestCache.Standalone,
conn_opts: [
host: "127.0.0.1",
port: 6379
]

# Cluster mode
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.ClientCluster,
# Client-side cluster mode
config :nebulex_redis_adapter, Nebulex.Adapters.Redis.TestCache.ClientSideCluster,
mode: :client_side_cluster,
client_side_cluster: [
nodes: [
Expand All @@ -34,7 +34,7 @@ config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.ClientCluster,
]

# Redis Cluster mode (with Redis >= 7)
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.RedisCluster,
config :nebulex_redis_adapter, Nebulex.Adapters.Redis.TestCache.RedisCluster,
mode: :redis_cluster,
redis_cluster: [
# Configuration endpoints
Expand All @@ -51,7 +51,7 @@ config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.RedisCluster,
]

# Redis Cluster mode with errors
config :nebulex_redis_adapter, NebulexRedisAdapter.TestCache.RedisClusterConnError,
config :nebulex_redis_adapter, Nebulex.Adapters.Redis.TestCache.RedisClusterConnError,
mode: :redis_cluster,
pool_size: 2,
redis_cluster: [
Expand Down
8 changes: 5 additions & 3 deletions coveralls.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
{
"skip_files": [
"lib/nebulex_redis_adapter/exceptions.ex",
"lib/nebulex_redis_adapter/helpers.ex",
"lib/nebulex_redis_adapter/serializer.ex",
"lib/nebulex/adapters/redis/helpers.ex",
"lib/nebulex/adapters/redis/serializer.ex",
"lib/nebulex/adapters/redis/serializer/serializable.ex",
"lib/nebulex/adapters/redis/options.ex",
"lib/nebulex/adapters/redis/error_formatter.ex",
"test/*"
],
"coverage_options": {
Expand Down
Loading
Loading