Skip to content

Commit

Permalink
Python: Add ZSCAN and HSCAN commands (valkey-io#1732)
Browse files Browse the repository at this point in the history 
* Added zscan command

* Added HSCAN command

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Debug tests

* Debug tests

* Debug tests

* Debug tests

* Debug tests

* Debug tests

* Debug tests

* Debug tests

* Debug tests

* Fixed tests and addressed comments

* Update python/python/tests/test_async_client.py

Co-authored-by: Yury-Fridlyand <[email protected]>

---------

Co-authored-by: Yury-Fridlyand <[email protected]>
  • Loading branch information
@GumpacG @Yury-Fridlyand
GumpacG and Yury-Fridlyand authored Jul 1, 2024
1 parent 6111494 commit 397c641
Show file tree
Hide file tree
Showing 6 changed files with 442 additions and 3 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/python.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
- uses: actions/checkout@v4
with:
submodules: recursive

- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
Expand Down Expand Up @@ -107,7 +107,7 @@ jobs:
# all installed dependencies and build files
source .env/bin/activate
pip install mypy types-protobuf
# Install the benchmark requirements
# Install the benchmark requirements
pip install -r ../benchmarks/python/requirements.txt
python -m mypy ..
Expand Down Expand Up @@ -169,7 +169,7 @@ jobs:
yum -y remove git
yum -y remove git-*
yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm
yum install -y git
yum install -y git
git --version
- uses: actions/checkout@v4
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@
* Python: Added LCS command ([#1716](https://github.com/aws/glide-for-redis/pull/1716))
* Python: Added WAIT command ([#1710](https://github.com/aws/glide-for-redis/pull/1710))
* Python: Added XAUTOCLAIM command ([#1718](https://github.com/aws/glide-for-redis/pull/1718))
* Python: Add ZSCAN and HSCAN commands ([#1732](https://github.com/aws/glide-for-redis/pull/1732))

### Breaking Changes
* Node: Update XREAD to return a Map of Map ([#1494](https://github.com/aws/glide-for-redis/pull/1494))
Expand Down
122 changes: 122 additions & 0 deletions python/python/glide/async_commands/core.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5578,6 +5578,128 @@ async def sscan(
await self._execute_command(RequestType.SScan, args),
)

async def zscan(
self,
key: str,
cursor: str,
match: Optional[str] = None,
count: Optional[int] = None,
) -> List[Union[str, List[str]]]:
"""
Iterates incrementally over a sorted set.
See https://valkey.io/commands/zscan for more details.
Args:
key (str): The key of the sorted set.
cursor (str): The cursor that points to the next iteration of results. A value of "0" indicates the start of
the search.
match (Optional[str]): The match filter is applied to the result of the command and will only include
strings that match the pattern specified. If the sorted set is large enough for scan commands to return
only a subset of the sorted set then there could be a case where the result is empty although there are
items that match the pattern specified. This is due to the default `COUNT` being `10` which indicates
that it will only fetch and match `10` items from the list.
count (Optional[int]): `COUNT` is a just a hint for the command for how many elements to fetch from the
sorted set. `COUNT` could be ignored until the sorted set is large enough for the `SCAN` commands to
represent the results as compact single-allocation packed encoding.
Returns:
List[Union[str, List[str]]]: An `Array` of the `cursor` and the subset of the sorted set held by `key`.
The first element is always the `cursor` for the next iteration of results. `0` will be the `cursor`
returned on the last iteration of the sorted set. The second element is always an `Array` of the subset
of the sorted set held in `key`. The `Array` in the second element is always a flattened series of
`String` pairs, where the value is at even indices and the score is at odd indices.
Examples:
# Assume "key" contains a sorted set with multiple members
>>> result_cursor = "0"
>>> while True:
... result = await redis_client.zscan("key", "0", match="*", count=5)
... new_cursor = str(result [0])
... print("Cursor: ", new_cursor)
... print("Members: ", result[1])
... if new_cursor == "0":
... break
... result_cursor = new_cursor
Cursor: 123
Members: ['value 163', '163', 'value 114', '114', 'value 25', '25', 'value 82', '82', 'value 64', '64']
Cursor: 47
Members: ['value 39', '39', 'value 127', '127', 'value 43', '43', 'value 139', '139', 'value 211', '211']
Cursor: 0
Members: ['value 55', '55', 'value 24', '24', 'value 90', '90', 'value 113', '113']
"""
args = [key, cursor]
if match is not None:
args += ["MATCH", match]
if count is not None:
args += ["COUNT", str(count)]

return cast(
List[Union[str, List[str]]],
await self._execute_command(RequestType.ZScan, args),
)

async def hscan(
self,
key: str,
cursor: str,
match: Optional[str] = None,
count: Optional[int] = None,
) -> List[Union[str, List[str]]]:
"""
Iterates incrementally over a hash.
See https://valkey.io/commands/hscan for more details.
Args:
key (str): The key of the set.
cursor (str): The cursor that points to the next iteration of results. A value of "0" indicates the start of
the search.
match (Optional[str]): The match filter is applied to the result of the command and will only include
strings that match the pattern specified. If the hash is large enough for scan commands to return only a
subset of the hash then there could be a case where the result is empty although there are items that
match the pattern specified. This is due to the default `COUNT` being `10` which indicates that it will
only fetch and match `10` items from the list.
count (Optional[int]): `COUNT` is a just a hint for the command for how many elements to fetch from the hash.
`COUNT` could be ignored until the hash is large enough for the `SCAN` commands to represent the results
as compact single-allocation packed encoding.
Returns:
List[Union[str, List[str]]]: An `Array` of the `cursor` and the subset of the hash held by `key`.
The first element is always the `cursor` for the next iteration of results. `0` will be the `cursor`
returned on the last iteration of the hash. The second element is always an `Array` of the subset of the
hash held in `key`. The `Array` in the second element is always a flattened series of `String` pairs,
where the value is at even indices and the score is at odd indices.
Examples:
# Assume "key" contains a hash with multiple members
>>> result_cursor = "0"
>>> while True:
... result = await redis_client.hscan("key", "0", match="*", count=3)
... new_cursor = str(result [0])
... print("Cursor: ", new_cursor)
... print("Members: ", result[1])
... if new_cursor == "0":
... break
... result_cursor = new_cursor
Cursor: 31
Members: ['field 79', 'value 79', 'field 20', 'value 20', 'field 115', 'value 115']
Cursor: 39
Members: ['field 63', 'value 63', 'field 293', 'value 293', 'field 162', 'value 162']
Cursor: 0
Members: ['field 420', 'value 420', 'field 221', 'value 221']
"""
args = [key, cursor]
if match is not None:
args += ["MATCH", match]
if count is not None:
args += ["COUNT", str(count)]

return cast(
List[Union[str, List[str]]],
await self._execute_command(RequestType.HScan, args),
)

@dataclass
class PubSubMsg:
"""
Expand Down
80 changes: 80 additions & 0 deletions python/python/glide/async_commands/transaction.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4036,6 +4036,86 @@ def sscan(

return self.append_command(RequestType.SScan, args)

def zscan(
self: TTransaction,
key: str,
cursor: str,
match: Optional[str] = None,
count: Optional[int] = None,
) -> TTransaction:
"""
Iterates incrementally over a sorted set.
See https://valkey.io/commands/zscan for more details.
Args:
key (str): The key of the sorted set.
cursor (str): The cursor that points to the next iteration of results. A value of "0" indicates the start of
the search.
match (Optional[str]): The match filter is applied to the result of the command and will only include
strings that match the pattern specified. If the sorted set is large enough for scan commands to return
only a subset of the sorted set then there could be a case where the result is empty although there are
items that match the pattern specified. This is due to the default `COUNT` being `10` which indicates
that it will only fetch and match `10` items from the list.
count (Optional[int]): `COUNT` is a just a hint for the command for how many elements to fetch from the
sorted set. `COUNT` could be ignored until the sorted set is large enough for the `SCAN` commands to
represent the results as compact single-allocation packed encoding.
Returns:
List[Union[str, List[str]]]: An `Array` of the `cursor` and the subset of the sorted set held by `key`.
The first element is always the `cursor` for the next iteration of results. `0` will be the `cursor`
returned on the last iteration of the sorted set. The second element is always an `Array` of the subset
of the sorted set held in `key`. The `Array` in the second element is always a flattened series of
`String` pairs, where the value is at even indices and the score is at odd indices.
"""
args = [key, cursor]
if match is not None:
args += ["MATCH", match]
if count is not None:
args += ["COUNT", str(count)]

return self.append_command(RequestType.ZScan, args)

def hscan(
self: TTransaction,
key: str,
cursor: str,
match: Optional[str] = None,
count: Optional[int] = None,
) -> TTransaction:
"""
Iterates incrementally over a hash.
See https://valkey.io/commands/hscan for more details.
Args:
key (str): The key of the set.
cursor (str): The cursor that points to the next iteration of results. A value of "0" indicates the start of
the search.
match (Optional[str]): The match filter is applied to the result of the command and will only include
strings that match the pattern specified. If the hash is large enough for scan commands to return only a
subset of the hash then there could be a case where the result is empty although there are items that
match the pattern specified. This is due to the default `COUNT` being `10` which indicates that it will
only fetch and match `10` items from the list.
count (Optional[int]): `COUNT` is a just a hint for the command for how many elements to fetch from the hash.
`COUNT` could be ignored until the hash is large enough for the `SCAN` commands to represent the results
as compact single-allocation packed encoding.
Returns:
List[Union[str, List[str]]]: An `Array` of the `cursor` and the subset of the hash held by `key`.
The first element is always the `cursor` for the next iteration of results. `0` will be the `cursor`
returned on the last iteration of the hash. The second element is always an `Array` of the subset of the
hash held in `key`. The `Array` in the second element is always a flattened series of `String` pairs,
where the value is at even indices and the score is at odd indices.
"""
args = [key, cursor]
if match is not None:
args += ["MATCH", match]
if count is not None:
args += ["COUNT", str(count)]

return self.append_command(RequestType.HScan, args)

def lcs(
self: TTransaction,
key1: str,
Expand Down
Loading

0 comments on commit 397c641

Please sign in to comment.