Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: limpyd/redis-limpyd
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.2.3
Choose a base ref
...
head repository: limpyd/redis-limpyd
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Dec 16, 2015

  1. Copy the full SHA
    dfd271c View commit details
  2. Bump to version 0.2.4

    twidi committed Dec 16, 2015
    Copy the full SHA
    520ee8e View commit details
  3. Merge pull request #93 from twidi/feature/locking-bug-with-redis-py-2.10

    Resolve a redis-py bug with decode_responses=True
    twidi committed Dec 16, 2015
    Copy the full SHA
    fddad21 View commit details

Commits on Feb 15, 2017

  1. Tell git to ignore more things

    twidi committed Feb 15, 2017
    Copy the full SHA
    67c78c3 View commit details
  2. Copy the full SHA
    476b6f2 View commit details
  3. Copy the full SHA
    f837b02 View commit details
  4. Copy the full SHA
    f5052f7 View commit details
  5. Remove support for python 2.6

    twidi committed Feb 15, 2017
    Copy the full SHA
    a45ea21 View commit details
  6. Copy the full SHA
    af64a9b View commit details
  7. Copy the full SHA
    dcf34f3 View commit details

Commits on Jan 26, 2018

  1. Copy the full SHA
    cb2ac53 View commit details
  2. Allow usage of debugger in collection

    This is done by restoring "state" (_slice and _len_mode) when quitting a
    function.
    When debugging, the debugger my call __len__ to display the actual
    variables and then change one of these values, so the state of the
    current debbuging is broken.
    Note that it's still not thread-safe but still better than before.
    twidi committed Jan 26, 2018
    Copy the full SHA
    ecccf56 View commit details
  3. Copy the full SHA
    f1f71ac View commit details
  4. Copy the full SHA
    c06a2ba View commit details
  5. Copy the full SHA
    9b059b4 View commit details
  6. Copy the full SHA
    2187f84 View commit details
  7. Copy the full SHA
    7a4312e View commit details
  8. ExtendedCollection.intersect can accept list and zset keys

    Before, only sets were allowed (without check, btw)
    twidi committed Jan 26, 2018
    1
    Copy the full SHA
    da4f2d0 View commit details
  9. Copy the full SHA
    3f8561f View commit details
  10. Copy the full SHA
    a9e0eb6 View commit details
  11. Extract all index related stuff into a standalone class

    This class, `EqualIndex` is used by default when the field is indexable.
    It is based on `BaseIndex` that will be the base for other indexes to
    come. Each index handle the indexing, deindexing, and the filtereing (by
    returning a redis key that will then be used by the collection)
    
    The new indexes handle a filter "suffix", for example, for the
    EqualIndex: "None" (no suffix) or "__eq".
    
    When such a suffix is encountered in a filter, the first index saying it
    can handle it will do.
    
    It's the same for uniqueness, each index says if it can or not handle
    uniqueness, and then the first one capable of will check for uniqueness
    (to avoid doing too much work if many indexes say thay can handle it: we
    trust the first one)
    
    For now, and maybe for later, the primary keys are still not indexable
    (they are managed in a special way), but we added the ability to use the
    prefix "__eq" for them too.
    
    For the basic collection, we changed the moment of the index is called
    to get the key: instead of the moment the filter is added, it's now when
    the collection is called. It doesn't change a lot of things for
    EqualIndex, but it may for more complex indexes that may have to do some
    computation, locally or at the redis level.
    
    Note that this commit does not change any existing test: it is 100%
    compatible with existing usage of limpyd
    twidi committed Jan 26, 2018
    1
    Copy the full SHA
    2808b20 View commit details
  12. Extract code to check redis version

    Also rename 'has_scripting' into 'support_scripting'
    twidi committed Jan 26, 2018
    Copy the full SHA
    bd405af View commit details
  13. Add a TextRangeIndex index

    It allows usage of these filter suffixes:
    - None
    - eq
    - lt
    - lte
    - gt
    - gte
    - startwith
    
    It uses the zrangebylex command from redis so it will raise if not
    supported by the current redis version (>=2.8.9)
    twidi committed Jan 26, 2018
    Copy the full SHA
    d62bb2e View commit details
  14. Create indexes instances only when needed

    For this we use a @cached_property decorator from django
    twidi committed Jan 26, 2018
    Copy the full SHA
    f5809cf View commit details
  15. Use lua in redis for range filtering

    Previously, we fetched the result from the edis zrangebylex call,
    extracted the primary keys, then sent them back to redis in a set or
    sorted set.
    The problems are:
    - it could be a lot of data to transfer
    - so a big cost in memory for the caller
    - and it could also be a long time blocking redis to return the data
    
    Now with the script, we simple tell redis to do this in lua: no data is
    transfered to the caller.
    And in the script, the fetch of data via zrangebylex is done by blocks,
    so it won't take a lot of memory
    twidi committed Jan 26, 2018
    Copy the full SHA
    92c641e View commit details
  16. Copy the full SHA
    f88074c View commit details
  17. Extract maximum stuff from TextRangeIndex to BaseRangeIndex

    This `BaseRangeIndex` will then be used for `NumberRangeIndex`
    twidi committed Jan 26, 2018
    Copy the full SHA
    5941127 View commit details
  18. Add a NumberRangeIndex index

    It allows usage of these filter suffixes:
    - None
    - eq
    - lt
    - lte
    - gt
    - gte
    
    It uses the zrangebyscore command from redis, using the value (numbers)
    as score.
    twidi committed Jan 26, 2018
    Copy the full SHA
    08712f9 View commit details
  19. Copy the full SHA
    f78da10 View commit details
  20. Copy the full SHA
    08dade3 View commit details
  21. Copy the full SHA
    693b0ad View commit details
  22. Adds ability to override behavior without subclassing

    Simple with a class method named `override` that can accept a prefix or
    transform.
    
    The prefix will allow to differentiate two same indexes, at the storage
    level and the query level ("myfield__prefix=...")
    
    The transform is a function that will transform the value to be stored.
    For example to store only the year of a date.
    twidi committed Jan 26, 2018
    Copy the full SHA
    ddcd7fe View commit details
  23. Add "multi-indexes" feature in contrib

    This allow to easily compose complexe indexes based on many ones, for
    example a DateTime index allowing filtering on date, time, year,
    month...
    twidi committed Jan 26, 2018
    Copy the full SHA
    7b314cc View commit details
  24. Add DateTime index in contrib

    twidi committed Jan 26, 2018
    Copy the full SHA
    e1f4cde View commit details
  25. Add doc about contrib multi-indexes

    This explains the concept with the example of `DateTimeIndex` using the
    real code used to create this index.
    twidi committed Jan 26, 2018
    Copy the full SHA
    55c2a2e View commit details

Commits on Jan 28, 2018

  1. Add SimpleDateTimeIndex

    twidi committed Jan 28, 2018
    Copy the full SHA
    f70fd28 View commit details

Commits on Jan 29, 2018

  1. Copy the full SHA
    6fb81bd View commit details
  2. Make sort(by='pk') actually work...

    pks are not stored as real fields so using `sort(by='pk')` couldn't work
    as expected.
    Now, calling `sort(by='pk')` (or the real name of the pk field is
    managed manually) is the same as doing `sort()`.
    Of course, `alpha=True` also works as expected
    twidi committed Jan 29, 2018
    Copy the full SHA
    0d69492 View commit details
  3. Add a way to clean/rebuild indexes

    Fixes #99
    twidi committed Jan 29, 2018
    Copy the full SHA
    9d33460 View commit details
  4. Copy the full SHA
    baa6024 View commit details
  5. Small pass on documentation

    twidi committed Jan 29, 2018
    Copy the full SHA
    ff306a9 View commit details
  6. Add support for "in" suffix in indexes

    Fixes #108
    twidi committed Jan 29, 2018
    Copy the full SHA
    1c0482c View commit details
  7. Correct and enhance collection slicing

    Some slicing were not managed at all, retrieving the wrong number of
    data.
    It is now corrected, with also more cases where we try to limit the data
    returned by redis
    To be sure that the data is always correctly returned, in the tests we
    compare the result of slicing a collection to the same slicing of a
    normal python list. Same for indexing (both in `__getitem__`)
    twidi committed Jan 29, 2018
    Copy the full SHA
    0debbfc View commit details
  8. Use set litterals

    Not supported in python 2.6 but now we support only python >=2.7
    twidi committed Jan 29, 2018
    Copy the full SHA
    33f1593 View commit details
  9. Copy the full SHA
    e21a27d View commit details
  10. Copy the full SHA
    64d478f View commit details
  11. Add authors file

    twidi committed Jan 29, 2018
    Copy the full SHA
    f890354 View commit details
  12. Copy the full SHA
    2473c65 View commit details
  13. Copy the full SHA
    0c28d9f View commit details
  14. Bump version to 1.0

    twidi committed Jan 29, 2018
    Copy the full SHA
    87556c5 View commit details
Showing with 10,164 additions and 1,567 deletions.
  1. +10 −1 .gitignore
  2. +7 −9 .travis.yml
  3. +8 −0 AUTHORS.rst
  4. +0 −31 CHANGELOG
  5. +122 −0 CHANGELOG.rst
  6. 0 LICENCE → LICENSE
  7. +1 −2 MANIFEST.in
  8. +22 −11 README.rst
  9. +21 −1 doc/README.md
  10. +29 −7 doc/about.rst
  11. +1 −0 doc/changelog.rst
  12. +311 −68 doc/collections.rst
  13. +9 −5 doc/conf.py
  14. +449 −149 doc/contrib.rst
  15. +46 −7 doc/database.rst
  16. +209 −140 doc/fields.rst
  17. +10 −22 doc/index.rst
  18. +143 −19 doc/models.rst
  19. +21 −5 limpyd/__init__.py
  20. +629 −151 limpyd/collection.py
  21. +229 −211 limpyd/contrib/collection.py
  22. +43 −6 limpyd/contrib/database.py
  23. +1,053 −0 limpyd/contrib/indexes.py
  24. +6 −7 limpyd/contrib/related.py
  25. +99 −19 limpyd/database.py
  26. +1 −0 limpyd/exceptions.py
  27. +543 −194 limpyd/fields.py
  28. +1,591 −0 limpyd/indexes.py
  29. +216 −36 limpyd/model.py
  30. +57 −6 limpyd/utils.py
  31. +0 −4 requirements-2.6.txt
  32. +1 −2 requirements.txt
  33. +22 −15 run_tests.py
  34. +61 −0 setup.cfg
  35. +2 −62 setup.py
  36. +127 −15 tests/base.py
  37. +316 −67 tests/collection.py
  38. +308 −148 tests/contrib/collection.py
  39. +63 −1 tests/contrib/database.py
  40. +1,366 −0 tests/contrib/indexes.py
  41. +49 −49 tests/contrib/related.py
  42. +0 −9 tests/fields/__init__.py
  43. +41 −17 tests/fields/hash.py
  44. +9 −9 tests/fields/instancehash.py
  45. +73 −0 tests/fields/list.py
  46. +12 −12 tests/fields/pk.py
  47. +63 −5 tests/fields/set.py
  48. +213 −13 tests/fields/sortedset.py
  49. +180 −3 tests/fields/string.py
  50. +1,232 −0 tests/indexes.py
  51. +1 −6 tests/lock.py
  52. +130 −14 tests/model.py
  53. +9 −9 tests/utils.py
11 changes: 10 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -9,5 +9,14 @@
*.orig
*~
doc/_build

# build stuff
*.egg-info
dist/*
dist/
build/

# pycharm
.idea/

# auto-venv
.venv
16 changes: 7 additions & 9 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -5,18 +5,16 @@ cache:
- $HOME/.cache/pip

python:
- "2.6"
- "2.7"
- "3.3"
- "3.4"

env:
- REDIS="redis==2.9.1"
- REDIS="redis==2.10.5"
- "3.5"
- "3.6"
- "3.7"
- "3.8"
- "pypy" # 2.7
- "pypy3" # 3.6

install:
- "pip install ."
- "pip install $REDIS"
- pip install .[tests]

script: "python run_tests.py"

8 changes: 8 additions & 0 deletions AUTHORS.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
``redis-limpyd`` is a project initiated by Yohan Boniface with the help of Stephane "Twidi" Angel,
now exclusively maintainned by Twidi.

Contributors
------------

* Yohan Boniface <b@enix.org>
* Stéphane "Twidi" Angel <https://www.twidi.com/>
31 changes: 0 additions & 31 deletions CHANGELOG

This file was deleted.

122 changes: 122 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
Changelog
=========

Release *2.1.2* - ``2020-05-05``
--------------------------------
* fix double-slicing with instances/values/values_list (like `collection[:3][2]`)

Release *v2.1.1* - ``2020-05-03``
---------------------------------
* speed up (more than x1000, no kidding) of limpyd model creation time

Release *v2.1* - ``2019-11-14``
-------------------------------
* Add new index `ScoredEqualIndex` in `limpyd.contrib.indexes` (to index a field with a score)
* Add new index `EqualIndexWith` in `limpyd.contrib.indexes` (to index many fields at once, unique together or not)
* Lot of rework of collections and indexing internals

Release *v2.0.1* - ``2019-10-11``
---------------------------------
* Fix versions support in documentation

Release *v2* - ``2019-10-11``
-----------------------------
* Support for redis-py >= 3 only
* Support for redis-server >= 3 only
* Breaking change: `zadd` value/scores cannot be passed as positional arguments anymore
* Breaking change: `zadd` flags other than `ch` are explicitely not supported
* Breaking change: `zincrby` arguments are swaped (`amount, value` instead of `value, amount`)
* Breaking change: Redis server with `LUA` scripting support is mandatory
* Breaking change: `set` flags other than `ex` and `px` are explicitely not supported
* Breaking change: `skip_exist_test` to ``Collection.instances()`` is renamed to `lazy`
* Breaking change: collections return generators instead of lists
* Breaking change: collections method (`filter`, `values`...) return a new collection instead of updating the current one
* Add `decrby`, `incrby` and `bitpos` to `StringField`
* Add expiring commands to all normal fields (not `InstanceHashField` and `*PKField`): `expire`, `pexpire`, `expireat`, `pexpireat`, `ttl`, `pttl`, `persite`. But `*expire*` commands can only be called on non-indexable fields
* Add `setex` and `psetex` to `StringField`. Can only be called on non-indexable fields.
* Deny `ex` and `px` flag to `set` if field is indexable
* Add support for `count` argument to `spop` (only for redis-server >= 3.2)
* Add new commands to `ListField`: `lcontains`, `lrank` and `lcount`, to know if a value is in the list, where, and how many times. This is done on the redis server side via lua scripting.
* Optimize deindexing when calling `hmset` or `hdel`
* Add `hstrlen` to `HashField`
* Add `zlexcount`, `zrangebylex`, `zremrangebylex` and `zrevrangebylex` to `SortedSetField`
* Add `zpopmax` and `zpopmin` `SortedSetField` (only for redis-server >= 5)
* When calling `instances` in a collection, the ones that may have raised a `DoesNotExist` exception are now skipped

Release *v1.3.1* - ``2019-10-11``
---------------------------------
* Resolve race condition in `get_or_connect`

Release *v1.3* - ``2019-09-22``
-------------------------------
* Official support for Python 3.7 and 3.8
* Remove support for Python 3.4

Release *v1.2* - ``2018-01-31``
-------------------------------
* Repair packaging

Release *v1.1* - ``2018-01-30``
-------------------------------
* BROKEN RELEASE, sorry
* Official support for redis-py 2.10.6
* Resolve two race conditions (``get`` and more important, ``pipeline``)
* Add *scan* methods for databases/models/instances/fields (sets, hsets, zsets)
* Add *sort* methods for sets, lists, zsets

Release *v1.0.1* - ``2018-01-30``
---------------------------------
* BROKEN RELEASE, sorry
* Official support for PyPy & PyPy3

Release *v1.0* - ``2018-01-29``
-------------------------------
* BROKEN RELEASE, sorry
* Add real indexing capabilities
* Correct/enhance slicing
* Remove support for python 3.3 (keeps 2.7 and 3.4 to 3.6)

Release *v0.2.4* - ``2015-12-16``
---------------------------------

* Locally solve a locking bug in redis-py

Release *v0.2.3* - ``2015-12-16``
---------------------------------

* Compatibility with Redis-py 2.10

Release *v0.2.2* - ``2015-06-12``
---------------------------------

* Compatibility with pip 6+

Release *v0.2.1* - ``2015-01-12``
---------------------------------

* Stop using dev version of "future"

Release *v0.2.0* - ``2014-09-07``
---------------------------------

* Adding support for python 3.3 and 3.4

Release *v0.1.3* - ``2013-09-07``
---------------------------------

* Add the missing 'hdel' command to the RedisModel class

Release *v0.1.2* - ``2013-08-30``
---------------------------------

* Add the missing 'delete' command to the HashField field

Release *v0.1.1* - ``2013-08-26``
---------------------------------

* Include only the "limpyd" package in setup.py (skip the tests)

Release *v0.1.0* - ``2013-02-12``
---------------------------------

* First public version
File renamed without changes.
3 changes: 1 addition & 2 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
include LICENSE
include README.rst
include requirements.txt
include requirements-2.6.txt
33 changes: 22 additions & 11 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
|PyPI Version| |Build Status|
|PyPI Version| |Build Status| |Doc Status|

======
Limpyd
======

`Limpyd` provides an **easy** way to store objects in `Redis <http://redis.io/>`_, **without losing the power and the control of the Redis API**, in a *limpid* way, with just as abstraction as needed.
``Limpyd`` provides an **easy** way to store objects in Redis_, **without losing the power and the control of the Redis API**, in a *limpid* way, with just as abstraction as needed.

Featuring:

- Don't care about keys, `limpyd` do it for you
- Don't care about keys, ``limpyd`` do it for you
- Retrieve objects from some of their attributes
- Retrieve objects collection
- CRUD abstraction
- Powerful indexing and filtering
- Keep the power of all the `Redis data types <http://redis.io/topics/data-types>`_ in your own code

Example of configuration:
@@ -66,19 +67,25 @@ So you can use it like this:
Install
=======

Python versions 2.6, 2.7, 3.3 and 3.4 are supported.
Redis-py versions >= 2.9.1, < 2.11 are supported.
Python versions ``2.7`` and ``3.5`` to ``3.8`` are supported (CPython and PyPy).

Redis-py_ versions ``>= 3`` are supported, with redis-server versions ``>= 3``.

.. code:: bash
pip install redis-limpyd
For Redis-py_ versions < 3, please use limpyd version ``1.3.1`` (or later in ``1.x`` versions)

Documentation
=============

See https://redis-limpyd.readthedocs.org/ for a full documentation.
See https://redis-limpyd.readthedocs.io/ for a full documentation

Changelog
=========

See `CHANGELOG.rst <CHANGELOG.rst>`_

Maintainers
===========
@@ -87,13 +94,17 @@ Maintainers
* `Yohan Boniface <https://github.com/yohanboniface/>`_


Extentions
Extensions
==========

* A bundle of great extensions: `Limpyd-extensions <https://github.com/twidi/redis-limpyd-extensions>`_
* A queue/task/job manager: `Limpyd-jobs <https://github.com/twidi/redis-limpyd-jobs>`_
* A bundle of great extensions: `Limpyd-extensions <https://github.com/limpyd/redis-limpyd-extensions>`_
* A queue/task/job manager: `Limpyd-jobs <https://github.com/limpyd/redis-limpyd-jobs>`_

.. |PyPI Version| image:: https://img.shields.io/pypi/v/redis-limpyd.png
:target: https://pypi.python.org/pypi/redis-limpyd
.. |Build Status| image:: https://travis-ci.org/yohanboniface/redis-limpyd.png
:target: https://travis-ci.org/yohanboniface/redis-limpyd
.. |Build Status| image:: https://travis-ci.org/limpyd/redis-limpyd.png?branch=master
:target: https://travis-ci.org/limpyd/redis-limpyd
.. |Doc Status| image:: https://readthedocs.org/projects/redis-limpyd/badge/
:target: http://redis-limpyd.readthedocs.io/en/latest/
.. _Redis: http://redis.io
.. _Redis-py: https://github.com/andymccurdy/redis-py
22 changes: 21 additions & 1 deletion doc/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,21 @@
For a better experience you may consult this documentation at https://redis-limpyd.readthedocs.org/
For a better experience you may consult this documentation at https://redis-limpyd.readthedocs.io/ where it will be automatically updated when a git tag of a stable release will be pushed.

It can be hosted locally, though.

To build it:

- install sphinx and the "read the doc" theme:

```bash
pip install .[doc]
```

- build the doc

```bash
python setup.py build_sphinx
```

- see it

go to ``file:///local/path/to/redis-limpyd/doc/_build/html/index.html``
36 changes: 29 additions & 7 deletions doc/about.rst
Original file line number Diff line number Diff line change
@@ -2,26 +2,48 @@
About
*****

`redis-limpyd` is a project initiated by `Yohan Boniface <https://github.com/yohanboniface/>`_, using python to store "models" in Redis_.
``redis-limpyd`` is a project initiated by `Yohan Boniface <https://github.com/yohanboniface/>`_, using python to store "models" in Redis_, with the help of `Stéphane «Twidi» Angel <https://www.twidi.com/>`_

The project can be found here: https://github.com/yohanboniface/redis-limyd

Yohan is helped in the developement by `Stéphane "Twidi" Angel <https://github.com/twidi/>`_, with lot of work on his branches, aimed to be integrated upstream via pull requests when we have time to review code.
You can found these branches here: https://github.com/twidi/redis-limpyd/branches (the `develop` branch is generally up to date with the work on all of them)
Source
======

If you want to help, please fork (`master` or a feature branch, not `develop`) and work on a branch with a comprehensive name, write tests (seriously, everything is severely tested in `limpyd`) and make a pull request.
The project is hosted on Github at https://github.com/limpyd/redis-limpyd


Install
=======

Python version 2.6, 2.7, 3.3 and 3.4 are supported.
Redis-py versions >= 2.9.1, < 2.11 are supported.
Python versions ``2.7`` and ``3.5`` to ``3.8`` are supported (CPython and PyPy).

Redis-py_ versions ``>= 3`` are supported, with redis-server versions ``>= 3``.

.. code:: bash
pip install redis-limpyd
For Redis-py_ versions < 3, please use limpyd version ``1.3.1`` (or later in ``1.x`` versions)


Participation
=============

If you want to help, please fork (``master`` or a feature branch, not ``develop``) and work on a branch with a comprehensive name, write tests (seriously, everything is severely tested in ``limpyd``) and make a pull request.


Maintainers
===========

* `Stéphane «Twidi» Angel <https://www.twidi.com/>`_ (main maintainer)
* `Yohan Boniface <https://github.com/yohanboniface/>`_ (creator)


Extensions
==========

* A bundle of great extensions: `Limpyd-extensions <https://github.com/limpyd/redis-limpyd-extensions>`_
* A queue/task/job manager: `Limpyd-jobs <https://github.com/limpyd/redis-limpyd-jobs>`_


.. _Redis: http://redis.io
.. _Redis-py: https://github.com/andymccurdy/redis-py
1 change: 1 addition & 0 deletions doc/changelog.rst
Loading