Python Scientist - Port of GitHub's Scientist - A library for carefully refactoring critical paths
This is a Python port of GitHub's Scientist which is a library for carefully refactoring critical paths.
Let's pretend you're changing the way you handle permissions in a large web app. Tests can help guide your refactoring, but you really want to compare the current and refactored behaviors under load.
By using an experiment from this library, you can try both ways (the old way and the new way), and you can make sure that the result is the same, and they both don't have issues (or at least the same issues).
When you wrap your change in an experiment, you will get that:
- It (the experiment) decides whether or not to run the new way (we call it the
try
path) based on your configuration, - Randomizes the order in which the old way (we call it the
use
path) andtry
paths are run, - Measures the durations of all behaviors,
- Compares the result of
try
to the result ofuse
, - Swallows (but records) any exceptions raised in the
try
path, and - Publishes all this information.
The use
block is also called the control. The try
block is also called the candidate.
If you don't declare any try
blocks, none of the Scientist machinery is invoked and the control value is always returned.
Here's a quick and dirty way to science
# Import the library
import pyentist
# Using a context manager makes it easy to setup
with pyentist.science('exp1') as e:
x = 10
y = 5
# You can change the configuration of the experiment here
e.should_raise_on_mismatch = True
# The `use` method accepts the function that will be used as a control
# case, the base case, the old case, etc.
e.use(lambda: x * y)
# The `try_candidate` method accepts the name of the candidate and the
# function that will be used in this candidate.
e.try_candidate('candidate1', lambda: sum(x for _ in range(y)))
# You can try more than one candidate if you want.
# You have access to the returned value by the control case
print(e.returned_value)
# You can also peek at e.result wich is an instance of pyentist.Result
More documentation coming soon
GNU General Public License v3 or later (GPLv3+)