-
Notifications
You must be signed in to change notification settings - Fork 799
SimpleCV 2.0 Planning
This is a scratch pad for SimpleCV 2.0 plans. It will probably eventually be rolled into a roadmap as we don't have one at the moment.
The main changes for SimpleCV 2.0 is to make it a true modular framework. The way the system works now is that everything is getting merged into the main core and although makes things simpler it's also very big and most people don't need everything that is built into simplecv. The idea is to have a core system that people write extensions, plugins, modules or whatever taxonomy you want to use. Then SimpleCV.org will list various modules that are hosted on github. This will make SimpleCV much more streamlined, unit test will live with the individual module, simplecv's core will be very stripped down. Things will be much easier to parallellize and you will only get the features you need. Come release time there will be bundled releases of SimpleCV, where as the SimpleCV organization will bundle simplecv core with many of the modules they think should be included by default. That way the bundled releases function similar to how the SimpleCV 1.x version works.
I envision the plugin system working similiar to how the flask web framework functions (ref: https://github.com/mitsuhiko/flask/blob/master/flask/exthook.py). That exact code could probably be borrowed. When SimpleCV loads it will use the same convention that flask uses with python system installed module with prefix "simplecv_". These module files will be functions that will at as hooks into the simplecv core functions. For example, the keypoint matching stuff would live in a module, each function would 'hook' into the core functionality like image by prefixing the function name with "image_". So the module would look something like:
def image_findKeypoints(*args, **kwargs):
# find keypoint code here
The module system would then know to monkey patch this onto the Image class, and when you had the module installed it would automatically append to the Image object.
For example:
> # Without keypoint module enabled
> img = Image('simplecv')
> img.findKeypoints()
> error: function doesn't exist
> # With keypoint module enabled
> img = Image('simplecv')
> img.findKeypoints()
> FeatureSet: 10 keypoints found
SimpleCV Core could also use a configuration file to determine what plugins it should enable or disable. There could always be a shell command to read / write this file to ensure file is correct syntax (whether it be yaml, ini file, etc).
> simplecv module list
Available modules on system:
Enabled | Name
----------------------------------
| Keypoints
x | image operations
visit simplecv.org/modules to find more
> simplecv module enable Keypoints
keypoint module enabled
> simplecv module list
Available modules on system:
Enabled | Name
----------------------------------
x | Keypoints
x | image operations
Others advantages to this system is that modules will be required to include unit test, performance scripts, as well as continuous integration support with something like Travis-CI (https://travis-ci.org).
Not 100% sure on this, but I think I would like to force the use of *args, **kwargs to every function call. I think this would be useful in the case of backward compatibility in the future (if that makes sense). For example, if we move to a modular system, and modules start to become interdependent (like the keypoint module depends on the image operations module to be installed), how can you ensure keyword parameters don't change, change order, etc. Using args,kwargs method you will never have to worry about parameter position. Although it does require you to pass keywords when calling the functions it still allows for defaults to be set if the keyword argument isn't passed it. It basically kills the ability to do something like:
> #old way
> img.binarize(50)
> #suggested new way
> img.binarize(size=50)
This way also allows multiple keywords to share the same value, so in the case of backward compatibility.
I would like to make SimpleCV 2.0 framework make much more use of things like multiprocessing module. Although most of the existing 1.x stuff is doable in realtime (~30fps), it could run so much faster if things were parallelized across the multiple cores that are available on most existing systems.
It would eventually be nice to get support for OpenCL as well, but much of the existing code base could be greatly improved with multiprocessing as well as replacing parts of code where for loops existing that could probably be generators, list comprehensions, maps, etc.
The feature class will be reworked to include all drawable objects and a larger number of features than are currently supported. For example drawing a rectangle will change from:
img.drawRectangle(x,y,w,h)
To something like:
rect = Rectangle(roi=(x,y,w,h),color=Color.RED,img=myImg)
rect.draw() # draw the rectangle manually
rect.crop().show()
img.drawObjects() # image knows about attached features.
For 2.0 the new ROI class functionality will move up to the feature class and all ROIs will be features (e.g. you can create a feature that is simply an ROI and then call crop). The base feature class will also provide an interface to all basic graphics operations such as scale, translate, rotate, etc. The base feature class should also support some abstract notions of position to improve filtering (e.g. above, below, left, right, on edge, centered, contains, does not contain, etc). Additionally the base feature class will support some basic notions of collections, such as splitting and merging which return a new Feature, but are non-destructive to the original feature. As a general rule all features will use lazy evaluaition for hefty computations (similar to many of the blob methods).
The rendering system needs a complete rewrite. Pygame has made it easy to use cross platform with a lot of functionality, it also has tons of issues running cross platform. The new rendering system will be completely pushed to the web browser. It only makes sense as things like ipython notebooks already exist in browser. And rendering cross platform in browser is a fairly well solved problem. The idea will be to couple it with SimpleCV-JS (https://github.com/sightmachine/simplecv-js). That way python functionality will be extendable as the browser becomes more universal. Although this may break support for things like internet explorer, the back up plan will be things like pygame, gtk, QT, Tk, or whatever your display engine could always be extended through a SimpleCV module. It appears with WebRTC that rendering in real time in the browser is completely doable now and will be the preferred method going forward. This not only reduces dependencies, it also makes things work cross platform much more seamlessly.
Not sure if rendering system will be a module or built into core. It may have to be built into core to make things easier, but can be decided after some design documents are put together.
The new simplecv 2.0 framework will require a quite a bit of system rewrite from how the existing 1.x version works. Although it will be a lot of work I believe it will be worth the effort. I don't think all of the existing 1.x base will have to disappear, there will just be a lot of shifting code around from existing core functions to modules.