Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement new database for running simple tests #4

Open
cyberixae opened this issue Feb 6, 2014 · 0 comments
Open

Implement new database for running simple tests #4

cyberixae opened this issue Feb 6, 2014 · 0 comments

Comments

@cyberixae
Copy link
Member

BSMR has a simple web filesystem (PetriFS) that is used for doing simple test runs. A slightly more complex database is needed for doing test runs that start from where the previous computation ended.

The input plugin for the new test database should read an entire directory of files. Feeding (filename, filedata) pairs to mappers. The output plugin for the directory needs to write the results into a directory by using the bucketId as the filename and the reducing result as the file contents. The factories that produce the plugins should take a dataset identifier as a parameter and select the correct directory as a result.

It should be possible to chain computations by giving the input plugin factory of a current run the dataset identifier that was given to the output plugin factory of a previous run. It should follow that the mapper input pairs of the second run are equal to the (bucketId, reducerOutput) of the first run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant