Skip to content

idearise/jpotatoe

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 

Repository files navigation

Jpotatoe: A simple JRuby script for one-way CouchDB 0.8.1 replication

This script is a modified version of potatoe.rb, an example Ruby script that
demonstrated basic CouchDB replication through the DbUpdateNotificationProcess
hook (couchdb.ini) setting. The original potatoe.rb script is found in this
blog post:

http://blog.idearise.com/2008/09/08/couchdb-081/

The original potatoe.rb script did not do batch updates or timed updates. The
Jpotatoe script does both and is meant to be used with JRuby.

However, I don’t think this script is a good way for real heavy CouchDB
replication. It hasn’t been tested with major loads, and there is no notion of
how it may behave. Furthermore, it just doesn’t seem right to do major
replication this way instead of doing it through a built-in CouchDB service or a
message queue.

So, why is this here? Hopefully, it will help anyone, who is thinking about doing
replication this way, to see what might be wrong with such an approach.
In any case, it might just be a waste of time. ;-)

Caveats and TODO

  • It is not known how JRuby startup times may affect any CouchDB update messages
    that are already being sent before the script process is ready to accept those
    messages.
  • CouchDB fires another “database update” message after a successful
    replication. This means that during its periodic time check the script may call
    for a subsequent replication even if there are no other database updates.
  • A CouchDB “database update” message doesn’t always mean a document was
    updated…See the previous point about CouchDB sending a database update
    message after a successful replication.
  • The script spawns a thread for each replication HTTP POST.
  • The thread locks are not on a “transaction” that includes the
    replication; they are on the minimum shared resource. This means that
    additional database updates may occur before replication has begun and will be
    end up as part of that replication.
  • The script doesn’t handle batch sizes or times for different databases.
  • CouchDB replication may be completely different in the future!

Example Usage

Batch size is the number of database updates that the script will wait for before
starting the replication process. A database update doesn’t always mean that a
document was updated!

The script will also start the replication process every X seconds to make sure
that the target database is updated every so often.

replicate_url = "http://192.168.0.4:5984/_replicate"
source = "http://192.168.0.4:5984/" # with trailing slash
target = "http://192.168.0.2:5984/" # with trailing slash
databases = { "mytestdb" => 0 } # { "name_in_quotes" => default_update_count }

config = { :replicate_url => replicate_url,
           :source => source,
           :target => target,
           :databases => databases,
           :batch_size => 50,
           :x_seconds => 1800 }
 
Jpotatoe.new(config).watch

About

Simple JRuby script for one-way CouchDB 0.8.1 replication

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages