Skip to content
snej edited this page Dec 30, 2011 · 25 revisions

Contents

  • What Is This?
  • Performance & Size
  • Compatibility
  • Implementation & Design
  • Choices, Choices

What Is This?

Q: What is TouchDB?

A: To quote from the Why TouchDB? document: “TouchDB is a lightweight CouchDB -compatible database engine suitable for embedding into mobile apps. Think of it this way: If CouchDB is MySQL, then TouchDB is SQLite.” For more details, read that page.

Q: What does it run on?

A: Currently, iOS 5+ and Mac OS X 10.7+. It is likely to be ported to Java/Android, and other ports are certainly possible.

Q: Can I use it yet?

A: Sure, if you understand that it’s currently (Dec. 2011) pre-alpha, i.e. incomplete, buggy, under-documented. By using it at this point you’ll probably be helping development of TouchDB as much as developing your app :)

Q: What’s the license?

A: TouchDB itself is released under the Apache License 2.0 (the same as CouchDB and Couchbase Mobile.)

Some libraries that TouchDB uses (FMDB, MYUtilities, and “CocoaHTTPServer”:“https://github.com/robbiehanson/CocoaHTTPServer”) have MIT or BSD licenses. We may seek relicensing or rewrite those parts of the code, to get everything under a single (Apache) license.

Q: Who’s behind this?

A: Couchbase. The coding so far is by Jens Alfke (snej), who is hoping to get contributions from other people soon :)

Performance & Size

Q: How big is TouchDB?

A: Right now (18 Dec 2011) it compiles to about 150k bytes of optimized ARM7 code. And the source code is about 4000 lines of Objective-C (plus another ~2500 lines from some existing utility libraries.)

By comparison, Couchbase Mobile 2.0 is about 3.5MB of code.

Q: How long does it take to start up?

A: Right now (18 Dec 2011) it takes about 100ms to initialize the library and open a small database. That’s a cold launch; if the app has been launched and quit recently, leaving stuff in cache, it’s more like 60ms. This is on an iPad 2; older devices will be a bit slower.

By comparison, Couchbase Mobile 2.0 takes about 5 seconds to initialize on the same hardware.

Q: How fast is it?

A: I don’t know yet; I haven’t done any real benchmarks or stress-testing, nor have I put much work into optimization. With small databases, everything is pretty much instant. The most complex thing I’ve run is pulling a database of about 3,000 revisions from either a local server or from IrisCouch; this seems to go about the same speed as CouchDB.

I’m sure that it will not scale as well as CouchDB to large data sets. But I also think it’s unlikely that mobile apps are going to need multi-gigabyte databases or millions of documents.

Q: How much data can it handle?

A: There aren’t any hard limits in TouchDB itself, nor to my knowledge in SQLite. The most likely practical limit is the available disk/flash storage on the device, and of course app responsiveness as query times increase (see above).

I’ve heard that Android has a 2GB file size limit; but this should be less of a problem for TouchDB than CouchDB, because the database file doesn’t grow as fast (it doesn’t need explicit compaction) and because it doesn’t store attachments inside the database file.

Compatibility

Q: Is TouchDB compatible with CouchDB?

A: It depends on what you mean by “compatible” :)

Q: OK, can it replicate with CouchDB servers?

A: Yes, its replication protocol is entirely compatible. That’s a very important goal. Apps using TouchDB will be able to sync with servers running Apache CouchDB or BigCouch or Couchbase Server, and with hosting services like Cloudant and IrisCouch.

(I’m using the future tense because, as of 30 Dec. 2011, not everything required for 100% compatibility, like filtered replication, has been implemented yet. And it hasn’t been tested with any particularly large or complex databases. But replication does work to and from CouchDB and IrisCouch.)

Q: Does it still have documents, revisions, attachments just like CouchDB?

A: Yes, the fundamental data model is also compatible. That’s pretty much required for replication compatibility, anyway.

At this point I don’t plan on supporting design documents, or list and show functions. They don’t make a lot of sense for mobile apps. (TouchDB does support map/reduce and validation functions, and will support filters, but they’re not stored in the database, because they’re native function pointers not JavaScript code.)

Q: Does it have conflict handling and revision-trees like CouchDB?

A: Yes. Revision trees are implemented, and preserved across replication, though I haven’t yet implemented all the APIs needed for apps to detect and resolve conflicts.

Q: Is the API compatible?

A: Sort of. The intention is that the primary API developers will use is a native one, to match their development language. The details of this haven’t been worked out yet — the classes in the implementation are probably going to remain internal private API, with the public API on iOS being CouchCocoa, either in its current form or rewritten to call TouchDB directly.

That being said, the REST API is supported, in a sense. The iOS implementation registers a custom URL protocol handler with the CFNetwork framework, which means that when the app uses the Cocoa NSURLConnection API to access “touchdb:” URLs, it talks to the TouchDB library in-process instead of to the network. There are no actual sockets involved at all. This makes it very easy to use existing code (such as CouchCocoa) with little or no change.

Q: Can’t you access CouchDB over HTTP?

A: There’s an experimental HTTP server extension. This will help enable TouchDB-to-TouchDB (P2P) replication as well as making testing easier. It’s not meant to be the primary way for an app to access TouchDB, though.

Q: So what things aren’t compatible?

A: TouchDB doesn’t currently support every detail of the CouchDB REST API. We’ll add support for things that turn out to be important to mobile apps, and will probably ignore things that aren’t (like /restart, /_config and /users.)

Other things that aren’t compatible: The database file format. Configuration options (no .ini files.) Functions in interpreted languages (JS or Erlang).

Q: What about CouchApps?

A: CouchApps running in PhoneGap are definitely an interesting mobile platform. TouchDB won’t directly support these the way Couchbase Mobile 2.0 does, but I think it could be extended to without too much trouble. Mostly someone would need to implement the missing design-document support, including a JavaScript interpreter.

Implementation & Design

Q: Why not work on optimizing CouchDB, instead of starting a new implementation?

A: I did, but it doesn’t seem possible to optimize CouchDB’s size and startup time enough for mobile apps. We were able to get some significant improvements in Couchbase Mobile 2.0, but beyond that, further shrinking and speeding-up possibilities seemed pretty minor (on the order of 5%) when what we wanted were orders of magnitude. Most of this was intrinsic in the fact that it’s implemented in an interpreted language (Erlang) whose runtime library needs to be bundled in.

Q: The code is all Objective-C. I thought this was supposed to be cross-platform?

A: The design is cross-platform. After considering various interpreted languages and cross-platform libraries, I decided they had too much overhead for mobile apps, or would make the implementation too awkward. So instead the goal is to have separate implementations for each platform, but share aspects like the SQL schema, class architecture and algorithms.

Q: Why not use a more widely-known language for the reference implementation?

A: True, from that standpoint it might have been better to start with a Java implementation for Android, since more people read Java fluently than read Objective-C. In the end it came down to the fact that I (Jens) like Objective-C better than Java, and have experience developing for iOS but not Android.

I’m pretty certain that a Java/Android implementation will follow pretty soon. I have also heard interest in an ANSI C implementation (possibly using the Apache Portable Runtime or GLib.)

Q: Is the implementation based on CouchDB’s, just ported from Erlang to native code?

A: No. I am honestly not very familiar with the CouchDB implementation. I don’t think it would have made sense to refer directly to the Erlang code anyway, as the way code is structured in that language (functional, highly parallelized, non-OOP) is very different from imperative OOP languages like Objective-C and Java.

And in any case, a lot of the code is of necessity very different because it’s using SQL for storage, rather than a custom append-only B-tree implementation.

Q: Does it use the same B-tree structures or file format as CouchDB?

A: No, it lets SQLite take care of file storage and indexing. TouchDB database files are regular SQLite databases. There’s documentation on the schema, if you’re curious.

Q: Why SQLite instead of a B-tree engine like Berkeley DB or Kyoto Cabinet?

A: Largely because SQLite is already available as a shared library on every platform we’re interested in; this keeps our code size down and simplifies the build process.

Additionally, both Berkeley and Kyoto have GPL-like licenses that are less friendly to commercial developers (especially iOS developers) and incompatible with the Apache license of TouchDB itself.

Choices, Choices

Q: What does this mean for Couchbase Mobile?

A: [Jens puts on his Couchbase hat] We don’t know yet. TouchDB was definitely created with the intention of replacing CouchDB as the engine inside Couchbase Mobile, to achieve some important order-of-magnitude improvements in size and startup time. As of this writing (Dec 2011) it’s still an investigation, pre-alpha, so it’s too early to say. The good news is that a switch should have minimal impact on apps that use Couchbase Mobile, because the higher-level APIs they use will remain the same.

Q: Why would I use this instead of CouchDB?

A: Because it’s a lot smaller, starts up a lot more quickly, and is easily embeddable into an app. Those are important factors for mobile app developers (and some desktop app developers too.) If you’re working on server-side software they probably don’t matter to you, or at least don’t outweigh the drawbacks.

Q: Why would I use CouchDB (or Couchbase Server, or BigCouch) instead of this?

A: Because CouchDB is much more mature, scales better, interoperates easily with other server processes using a REST API, and runs on more platforms.

Q: Why would I use this instead of iOS’s CoreData framework?

A: Basically for the same reasons that are already outlined in the Couchbase Mobile marketing materials. The primary one is world-class, highly-flexible data sync capabilities that go way beyond what you can get from iCloud. Another factor is that the CouchCocoa API is (we think) simpler and easier to use than CoreData’s.

Q: Why would I use this instead of working directly with SQLite (or an adapter like FMDB)?

A: As with the previous comparison to CoreData: the big reason is sync. If your users want to work with their data on multiple devices or platforms (including the Web), or have it transparently backed up, the replication capabilities in TouchDB and CouchDB will make it very easy compared to the pain of implementing sync yourself, or trying to duct-tape your custom SQLite database to the iCloud APIs.