Lurker

Generates API schemas, validates code against them and creates a handy web interface for testing the API. Works on Rails 3.2, 4.0, 4.1, 4.2 & Ruby 1.9.3, 2.0.0, 2.1.5, 2.2.0, 2.3.0

Installation

Add my public key:

gem cert --add <(curl -Ls https://raw.github.com/razum2um/lurker/master/certs/razum2um.pem)

$ gem install lurker                 # without key
$ gem install lurker -P HighSecurity # secure, with key added

Or this line to your application's Gemfile:

gem 'lurker'

Inject into test_helper.rb or spec_helper.rb:

require 'lurker/spec_helper'

Usage

Wrap your integration test code, which does request like this

Lurker::Spy.on do
  get "/api/v1/users.json"
end

And run the specs and commit your schemas. That's all, easy!

RSpec usage

Write your RSpec controller or request specs as usual, but add :lurker mark (like documented controller example or request spec example).

it "lists users", :lurker do
  get "/api/v1/users.json"
end

NOTE: If you use rspec-rails, it should be required first

require 'rspec/rails'
require 'lurker/spec_helper'

Minitest usage

You can use minitest-around to wrap your test classes like this:

class DestroyRepoTest < ActionDispatch::IntegrationTest
  def around(&block)
    Lurker::Spy.on(&block)
  end
end

You also can wrap any block with API requests manually.

Schemas

Please, commit your files under Rails.root/lurker directory. Feel free to edit them according to json-schema standard. It can be very strict and flexible if you wish: see an example, but scaffolded schemas are pretty good by default.

A  lurker/ExampleApp.service.yml
A  lurker/api/v1/users-GET.json.yml
A  lurker/api/v1/users/__user_id/repos-GET.json.yml

I also advise you to look on Understanding JSON Schema book, it is up-to-date with draft4 and contains lot of examples.

Profit!

Now, every test run lurker will look into your requests and validate them, and it fails if your code changes the API!

Failure/Error: post :create [...]
Lurker::ValidationError:
  Request
  - The property '#/' contains additional properties ["social_network"] outside of the schema
    when none are allowed in schema file:///.../lurker/api/v1/users-POST.json.yml#
  Response
  - The property '#/user/last_sign_in_at' of type String did not match the following type:
    null in schema file:///.../lurker/api/v1/users-POST.json.yml#

The generation of live-documentation is pretty simple:

bin/lurker convert           # builds html under `Rails.root/public/lurker` to be served under `/lurker` url
bin/lurker convert -f pdf    # builds `Rails.root/public/lurker/snake_cased_name.pdf`

For different document root or serving URL prefix use -o and -u options accordingly.

If you want to provide additional documentation for your API (and you probably should), you can use separate Markdown files in the schema folder. To generate documentation stubs for the current schema:

bin/lurker init_docs         # generate documentation stubs for the current schema

Let's run your rails s and visit http://localhost:3000/lurker/ (or see demo for example)

Now, you can test your API on-line (for real)

Demo application

You can run this to get the demo running locally:

git clone https://github.com/razum2um/lurker.git
cd lurker
bundle exec appraisal rails-42 bundle install
bundle exec appraisal rails-42 rake build_example_docs
cd tmp/lurker_app_rails_42
bin/rails s

Lurker supports multiple domains (usually staging and production) and can be deployed statically everywhere as well as be served by the current Rails instance.

For example:

• Github Pages is deployed statically; no API endpoint
• Custom domain HTML deployed under nginx; passenger serves demo api production endpoint
• Heroku HTML is served by unicorn as well as staging api endpoint in Sandbox mode

Features

• Autoscaffolding for non-covered API endpoints
• Autotesting for covered endpoint once written (both request & response!)
• Pretty HTML documentation based on your schemas
• Pretty submit form to test API endpoints (live) based on schemas (enter a name & press "Submit")
• Handling URLs with dynamic segments (such as api/v1/:user_id/repos)
• JSON-Schema partials, also in YAML format (demo)
• Generation PDF documentation (NOTE: add gem 'pdfkit' to Gemfile)
• Multiple docs for many usecases (e.g :lurker => '...')
• ERB support inside .json.yml.erb
• Insert custom content in Markdown into index.html (NOTE: add gem 'kramdown' to Gemfile)
• Syntax highlighting for sample response (add gem 'execjs' to Gemfile)
• Separate API-services generated within one test suite
• Capistrano integration
• JSON-Schema draft-v4 support
• Static site deploy and milti-domain support
• Builtin Rack middleware Lurker::Server.to_rack serves cached digested assets
• RSpec & Minitest support

Related gems

• Swarker converts Lurker schemas to Swagger schema

Token authentication with sandbox

Lurker::Sandbox allows you to test services with token authentication:

# make sure it's not production!
# e.g. config/environtents/staging.rb
config.middleware.use Lurker::Sandbox

E.g. demo application on Heroku runs with it: when creating, updating repos or users ids getting increased, but if you look into GET #index, new items are NOT showing up. This is NOT a bug! Sequences in PostgreSQL are increasing notwithstanding ROLLBACK is called. As such:

• run all your specs with the same testing token
• ensure the same token to be accepted on your demo application
• insert Lurker::Sandbox and the recorded examples should be ok to submit again

Contributions

I try to use Waffle to develop this gem, if you want to help:

• look on "Ready" section
• drag an issue to "In Progress" and assign to yourself
• have fun!

NOTE: to get new version of bundled bootstrap or update JS/CSS, don't touch files under lib/lurker/templates/public - they are autogenerated and copied to static generated site while bin/lurker convert

bundle exec appraisal rails-42 rake assets:precompile # to build them

Don't commit lib/lurker/templates/public/**/* to avoid conflicts.

NOTE: if you write features keep in mind to generate different files with aruba because they are kept in the lurker_app directory to be deployed as a demo. Please, write features in a way to generate new relevant lurker/**/*.json.yml files

NOTE: template partial submit_form.html.erb and it's partials is a big jsx script for React so there are <label htmlFor instead of <label for> and <div className instead of <div class

Different Rails versions

We use appraisal to test the lib with different Rails versions (from 3.2 up to 4.2):

# run this to install all gems for all the supported rails versions:
bundle exec appraisal install

# then prefix any rake command with `bundle exec appraisal` and rails version
bundle exec appraisal rails-42 rake build_example_docs

Test apps are getting generated into their's directory under tmp/lurker_app... prefix. They also use separate databases for each rails version both dev & test env - all prefixed with lurker_app

Demo app == Testing app

Currently, the testing application is using PostgreSQL because the same testing app is deployed to serve demo purposes.

This is also the reason not to delete anything under lurker directory between feature tests and using different API endpoints of the testing app. To run cucumber with clean lurker & public/lurker directories run:

CLEAN=1 bundle exec appraisal rails-42 cucumber features

Beware while writing your feature tests for PRs.

Acknowledgments

This gem is quite opinionated and relies on rails - if you're interested in anything else, please take a look at api_taster or fdoc, This gem is heavily inspired by them. Thanks, @square & @fredwu

Also thanks to

• Andrey Deryabin for advice
• React.js for reactive UI
• highlight.js for syntax highlighting