Skip to content

Home

Jump to bottom Edit New page
darwin edited this page Sep 13, 2010 · 36 revisions

FireLogger Protocol

Hey boy! so you want to implement FireLogger support for your favorite language?

Overview

FireLogger content is sent from server to client via HTTP headers. To be safe it is BASE64 encoded UTF-8 JSON broken into multiple FireLogger headers.

Server can send multiple JSONs in one response, I call it packets. That is why each FireLogger header has hex id which should be assigned by server for every new packet (for example FirePython generates this as random ID).

These are main reasons:

  1. we don’t want to interfere with response content (it may be anything from HTML, javascript, CSS or unknown binary data)
  2. to keep it simple and safe (opening other communication channel with server would be from non-trivial to impossible – you know firewalls and such)
  3. JSON is easy for FireLogger addon (Javascript) and every server-side environment has libraries for serialization into JSON, so no big deal here

Request Headers

You have to take care about incoming request headers in your server side library:

  • X-FireLogger – presence of this header says that FireLogger is ready to receive your packets, value is FireLogger version
    • you must emit packets only if you encounter this header
    • you should warn user if there is version mismatch between your library and FireLogger
  • X-FireLoggerAuth – authentication token
    • you should check against this token if you have password to protect server-side

Authentication

The auth token is md5_hex(#FireLoggerPassword#[password]#), of course replace [password] with actual password.

  • see “FirePython implementation” here: http://github.com/darwin/firepython/blob/34435b7eea4a12fd63a51ed227b154d502b63517/utils.py#L26
  • see “FireLogger implementation” here: http://github.com/darwin/firelogger/blob/a06ad4e45f33f50cd91310ac518a5dfe8436b3ab/firefox/chrome/content/firelogger.js#L237

Reponse

You can render more packets into response, each with new id.

Packet structure is following:




{
  errors: [...],  // optional
  logs: [...] // optional
}
  • errors – array of internal exceptions (maybe you don’t want to use this at all)
  • logs – array of actual log records

Example of packet with one log record is following:


{
  "logs": [{
    "args": {
      "py/tuple": ["/", {
        "action": "index",
        "controller": "welcome"
      },
      {
        "multi": {
          "dicts": {
            "py/tuple": [{
              "_items": "[('action', u'index'), ('controller', u'welcome')]",
              "_": "MultiDict([('action', u'index'), ('controller', u'welcome')])"
            },
            {
              "reason": "'Not a POST request'",
              "_": ""
            }]
          },
          "_": "NestedMultiDict([('action', u'index'), ('controller', u'welcome')])"
        },
        "errors": "ignore",
        "decode_keys": true,
        "_": "UnicodeMultiDict([(u'action', u'index'), (u'controller', u'welcome')])",
        "encoding": "utf-8"
      }]
    },
    "name": "www",
    "thread": -1610029280,
    "level": "debug",
    "process": 64884,
    "timestamp": 1237161193396626,
    "threadName": "MainThread",
    "pathname": "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/logging/__init__.py",
    "lineno": 1327,
    "template": "Dispatching %s to %s (%s)",
    "time": "23:53:13.396",
    "exc_text": null,
    "message": "Dispatching / to {'action': u'index', 'controller': u'welcome'} (UnicodeMultiDict([(u'action', u'index'), (u'controller', u'welcome')]))"
  }]
}

Important log fields:

  • message – plain log message as it would be logged by your text logger
  • template – template version of message, arguments should be marked as %X, where FireLogger doesn’t care about X
  • args – array of arguments to be replaced in template. it is your hard work to provide FireLogger with detailed representation of arguments as structured data so user can drill it down in the Watches window. note: in current example, there is “py/tuple” structure wrapping actual array. This is specific to Python jsonpickle library, FireLogger can unwrap it, but you should send plain array in this case.
  • level – debug level, one of debug,info,warning,error,exception
  • timestamp – unix timestamp of log record (sorting)
  • time – user friendly time to be displayed with log record
  • name – logger name – see green bubbles on the right of each log record
  • pathname, lineno – log line location in source file

Other fields are probably not used. These fields are visible to user when he right-clicks on log record and select “inspect in DOM tab” so you can add your own properties if you find it useful.

Exceptions:

In case of exception, FireLogger is able to provide back trace, call stack or call it in whatever you like. It looks like this:


{
  "logs": [{
    "args": {
      "py/tuple": []
    },
    "name": "www",
    "thread": -1610029280,
    "level": "error",
    "process": 64884,
    "timestamp": 1237167142975769,
    "threadName": "MainThread",
    "pathname": "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/logging/__init__.py",
    "lineno": 1294,
    "template": {
      "_": "Exception('Test exception',)"
    },
    "time": "01:32:22.975",
    "exc_text": "Traceback (most recent call last):
 File " / Users / darwin / code / hed / src / bootstrap.py ", line 162, in __call__
 handler.get(*groups)
 File " / Users / darwin / code / hed / src / bootstrap.py ", line 111, in get
 self.route(*args, **kvargs)
 File " / Users / darwin / code / hed / src / bootstrap.py ", line 86, in route
 raise Exception("
    Test exception ")
Exception: Test exception",
    "exc_info": {
      "py/tuple": [{
        "py/type": "exceptions.Exception"
      },
      {
        "py/ref": "/logs/template"
      },
      [{
        "py/tuple": ["/Users/darwin/code/hed/src/bootstrap.py", 162, "__call__", "handler.get(*groups)"]
      },
      {
        "py/tuple": ["/Users/darwin/code/hed/src/bootstrap.py", 111, "get", "self.route(*args, **kvargs)"]
      },
      {
        "py/tuple": ["/Users/darwin/code/hed/src/bootstrap.py", 86, "route", "raise Exception("
        Test exception ")"]
      }]]
    },
    "message": "Test exception
Traceback (most recent call last):
 File " / Users / darwin / code / hed / src / bootstrap.py ", line 162, in __call__
 handler.get(*groups)
 File " / Users / darwin / code / hed / src / bootstrap.py ", line 111, in get
 self.route(*args, **kvargs)
 File " / Users / darwin / code / hed / src / bootstrap.py ", line 86, in route
 raise Exception("
    Test exception ")
Exception: Test exception",
    "exc_frames": [{
      "logging": "",
      "show_error": "",
      "start_response": "",
      "self": "",
      "base64": "",
      "request": "",
      "method": "'GET'",
      "sys": "",
      "handler": "",
      "environ": "{'wsgi.version': (1, 0), 'HTTP_COOKIE': 'session=aghwYWdlYm91dHIOCxIHU2Vzc2lvbhjbAgw; pbs=aghwYWdlYm91dHIOCxIHU2Vzc2lvbhjbAgw', 'SERVER_SOFTWARE': 'Development/1.0', 'SCRIPT_NAME': '', 'REQUEST_METHOD': 'GET', 'PATH_INFO': '/', 'SERVER_PROTOCOL': 'HTTP/1.0', 'QUERY_STRING': '', 'CONTENT_LENGTH': '', 'HTTP_ACCEPT_CHARSET': 'ISO-8859-1,utf-8;q=0.7,*;q=0.7', 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.7) Gecko/2009021906 Firefox/3.0.7', 'HTTP_CONNECTION': 'close', 'SERVER_NAME': 'www.pagebout.local', 'REMOTE_ADDR': '192.168.5.2', 'HTTP_X_FIRELOGGERAUTH': 'dafd69266b2ad83f097eb7b8ee814d76', 'wsgi.url_scheme': 'http', 'PATH_TRANSLATED': '/Users/darwin/code/hed/src/main.py', 'SERVER_PORT': '5000', 'CONTENT_TYPE': 'application/x-www-form-urlencoded', 'CURRENT_VERSION_ID': 'alpha.1', 'wsgi.input': , 'HTTP_HOST': 'www.pagebout.local', 'wsgi.multithread': False, 'TZ': 'UTC', 'HTTP_CACHE_CONTROL': 'max-age=0', 'USER_EMAIL': '', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'HTTP_X_FIRELOGGER': '0.3', 'APPLICATION_ID': 'pagebout', 'GATEWAY_INTERFACE': 'CGI/1.1', 'wsgi.run_once': True, 'wsgi.errors': ', mode 'w' at 0x170b0>, 'wsgi.multiprocess': False, 'HTTP_ACCEPT_LANGUAGE': 'en-us,en;q=0.5', 'AUTH_DOMAIN': 'gmail.com'}",
      "groups": "[]",
      "root": "",
      "response": ""
    },
    {
      "kvargs": "{}",
      "args": "()",
      "self": ""
    },
    {
      "self": ""
    }]
  }]
}
  • exc_info – array of 3 items (actually Python’s exc_info structure), the last is list of stack frames, note: again wrapped in py/tuple which is not needed
  • exc_text – is not used, instead template is used, see again there is a hack with “_”, expected is string, but in case of exception also “_” does the work
  • exc_frames – contains locals for all stack frames listed in exc_info2

And that’s it!

This protocol has started as a lazy serialization of Python log and exception structures. I know it is not very nice, but who cares? The project started as a quick hack to test if I can do it in a few hours. It worked but it took much more to get into in current state. But protocol stayed Pythonic. I’m open to extend or change this protocol if you need. Good luck.

And don’t forget you have reference implementation here

Add a custom footer
Add a custom sidebar
Clone this wiki locally