Debugging application startup problems

Sometimes an application refuses to startup. When this happens, Phusion Passenger reports one of these messages:

An error occurred while starting up the preloader. It sent an unknown response type "".
An error occurred while starting up the preloader: it did not write a startup response in time.
An error occurred while starting the web application: it did not write a handshake message in time.
An error occurred while starting the web application. It sent an unknown response type "".

Phusion Passenger reports this error if the application did not finish initializing within a time limit, or if it exited without sending Phusion Passenger a message that says "I've initialized successfully!".

The spawning process explained

To understand what's going on, consider what Phusion Passenger does under the hood. Phusion Passenger spawns an application by running one of the following command:

• If your shell is bash: bash -l -c '/path-to/SpawnPreparer /path-to-loader-or-preloader'
• If your shell is not bash: /path-to/SpawnPreparer /path-to-loader-or-preloader

So when launching your application, there are at most 4 components at work:

1. If your shell is bash, then everything else is executed through bash, by passing the -l (login shell) flag to bash. This causes bash to load its startup files, e.g. bashrc, profile, etc. The reason why we do this is because a lot of users try to set environment variables in their bashrc, and they expect these environment variables to be picked up by applications spawned by Phusion Passenger. Unfortunately environment variables don't work that way, but we support it anyway because it is good for usability.
2. Bash immediately invokes the Phusion Passenger SpawnPreparer executable. Or, if the user's shell is not bash, SpawnPreparer is invoked directly. This executable is responsible for setting certain environment variables, current working directory, and other process environmental conditions. When SpawnPreparer is done, it executes the loader or the preloader.
3. If passenger_spawn_method is set to smart (the default), and there is a preloader available for the application's programming language, then this step executes the language-specific preloader. If either of the previous conditions are not met (and thus the passenger_spawn_method is automatically forced to direct), then this step executes the language-specific loader.
   • A preloader loads the application code in memory, and forks off worker processes every time Phusion Passenger needs another application process, which is how the "smart" spawning method is implemented.
   • A loader also loads the application code. But unlike the preloader it does not fork off worker processes. Every time Phusion Passenger needs a process, it starts another loader that fully loads the application code.

This is documented in detail in Spawning methods explained. 4. The loader or preloader loads your application by evaluating its startup file (Ruby: config.ru; Python: passenger_wsgi.py; Node.js: passenger_node.js).

In any of the above 4 steps, something can go wrong. In versions prior to Phusion Passenger 3, and in other application servers such as WEBrick or (G)Unicorn, steps 1 and 2 don't exist, which is why your application may startup correctly under those servers.

Possible causes of problems

Stdout redirection

Phusion Passenger uses the application's stdout for communication with the application. This means that if, during any of those steps, stdout is closed, overwritten or redirected to a file, then Phusion Passenger loses its means to communicate with the application. After a while, Phusion Passenger concludes that the application fails to start up, and reports an error.

It is however safe to write data to stdout as long as the data does not begin with !> . Phusion Passenger uses !> to mark internal communication messages.

Solution for this problem: ensure that your application and your bash startup files never redirect stdout to a file, or otherwise modify that file descriptor.

To test whether your something along the spawning chain messed with stdout, add a simple print command to the beginning of startup file. If stdout is intact, then Phusion Passenger will redirect that message to the web server error log, which you will see immediately.

• Ruby (config.ru): puts "Loading config.ru!"
• Python (passenger_wsgi.py): print("Loading passenger_wsgi.py!")
• Node.js (passenger_node.js): require("sys").puts("Loading passenger_node.js!");