Workflows

Contents

• Git workflows
• Updating code on the production server
• Resolving an internal server error
• Running South migrations
• Creating South migrations

Git workflows

See the page on Git Workflows.

Updating code on the production server

Requirements before proceeding:

• You will need a user account on your production server.
• You may need to use sudo to have permission to run certain commands. For example, if apachectl start yields a "permission denied" error, then sudo apachectl start might work. To use sudo, you have to be added as a sudoer on the server.
  • Using sudo on git commands, however, is generally a bad idea. See the below requirements to set up Git permissions with your production server username.
• To use Git on the production server, you must have a Git credential helper or SSH key set up on the production server. See GitHub's guide for more info (make sure to select the server's operating system at the top of the page).
• You will need write permission for the directories in the Git repository. To ensure that all project collaborators can write to the directories, you should make sure all of these directories have g+w (group write) permission and are under a group which all project collaborators are part of. Suppose your group is called admin, and your project root directory is CoralNet. Then you would run sudo chgrp -R admin CoralNet and sudo chmod -R g+w CoralNet.

Now, whenever you need to update code on the production server, you'd do the following:

1. SSH into the production server machine, and locate the directory with the project code.
2. Put up the maintenance message. Set the maintenance time to be at least several minutes after the current time. That way, users have some advance warning before you actually start messing with the site. More info on the maintenance message: here
3. Wait until your specified maintenance time begins.
4. Stop the server with apachectl stop. This is done because updating code while the server is running can temporarily leave the server code in an inconsistent state, which can lead to some very weird internal server errors. (Note that this is only the case when settings.DEBUG = False, which it should be for production servers.)
5. Follow the Updating procedure on the Git workflows page. This'll be simple because you almost never develop code on the production server; you just update it. In particular, git fetch origin, git checkout -- <path to file>, and git rebase origin/master are probably all you will need.
6. If there are any new Python packages or other packages to install, then install them.
7. If there are any new settings to specify in settings_2.py, or new task helper functions to create in images/task_helpers.py, then do that.
8. If any static files (CSS, Javascript, etc.) were added or changed, run python manage.py collectstatic to serve those new static files.
9. If there are any new South migrations to run, then run those.
10. Start the server again with apachectl start.
11. If possible, quickly test the new functionality to check that's it's working.
12. Take down the maintenance message.

Resolving an internal server error

Almost always, when you need to resolve an internal server error, you'll need to know the error traceback.

• Sentry logs errors and their tracebacks, so if an error occurred, check the Sentry client, which is at /sentry . Log in as a site admin, using the same username and password as an admin account on the website.
• Occasionally, there will be an error that Sentry cannot log, or an error that prevents the Sentry client from being reachable. In that case, you'll need to check the Apache error log. The Apache error log location is configured with the ErrorLog directive, and this directive's value should be in one of the Apache configuration files (ending in .conf).

Special cases:

• If an internal server error is only occurring on the production server (not on any development machines), and/or intermittently occurring, then it's possible that someone forgot to do step 3 in "Updating code on the production server". Try restarting Apache (apachectl restart) and see if that stops the error from happening again.

Running South migrations

The South documentation is pretty readable and is a great resource to understand what South does and how to use it.

To get at least the basics of what database migrations are and what South does about them, read: What are migrations?, and at least skim the Tutorial's Part 1 and Part 2. It's also recommended to read the whole Tutorial, though; it could spare you some confusion later on.

Here's a quick run-down on running South migrations:

0. Check which migrations have yet to be run by typing python manage.py migrate --list (or python manage.py migrate <appname> --list to just see migrations for a particular app).
1. Run the migrations that you haven't yet run. You can run these:
   • All at once: python manage.py migrate
   • One app at a time: python manage.py migrate <appname>
   • One migration at a time. If you've already run up to migration 0003 in your app and the migrations go up to 0005, you can use python manage.py migrate <appname> 0004 to just run migration 0004 and not 0005.
   • In a dry-run mode, meaning you'll "test" the migrations without actually applying them to your database: python manage.py migrate --db-dry-run. Note that certain migrations cannot be dry-run, however, in which case this command would do nothing for that migration.
   • As backwards migrations: if you've just run migration 0003 and you want to roll back, you can do python manage.py migrate <appname> 0002 to do so.

Notes:

• See the database-specific issues. CoralNet uses MySQL, so if something goes wrong during a migration, South will probably turn up a long and fairly scary-looking error message, saying that your database may now be in an inconsistent state that is "between" migrations. This can be a pain to sort out. If this happens, then before trying anything else, you may want to go into the MySQL command line and examine your tables manually to get a sense of what has happened (i.e. are you really in a state that's between migrations, or did the migration just fail at the start, meaning you can just figure out what's wrong and re-run the migration?).

Creating South migrations

If you haven't already, see the "Running South migrations" section for some basic info on migrations.

The following procedure applies mainly to schema migrations. For data migrations (not as common), see Part 3 of the South Tutorial. Also note that the following procedure only covers the most common kinds of schema migrations. Read the rest of the South Tutorial for more special cases, such as custom fields and migration dependencies.

0. Change one of the Django apps' models code.
   • You'll need to go through the following steps to create a migration only if your change requires corresponding changes at the database level. This includes adding or deleting a model, adding or deleting a field, or making a field have null=True or null=False. It does not include some things like changing help text, or changing whether a field is editable in Django ModelForms that use that model.
1. Run python manage.py schemamigration <appname> --auto for the app whose model code you changed. This will start the process of auto-generating a migration file.
2. If you're adding or removing a field, or making a field accept null or not accept null, you may be asked to provide a default value for that field, so do so. You'll probably also be asked if you want the default value to apply only when this migration is run (a "one-off" default value), or if you want that default value to apply to this field from now on. Answer according to your needs.
   • You may wonder why you need to supply a default value even when removing the field. The reason is that if you apply the backwards migration, you'll be re-adding that field, and adding the field requires a default value.
3. The newly generated migration file will be in <appname>/migrations.
4. Run the migration as described in the "Running South migrations" section.
5. If you want to make extra sure that things will go smoothly (not a bad idea with migrations), try the backwards migration, too. So a thorough test procedure would involve running forwards, backwards, then forwards again so you're up to date.
6. When you're sure you're going to keep this migration, don't forget to add this migration file to Git version control so that other developers (and the server) can run the migration as well.

Notes:

• MySQL may have issues during the migration if you change a field from accepting null values (null=True) to not accepting null values (null=False). So if you do this, be aware that you may have to deal with an "inconsistent state" error.