Release 3.14.0

• Added support for the HTCondor Batch System
• Updated documentation.
• Added the protobuf dependency for mesos.
• Updates for CWL (scheduling, update CWL ref runner, update conformance tests, expose cwltool's tempdir-prefix, expose user-space-docker-cmd, and always import links for CWL)
• Added an "all" option to install all extras.
• Check for venv in python 2 and 3.

Bug Fixes

• Added buffering for local file transfers to fix MemoryError bugs
• Fix race conditions for SLURM, GridEngine, Mesos, and Torque batch systems.
• Updated file size determination on Apple products.
• Fixed a bug when making the docs through Sphinx.
• Amended memory limit and reservation in LSF.
• Removed cwl-runner command, as it was interfering with the native CWL command.

Thank you to our external contributors: @mr-c (big thanks for all of the CWL support!), @chapmanb, @ryanking, @jmchilton, @jorvis, @evan-wehi, @brainstorm, @jsmedmar

Release 3.13.0

• WDL Alpha support
• Job store symlinking
• Updated cwltool support (version; fixes; testing)
• Better logging
• Option to get stderr from subprocessDockerCall
• Integration testing with ENCODE
• Cactus with autoscaling

Bug Fixes

• Fixed static provisioning
• Fixed launch-cluster bug when terminal size was too small
• Removed AWS specific code from the abstract provisioner

Thank you to our external contributors: @mr-c, @chapmanb, @ryanking, @jmchilton, @jorvis, @evan-wehi, @brainstorm, @jsmedmar

Release 3.12.0

• Forkless mode for debugging
• Enabled Travis builds with a subset of tests (less than 15 minutes).
• DockerCall function changed to use docker API.
• Updated to Azure version 2.0.
• Added tempDir convenience function to job and made job.log an alias for fileStore.LogToMaster.
• Enhanced toil-cwl-runner to support SoftwareRequirements & BioContainers.
• Increased default retry count to 1.
• Deprecated cgcloud.
• Updated EC2 instance types.
• Upgraded pip to 9.0.1 and setup tools to 36.5.0 in appliance build.

Bug fixes

• Parasol parses jobStore path correctly.
• Parasol doesn't attempt to remove jobs that it didn't launch.
• Fixed problem with dynamic batch systems not always being created.
• Fix workflow effort being wasted if a single export fails.
• Added a default value for maxNodes.
• Fixed py.test collection errors when running locally.

Thank you to our external contributors: @jmchilton, @jorvis, @evan-wehi, @mr-c, @brainstorm

Release 3.11.0

• Allow batch creation of jobs.
• CWL: run internal jobs locally, not on cluster.
• TOIL_TEST_QUICK environment variable for running a subset of tests.
• Allow clusters with multiple node types.
• Put jobName into the _toil_worker arguments.
• GridEngine: cache instead of block for statePollingWait

Bug Fixes

• CWL: avoid early removal of the temporary outdir
• Fixed AWS readFileStream deadlock on partial reads.
• Use a socket to get the public-facing IP

Thank you to our external contributors: @chapmanb, @ryanking, @mr-c

Release 3.10.1

Adds missed commits that should have been in 3.10.0.

Release 3.10.0

• Full CWL 1.0.1 support.
• Improved documentation with better examples.
• Added option to link files when importing into fileJobStore.
• Allowed some interactive support
• Improved local testing targets.
• Sleep support for all state-querying functions in batch.
• Opened up UDP traffic within AWS clusters.
• Set mesosMasterAddress default to private IP.
• CWL: symlink jobStore files instead of copying.

Bug Fixes

• Fixed getEmptyFileStoreID.
• Automatically use the correct LSF units.
• Fixed failure in resolve_indirect_inner in cwltoil.py.
• Fixed docker tests failing on Mac.
• Stopped log level option defaults from clobbering each other.
• Various other minor bug fixes.

Thank you to our external contributors:
@mr-c, @tetron, @liuxf09, @ryanking, @OliverEvans96, @alaindomissy, @brainstrom, @chapmanb, @nazamy, @jmchilton, @stefanoberri

releases/3.9.0

• Added a plugin batch systems that abstracts option settings for batch systems.
• Fixed mysterious fail with rsync --protect-args on OS-X.
• Added environment variable TOIL_AWS_KEYNAME for setting AWS credentials during tests.
• Tests now timeout with pytest rather than Jenkins.
• Added support for PBSPro (Torque's proprietary version).
• Fixed resolution of symlinks for hard links bug.
• Added an option to set root volume on leader and worker nodes.
• Fixed autoscaler crash.
• Added environment variable TOIL_TORQUE_ARGS for setting Torque arguments.
• Fixed hot-deployment with python-3-supporting libraries.
• Fixed race in ClusterScaler test.
• Several logging improvements and fixes.
• Pip show now displays license info.

Release 3.8.0

• Don't error if job hasn't been submitted yet (resolves #1666)
• Match doc to reality for --preserve-environment
• CWL: symlink jobStore files instead of copying
• When killing tasks, make sure to reap child processes in the process groups
• Better error checking in piped docker commands
• Fix case where a job has been put in currentjobs, but not yet dispatc…
• Use hard links instead of copied files on import to file job store
• CWL: Add support for InitalWorkDirRequirement
• Set proper value for service job preemptable requirement
• Use chunked transfer in AWS job store importing
• Lower timeout for domain binding in Toil Clean
• Add import os.

3.7.0

• Updated documentation at http://toil.readthedocs.io/en/latest/
• Improved Azure support
• Remove upper limit on size of jobs in AWS job store
• CWL workflows can now take advantage of Toil Autoscaling
• CWLToil bug fixes and enhancements
• Toil Appliance updated to Mesos version 1.0.1
• Improve scalability of Mesos batch system
• Better error message for unfulfilled promises
• Fix handling of checkpointed jobs
• Fix service job deadlock detection
• Allow disabling of hot deployment mechanism

3.5.2

Hotfix release for 3.5.x

• Fixes job state discovery race
• Fixes race between job store file exporting and caching
• Fixes bugs in Slurm batch system