- Added optional
resume_vm
field to/snapshot/load
API call. - Added support for block rate limiter PATCH.
- Added devtool test
-c|--cpuset-cpus
flag for cpus confinement when tests run. - Added devtool test
-m|--cpuset-mems
flag for memory confinement when tests run. - Added the virtio traditional memory ballooning device.
- Added a mechanism to handle vCPU/VMM errors that result in process termination.
- Change the information provided in
DescribeInstance
command to provide microVM state information (Not started/Running/Paused) instead of whether it's started or not. - Removed the jailer
--extra-args
parameter. It was a noop, having been replaced by the--
separator for extra arguments. - Changed the output of the
--version
command line parameter to include a list of supported snapshot data format versions for the firecracker binary. - Increased the maximum number of virtio devices from 11 to 19.
- Added a new check that prevents creating v0.23 snapshots when more than 11 devices are attached.
- Fixed inconsistency in YAML file InstanceInfo definition
- Added metric for throttled block device events.
- Added metrics for counting rate limiter throttling events.
- Added metric for counting MAC address updates.
- Added metrics for counting TAP read and write errors.
- Added metrics for counting RX and TX partial writes.
- Added metrics that measure the duration of pausing and resuming the microVM, from the VMM perspective.
- Added metric for measuring the duration of the last full/diff snapshot created, from the VMM perspective.
- Added metric for measuring the duration of loading a snapshot, from the VMM perspective.
- Added metrics that measure the duration of pausing and resuming the microVM, from the API (user) perspective.
- Added metric for measuring the duration of the last full/diff snapshot created, from the API (user) perspective.
- Added metric for measuring the duration of loading a snapshot, from the API (user) perspective.
- Added
track_dirty_pages
field tomachine-config
. If enabled, Firecracker can create incremental guest memory snapshots by saving the dirty guest pages in a sparse file. - Added a new API call,
PATCH /vm
, for changing the microVM state (toPaused
orResumed
). - Added a new API call,
PUT /snapshot/create
, for creating a full or diff snapshot. - Added a new API call,
PUT /snapshot/load
, for loading a snapshot. - Added new jailer command line argument
--cgroup
which allow the user to specify the cgroups that are going to be set by the Jailer. - Added full support for AMD CPUs (General Availability). More details here.
- Boot time on AMD achieves the desired performance (i.e under 150ms).
- The logger
level
field is now case-insensitive. - Disabled boot timer device after restoring a snapshot.
- Enabled boot timer device only when specifically requested, by using the
--boot-timer
dedicated cmdline parameter. - firecracker and jailer
--version
now gets updated on each devtool build to the output ofgit describe --dirty
, if the git repo is available. - MicroVM process is only attached to the cgroups defined by using
--cgroups
or the ones defined indirectly by using--node
. - Changed
devtool build
to build jailer binary formusl
only targets. Building jailer binary fornon-musl
targets have been removed.
- Added a new API call,
PUT /metrics
, for configuring the metrics system. - Added
app_name
field in InstanceInfo struct for storing the application name. - New command-line parameters for
firecracker
, named--log-path
,--level
,--show-level
and--show-log-origin
that can be used for configuring the Logger when starting the process. When using this method for configuration, only--log-path
is mandatory. - Added a guide for updating the dev container image.
- Added a new API call,
PUT /mmds/config
, for configuring theMMDS
with a custom valid link-local IPv4 address. - Added experimental JSON response format support for MMDS guest applications requests.
- Added metrics for the vsock device.
- Added
devtool strip
command which removes debug symbols from the release binaries. - Added the
tx_malformed_frames
metric for the virtio net device, emitted when a TX frame missing the VNET header is encountered.
- Added
--version
flag to both Firecracker and Jailer. - Return
405 Method Not Allowed
MMDS response for non HTTPGET
MMDS requests originating from guest. - Fixed folder permissions in the jail (#1802).
- Any number of whitespace characters are accepted after ":" when parsing HTTP headers.
- Potential panic condition caused by the net device expecting to find a VNET header in every frame.
- Potential crash scenario caused by "Content-Length" HTTP header field accepting negative values.
- Fixed #1754 - net: traffic blocks when running ingress UDP performance tests with very large buffers.
- Updated CVE-2019-3016 mitigation information in Production Host Setup
- In case of using an invalid JSON as a 'config-file' for Firecracker, the process will exit with return code 152.
- Removed the
testrun.sh
wrapper. - Removed
metrics_fifo
field from the logger configuration. - Renamed
log_fifo
field from LoggerConfig tolog_path
andmetrics_fifo
field from MetricsConfig tometrics_path
. PATCH /drives/{id}
only allowed post-boot. UsePUT
for pre-boot updates to existing configurations.PATCH /network-interfaces/{id}
only allowed post-boot. UsePUT
for pre-boot updates to existing configurations.- Changed returned status code from
500 Internal Server Error
to501 Not Implemented
, for queries on the MMDS endpoint in IMDS format, when the requested resource value type is unsupported. - Allowed the MMDS data store to be initialized with all supported JSON types. Retrieval of these values within the guest, besides String, Array, and Dictionary, is only possible in JSON mode.
PATCH
request on/mmds
before the data store is initialized returns403 BadRequest
.- Segregated MMDS documentation in MMDS design documentation and MMDS user guide documentation.
- Support for booting with an initial RAM disk image. This image can be
specified through the new
initrd_path
field of the/boot-source
API request.
- Fixed #1469 - Broken GitHub location for Firecracker release binary.
- The jailer allows changing the default api socket path by using the extra arguments passed to firecracker.
- Fixed #1456 - Occasional KVM_EXIT_SHUTDOWN and bad syscall (14) during VM shutdown.
- Updated the production host setup guide with steps for addressing CVE-2019-18960.
- The HTTP header parsing is now case insensitive.
- The
put_api_requests
andpatch_api_requests
metrics for net devices were un-swapped.
- Removed redundant
--seccomp-level
jailer parameter since it can be simply forwarded to the Firecracker executable using "end of command options" convention. - Removed
memory.dirty_pages
metric. - Removed
options
field from the logger configuration. - Decreased release binary size by ~15%.
- Changed default API socket path to
/run/firecracker.socket
. This path also applies when running with the jailer. - Disabled KVM dirty page tracking by default.
- Removed redundant RescanBlockDevice action from the /actions API.
The functionality is available through the PATCH /drives API.
See
docs/api_requests/patch-block.md
.
- Added support for GICv2.
- Fixed CVE-2019-18960 - Fixed a logical error in bounds checking performed on vsock virtio descriptors.
- Fixed #1283 - Can't start a VM in AARCH64 with vcpus number more than 16.
- Fixed #1088 - The backtrace are printed on
panic
, no longer causing a seccomp fault. - Fixed #1375 - Change logger options type from
Value
toVec<LogOption>
to prevent potential unwrap on None panics. - Fixed #1436 - Raise interrupt for TX queue used descriptors
- Fixed #1439 - Prevent achieving 100% cpu load when the net device rx is throttled by the ratelimiter
- Fixed #1437 - Invalid fields in rate limiter related API requests are now failing with a proper error message.
- Fixed #1316 - correctly determine the size of a virtio device backed by a block device.
- Fixed #1383 - Log failed api requests.
- Decreased release binary size by 10%.
- New command-line parameter for
firecracker
, named--no-api
, which will disable the API server thread. If set, the user won't be able to send any API requests, neither before, nor after the vm has booted. It must be paired with--config-file
parameter. Also, when API server is disabled, MMDS is no longer available now. - New command-line parameter for
firecracker
, named--config-file
, which represents the path to a file that contains a JSON which can be used for configuring and starting a microVM without sending any API requests. - The jailer adheres to the "end of command options" convention, meaning
all parameters specified after
--
are forwarded verbatim to Firecracker. - Added
KVM_PTP
support to the recommended guest kernel config. - Added entry in FAQ.md for Firecracker Guest timekeeping.
- Vsock API call:
PUT /vsocks/{id}
changed toPUT /vsock
and no longer appear to support multiple vsock devices. Any subsequent calls to this API endpoint will override the previous vsock device configuration. - Removed unused 'Halting' and 'Halted' instance states.
- Vsock host-initiated connections now implement a trivial handshake protocol. See the vsock doc for details. Related to: #1253, #1432, #1443
- Fixed serial console on aarch64 (GitHub issue #1147).
- Upon panic, the terminal is now reset to canonical mode.
- Explicit error upon failure of vsock device creation.
- The failure message returned by an API call is flushed in the log FIFOs.
- Insert virtio devices in the FDT in order of their addresses sorted from low to high.
- Enforce the maximum length of the network interface name to be 16 chars as specified in the Linux Kernel.
- Changed the vsock property
id
tovsock_id
so that the API client can be successfully generated from the swagger definition.
- New device: virtio-vsock, backed by Unix domain sockets (GitHub issue #650).
See
docs/vsock.md
.
- No error is thrown upon a flush metrics intent if logger has not been configured.
- Updated the documentation for integration tests.
- Fixed high CPU usage before guest network interface is brought up (GitHub issue #1049).
- Fixed an issue that caused the wrong date (month) to appear in the log.
- Fixed a bug that caused the seccomp filter to reject legit syscalls in some rare cases (GitHub issue #1206).
- Docs: updated the production host setup guide.
- Docs: updated the rootfs and kernel creation guide.
- Removed experimental support for vhost-based vsock devices.
- New API call:
PATCH /machine-config/
, used to update VM configuration, before the microVM boots. - Added an experimental swagger definition that includes the specification for the vsock API call.
- Added a signal handler for
SIGBUS
andSIGSEGV
that immediately terminates the process upon intercepting the signal. - Added documentation for signal handling utilities.
- Added [alpha] aarch64 support.
- Added metrics for successful read and write operations of MMDS, Net and Block devices.
vcpu_count
,mem_size_mib
andht_enabled
have been changed to be mandatory forPUT
requests on/machine-config/
.- Disallow invalid seccomp levels by exiting with error.
- Incorrect handling of bind mounts within the jailed rootfs.
- Corrected the guide for
Alpine
guest setup.
- Added [alpha] AMD support.
- New
devtool
command:prepare_release
. This updates the Firecracker version, crate dependencies and credits in preparation for a new release. - New
devtool
command:tag
. This creates a new git tag for the specified release number, based on the changelog contents. - New doc section about building with glibc.
- Dropped the JSON-formatted
context
command-line parameter from Firecracker in favor of individual classic command-line parameters. - When running with
jailer
the location of the API socket has changed to<jail-root-path>/api.socket
(API socket was moved inside the jail). PUT
andPATCH
requests on/mmds
with data containing any value type other thanString
,Array
,Object
will returns status code 400.- Improved multiple error messages.
- Removed all kernel modules from the recommended kernel config.
- Corrected the seccomp filter when building with glibc.
- Removed the
seccomp.bad_syscalls
metric.
- Corrected the conditional compilation of the seccomp rule for
madvise
.
- A
madvise
call issued by themusl
allocator was added to the seccomp whitelist to prevent Firecracker from terminating abruptly when allocating memory in certain conditions.
- New API action: SendCtrlAltDel, used to initiate a graceful shutdown, if the guest has driver support for i8042 and AT Keyboard. See the docs for details.
- New metric counting the number of egress packets with a spoofed MAC:
net.tx_spoofed_mac_count
. - New API call:
PATCH /network-interfaces/
, used to update the rate limiters on a network interface, after the start of a microVM.
- Added missing
vmm_version
field to the InstanceInfo API swagger definition, and marked several other mandatory fields as such. - New default command line for guest kernel:
reboot=k panic=1 pci=off nomodules 8250.nr_uarts=0 i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd
.
- virtio-blk: VIRTIO_BLK_T_FLUSH now working as expected.
- Vsock devices can be attached when starting Firecracker using the jailer.
- Vsock devices work properly when seccomp filtering is enabled.
- Documentation for development environment setup on AWS in
dev-machine-setup.md
. - Documentation for microVM networking setup in
docs/network-setup.md
. - Limit the maximum supported vCPUs to 32.
- Log the app version when the
Logger
is initialized. - Pretty print panic information.
- Firecracker terminates with exit code 148 when a non-whitelisted syscall is intercepted.
- Fixed build with the
vsock
feature.
- Documentation for Logger API Requests in
docs/api_requests/logger.md
. - Documentation for Actions API Requests in
docs/api_requests/actions.md
. - Documentation for MMDS in
docs/mmds.md
. - Flush metrics on request via a PUT
/actions
with theaction_type
field set toFlushMetrics
.
- Updated the swagger definition of the
Logger
to specify the required fields and provide default values for optional fields. - Default
seccomp-level
is2
(was previously 0). - API Resource IDs can only contain alphanumeric characters and underscores.
- Seccomp filters are now applied to all Firecracker threads.
- Enforce minimum length of 1 character for the jailer ID.
- Exit with error code when starting the jailer process fails.
- Removed
InstanceHalt
from the list of possible actions.
- The
/logger
API has a new field calledoptions
. This is an array of strings that specify additional logging configurations. The only supported value isLogDirtyPages
. - When the
LogDirtyPages
option is configured viaPUT /logger
, a new metric calledmemory.dirty_pages
is computed as the number of pages dirtied by the guest since the last time the metric was flushed. - Log messages on both graceful and forceful termination.
- Availability of the list of dependencies for each commit inside the code base.
- Documentation on vsock experimental feature and host setup recommendations.
PUT
requests on/mmds
always return 204 on success.PUT
operations on/network-interfaces
API resources no longer accept the previously requiredstate
parameter.- The jailer starts with
--seccomp-level=2
(was previously 0) by default. - Log messages use
anonymous-instance
as instance id if none is specified.
- Fixed crash upon instance start on hosts without 1GB huge page support.
- Fixed "fault_message" inconsistency between Open API specification and code base.
- Ensure MMDS compatibility with C5's IMDS implementation.
- Corrected the swagger specification to ensure
OpenAPI 2.0
compatibility.
- Apache-2.0 license
- Docs:
- Experimental vhost-based vsock implementation.
- Improved MMDS network stack performance.
- If the logging system is not yet initialized (via
PUT /logger
), log events are now sent to stdout/stderr. - Moved the
instance_info_fails
metric underget_api_requests
- Improved readme and added links to more detailed information, now featured in subject-specific docs.
- Fixed bug in the MMDS network stack, that caused some RST packets to be sent without a destination.
- Fixed bug in
PATCH /drives
, whereby the ID in the path was not checked against the ID in the body.
- The Swagger definition was corrected.
- Each Firecracker process has an associated microVM Metadata Store (MMDS). Its
contents can be configured using the
/mmds
API resource.
- The boot source is specified only with the
kernel_image_path
and the optional parameterboot_args
. All other fields are removed. - The
path_on_host
property in the drive specification is now marked as mandatory. - PATCH drive only allows patching/changing the
path_on_host
property. - All PUT and PATCH requests return the status code 204.
- CPUID brand string (aka model name) now includes the host CPU frequency.
- API requests which add guest network interfaces have an additional parameter,
allow_mmds_requests
which defaults tofalse
. - Stopping the guest (e.g. using the
reboot
command) also terminates the Firecracker process. When the Firecracker process ends for any reason, (other thankill -9
), metrics are flushed at the very end. - On startup
jailer
closes all inherited file descriptors based onsysconf(_SC_OPEN_MAX)
except input, output and error. - The microVM ID prefixes each Firecracker log line. This ID also appears
in the process
cmdline
so it's now possible tops | grep <ID>
for it.
- Seccomp filtering is configured via the
--seccomp-level
jailer parameter. - Firecracker logs the starting addresses of host memory areas provided as guest memory slots to KVM.
- The metric
panic_count
gets incremented to signal that a panic has occurred. - Firecracker logs a backtrace when it crashes following a panic.
- Added basic instrumentation support for measuring boot time.
StartInstance
is a synchronous API request (it used to be an asynchronous request).
- Ensure that fault messages sent by the API have valid JSON bodies.
- Use HTTP response code 500 for internal Firecracker errors, and 400 for user errors on InstanceStart.
- Serialize the machine configuration fields to the correct data types (as specified in the Swagger definition).
- NUMA node assignment is properly enforced by the jailer.
- The
is_root_device
andis_read_only
properties are now marked as required in the Swagger definition ofDrive
object properties.
GET
requests on the/actions
API resource are no longer supported.- The metrics associated with asynchronous actions have been removed.
- Remove the
action_id
parameter forInstanceStart
, both from the URI and the JSON request body.
- The jailer can now be configured to enter a preexisting network namespace, and to run as a daemon.
- Enabled PATCH operations on
/drives
resources.
- The microVM
id
supplied to the jailer may now contain alphanumeric characters and hyphens, up to a maximum length of 64 characters. - Replaced the
permissions
property of/drives
resources with a boolean. - Removed the
state
property of/drives
resources.
- Rate limiting functionality allows specifying an initial one time burst size.
- Firecracker can now boot from an arbitrary boot partition by specifying its unique id in the driver's API call.
- Block device rescan is triggered via a PUT
/actions
with the drive ID in the action body'spayload
field and theaction_type
field set toBlockDeviceRescan
.
- Removed
noapic
from the default guest kernel command line. - The
action_id
parameter is no longer required for synchronous PUT requests to/actions
. - PUT requests are no longer allowed on
/drives
resources after the guest has booted.
- Fixed guest instance kernel loader to accelerate vCPUs launch and consequently guest kernel boot.
- Fixed network emulation to improve IO performance.
- Firecracker uses two different named pipes to record human readable logs and metrics, respectively.
- Seccomp filtering can be enabled via setting the
USE_SECCOMP
environment variable. - It is possible to supply only a partial specification when attaching a rate limiter (i.e. just the bandwidth or ops parameter).
- Errors related to guest network interfaces are now more detailed.
- Fixed a bug that was causing Firecracker to panic whenever a
PUT
request was sent on an existing network interface. - The
id
parameter of thejailer
is required to be an RFC 4122-compliant UUID. - Fixed an issue which caused the network RX rate limiter to be more restrictive than intended.
- API requests which contain unknown fields will generate an error.
- Fixed an issue related to high CPU utilization caused by improper
KVM PIT
configuration. - It is now possible to create more than one network tun/tap interface inside a jailed Firecracker.
- Added metrics for API requests, VCPU and device actions for the serial
console (
UART
), keyboard (i8042
), block and network devices. Metrics are logged every 60 seconds. - A CPU features template for C3 is available, in addition to the one for T2.
- Seccomp filters restrict Firecracker from calling any other system calls than
the minimum set it needs to function properly. The filters are enabled by
setting the
USE_SECCOMP
environment variable to 1 before running Firecracker. - Firecracker can be started by a new binary called
jailer
. The jailer takes as command line arguments a unique ID, the path to the Firecracker binary, the NUMA node that Firecracker will be assigned to and auid
andgid
for Firecracker to run under. It sets up achroot
environment and acgroup
, and calls exec to morph into Firecracker.
- In case of failure, the metrics and the panic location are logged before aborting.
- Metric values are reset with every flush.
CPUTemplate
is now calledCpuTemplate
in order to work seamlessly with the swagger code generator for Go.firecracker-beta.yaml
is now calledfirecracker.yaml
.
- Handling was added for several untreated KVM exit scenarios, which could have led to panic.
- Fixed a bug that caused Firecracker to crash when attempting to disable the
IA32_DEBUG_INTERFACE MSR
flag in the T2 CPU features.
- Removed a leftover file generated by the logger unit tests.
- Removed
firecracker-v1.0.yaml
.
- The CPU Template can be set with an API call on
PUT /machine-config
. The only available template is T2. - Hyperthreading can be enabled/disabled with an API call on
PUT /machine-config
. By default, hyperthreading is disabled. - Added boot time performance test (
tests/performance/test_boottime.py
). - Added Rate Limiter for VirtIO/net and VirtIO/net devices. The Rate Limiter
uses two token buckets to limit rate on bytes/s and ops/s. The rate limiter
can be (optionally) configured per drive with a
PUT
on/drives/{drive_id}
and per network interface with aPUT
on/network-interface/{iface_id}
. - Implemented pre-boot PUT updates for
/boot-source
,/drives
,/network-interfaces
and/vsock
. - Added integration tests for
PUT
updates.
- Moved the API definition (
swagger/firecracker-beta.yaml
) to theapi_server
crate. - Removed
"console=ttyS0"
and added"8250.nr_uarts=0"
to the default kernel command line to decrease the boot time. - Changed the CPU topology to have all logical CPUs on a single socket.
- Removed the upper bound on CPU count as with musl there is no good way to get the total number of logical processors on a host.
- Build time tests now print the full output of commands.
- Disabled the Performance Monitor Unit and the Turbo Boost.
- Check the expected KVM capabilities before starting the VM.
- Logs now have timestamps.
testrun.sh
can run on platforms with more than one package manager by setting the package manager via a command line parameter (-p
).- Allow correct set up of multiple network-interfaces with auto-generated MAC.
- Fixed sporadic bug in VirtIO which was causing lost packages.
- Don't allow
PUT
requests with empty body on/machine-config
. - Deny
PUT
operations after the microvm boots (exception: the temporarily fix for live resize of block devices).
- Removed examples crate. This used to have a Python example of starting
Firecracker. This is replaced by
test_api.py
integration tests. - Removed helper scripts for getting coverage and coding style errors. These
were replaced by
test_coverage.py
andtest_style.py
test integration tests. - Removed
--vmm-no-api
command line option. Firecracker can only be started via the API.
- Users can interrogate the Machine Configuration (i.e. vcpu count and memory
size) using a
GET
request on/machine-config
. - The logging system can be configured through the API using a
PUT
on/logger
. - Block devices support live resize by calling
PUT
with the same parameters as when the block was created. - Release builds have Link Time Optimization (LTO) enabled.
- Firecracker is built with
musl
, resulting in a statically linked binary. - More in-tree integration tests were added as part of the continuous integration system.
- The vcpu count is enforced to
1
or an even number. - The Swagger definition of rate limiters was updated.
- Syslog-enabled logs were replaced with a host-file backed mechanism.
- The host topology of the CPU and the caches is not leaked into the microvm anymore.
- Boot time was improved by advertising the availability of the TSC deadline timer.
- Fixed an issue which prevented Firecracker from working on 4.14 (or newer) host kernels.
- Specifying the MAC address for an interface through the API is optional.
- Removed support for attaching vsock devices.
- Removed support for building Firecracker with glibc.
- Users can now interrogate Instance Information (currently just instance state) through the API.
- Renamed
api/swagger/all.yaml
toapi/swagger/firecracker-v1.0.yaml
which specifies targeted API support for Firecracker v1.0. - Renamed
api/swagger/firecracker-v0.1.yaml
toapi/swagger/firecracker-beta.yaml
which specifies the currently supported API. - Users can now enforce that an emulated block device is read-only via the API.
To specify whether a block device is read-only or read-write, an extra
"permissions" field was added to the Drive definition in the API. The root
filesystem is automatically mounted in the guest OS as
ro
/rw
according to the specified "permissions". It's the responsibility of the user to mount any other read-only block device as such within the guest OS. - Users can now stop the guest VM using the API. Actions of type
InstanceHalt
are now supported via the API.
- Added support for
getDeviceID()
invirtIO-block
. Without this, the guest Linux kernel would complain at boot time that the operation is unsupported. stdin
control is returned to the Firecracker process when guest VM is inactive. Raw modestdin
is forwarded to the guest OS when guest VM is running.
- Removed
api/swagger/actions.yaml
. - Removed
api/swagger/devices.yaml
. - Removed
api/swagger/firecracker-mvp.yaml
. - Removed
api/swagger/limiters.yaml
.
- Users can now specify the MAC address of a guest network interface via the
PUT
network interface API request. Previously, the guest MAC address parameter was ignored.
- Fixed a guest memory allocation issue, which previously led to a potentially significant memory chunk being wasted.
- Fixed an issue which caused compilation problems, due to a compatibility breaking transitive dependency in the tokio suite of crates.
- One-process virtual machine manager (one Firecracker per microVM).
- RESTful API running on a unix socket. The API supported by v0.1 can be found
at
api/swagger/firecracker-v0.1.yaml
. - Emulated keyboard (
i8042
) and serial console (UART
). The microVM serial console input and output are connected to those of the Firecracker process (this allows direct console access to the guest OS). - The capability of mapping an existing host tun-tap device as a VirtIO/net device into the microVM.
- The capability of mapping an existing host file as a GirtIO/block device into the microVM.
- The capability of creating a VirtIO/vsock between the host and the microVM.
- Default demand fault paging & CPU oversubscription.