Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split instance state into Instance and VMM tables #4194

Merged
merged 59 commits into from
Oct 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
4f56437
Define vmm table & update instance table
gjcolombo Sep 25, 2023
ed4f8e0
update DB model types for instances and VMMs
gjcolombo Sep 25, 2023
75a2f6a
redefine internal instance/vmm runtime state types
gjcolombo Sep 25, 2023
27eb921
Update sled-agent params types
gjcolombo Sep 26, 2023
f1ada46
sled agent: rework instance state management
gjcolombo Sep 26, 2023
cae4922
Rework InstanceInner -> InstanceStates contract & termination logic
gjcolombo Sep 27, 2023
74257ae
Rework simulated instances
gjcolombo Sep 27, 2023
a5c3488
Fix up simulated sled agent collection tests
gjcolombo Sep 27, 2023
4bcced2
Plumb parameters from sled agent entry points
gjcolombo Sep 27, 2023
39b7e64
sled agent: clean up test transcription errors
gjcolombo Sep 27, 2023
70cc2fb
update sled agent OpenAPI spec
gjcolombo Sep 27, 2023
854c77f
sled agent: reimplement generated-type From impls
gjcolombo Sep 27, 2023
46560b2
nexus: adapt some db queries to instance/vmm split
gjcolombo Sep 27, 2023
df011d5
Draft CTE for updating an instance/VMM in a single statement
gjcolombo Sep 27, 2023
896ec96
nexus: fix network interface queries
gjcolombo Sep 28, 2023
f5241c3
clean up test code that assumes instances always have sleds
gjcolombo Sep 28, 2023
94021a0
nexus: return instance/vmm info tuples; fix up serial console APIs
gjcolombo Sep 29, 2023
38b3939
Fix handling of instance/vmm state changes from sled agent
gjcolombo Sep 29, 2023
7ec7f84
Update Nexus::handle_instance_put_result
gjcolombo Sep 29, 2023
8dfdfdd
Update Nexus::instance_request_state
gjcolombo Sep 29, 2023
6dfeae1
Remove Nexus::instance_sled
gjcolombo Sep 29, 2023
3952f90
Rework instance create saga
gjcolombo Sep 29, 2023
bcc2f3b
Update instance start saga
gjcolombo Sep 29, 2023
0edf89c
Update instance migration saga
gjcolombo Sep 29, 2023
d5ab2f2
Update instance delete saga
gjcolombo Sep 29, 2023
cafa141
Update disk snapshot saga
gjcolombo Sep 29, 2023
781503e
Fix build errors
gjcolombo Sep 29, 2023
28683b8
cleanup: use dendrite deletion helper from start saga undo
gjcolombo Sep 30, 2023
bac9309
Actually check reservoir space when allocating to sleds
gjcolombo Sep 30, 2023
b3df607
bugfix: reorder instance columns in Diesel table schema
gjcolombo Sep 30, 2023
fd30291
bugfix: only query ID in 'find' prongs of instance/vmm update CTE
gjcolombo Sep 30, 2023
902c0a0
bugfix: use a valid state name in instance state subquery
gjcolombo Sep 30, 2023
7ffd8ef
Update Nexus OpenAPI spec
gjcolombo Sep 30, 2023
db4fa9c
Re-enable sled agent -> Nexus updates; fix tests
gjcolombo Oct 1, 2023
6534c38
Merge fix: use connections instead of pool in new code
gjcolombo Oct 2, 2023
521163f
remove dead code
gjcolombo Oct 2, 2023
6114aec
Update omdb to handle vmm optionality
gjcolombo Oct 2, 2023
fe709ad
clippy
gjcolombo Oct 2, 2023
5285226
omdb: fix expected schema version in tests
gjcolombo Oct 2, 2023
b8709e7
Don't abort instance monitor task on RunningState drop
gjcolombo Oct 3, 2023
f635ba1
Report Propolis Stopped state to Nexus as Stopping
gjcolombo Oct 3, 2023
71dc6e9
Remove instances from instance manager if terminated before starting
gjcolombo Oct 3, 2023
9a93a78
fix doc build
gjcolombo Oct 3, 2023
20c0d39
Merge branch 'main' into gjcolombo/your-vmm-table-is-ready
gjcolombo Oct 6, 2023
98bd378
Remove immutable data from VmmRuntimeState API struct
gjcolombo Oct 6, 2023
f20a6d8
Remove useless parameter from db::model::Instance constructor
gjcolombo Oct 6, 2023
f845e3a
clean up instance update-and-check CTE
gjcolombo Oct 6, 2023
32358bf
use Running as the state for instances w/active VMMs
gjcolombo Oct 6, 2023
3f41b17
clarify remarks re safety of rude termination
gjcolombo Oct 6, 2023
e688552
break schema upgrade into multiple statements
gjcolombo Oct 6, 2023
78f72e5
Standardize on "propolis_id" name in most places
gjcolombo Oct 6, 2023
d2a0636
make Nexus sole owner of Instance records' state values
gjcolombo Oct 7, 2023
b8cbde5
amend schema update README to suggest named constraints
gjcolombo Oct 7, 2023
8923fb4
improve comment on instance_and_vmm_update_runtime
gjcolombo Oct 7, 2023
e5b2a54
add issue number to TODO
gjcolombo Oct 7, 2023
2933c4b
clarify doc comment
gjcolombo Oct 9, 2023
6b5dfd3
Merge branch 'main' into gjcolombo/your-vmm-table-is-ready
gjcolombo Oct 11, 2023
ec65227
Merge branch 'main' into gjcolombo/your-vmm-table-is-ready
gjcolombo Oct 12, 2023
acbaf43
pick up Diesel error changes in new code
gjcolombo Oct 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 27 additions & 16 deletions clients/nexus-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,22 +88,41 @@ impl From<omicron_common::api::internal::nexus::InstanceRuntimeState>
s: omicron_common::api::internal::nexus::InstanceRuntimeState,
) -> Self {
Self {
run_state: s.run_state.into(),
sled_id: s.sled_id,
propolis_id: s.propolis_id,
dst_propolis_id: s.dst_propolis_id,
propolis_addr: s.propolis_addr.map(|addr| addr.to_string()),
gen: s.gen.into(),
migration_id: s.migration_id,
propolis_gen: s.propolis_gen.into(),
ncpus: s.ncpus.into(),
memory: s.memory.into(),
hostname: s.hostname,
propolis_id: s.propolis_id,
time_updated: s.time_updated,
}
}
}

impl From<omicron_common::api::internal::nexus::VmmRuntimeState>
for types::VmmRuntimeState
{
fn from(s: omicron_common::api::internal::nexus::VmmRuntimeState) -> Self {
Self {
gen: s.gen.into(),
state: s.state.into(),
time_updated: s.time_updated,
}
}
}

impl From<omicron_common::api::internal::nexus::SledInstanceState>
for types::SledInstanceState
{
fn from(
s: omicron_common::api::internal::nexus::SledInstanceState,
) -> Self {
Self {
instance_state: s.instance_state.into(),
propolis_id: s.propolis_id,
vmm_state: s.vmm_state.into(),
}
}
}

impl From<omicron_common::api::external::InstanceState>
for types::InstanceState
{
Expand All @@ -124,14 +143,6 @@ impl From<omicron_common::api::external::InstanceState>
}
}

impl From<omicron_common::api::external::InstanceCpuCount>
for types::InstanceCpuCount
{
fn from(s: omicron_common::api::external::InstanceCpuCount) -> Self {
Self(s.0)
}
}

impl From<omicron_common::api::external::Generation> for types::Generation {
fn from(s: omicron_common::api::external::Generation) -> Self {
Self(i64::from(&s) as u64)
Expand Down
38 changes: 24 additions & 14 deletions clients/sled-agent-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,9 @@ impl From<omicron_common::api::internal::nexus::InstanceRuntimeState>
s: omicron_common::api::internal::nexus::InstanceRuntimeState,
) -> Self {
Self {
run_state: s.run_state.into(),
sled_id: s.sled_id,
propolis_id: s.propolis_id,
dst_propolis_id: s.dst_propolis_id,
propolis_addr: s.propolis_addr.map(|addr| addr.to_string()),
migration_id: s.migration_id,
propolis_gen: s.propolis_gen.into(),
ncpus: s.ncpus.into(),
memory: s.memory.into(),
hostname: s.hostname,
gen: s.gen.into(),
time_updated: s.time_updated,
}
Expand Down Expand Up @@ -85,22 +78,39 @@ impl From<types::InstanceRuntimeState>
{
fn from(s: types::InstanceRuntimeState) -> Self {
Self {
run_state: s.run_state.into(),
sled_id: s.sled_id,
propolis_id: s.propolis_id,
dst_propolis_id: s.dst_propolis_id,
propolis_addr: s.propolis_addr.map(|addr| addr.parse().unwrap()),
migration_id: s.migration_id,
propolis_gen: s.propolis_gen.into(),
ncpus: s.ncpus.into(),
memory: s.memory.into(),
hostname: s.hostname,
gen: s.gen.into(),
time_updated: s.time_updated,
}
}
}

impl From<types::VmmRuntimeState>
for omicron_common::api::internal::nexus::VmmRuntimeState
{
fn from(s: types::VmmRuntimeState) -> Self {
Self {
state: s.state.into(),
gen: s.gen.into(),
time_updated: s.time_updated,
}
}
}

impl From<types::SledInstanceState>
for omicron_common::api::internal::nexus::SledInstanceState
{
fn from(s: types::SledInstanceState) -> Self {
Self {
instance_state: s.instance_state.into(),
propolis_id: s.propolis_id,
vmm_state: s.vmm_state.into(),
}
}
}

impl From<types::InstanceState>
for omicron_common::api::external::InstanceState
{
Expand Down
31 changes: 1 addition & 30 deletions common/src/api/external/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -739,6 +739,7 @@ pub enum ResourceType {
UpdateableComponent,
UserBuiltin,
Zpool,
Vmm,
}

// IDENTITY METADATA
Expand Down Expand Up @@ -866,25 +867,6 @@ impl InstanceState {
InstanceState::Destroyed => "destroyed",
}
}

/// Returns true if the given state represents a fully stopped Instance.
/// This means that a transition from an !is_stopped() state must go
/// through Stopping.
pub fn is_stopped(&self) -> bool {
match self {
InstanceState::Starting => false,
InstanceState::Running => false,
InstanceState::Stopping => false,
InstanceState::Rebooting => false,
InstanceState::Migrating => false,

InstanceState::Creating => true,
InstanceState::Stopped => true,
InstanceState::Repairing => true,
InstanceState::Failed => true,
InstanceState::Destroyed => true,
}
}
}

/// The number of CPUs in an Instance
Expand Down Expand Up @@ -912,17 +894,6 @@ pub struct InstanceRuntimeState {
pub time_run_state_updated: DateTime<Utc>,
}

impl From<crate::api::internal::nexus::InstanceRuntimeState>
for InstanceRuntimeState
{
fn from(state: crate::api::internal::nexus::InstanceRuntimeState) -> Self {
InstanceRuntimeState {
run_state: state.run_state,
time_run_state_updated: state.time_updated,
}
}
}

/// View of an Instance
#[derive(ObjectIdentity, Clone, Debug, Deserialize, Serialize, JsonSchema)]
pub struct Instance {
Expand Down
69 changes: 44 additions & 25 deletions common/src/api/internal/nexus.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,40 +29,59 @@ pub struct DiskRuntimeState {
pub time_updated: DateTime<Utc>,
}

/// Runtime state of the Instance, including the actual running state and minimal
/// metadata
///
/// This state is owned by the sled agent running that Instance.
/// The "static" properties of an instance: information about the instance that
/// doesn't change while the instance is running.
#[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)]
pub struct InstanceRuntimeState {
/// runtime state of the Instance
pub run_state: InstanceState,
/// which sled is running this Instance
pub sled_id: Uuid,
/// which propolis-server is running this Instance
pub propolis_id: Uuid,
/// the target propolis-server during a migration of this Instance
pub dst_propolis_id: Option<Uuid>,
/// address of propolis-server running this Instance
pub propolis_addr: Option<SocketAddr>,
/// migration id (if one in process)
pub migration_id: Option<Uuid>,
/// The generation number for the Propolis and sled identifiers for this
/// instance.
pub propolis_gen: Generation,
/// number of CPUs allocated for this Instance
pub struct InstanceProperties {
pub ncpus: InstanceCpuCount,
/// memory allocated for this Instance
pub memory: ByteCount,
/// RFC1035-compliant hostname for the Instance.
/// RFC1035-compliant hostname for the instance.
// TODO-cleanup different type?
pub hostname: String,
/// generation number for this state
}

/// The dynamic runtime properties of an instance: its current VMM ID (if any),
/// migration information (if any), and the instance state to report if there is
/// no active VMM.
#[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)]
pub struct InstanceRuntimeState {
/// The instance's currently active VMM ID.
pub propolis_id: Option<Uuid>,
/// If a migration is active, the ID of the target VMM.
pub dst_propolis_id: Option<Uuid>,
/// If a migration is active, the ID of that migration.
pub migration_id: Option<Uuid>,
/// Generation number for this state.
pub gen: Generation,
/// timestamp for this information
/// Timestamp for this information.
pub time_updated: DateTime<Utc>,
}

/// The dynamic runtime properties of an individual VMM process.
#[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)]
pub struct VmmRuntimeState {
/// The last state reported by this VMM.
pub state: InstanceState,
/// The generation number for this VMM's state.
pub gen: Generation,
/// Timestamp for the VMM's state.
pub time_updated: DateTime<Utc>,
}

/// A wrapper type containing a sled's total knowledge of the state of a
/// specific VMM and the instance it incarnates.
gjcolombo marked this conversation as resolved.
Show resolved Hide resolved
#[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)]
pub struct SledInstanceState {
/// The sled's conception of the state of the instance.
pub instance_state: InstanceRuntimeState,

/// The ID of the VMM whose state is being reported.
pub propolis_id: Uuid,

/// The most recent state of the sled's VMM process.
pub vmm_state: VmmRuntimeState,
}

// Oximeter producer/collector objects.

/// Information announced by a metric server, used so that clients can contact it and collect
Expand Down
Loading
Loading