Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'mso_schema_site_contract_service_graph' resource fails validation by NDO (DCNE-95) #273

Open
andbyrne opened this issue Jun 4, 2024 · 3 comments · Fixed by #284
Open
Assignees
Labels
bug Something isn't working jira-sync Sync this issue to Jira

Comments

@andbyrne
Copy link

andbyrne commented Jun 4, 2024

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v1.5.7

  • provider registry.terraform.io/ciscodevnet/mso v1.2.0

MSO version

  • V 4.3.1.1008

APIC version and APIC Platform for Site Level Resources

  • V 6.0.5h on-prem

Affected Resource(s)

  • mso_schema_site_service_graph
  • mso_schema_site_contract_service_graph

Terraform Configuration Files

locals {
  tenant    = "TENANT1"
  tenant_id = mso_tenant.tenant.id
  schema    = "SCHEMA1"
  schema_id = mso_schema.schema.id
  template  = "TEMPLATE1"
  site1_id  = "61469febeea713e579d584bf"
  site2_id  = "6146a02beea713e579d584c0"
}

terraform {
  required_providers {
    mso = {
      source = "CiscoDevNet/mso"
    }
  }
}

provider "mso" {
  insecure = true
  platform = "nd"
}

resource "mso_tenant" "tenant" {
  name              = local.tenant
  display_name      = local.tenant
  orchestrator_only = true

  site_associations {
    site_id = local.site1_id
  }
  site_associations {
    site_id = local.site2_id
  }

  user_associations {
    user_id = "48d105bdfbc49a5fcf39a10b961386e61dfe40085cb0315d818b761c5735dafa"
  }
  user_associations {
    user_id = "7f9499f324ce1a343206f0d1aef3637da2051dc6b2520374dd55a78a14b51764"
  }
}

resource "mso_schema" "schema" {
  name = local.schema

  template {
    name          = local.template
    display_name  = local.template
    template_type = "aci_multi_site"
    tenant_id     = local.tenant_id
  }
}

resource "mso_schema_site" "schema_site1" {
  schema_id           = local.schema_id
  site_id             = local.site1_id
  template_name       = local.template
  undeploy_on_destroy = true
}

resource "mso_schema_site" "schema_site2" {
  schema_id           = local.schema_id
  site_id             = local.site2_id
  template_name       = local.template
  undeploy_on_destroy = true
}

resource "mso_schema_template_vrf" "schema_template_vrf" {
  ip_data_plane_learning        = "enabled"
  layer3_multicast              = false
  name                          = "VRF1"
  display_name                  = "VRF1"
  preferred_group               = false
  schema_id                     = local.schema_id
  site_aware_policy_enforcement = true
  template                      = local.template
  vzany                         = true
}

resource "mso_schema_template_bd" "schema_template_bd" {
  arp_flooding                    = false
  intersite_bum_traffic           = false
  ipv6_unknown_multicast_flooding = "flood"
  layer2_stretch                  = true
  layer2_unknown_unicast          = "proxy"
  layer3_multicast                = false
  multi_destination_flooding      = "flood_in_bd"
  name                            = "BD1"
  display_name                    = "BD1"
  optimize_wan_bandwidth          = false
  schema_id                       = local.schema_id
  template_name                   = local.template
  unicast_routing                 = true
  unknown_multicast_flooding      = "flood"
  vrf_name                        = "VRF1"
  depends_on                      = [mso_schema_template_vrf.schema_template_vrf]

}

resource "mso_schema_template_bd_subnet" "schema_template_bd_subnet" {
  bd_name            = "BD1"
  ip                 = "10.1.1.1/24"
  no_default_gateway = false
  primary            = false
  querier            = false
  schema_id          = local.schema_id
  scope              = "private"
  shared             = false
  template_name      = local.template
  depends_on         = [mso_schema_template_bd.schema_template_bd]

}

resource "mso_schema_template_filter_entry" "schema_template_filter_entry" {
  arp_flag             = "unspecified"
  destination_from     = "unspecified"
  destination_to       = "unspecified"
  entry_display_name   = "IP"
  entry_name           = "IP"
  ether_type           = "ip"
  ip_protocol          = "unspecified"
  match_only_fragments = false
  name                 = "FLT1"
  display_name         = "FLT1"
  schema_id            = local.schema_id
  source_from          = "unspecified"
  source_to            = "unspecified"
  stateful             = false
  tcp_session_rules = [
    "unspecified",
  ]
  template_name = local.template
}


resource "mso_schema_template_contract" "schema_template_contract" {
  contract_name = "CON1"
  filter_type   = "bothWay"
  schema_id     = local.schema_id
  scope         = "context"
  template_name = local.template
  filter_relationship {
    filter_name = "FLT1"
    filter_type = "bothWay"
  }
  depends_on = [mso_schema_template_filter_entry.schema_template_filter_entry]
}


resource "mso_schema_template_vrf_contract" "schema_template_vrf_contract_consumer" {
  contract_name     = "CON1"
  relationship_type = "consumer"
  schema_id         = local.schema_id
  template_name     = local.template
  vrf_name          = "VRF1"
  depends_on        = [mso_schema_template_contract.schema_template_contract]

}

resource "mso_schema_template_vrf_contract" "schema_template_vrf_contract_provider" {
  contract_name     = "CON1"
  relationship_type = "provider"
  schema_id         = local.schema_id
  template_name     = local.template
  vrf_name          = "VRF1"
  depends_on        = [mso_schema_template_contract.schema_template_contract]

}

resource "mso_schema_template_service_graph" "schema_template_service_graph" {
  schema_id          = local.schema_id
  service_graph_name = "SG1"
  template_name      = local.template

  service_node {
    type = "firewall"
  }
}

resource "mso_schema_template_contract_service_graph" "schema_template_contract_service_graph" {
  contract_name      = "CON1"
  schema_id          = local.schema_id
  service_graph_name = "SG1"
  template_name      = local.template

  node_relationship {
    consumer_connector_bd_name = "BD1"
    provider_connector_bd_name = "BD1"
  }

  depends_on = [mso_schema_template_service_graph.schema_template_service_graph, mso_schema_template_contract.schema_template_contract]

}

resource "mso_schema_site_service_graph" "schema_site_service_graph_site1" {
  schema_id          = local.schema_id
  service_graph_name = "SG1"
  site_id            = local.site1_id
  template_name      = local.template

  service_node {
    device_dn = "uni/tn-${local.tenant}/lDevVip-FW1"
  }

  depends_on = [mso_schema_template_service_graph.schema_template_service_graph]
}

resource "mso_schema_site_service_graph" "schema_site_service_graph_site2" {
  schema_id          = local.schema_id
  service_graph_name = "SG1"
  site_id            = local.site2_id
  template_name      = local.template

  service_node {
    device_dn = "uni/tn-${local.tenant}/lDevVip-FW1"
  }

  depends_on = [mso_schema_template_service_graph.schema_template_service_graph]
}

resource "mso_schema_site_contract_service_graph" "schema_site_contract_service_graph_site1" {
  contract_name      = "CON1"
  schema_id          = local.schema_id
  service_graph_name = "SG1"
  site_id            = local.site1_id
  template_name      = local.template

  node_relationship {
    consumer_connector_cluster_interface      = "INT1"
    consumer_connector_redirect_policy        = "PBR1"
    consumer_connector_redirect_policy_tenant = local.tenant
    provider_connector_cluster_interface      = "INT1"
    provider_connector_redirect_policy        = "PBR1"
    provider_connector_redirect_policy_tenant = local.tenant
  }
  depends_on = [mso_schema_template_contract_service_graph.schema_template_contract_service_graph]
}

resource "mso_schema_site_contract_service_graph" "schema_site_contract_service_graph_site2" {
  contract_name      = "CON1"
  schema_id          = local.schema_id
  service_graph_name = "SG1"
  site_id            = local.site2_id
  template_name      = local.template

  node_relationship {
    consumer_connector_cluster_interface      = "INT1"
    consumer_connector_redirect_policy        = "PBR1"
    consumer_connector_redirect_policy_tenant = local.tenant
    provider_connector_cluster_interface      = "INT1"
    provider_connector_redirect_policy        = "PBR1"
    provider_connector_redirect_policy_tenant = local.tenant
  }
  depends_on = [mso_schema_template_contract_service_graph.schema_template_contract_service_graph]
}

Debug Output

mso_schema_site_service_graph:

<snip>
2024-07-10T16:45:39.720+1000 [ERROR] vertex "mso_schema_site_service_graph.schema_site_service_graph_site1" error: No Template found
2024-07-10T16:45:39.727+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:45:39 [DEBUG] Exit from do method
2024-07-10T16:45:39.727+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:45:39 graphcont err No Template found
2024-07-10T16:45:39.727+1000 [ERROR] vertex "mso_schema_site_service_graph.schema_site_service_graph_site1 (expand)" error: No Template found
<snip>
2024-07-10T16:45:39.759+1000 [ERROR] vertex "mso_schema_site_service_graph.schema_site_service_graph_site2" error: No Template found
2024-07-10T16:45:39.767+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:45:39 [DEBUG] Exit from do method
2024-07-10T16:45:39.767+1000 [ERROR] vertex "mso_schema_site_service_graph.schema_site_service_graph_site2 (expand)" error: No Template found
2024-07-10T16:45:39.767+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:45:39 graphcont err No Template found
2024-07-10T16:45:39.770+1000 [INFO]  backend/local: plan operation completed

mso_schema_site_contract_service_graph:

2024-07-10T16:52:42.170+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:52:42 [TRACE] HTTP Request Method and URL: PATCH https://xxx/mso/api/v1/schemas/668e2ef2bbcfd63ff84a6a75?validate=false
2024-07-10T16:52:42.170+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:52:42 [TRACE] HTTP Request Body: {[{"op":"add","path":"/sites/6146a02beea713e579d584c0-TEMPLATE1/contracts/CON1/serviceGraphRelationship","value":{"serviceGraphRef":{"schemaId":"668e2ef2bbcfd63ff84a6a75","serviceGraphName":"SG1","templateName":"TEMPLATE1"},"serviceNodesRelationship":[{"consumerConnector":{"clusterInterface":{"dn":"uni/tn-TENANT1/lDevVip-FW1/lIf-INT1"},"redirectPolicy":{"dn":"uni/tn-TENANT1/svcCont/svcRedirectPol-PBR1"},"subnets":[]},"providerConnector":{"clusterInterface":{"dn":"uni/tn-TENANT1/lDevVip-FW1/lIf-INT1"},"redirectPolicy":{"dn":"uni/tn-TENANT1/svcCont/svcRedirectPol-PBR1"}},"serviceNodeRef":{"schemaId":"668e2ef2bbcfd63ff84a6a75","serviceGraphName":"SG1","serviceNodeName":"node1","templateName":"TEMPLATE1"}}]}}]}
2024-07-10T16:52:42.720+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:52:42 [DEBUG] HTTP Request: PATCH https://xxx/mso/api/v1/schemas/668e2ef2bbcfd63ff84a6a75?validate=false
2024-07-10T16:52:42.720+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:52:42 [DEBUG] HTTP Response: 400 400 Bad Request &{400 Bad Request 400 HTTP/1.1 1 1 map[Content-Language:[en] Content-Length:[228] Content-Type:[application/json] Date:[Wed, 10 Jul 2024 06:52:42 GMT] Server:[nginx] Strict-Transport-Security:[max-age=31536000; includeSubDomains] X-Ratelimit-Limit:[-1] X-Ratelimit-Remaining:[-1] X-Ratelimit-Reset:[1559582945]] 0x140005cd180 228 [] false false map[] 0x14000294a00 0x14000540160}
2024-07-10T16:52:42.720+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:52:42 [DEBUG] HTTP response unique string PATCH https://xxx/mso/api/v1/schemas/668e2ef2bbcfd63ff84a6a75?validate=false {"code":400,"message":"site-local Contract CON1 in site SITE2 in template TEMPLATE1: Provider connector in TEMPLATE1 must have uniform redirect policy configured on all sites for node1 and contract CON1","info":null}
2024-07-10T16:52:42.720+1000 [DEBUG] provider.terraform-provider-mso_v1.2.0: 2024/07/10 16:52:42 [DEBUG] Exit from do method
2024-07-10T16:52:42.724+1000 [DEBUG] State storage *statemgr.Filesystem declined to persist a state snapshot
2024-07-10T16:52:42.725+1000 [ERROR] vertex "mso_schema_site_contract_service_graph.schema_site_contract_service_graph_site2" error: "site-local Contract CON1 in site SITE2 in template TEMPLATE1: Provider connector in TEMPLATE1 must have uniform redirect policy configured on all sites for node1 and contract CON1"{}

Expected Behavior

mso_schema_site_service_graph:
This resource should not be dependent on other resources having already been applied in order for a plan to be successful.

mso_schema_site_contract_service_graph:
A separate PATCH operation for each site should be accepted by NDO

Actual Behavior

mso_schema_site_service_graph:
This resource fails to plan/apply unless dependent resources (such as mso_schema and mso_schema_template_service_graph) have already been applied.

mso_schema_site_contract_service_graph:
NDO appears to have introduced a validation rule that won't permit sites in this resource to be configured in separate PATCH requests.

It looks like it will be necessary to update this resource to allow the configuration for all sites to be specified under one instance of the resource.

@akinross akinross added the bug Something isn't working label Jun 4, 2024
@andbyrne
Copy link
Author

andbyrne commented Jun 5, 2024

Also tested NDO v4.2(3e) which suffers from the same issue.

@lhercot
Copy link
Member

lhercot commented Jul 13, 2024

The fix has been released as part of v1.2.1

@andbyrne
Copy link
Author

Only mso_schema_site_service_graph has been resolved. The problem with mso_schema_site_contract_service_graph for which this issue was originally raised has not yet been resolved.

@lhercot lhercot reopened this Jul 18, 2024
@samiib samiib added the jira-sync Sync this issue to Jira label Aug 14, 2024
@github-actions github-actions bot changed the title 'mso_schema_site_contract_service_graph' resource fails validation by NDO 'mso_schema_site_contract_service_graph' resource fails validation by NDO (DCNE-95) Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working jira-sync Sync this issue to Jira
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants