Skip to content

Commit

Permalink
POL-1378 Linting Updates: Cost Policies: Google (#2736)
Browse files Browse the repository at this point in the history
* update

* fix

* update

* fix

* update

* fix

* update

* update

* update

* update

* update

* update

* update

* update

* update
  • Loading branch information
XOmniverse authored Oct 16, 2024
1 parent 8b318a7 commit 1892db3
Show file tree
Hide file tree
Showing 47 changed files with 284 additions and 334 deletions.
2 changes: 1 addition & 1 deletion cost/google/cheaper_regions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,4 @@ The [Provider-Specific Credentials](https://docs.flexera.com/flexera/EN/Automati

## Cost

This Policy Template does not incur any cloud costs.
This policy template does not incur any cloud costs.
8 changes: 8 additions & 0 deletions cost/google/cloud_run_anomaly_detection/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## v0.1.3

- Minor code improvements to conform with current standards. Functionality unchanged.

## v0.1.2

- fixed spelling errors in parameters

## v0.1.1

- fixed link to README in policy description
Expand Down
42 changes: 24 additions & 18 deletions cost/google/cloud_run_anomaly_detection/README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,14 @@
# Google Cloud Run Anomaly Detection

## What it does
## What It Does

This Policy uses Google Cloud Metrics data to identify anomalies for Cloud Run services using the [Standard Score (aka `Z-score`)](https://en.wikipedia.org/wiki/Standard_score) statistical method.

> The standard score is the number of standard deviations by which the value of a raw score (i.e., an observed value or data point) is above or below the mean value of what is being observed or measured.
This policy only uses Google Cloud Metric data and is designed to notify of anomalies <24 hours -- specifically before cost and usage data is available.

## Prerequisites

This Policy Template uses [Credentials](https://docs.flexera.com/flexera/EN/Automation/ManagingCredentialsExternal.htm) for authenticating to datasources -- in order to apply this policy you must have a Credential registered in the system that is compatible with this policy. If there are no Credentials listed when you apply the policy, please contact your Flexera Org Admin and ask them to register a Credential that is compatible with this policy. The information below should be consulted when creating the credential(s).

- [**Google Cloud Credential**](https://docs.flexera.com/flexera/EN/Automation/ProviderCredentials.htm#automationadmin_4083446696_1121577) (*provider=gce*) which has the following:

- Permissions
- `resourcemanager.projects.get`
- `compute.regions.list`
- `run.services.list`
- `monitoring.timeSeries.list`

The [Provider-Specific Credentials](https://docs.flexera.com/flexera/EN/Automation/ProviderCredentials.htm) page in the docs has detailed instructions for setting up Credentials for the most common providers.

### Input Parameters
## Input Parameters

This policy has the following input parameters required when launching the policy.

Expand All @@ -34,9 +20,29 @@ This policy has the following input parameters required when launching the polic
- *Lookback Aggregation Period* - The time period to aggregate the metric data
- *Metric Name* - The name of the metric to monitor for anomalies
- *Threshold For Z-score* - The threshold for Z-scale, which is the number of consequent anomaly events to trigger an incident (i.e. 1, 2, 3)
- *Threshold For Consequtive Anomalies* - Number of Consqutive Anomalies to trigger an incident
- *Threshold For Consecutive Anomalies* - Number of Consecutive Anomalies to trigger an incident
- *Email addresses* - A list of email addresses to notify

## Policy Actions

- Send an email report

## Prerequisites

This Policy Template uses [Credentials](https://docs.flexera.com/flexera/EN/Automation/ManagingCredentialsExternal.htm) for authenticating to datasources -- in order to apply this policy you must have a Credential registered in the system that is compatible with this policy. If there are no Credentials listed when you apply the policy, please contact your Flexera Org Admin and ask them to register a Credential that is compatible with this policy. The information below should be consulted when creating the credential(s).

- [**Google Cloud Credential**](https://docs.flexera.com/flexera/EN/Automation/ProviderCredentials.htm#automationadmin_4083446696_1121577) (*provider=gce*) which has the following:
- `resourcemanager.projects.get`
- `compute.regions.list`
- `run.services.list`
- `monitoring.timeSeries.list`

The [Provider-Specific Credentials](https://docs.flexera.com/flexera/EN/Automation/ProviderCredentials.htm) page in the docs has detailed instructions for setting up Credentials for the most common providers.

## Supported Clouds

- Google

## Cost

This Policy Template does not incur any cloud costs.
This policy template does not incur any cloud costs.
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ category "Cost"
default_frequency "daily"
info(
publish: "false",
version: "0.1.1",
version: "0.1.3",
provider: "Google",
service: "Cloud Run",
policy_set: "Anomaly Detection"
Expand All @@ -19,15 +19,14 @@ info(
###############################################################################

parameter "param_email" do
category "Policy Settings"
type "list"
category "Policy Settings"
label "Email addresses to notify"
description "Email addresses of the recipients you wish to notify when new incidents are created"
default []
end

parameter "param_projects_allow_or_deny" do
category "Filters"
type "string"
category "Filters"
label "Allow/Deny Projects"
Expand All @@ -37,7 +36,6 @@ parameter "param_projects_allow_or_deny" do
end

parameter "param_projects_list" do
category "Filters"
type "list"
category "Filters"
label "Allow/Deny Projects List"
Expand All @@ -46,7 +44,6 @@ parameter "param_projects_list" do
end

parameter "param_regions_allow_or_deny" do
category "Filters"
type "string"
category "Filters"
label "Allow/Deny Regions"
Expand All @@ -56,7 +53,6 @@ parameter "param_regions_allow_or_deny" do
end

parameter "param_regions_list" do
category "Filters"
type "list"
category "Filters"
label "Allow/Deny Regions List"
Expand All @@ -65,48 +61,48 @@ parameter "param_regions_list" do
end

parameter "param_metric_lookback_time" do
type "string"
category "Statistics"
label "Lookback Time Period"
type "string"
description "The time period to look back for anomalies. The longer the time period, the more accurate the anomaly detection will be."
default "Last 1 day"
allowed_values ["Last 15 minutes", "Last 30 minutes", "Last 1 hour", "Last 3 hours", "Last 6 hours", "Last 12 hours", "Last 1 day", "Last 2 days", "Last 7 days", "Last 14 days"]
default "Last 1 day"
end

parameter "param_metric_aggregation" do
type "string"
category "Statistics"
label "Lookback Aggregation Period"
type "string"
description "The time period to aggregate the metric data"
default "1 minute"
allowed_values ["1 minute", "5 minutes", "10 minutes", "15 minutes", "30 minutes", "1 hour", "3 hours", "6 hours", "12 hours", "1 day"]
default "1 minute"
end

parameter "param_threshold_metric_name" do
type "string"
category "Statistics"
label "Metric Name"
type "string"
description "The name of the metric to monitor for anomalies"
default "run.googleapis.com/container/billable_instance_time"
allowed_values ["run.googleapis.com/container/billable_instance_time", "run.googleapis.com/container/instance_count", "run.googleapis.com/container/memory", "run.googleapis.com/container/cpu"]
default "run.googleapis.com/container/billable_instance_time"
end

parameter "param_threshold_zscore" do
type "number"
category "Filters"
label "Threshold For Z-Score"
type "number"
description "The Z-score is the number of standard deviations by which the value of a raw score (i.e., an observed value or data point) is above or below the mean value of what is being observed or measured"
default 3
min_value 0
default 3
end

parameter "param_threshold_consequtive_anomalies" do
category "Filters"
label "Threshold For Consequtive Anomalies"
type "number"
description "Number of Consequtive Anomalies to trigger an incident"
default 0
category "Filters"
label "Threshold For Consecutive Anomalies"
description "Number of Consecutive Anomalies to trigger an incident"
min_value 0
default 0
end

###############################################################################
Expand Down Expand Up @@ -151,9 +147,9 @@ script "js_param_values", type: "javascript" do
// Using param_metric_lookback_time, calculate the start and end time for the interval
var end_time = new Date()
// Set the seconds and milliseconds to 0 to make the timestamp more human-readable
end_time.setSeconds(0,0)
end_time.setSeconds(0, 0)
// Default start_time to Last 1 day
var start_time = new Date(end_time.getTime() - 24*60*60000)
var start_time = new Date(end_time.getTime() - 24 * 60 * 60000)
// Override default if param value matches mapping in switch
switch (param_metric_lookback_time) {
case "Last 15 minutes":
Expand Down Expand Up @@ -317,10 +313,9 @@ datasource "ds_google_cloud_run_services" do
iterate $ds_google_regions_filtered
request do
auth $auth_google
pagination $pagination_google
host "run.googleapis.com" # v2 can use Global endpoint https://cloud.google.com/run/docs/reference/rest#rest_endpoints
verb "GET"
path join(["/v2/projects/", val(iter_item, "projectId"), "/locations/", val(iter_item, "region"), "/services"])
pagination $pagination_google
end
result do
encoding "json"
Expand All @@ -341,14 +336,14 @@ datasource "ds_cloud_run_utilization" do
request do
auth $auth_google
host "monitoring.googleapis.com"
path join(["/v3/projects/",val(iter_item,"projectId"),"/timeSeries"])
query "aggregation.alignmentPeriod", join(['+',val($ds_param_values,"aggregation"),"s"],"")
path join(["/v3/projects/", val(iter_item, "projectId"), "/timeSeries"])
query "aggregation.alignmentPeriod", join(['+', val($ds_param_values, "aggregation"), "s"], "")
query "aggregation.crossSeriesReducer", "REDUCE_SUM"
query "aggregation.groupByFields", 'resource.label."service_name"'
query "aggregation.perSeriesAligner", "ALIGN_RATE"
query "filter", join(['metric.type="',$param_threshold_metric_name,'" resource.type="cloud_run_revision" resource.label."service_name"="',last(split(val(iter_item, "name"), "/services/")),'" AND resource.labels."location"="',val(iter_item, "region"),'" AND resource.labels."service_name"="',last(split(val(iter_item, "name"), "/services/")),'"'],"")
query "filter", join(['metric.type="', $param_threshold_metric_name, '" resource.type="cloud_run_revision" resource.label."service_name"="', last(split(val(iter_item, "name"), "/services/")), '" AND resource.labels."location"="', val(iter_item, "region"), '" AND resource.labels."service_name"="', last(split(val(iter_item, "name"), "/services/")), '"'], "")
query "interval.startTime", val($ds_param_values, "start_time")
query "interval.endTime", val($ds_param_values, "end_time")
query "interval.endTime", val($ds_param_values, "end_time")
end
result do
encoding "json"
Expand Down Expand Up @@ -437,14 +432,9 @@ policy "pol_utilization" do
validate_each $ds_cloud_run_utilization_anomalies do
summary_template "Google Cloud Run Anomaly Detection: {{ len data }} Anomalies In {{ parameters.param_metric_lookback_time }} For {{ parameters.param_threshold_metric_name }} Metric"
detail_template " "
check lt( val(val(item, "value"), "consequtiveAnomalyIndex") , $param_threshold_consequtive_anomalies) # Check that the consequtiveAnomalyIndex is less than the param_consequtiveAnomaly_threshold
escalate $email
check lt(val(val(item, "value"), "consequtiveAnomalyIndex"), $param_threshold_consequtive_anomalies) # Check that the consequtiveAnomalyIndex is less than the param_consequtiveAnomaly_threshold
escalate $esc_email
export do
# resource_level true
# field "id" do
# label "Resource ID"
# path "service_name"
# end
field "param_threshold_consequtive_anomalies" do
label "Policy Parameter Threshold Consequtive Anomalies"
path "parameters.param_threshold_consequtive_anomalies"
Expand Down Expand Up @@ -497,12 +487,11 @@ policy "pol_utilization" do
end
end


###############################################################################
# Escalations
###############################################################################

escalation "email" do
escalation "esc_email" do
automatic true
label "Send Email"
description "Send incident email"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ The following policy actions are taken on any resources found to be out of compl

## Cost

This Policy Template does not launch any instances, and so does not incur any cloud costs.
This policy template does not incur any cloud costs.

## API Quotas

Expand Down
4 changes: 4 additions & 0 deletions cost/google/cloudsql_rightsizing/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## v2.10.2

- Minor code improvements to conform with current standards. Functionality unchanged.

## v2.10.1

- Deprecated: This policy is no longer being updated. Please see policy README for more information.
Expand Down
6 changes: 3 additions & 3 deletions cost/google/cloudsql_rightsizing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

This policy is no longer being updated. The [Google Rightsize Cloud SQL Recommender](https://github.com/flexera-public/policy_templates/tree/master/cost/google/rightsize_cloudsql_recommendations/) policy should be used instead.

## What it does
## What It Does

This Policy Template checks Google Cloud SQL instances based on provided CPU threshold over a 30 day average and resizes them after approval.

## Functional Details
## How It Works

- This policy identifies all Google CloudSQL instances reporting performance metrics to stackdriver whose CPU utilization is below the thresholds set in the **Average used CPU % - Downsize Threshold** and **Average used CPU % - Upsize Threshold** parameters.
- If APIs & Services are not enabled for a project, the policy will skip that particular project. On the next run if APIs & Services are enabled, then the project will be considered for execution.
Expand Down Expand Up @@ -56,4 +56,4 @@ Required permissions in the provider:

## Cost

This Policy Template does not incur any cloud costs.
This policy template does not incur any cloud costs.
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ severity "medium"
category "Cost"
default_frequency "daily"
info(
version: "2.10.1",
version: "2.10.2",
provider: "GCE",
service: "SQL",
policy_set: "Rightsize Database Services",
Expand Down Expand Up @@ -190,7 +190,6 @@ end

datasource "ds_google_instance_size_map" do
request do
verb "GET"
host "raw.githubusercontent.com"
path "/flexera/policy_templates/e412d4ca5dc191f15fcf6cc532dc2d31de44f3ee/data/google/instance_types.json"
header "User-Agent", "RS Policies"
Expand Down
2 changes: 1 addition & 1 deletion cost/google/cud_expiration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,4 @@ Additionally, this Policy Template requires that several APIs be enabled in your

## Cost

This Policy Template does not incur any cloud costs.
This policy template does not incur any cloud costs.
5 changes: 5 additions & 0 deletions cost/google/cud_recommendations/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## v4.2.1

- Added `Service` field to incident table
- Minor code improvements to conform with current standards.

## v4.2

- Updated policy to use new source for currency information. Policy functionality is unchanged.
Expand Down
Loading

0 comments on commit 1892db3

Please sign in to comment.