CHEF-12243-MAGIC-MODULE-Dataproc-Jobs-Resource Implementation

README.md

@@ -291,6 +291,7 @@ The following resources are available in the InSpec GCP Profile
 | [google_dataflow_project_location_job](docs/resources/google_dataflow_project_location_job.md)                                                 | [google_dataflow_project_location_jobs](docs/resources/google_dataflow_project_location_jobs.md)                                                         |
 | [google_dataproc_autoscaling_policy](docs/resources/google_dataproc_autoscaling_policy.md)                                                 | [google_dataproc_autoscaling_policies](docs/resources/google_dataproc_autoscaling_policies.md)                                                         |
 | [google_dataproc_cluster](docs/resources/google_dataproc_cluster.md)                                                 | [google_dataproc_clusters](docs/resources/google_dataproc_clusters.md)                                                         |
+| [google_dataproc_job](docs/resources/google_dataproc_job.md)                                                 | [google_dataproc_jobs](docs/resources/google_dataproc_jobs.md)                                                         |
 | [google_dataproc_workflow_template](docs/resources/google_dataproc_workflow_template.md)                                                 | [google_dataproc_workflow_templates](docs/resources/google_dataproc_workflow_templates.md)                                                         |
 | [google_dns_managed_zone](docs/resources/google_dns_managed_zone.md)                                                 | [google_dns_managed_zones](docs/resources/google_dns_managed_zones.md)                                                         |
 | [google_dns_resource_record_set](docs/resources/google_dns_resource_record_set.md)                                   | [google_dns_resource_record_sets](docs/resources/google_dns_resource_record_sets.md)                                           |

docs/resources/google_dataproc_jobs.md

@@ -0,0 +1,52 @@
+---
+title = "google_dataproc_jobs Resource"
+platform = "gcp"
+---
+
+## Syntax
+A `google_dataproc_jobs` is used to test a Google Job resource
+
+## Examples
+```
+  describe google_dataproc_jobs(project_id: 'value_project_id', region: 'value_region') do
+    it { should exist }
+    its('driver_output_resource_uris') { should include 'value_driveroutputresourceuri' }
+    its('driver_control_files_uris') { should include 'value_drivercontrolfilesuri' }
+    its('job_uuids') { should include 'value_jobuuid' }
+  end
+```
+
+## Properties
+Properties that can be accessed from the `google_dataproc_jobs` resource:
+
+See [google_dataproc_job.md](google_dataproc_job.md) for more detailed information
+* `references`: an array of `google_dataproc_job` reference
+* `placements`: an array of `google_dataproc_job` placement
+* `hadoop_jobs`: an array of `google_dataproc_job` hadoop_job
+* `spark_jobs`: an array of `google_dataproc_job` spark_job
+* `pyspark_jobs`: an array of `google_dataproc_job` pyspark_job
+* `hive_jobs`: an array of `google_dataproc_job` hive_job
+* `pig_jobs`: an array of `google_dataproc_job` pig_job
+* `spark_r_jobs`: an array of `google_dataproc_job` spark_r_job
+* `spark_sql_jobs`: an array of `google_dataproc_job` spark_sql_job
+* `presto_jobs`: an array of `google_dataproc_job` presto_job
+* `trino_jobs`: an array of `google_dataproc_job` trino_job
+* `flink_jobs`: an array of `google_dataproc_job` flink_job
+* `statuses`: an array of `google_dataproc_job` status
+* `status_histories`: an array of `google_dataproc_job` status_history
+* `yarn_applications`: an array of `google_dataproc_job` yarn_applications
+* `driver_output_resource_uris`: an array of `google_dataproc_job` driver_output_resource_uri
+* `driver_control_files_uris`: an array of `google_dataproc_job` driver_control_files_uri
+* `labels`: an array of `google_dataproc_job` labels
+* `schedulings`: an array of `google_dataproc_job` scheduling
+* `job_uuids`: an array of `google_dataproc_job` job_uuid
+* `dones`: an array of `google_dataproc_job` done
+* `driver_scheduling_configs`: an array of `google_dataproc_job` driver_scheduling_config
+
+## Filter Criteria
+This resource supports all of the above properties as filter criteria, which can be used
+with `where` as a block or a method.
+
+## GCP Permissions
+
+Ensure the [Cloud Dataproc API](https://console.cloud.google.com/apis/library/dataproc.googleapis.com) is enabled for the current project.

libraries/google/dataproc/property/job_driver_scheduling_config.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobDriverSchedulingConfig
+        attr_reader :memory_mb
+
+        attr_reader :vcores
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @memory_mb = args['memoryMb']
+          @vcores = args['vcores']
+        end
+
+        def to_s
+          "#{@parent_identifier} JobDriverSchedulingConfig"
+        end
+      end
+    end
+  end
+end

libraries/google/dataproc/property/job_flink_job.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+require 'google/dataproc/property/job_flink_job_logging_config'
+require 'google/dataproc/property/job_flink_job_logging_config_driver_log_levels'
+require 'google/dataproc/property/job_flink_job_properties'
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobFlinkJob
+        attr_reader :main_jar_file_uri
+
+        attr_reader :main_class
+
+        attr_reader :args
+
+        attr_reader :jar_file_uris
+
+        attr_reader :savepoint_uri
+
+        attr_reader :properties
+
+        attr_reader :logging_config
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @main_jar_file_uri = args['mainJarFileUri']
+          @main_class = args['mainClass']
+          @args = args['args']
+          @jar_file_uris = args['jarFileUris']
+          @savepoint_uri = args['savepointUri']
+          @properties = GoogleInSpec::Dataproc::Property::JobFlinkJobProperties.new(args['properties'], to_s)
+          @logging_config = GoogleInSpec::Dataproc::Property::JobFlinkJobLoggingConfig.new(args['loggingConfig'], to_s)
+        end
+
+        def to_s
+          "#{@parent_identifier} JobFlinkJob"
+        end
+      end
+    end
+  end
+end

libraries/google/dataproc/property/job_flink_job_logging_config.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+require 'google/dataproc/property/job_flink_job_logging_config_driver_log_levels'
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobFlinkJobLoggingConfig
+        attr_reader :driver_log_levels
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @driver_log_levels = GoogleInSpec::Dataproc::Property::JobFlinkJobLoggingConfigDriverLogLevels.new(args['driverLogLevels'], to_s)
+        end
+
+        def to_s
+          "#{@parent_identifier} JobFlinkJobLoggingConfig"
+        end
+      end
+    end
+  end
+end

libraries/google/dataproc/property/job_flink_job_logging_config_driver_log_levels.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobFlinkJobLoggingConfigDriverLogLevels
+        attr_reader :additional_properties
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @additional_properties = args['additionalProperties']
+        end
+
+        def to_s
+          "#{@parent_identifier} JobFlinkJobLoggingConfigDriverLogLevels"
+        end
+      end
+    end
+  end
+end

libraries/google/dataproc/property/job_flink_job_properties.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobFlinkJobProperties
+        attr_reader :additional_properties
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @additional_properties = args['additionalProperties']
+        end
+
+        def to_s
+          "#{@parent_identifier} JobFlinkJobProperties"
+        end
+      end
+    end
+  end
+end

libraries/google/dataproc/property/job_hadoop_job.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+require 'google/dataproc/property/job_hadoop_job_logging_config'
+require 'google/dataproc/property/job_hadoop_job_logging_config_driver_log_levels'
+require 'google/dataproc/property/job_hadoop_job_properties'
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobHadoopJob
+        attr_reader :main_jar_file_uri
+
+        attr_reader :main_class
+
+        attr_reader :args
+
+        attr_reader :jar_file_uris
+
+        attr_reader :file_uris
+
+        attr_reader :archive_uris
+
+        attr_reader :properties
+
+        attr_reader :logging_config
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @main_jar_file_uri = args['mainJarFileUri']
+          @main_class = args['mainClass']
+          @args = args['args']
+          @jar_file_uris = args['jarFileUris']
+          @file_uris = args['fileUris']
+          @archive_uris = args['archiveUris']
+          @properties = GoogleInSpec::Dataproc::Property::JobHadoopJobProperties.new(args['properties'], to_s)
+          @logging_config = GoogleInSpec::Dataproc::Property::JobHadoopJobLoggingConfig.new(args['loggingConfig'], to_s)
+        end
+
+        def to_s
+          "#{@parent_identifier} JobHadoopJob"
+        end
+      end
+    end
+  end
+end

libraries/google/dataproc/property/job_hadoop_job_logging_config.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+require 'google/dataproc/property/job_hadoop_job_logging_config_driver_log_levels'
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobHadoopJobLoggingConfig
+        attr_reader :driver_log_levels
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @driver_log_levels = GoogleInSpec::Dataproc::Property::JobHadoopJobLoggingConfigDriverLogLevels.new(args['driverLogLevels'], to_s)
+        end
+
+        def to_s
+          "#{@parent_identifier} JobHadoopJobLoggingConfig"
+        end
+      end
+    end
+  end
+end

libraries/google/dataproc/property/job_hadoop_job_logging_config_driver_log_levels.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: false
+
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    Type: MMv1     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
+module GoogleInSpec
+  module Dataproc
+    module Property
+      class JobHadoopJobLoggingConfigDriverLogLevels
+        attr_reader :additional_properties
+
+        def initialize(args = nil, parent_identifier = nil)
+          return if args.nil?
+          @parent_identifier = parent_identifier
+          @additional_properties = args['additionalProperties']
+        end
+
+        def to_s
+          "#{@parent_identifier} JobHadoopJobLoggingConfigDriverLogLevels"
+        end
+      end
+    end
+  end
+end