CloudSQL MySQL Plugin e2e test cases #199
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <!-- | ||
| Copyright © 2015-2019 Cask Data, Inc. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); you may not | ||
| use this file except in compliance with the License. You may obtain a copy of | ||
| the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | ||
| WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | ||
| License for the specific language governing permissions and limitations under | ||
| the License. | ||
| --> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" | ||
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <groupId>io.cdap.plugin</groupId> | ||
| <artifactId>e2e-test</artifactId> | ||
| <version>2.9.0-SNAPSHOT</version> | ||
|
There was a problem hiding this comment. should it 1.9.0-SNAPSHOT? Line 23 in 543dee0 |
||
| <properties> | ||
| <maven.compiler.source>8</maven.compiler.source> | ||
| <maven.compiler.target>8</maven.compiler.target> | ||
| </properties> | ||
| <profiles> | ||
| <profile> | ||
| <id>e2e-tests</id> | ||
| <properties> | ||
| <testSourceLocation>e2e-test/src/test</testSourceLocation> | ||
| </properties> | ||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-surefire-plugin</artifactId> | ||
| <version>2.18.1</version> | ||
| <configuration> | ||
| <skipTests>true</skipTests> | ||
| </configuration> | ||
| </plugin> | ||
|
|
||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-failsafe-plugin</artifactId> | ||
| <version>3.0.0-M5</version> | ||
| <configuration> | ||
| <includes> | ||
| <include>TestRunner.java</include> | ||
| </includes> | ||
| </configuration> | ||
| <executions> | ||
| <execution> | ||
| <goals> | ||
| <goal>integration-test</goal> | ||
| </goals> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
|
|
||
| <plugin> | ||
| <groupId>net.masterthought</groupId> | ||
| <artifactId>maven-cucumber-reporting</artifactId> | ||
| <version>5.5.0</version> | ||
|
|
||
| <executions> | ||
| <execution> | ||
| <id>execution</id> | ||
| <phase>verify</phase> | ||
| <goals> | ||
| <goal>generate</goal> | ||
| </goals> | ||
| <configuration> | ||
| <projectName>Cucumber Reports</projectName> <!-- Replace with project name --> | ||
| <outputDirectory>target/cucumber-reports/advanced-reports</outputDirectory> | ||
| <buildNumber>1</buildNumber> | ||
| <skip>false</skip> | ||
| <inputDirectory>${project.build.directory}/cucumber-reports</inputDirectory> | ||
| <jsonFiles> <!-- supports wildcard or name pattern --> | ||
| <param>**/*.json</param> | ||
| </jsonFiles> <!-- optional, defaults to outputDirectory if not specified --> | ||
| <classificationDirectory>${project.build.directory}/cucumber-reports</classificationDirectory> | ||
| <checkBuildResult>true</checkBuildResult> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| <dependencies> | ||
| <dependency> | ||
| <groupId>io.cdap.tests.e2e</groupId> | ||
| <artifactId>cdap-e2e-framework</artifactId> | ||
| <version>0.0.1-SNAPSHOT</version> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>mysql</groupId> | ||
| <artifactId>mysql-connector-java</artifactId> | ||
| <version>8.0.26</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.springframework</groupId> | ||
| <artifactId>spring-jdbc</artifactId> | ||
| <version>5.3.13</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.google.cloud.sql</groupId> | ||
| <artifactId>mysql-socket-factory-connector-j-8</artifactId> | ||
| <version>1.0.16</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.zaxxer</groupId> | ||
| <artifactId>HikariCP</artifactId> | ||
| <version>4.0.3</version> | ||
| </dependency> | ||
| </dependencies> | ||
| </profile> | ||
| </profiles> | ||
| </project> | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| Feature: CloudSQLMySQl Sink Design Time | ||
|
|
||
| @CLDMYSQL @TC-Mandatory-fields | ||
| Scenario Outline:Verify CloudSQLMYSQL Sink properties validation errors for mandatory fields | ||
|
There was a problem hiding this comment. for clarify, can we add source and sink test features in separate directories? |
||
| Given Open DataFusion Project to configure pipeline | ||
| When Target is CloudSQLMySQL | ||
| Then Open CloudSQLMySQL Properties | ||
| Then Enter the CloudSQLMySQL Sink Properties with blank property "<property>" | ||
| Then Validate mandatory property error for "<property>" | ||
| Examples: | ||
| | property | | ||
| | referenceName | | ||
| | database | | ||
| | connectionName | | ||
| | tableName | | ||
| | jdbcPluginName | | ||
|
|
||
| @CLDMYSQL @TC-Invalid-TestData-for-DriverName_Field: | ||
| Scenario: TC-CLDMYSQL-DSGN-02:Verify Driver Name field validation error with invalid test data | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Target is CloudSQLMySQL | ||
| Then Open CloudSQLMySQL Properties | ||
| Then Enter Reference Name "clsReferenceNameValid" & Database Name "clsDatabaseName" with Test Data | ||
| Then Enter Table Name "clsTableNameBQCS" and Connection Name "clsConnectionNameValid" | ||
| Then Validate Connector properties | ||
| Then Enter Driver Name with Invalid value for Driver name field "clsDriverNameInvalid" | ||
| Then Verify invalid Driver name error message is displayed for Driver "clsDriverNameInvalid" | ||
| Then Verify plugin validation fails with error | ||
| Then Close the CloudSQLMySQL Properties | ||
|
|
||
| @CLDMYSQL @TC-Invalid-TestData-for-ReferenceName&ConnectionName | ||
| Scenario: TC-CLDMYSQL-DSGN-03:Verify properties validation errors for invalid test data for Reference name & connection name | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Target is CloudSQLMySQL | ||
| Then Enter Reference Name & Connection Name with Invalid Test Data in Sink | ||
| Then Verify Reference Name "clsReferenceNameInvalid" Field with Invalid Test Data | ||
| Then Verify Connection Name "clsConnectionNameInvalid" fields with Invalid Test Data | ||
| Then Enter Connection Name with private instance type | ||
| Then Verify Connection Name with private instance type "clsConnectionNameInvalid" | ||
| Then Close the CloudSQLMySQL Properties | ||
|
|
||
| @CLDMYSQL @TC-Add-Comments | ||
| Scenario: TC-CLDMYSQL-DSGN-04:Verify the Add Comments functionality for CloudSQL MySQL connector | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Target is CloudSQLMySQL | ||
| Then Add and Save Comments sink "clsPluginComment" | ||
| Then Validate Sink Comment | ||
|
|
||
| @CLDMYSQL @TC-Edit-Comments | ||
| Scenario: TC-CLDMYSQL-DSGN-05:Verify the Edit added Comments functionality for CloudSQL MySQL connector | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Target is CloudSQLMySQL | ||
| Then Add and Save Comments sink "clsPluginComment" | ||
| Then Edit Sink Comments "clsPluginUpdateComment" | ||
| Then Validate Sink Update Comment | ||
|
|
||
| @CLDMYSQL @TC-Delete-Comments | ||
| Scenario: TC-CLDMYSQL-DSGN-06:Verify the Delete added Comments functionality for CloudSQL MySQL connector | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Target is CloudSQLMySQL | ||
| Then Add and Save Comments sink "clsPluginComment" | ||
| Then Edit Sink Comments "clsPluginUpdateComment" | ||
| Then Delete Comments | ||
| Then Validate Comment has been deleted successfully | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| Feature: BigQuery to CloudSqlMySql and GCS to CloudSqlMySql Runtime | ||
|
|
||
| @CLDMYSQL @TC-Runtime-BigQuery-to-CLOUDSqlMYSQL: | ||
| Scenario: TC-CLDMYSQL-RNTM-01:Verify user is able to transferred data from BigQuery to CloudSqlMysql with mandatory fields | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Source is BigQuery | ||
| When Target is CloudSQLMySQL | ||
| Then Link BigQuery to CloudSQLMySQL to establish connection | ||
| Then Enter the Source BigQuery Properties for table "clsTableNameBQ" | ||
| Then Validate the Schema | ||
| Then Verify the Connector status | ||
| Then Close the BigQuery Properties | ||
| Then Enter the sink CloudSQLMySQL Properties for table "clsTableNameBQCS" | ||
| Then Click on Validate button | ||
| Then Verify the Connector status | ||
| Then Close the CloudSQLMySQL Properties | ||
| Then Save the pipeline | ||
| Then Preview and run the pipeline | ||
| Then Verify the preview of pipeline is "success" | ||
| Then Click on PreviewData for CloudSQL MySQL | ||
| Then Verify Preview output schema matches the outputSchema captured in properties | ||
| Then Close the Preview | ||
| Then Pre records count from CloudSQLMySQL table "clsTableNameBQCS" | ||
| Then Deploy the pipeline | ||
| Then Run the Pipeline in Runtime | ||
| Then Wait till pipeline is in running state | ||
| Then Open and capture Logs | ||
| Then Verify the pipeline status is "Succeeded" | ||
| Then Post records count from CloudSQLMySQL table "clsTableNameBQCS" | ||
| Then Validate the output record count | ||
| Then Validate successMessage is displayed | ||
| Then Validate the count of records transferred from BigQuery "clsTableNameBQ" to CloudSqlMySql "clsTableNameBQCS" | ||
|
There was a problem hiding this comment. In addition to checking the number of records transferred, also validate that the correct data is written to the sink in all the test cases |
||
|
|
||
| @CLDMYSQL @TC-Runtime-BigQuery-to-CLOUDSqlMYSQL: | ||
| Scenario: TC-CLDMYSQL-RNTM-02:Verify user is able to transferred data from BigQuery to CloudSqlMysql with filter | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Source is BigQuery | ||
| When Target is CloudSQLMySQL | ||
| Then Link BigQuery to CloudSQLMySQL to establish connection | ||
| Then Enter the Source BigQuery Properties for table "clsTableNameBQ1" | ||
| Then Enter the Source BigQuery with filter "clsFilterBigQuery" option | ||
| Then Validate the Schema | ||
| Then Verify the Connector status | ||
| Then Close the BigQuery Properties | ||
| Then Enter the sink CloudSQLMySQL Properties for table "clsTableNameBQCS1" | ||
| Then Click on Validate button | ||
| Then Verify the Connector status | ||
| Then Close the CloudSQLMySQL Properties | ||
| Then Save the pipeline | ||
| Then Preview and run the pipeline | ||
| Then Verify the preview of pipeline is "success" | ||
| Then Click on PreviewData for CloudSQL MySQL | ||
| Then Verify Preview output schema matches the outputSchema captured in properties | ||
| Then Close the Preview | ||
| Then Pre records count from CloudSQLMySQL table "clsTableNameBQCS1" | ||
| Then Deploy the pipeline | ||
| Then Run the Pipeline in Runtime | ||
| Then Wait till pipeline is in running state | ||
| Then Open and capture Logs | ||
| Then Verify the pipeline status is "Succeeded" | ||
| Then Post records count from CloudSQLMySQL table "clsTableNameBQCS1" | ||
| Then Validate the output record count | ||
| Then Validate successMessage is displayed | ||
| Then Validate the count of records transferred from BigQuery "clsTableNameBQ1" to CloudSqlMySql with filter "clsFilterBigQuery" | ||
|
|
||
| @CLDMYSQL @TC-Runtime-GCS-to-CLOUDSQLMYSQL: | ||
| Scenario: TC-CLDMYSQL-RNTM-03:Verify user is able to transferred data from GCS to CloudSQLMySQL with mandatory fields | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Source is GCS bucket | ||
| When Target is CloudSQLMySQL | ||
| Then Link GCS to CloudSQLMySQL to establish connection | ||
| Then Enter the GCS Properties with "clsBucket" GCS bucket and format "clsFormatType" | ||
| Then Validate the Schema | ||
| Then Verify the Connector status | ||
| Then Close the GCS Properties | ||
| Then Enter the sink CloudSQLMySQL Properties for table "clsTableNameGCSCS" | ||
| Then Click on Validate button | ||
| Then Verify the Connector status | ||
| Then Close the CloudSQLMySQL Properties | ||
| Then Save the pipeline | ||
| Then Preview and run the pipeline | ||
| Then Verify the preview of pipeline is "success" | ||
| Then Click on PreviewData for CloudSQL MySQL | ||
| Then Verify Preview output schema matches the outputSchema captured in properties | ||
| Then Close the Preview | ||
| Then Deploy the pipeline | ||
| Then Run the Pipeline in Runtime | ||
| Then Wait till pipeline is in running state | ||
| Then Open and capture Logs | ||
| Then Verify the pipeline status is "Succeeded" | ||
| Then Validate the output record count | ||
| Then Validate successMessage is displayed | ||
|
|
||
| @CLDMYSQL @TC-Runtime-GCS-to-CLOUDSQLMYSQL: | ||
| Scenario: TC-CLDMYSQL-RNTM-04:Verify user is able to Duplicate the pipeline | ||
| Given Open DataFusion Project to configure pipeline | ||
| When Source is GCS bucket | ||
| When Target is CloudSQLMySQL | ||
| Then Link GCS to CloudSQLMySQL to establish connection | ||
| Then Enter the GCS Properties with "clsBucket" GCS bucket and format "clsFormatType" | ||
| Then Validate the Schema | ||
| Then Verify the Connector status | ||
| Then Close the GCS Properties | ||
| Then Enter the sink CloudSQLMySQL Properties for table "clsTableNameGCSCS" | ||
| Then Click on Validate button | ||
| Then Verify the Connector status | ||
| Then Close the CloudSQLMySQL Properties | ||
| Then Save the pipeline | ||
| Then Preview and run the pipeline | ||
| Then Verify the preview of pipeline is "success" | ||
| Then Click on PreviewData for CloudSQL MySQL | ||
| Then Verify Preview output schema matches the outputSchema captured in properties | ||
| Then Close the Preview | ||
| Then Deploy the pipeline | ||
| Then Run the Pipeline in Runtime | ||
| Then Wait till pipeline is in running state | ||
| Then Verify the pipeline status is "Succeeded" | ||
| Then Validate the output record count | ||
| Then Create Duplicate pipeline | ||
| Then Validate studio is opened with duplicate pipeline | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2022