Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC: using local abstractions sources instead of packages #40

Closed

Conversation

Yury-Fridlyand
Copy link

@Yury-Fridlyand Yury-Fridlyand commented Aug 31, 2022

Abstractions lib sourceshttps://github.com/[opensearch-project/opensearch-net-abstractions](https://github.com/opensearch-project/opensearch-net-abstractions) were copied into .net client to use in tests. No need any more to store local nuget packages of abstractions, release them or worry about outdated/broken references to them.
Few fixes were made to let workflows pass.

In case if this approach approved this PR should be merged to upstream/main after closing opensearch-project#51.

* Update packages.

Signed-off-by: Yury-Fridlyand <[email protected]>

* Cleanup - removing stale dependencies.

Signed-off-by: Yury-Fridlyand <[email protected]>

Signed-off-by: Yury-Fridlyand <[email protected]>
@Yury-Fridlyand
Copy link
Author

Yury-Fridlyand commented Aug 31, 2022

UPDATED
Output from git diff --diff-filter=M opensearch-net-abstractions/ opensearch-net/abstractions/

diff --git a/opensearch-net-abstractions/README.md b/opensearch-net/abstractions/README.md
index 2765381..88b8405 100755
--- a/opensearch-net-abstractions/README.md
+++ b/opensearch-net/abstractions/README.md
@@ -17,8 +17,6 @@

 You've reached the home repository for several auxiliary projects from the .NET team within OpenSearch.

-Pre-release packages of these projects are available at [OpenSearch NuGet Package Repository](https://www.github.com/opensearch-project/packages). See [GitHub documentation](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-nuget-registry#installing-a-package) for instructions on how to access them.
-
 Current projects:

 ### [OpenSearch.OpenSearch.Managed](src/OpenSearch.OpenSearch.Managed/README.md)
diff --git a/opensearch-net-abstractions/src/Directory.Build.props b/opensearch-net/abstractions/src/Directory.Build.props
index 968df46..9612245 100755
--- a/opensearch-net-abstractions/src/Directory.Build.props
+++ b/opensearch-net/abstractions/src/Directory.Build.props
@@ -5,10 +5,7 @@
     <Authors>OpenSearch Project and contributors</Authors>
     <Copyright>OpenSearch</Copyright>
     <PackageLicenseExpression>Apache-2.0</PackageLicenseExpression>
-    <RepositoryUrl>https://github.com/opensearch-project/opensearch-net-abstractions</RepositoryUrl>
     <RepositoryType>Git</RepositoryType>
-    <PackageProjectUrl>https://github.com/opensearch-project/opensearch-net-abstractions</PackageProjectUrl>
-    <PackageReleaseNotes>https://github.com/opensearch-project/opensearch-net-abstractions/releases</PackageReleaseNotes>


     <SignAssembly>true</SignAssembly>
@@ -19,19 +16,6 @@
     <PackageIcon>nuget-icon.png</PackageIcon>

   </PropertyGroup>
-
-  <ItemGroup>
-    <Content Include="../../nuget-icon.png" CopyToOutputDirectory="PreserveNewest">
-      <Link>nuget-icon.png</Link>
-      <Pack>True</Pack>
-      <PackagePath>nuget-icon.png</PackagePath>
-    </Content>
-    <None Include="../../LICENSE.txt" CopyToOutputDirectory="PreserveNewest">
-      <Pack>True</Pack>
-      <PackagePath>LICENSE.txt</PackagePath>
-    </None>
-  </ItemGroup>
-
   <ItemGroup>
     <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0" PrivateAssets="All"/>
     <PackageReference Include="ConfigureAwaitChecker.Analyzer" Version="4.0.0" PrivateAssets="All" />
diff --git a/opensearch-net-abstractions/src/OpenSearch.OpenSearch.Ephemeral/OpenSearch.OpenSearch.Ephemeral.csproj b/opensearch-net/abstractions/src/OpenSearch.OpenSearch.Ephemeral/OpenSearch.OpenSearch.Ephemeral.csproj
old mode 100644
new mode 100755
index a4ec93b..994e437
--- a/opensearch-net-abstractions/src/OpenSearch.OpenSearch.Ephemeral/OpenSearch.OpenSearch.Ephemeral.csproj
+++ b/opensearch-net/abstractions/src/OpenSearch.OpenSearch.Ephemeral/OpenSearch.OpenSearch.Ephemeral.csproj
@@ -4,6 +4,7 @@
     <TargetFrameworks>netstandard2.0;net461</TargetFrameworks>
     <Description>Provides an EphemeralCluster implementation that can download/bootstrap/run a throwaway customizable OpenSearch cluster</Description>
     <PackageTags>opensearch,opensearch,cluster,ephemeral</PackageTags>
+    <IsPackable>false</IsPackable>
   </PropertyGroup>
   <ItemGroup>
     <PackageReference Include="SharpZipLib.NETStandard" Version="1.0.7" />
@@ -12,4 +13,4 @@
   <ItemGroup>
     <ProjectReference Include="..\OpenSearch.OpenSearch.Managed\OpenSearch.OpenSearch.Managed.csproj" />
   </ItemGroup>
-</Project>
\ No newline at end of file
+</Project>
diff --git a/opensearch-net-abstractions/src/OpenSearch.OpenSearch.Managed/OpenSearch.OpenSearch.Managed.csproj b/opensearch-net/abstractions/src/OpenSearch.OpenSearch.Managed/OpenSearch.OpenSearch.Managed.csproj
old mode 100644
new mode 100755
index 1778c51..91a10c3
--- a/opensearch-net-abstractions/src/OpenSearch.OpenSearch.Managed/OpenSearch.OpenSearch.Managed.csproj
+++ b/opensearch-net/abstractions/src/OpenSearch.OpenSearch.Managed/OpenSearch.OpenSearch.Managed.csproj
@@ -7,13 +7,13 @@
       Also ships with an cluster abstraction that can start one or more OpenSearchNode's
     </Description>
     <PackageTags>opensearch,opensearch,cluster,observable,rx</PackageTags>
+    <IsPackable>false</IsPackable>
   </PropertyGroup>
   <ItemGroup>
     <PackageReference Include="Proc" Version="0.6.1" />
-    <!--<ProjectReference Include="..\..\..\..\proc\src\Proc\Proc.csproj" />-->
     <PackageReference Include="System.Net.Http" Version="4.3.4" />
   </ItemGroup>
   <ItemGroup>
     <ProjectReference Include="..\OpenSearch.Stack.ArtifactsApi\OpenSearch.Stack.ArtifactsApi.csproj" />
   </ItemGroup>
-</Project>
\ No newline at end of file
+</Project>
diff --git a/opensearch-net-abstractions/src/OpenSearch.OpenSearch.Xunit/OpenSearch.OpenSearch.Xunit.csproj b/opensearch-net/abstractions/src/OpenSearch.OpenSearch.Xunit/OpenSearch.OpenSearch.Xunit.csproj
old mode 100644
new mode 100755
index 3892ea3..77b3d37
--- a/opensearch-net-abstractions/src/OpenSearch.OpenSearch.Xunit/OpenSearch.OpenSearch.Xunit.csproj
+++ b/opensearch-net/abstractions/src/OpenSearch.OpenSearch.Xunit/OpenSearch.OpenSearch.Xunit.csproj
@@ -2,9 +2,9 @@
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
     <TargetFrameworks>netstandard2.0;net461</TargetFrameworks>
-    <IsPackable>True</IsPackable>
     <Description>Provides an Xunit test framework allowing you to run integration tests against local ephemeral OpenSearch clusters</Description>
     <PackageTags>opensearch,opensearch,xunit,cluster,integration,test,ephemeral</PackageTags>
+    <IsPackable>false</IsPackable>
   </PropertyGroup>
   <ItemGroup>
     <PackageReference Include="xunit" Version="2.4.2" />
@@ -12,4 +12,4 @@
   <ItemGroup>
     <ProjectReference Include="..\OpenSearch.OpenSearch.Ephemeral\OpenSearch.OpenSearch.Ephemeral.csproj" />
   </ItemGroup>
-</Project>
\ No newline at end of file
+</Project>
diff --git a/opensearch-net-abstractions/src/OpenSearch.Stack.ArtifactsApi/OpenSearch.Stack.ArtifactsApi.csproj b/opensearch-net/abstractions/src/OpenSearch.Stack.ArtifactsApi/OpenSearch.Stack.ArtifactsApi.csproj
old mode 100644
new mode 100755
index c228e37..dae96f5
--- a/opensearch-net-abstractions/src/OpenSearch.Stack.ArtifactsApi/OpenSearch.Stack.ArtifactsApi.csproj
+++ b/opensearch-net/abstractions/src/OpenSearch.Stack.ArtifactsApi/OpenSearch.Stack.ArtifactsApi.csproj
@@ -3,6 +3,7 @@
         <TargetFrameworks>netstandard2.0;net461</TargetFrameworks>
         <Description>Provides a set of classes to resolve the location of OpenSearch stack products in various stages: released, snapshot and build candidates</Description>
         <PackageTags>opensearch,opensearch,stack,versioning,artifacts</PackageTags>
+        <IsPackable>false</IsPackable>
     </PropertyGroup>

     <ItemGroup>
diff --git a/opensearch-net-abstractions/src/OpenSearch.Stack.ArtifactsApi/Platform/OsMonikers.cs b/opensearch-net/abstractions/src/OpenSearch.Stack.ArtifactsApi/Platform/OsMonikers.cs
old mode 100644
new mode 100755
index 2bf89b5..127b14d
--- a/opensearch-net-abstractions/src/OpenSearch.Stack.ArtifactsApi/Platform/OsMonikers.cs
+++ b/opensearch-net/abstractions/src/OpenSearch.Stack.ArtifactsApi/Platform/OsMonikers.cs
@@ -50,7 +50,7 @@ namespace OpenSearch.Stack.ArtifactsApi.Platform
                        if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX)) return OSPlatform.OSX;
                        if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux)) return OSPlatform.Linux;
                        throw new Exception(
-                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net-abstractions");
+                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net");
                }

                /// <summary>
@@ -78,7 +78,7 @@ namespace OpenSearch.Stack.ArtifactsApi.Platform
                                if (architecture == Architecture.Arm64) return "linux-arm64";
                        }
                        throw new Exception(
-                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net-abstractions");
+                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net");
                }

                public static string GetPlatformArchiveExtension(OSPlatform platform)
@@ -87,7 +87,7 @@ namespace OpenSearch.Stack.ArtifactsApi.Platform
                        if (platform == OSPlatform.OSX) return "tar.gz";
                        if (platform == OSPlatform.Windows) return "zip";
                        throw new Exception(
-                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net-abstractions");
+                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net");

                }
                public static string CurrentPlatformArchiveExtension()
@@ -104,7 +104,7 @@ namespace OpenSearch.Stack.ArtifactsApi.Platform
                        if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux)) return $"{Linux}-{intelX86Suffix}";

                        throw new Exception(
-                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net-abstractions");
+                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net");
                }

                internal static string CurrentPlatformSearchFilter()
@@ -114,7 +114,7 @@ namespace OpenSearch.Stack.ArtifactsApi.Platform
                        if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux)) return "tar";

                        throw new Exception(
-                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net-abstractions");
+                               $"{RuntimeInformation.OSDescription} is currently not supported please open an issue @opensearch-project/opensearch-net");
                }
        }
 }

@wbeckler
Copy link

Process-wise, this looks like a major improvement.

@Yury-Fridlyand Yury-Fridlyand force-pushed the dev-os-2.0-integ-local-abstractions branch from 865c854 to e0e8e4e Compare August 31, 2022 19:55
abstractions/Directory.Build.props Outdated Show resolved Hide resolved
abstractions/dotnet-tools.json Outdated Show resolved Hide resolved
abstractions/global.json Outdated Show resolved Hide resolved
abstractions/src/Directory.Build.props Outdated Show resolved Hide resolved
Yury-Fridlyand and others added 2 commits August 31, 2022 18:36
* Update nuget packages (opensearch-project#88)
* Cleanup - removing stale dependencies.
* Fix 2 typos in scripts project
* Update `scripts` project file to include all relevant objects.
* Rename `master` node role to `cluster_manager` as it was done in OpenSearch.
    Ref: opensearch-project/OpenSearch#2480
* Remove validation for indices segments stats.
    `OpenSearch` 2.0 uses newer version of `Lucene` (9.0) which doesn't provide segments stats info.
    Ref: opensearch-project/OpenSearch#2029 opensearch-project/OpenSearch#1109
    See also history for `server/src/main/java/org/opensearch/index/engine/SegmentsStats.java` in `OpenSearch` repo.
* Remove tests for `_type` validation in mapping APIs as it was removed from `OpenSearch`.
    Ref: opensearch-project/OpenSearch#2238 opensearch-project/OpenSearch#2480
* Remove usage of deprecated `search.remote` settings.
    Ref: opensearch-project/OpenSearch#1870
* Update abstractions package - patch to support OpenSearch 2.0. Update integration workflow to run tests on OpenSearch 2.0.
* Rename `master_timeout` to `cluster_manager_timeout` in all APIs where it is used.
* Enrich comments to already renamed `CatMaster`/`CatClusterManager` API.
* Rename in `/_cluster/stats`/`cluster.stats` and `/_cluster/state`/`cluster.state`.
* Add deprecation info.
* Rename in comments.
* Rename in test data.
* Renamings in tests including `MasterEligible`, but mark it is obsolete.
* Rename branch reference in scripting.
* Mark `indices.exists_type`/`TypeExists` APIs as deprecated.
* Update compatibility matrix and include it into `sln` file.
* Add deprecation notice to all reference of `include_type_name`/`IncludeTypeName`.
* Update compatibility matrix.
* Remove `OpenDistro` compatibility notice.
* Update repo link.
* Add small README for each project being released.
* Address PR opensearch-project#51 feedback.

Signed-off-by: Yury-Fridlyand <[email protected]>

Signed-off-by: Yury-Fridlyand <[email protected]>
* Correctly handle expected license headers of differing lengths

Signed-off-by: Thomas Farr <[email protected]>

* Shorten expected license header

Signed-off-by: Thomas Farr <[email protected]>

* Fix `*.cs` license header blocks to match expected.

```sh
perl -i -p0e 's|license\.\n\*\n\* Modifications|license.\n*/\n/*\n* Modifications|s' $(find . -iname '*.cs')
```

Signed-off-by: Thomas Farr <[email protected]>

Signed-off-by: Thomas Farr <[email protected]>
@Yury-Fridlyand Yury-Fridlyand changed the base branch from dev-os-2.0-integ to main August 31, 2022 23:24
@Yury-Fridlyand Yury-Fridlyand changed the base branch from main to dev-os-2.0-integ August 31, 2022 23:27
… `main/HEAD` @ 63bac1681.

* Copy abstractions sources
* Adopt the abstractions migration
* Merge misc files

Signed-off-by: Yury-Fridlyand <[email protected]>
@Yury-Fridlyand Yury-Fridlyand force-pushed the dev-os-2.0-integ-local-abstractions branch from e1a5fa5 to 6d39fde Compare September 1, 2022 16:46
MaxKsyunz and others added 2 commits September 2, 2022 14:38
…ions-no-build

Integrate abstractions projects into client build.
@Yury-Fridlyand
Copy link
Author

This PR review confirmed that this PoC has the right approach. All changes were implemented in scope of opensearch-project#92 and #41.

@Yury-Fridlyand Yury-Fridlyand deleted the dev-os-2.0-integ-local-abstractions branch September 6, 2022 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants