Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Scala client for certified events #2045

Merged
merged 54 commits into from
Oct 10, 2023

Conversation

saileshbaidya
Copy link
Contributor

Related Issues/PRs

#xxx

What changes are proposed in this pull request?

Scala client for certified events api hosted on ML workload Admin endpoint.

How is this patch tested?

  • I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Does this PR change any dependencies?

  • [* ] No. You can skip this section.
  • Yes. Make sure the dependencies are resolved correctly, and list changes here.

Does this PR add a new feature? If so, have you added samples on website?

  • [ *] No. You can skip this section.
  • Yes. Make sure you have added samples following below steps.
  1. Find the corresponding markdown file for your new feature in website/docs/documentation folder.
    Make sure you choose the correct class estimators/transformers and namespace.
  2. Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
  3. Make sure the DocTable points to correct API link.
  4. Navigate to website folder, and run yarn run start to make sure the website renders correctly.
  5. Don't forget to add <!--pytest-codeblocks:cont--> before each python code blocks to enable auto-tests for python samples.
  6. Make sure the WebsiteSamplesTests job pass in the pipeline.

@github-actions
Copy link

github-actions bot commented Aug 8, 2023

Hey @saileshbaidya 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary by GPT-4

The code changes include updates to the logging functionality in the SynapseML library. The updates include:

  • Adding new classes and methods for handling JWT (JSON Web Tokens) and making HTTP requests.
  • Updating existing logging methods to include a new parameter logCertifiedEvent which defaults to false. This parameter is used to determine whether or not to log a certified event.
  • Adding new tests for the JWT token parser and host endpoint utilities.

The changes also remove an unnecessary newline character from two files: .gitignore and environment.yml.

Suggestions

The changes in this PR look good. However, there are a few suggestions for improvement:

  1. In the SynapseMLLogging.scala file, the logBase method has been updated to include a new parameter logCertifiedEvent. It would be helpful to add comments explaining what this new parameter is for and how it affects the method's behavior.

  2. In the same file, the logVerb method has been updated to include two new parameters: columns and logCertifiedEvent. Again, adding comments explaining these parameters would be beneficial.

  3. The newly added files under the logging/Usage directory lack comments explaining their purpose and functionality. Adding such comments would improve code readability and maintainability.

  4. In the test files (FabricTokenParserTests.scala, HostEndpointUtilsTests.scala), it would be helpful to add more detailed descriptions of what each test is checking for in a comment above each test.

  5. There are several instances where lines of code have been removed from .gitignore, environment.yml, and manifest.yaml. If these changes are intentional, it might be helpful to explain why in the PR description.

  6. Lastly, ensure that all new methods have appropriate unit tests to maintain code coverage.

@saileshbaidya saileshbaidya changed the title [Draft] scala client for certified events Chore: [Draft] scala client for certified events Aug 8, 2023
@saileshbaidya saileshbaidya changed the title Chore: [Draft] scala client for certified events chore: [Draft] scala client for certified events Aug 8, 2023
build.sbt Outdated Show resolved Hide resolved
@saileshbaidya
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@codecov-commenter
Copy link

codecov-commenter commented Aug 17, 2023

Codecov Report

Merging #2045 (6a760a9) into master (21cff04) will decrease coverage by 10.86%.
The diff coverage is 41.17%.

❗ Current head 6a760a9 differs from pull request most recent head 5ef7c59. Consider uploading reports for the commit 5ef7c59 to get more accurate results

@@             Coverage Diff             @@
##           master    #2045       +/-   ##
===========================================
- Coverage   87.02%   76.16%   -10.86%     
===========================================
  Files         306      309        +3     
  Lines       16162    16259       +97     
  Branches      824      831        +7     
===========================================
- Hits        14065    12384     -1681     
- Misses       2097     3875     +1778     
Files Coverage Δ
...ft/azure/synapse/ml/logging/SynapseMLLogging.scala 91.66% <91.66%> (-0.79%) ⬇️
...re/synapse/ml/logging/common/PlatformDetails.scala 70.58% <70.58%> (ø)
...ft/azure/synapse/ml/logging/fabric/RESTUtils.scala 0.00% <0.00%> (ø)
...napse/ml/logging/fabric/CertifiedEventClient.scala 33.92% <33.92%> (ø)

... and 61 files with indirect coverage changes

@saileshbaidya
Copy link
Contributor Author

/azp run

@saileshbaidya
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mhamilton723
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

.gitignore Outdated Show resolved Hide resolved
environment.yml Outdated Show resolved Hide resolved
@mhamilton723
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mhamilton723 mhamilton723 merged commit 9a4800c into microsoft:master Oct 10, 2023
63 of 68 checks passed
@saileshbaidya saileshbaidya deleted the saibai/ScalaClientCE branch October 10, 2023 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants