Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-10782. Replace GSON with Jackson in EventQueue #7247

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

weimingdiit
Copy link
Contributor

What changes were proposed in this pull request?

Replace GSON with Jackson in EventQueue, which converts protobuf messages to JSON when trace level logging is enabled.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10782

How was this patch tested?

unit tests, manual tests

@weimingdiit weimingdiit marked this pull request as ready for review September 29, 2024 09:11
@weimingdiit
Copy link
Contributor Author

@adoroszlai Could you review this pr?

@adoroszlai
Copy link
Contributor

Thanks @weimingdiit for the patch.

Getting two kinds of errors:

ERROR events.EventQueue: Error serializing payload: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: org.apache.hadoop.hdds.scm.server.SCMDatanodeHeartbeatDispatcher$ContainerReportFromDatanode["report"]->org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos$ContainerReportsProto["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])

Solved this by setting:

.disable(SerializationFeature.FAIL_ON_EMPTY_BEANS)

on TRACING_SERIALIZER.

ERROR events.EventQueue: Error serializing payload: Direct self-reference leading to cycle (through reference chain: org.apache.hadoop.hdds.scm.server.SCMDatanodeHeartbeatDispatcher$PipelineReportFromDatanode["report"]->org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos$PipelineReportsProto["unknownFields"]->com.google.protobuf.UnknownFieldSet["defaultInstanceForType"])

I haven't found a quick solution for this.

Please also see #6500 (comment) for previous attempt to replace GSON in EventQueue.

@weimingdiit
Copy link
Contributor Author

@adoroszlai Thanks for your comment, I will try to solve the second question

@weimingdiit weimingdiit force-pushed the HDDS-10782 branch 2 times, most recently from 7935983 to c68fb93 Compare September 30, 2024 07:56
HDDS-10782. Replace GSON with Jackson in EventQueue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants