Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #323] Support the expansion of metrics data statistics in RocketMQ-Spring. #324

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

heihaozi
Copy link
Contributor

@heihaozi heihaozi commented Dec 9, 2020

What is the purpose of the change

Support the expansion of metrics data statistics in RocketMQ-Spring. This feature can enable:

  • Provide standard and diverse metrics representation(e.g. percentile, gauge, histogram).
  • Standardization of monitoring API and transport server API, and adapt to open-source monitoring components (e.g. Prometheus).
  • Integration with open-source visualization solution(e.g. Grafana).

Brief changelog

Created an interface called MetricExtension to record the number of messages produced or consumed. By obtaining the implementation of this interface through SPI, developers can easily write their own metrics data statistics method.

Verifying this change

Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

  • Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist.
  • Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test to make sure integration-test pass.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

@vongosling vongosling added this to the 2.1.2 milestone Dec 10, 2020
@@ -176,9 +176,10 @@
<exclude>.github/**</exclude>
<exclude>src/test/resources/certs/*</exclude>
<exclude>src/test/**/*.log</exclude>
<exclude>src/test/resources/META-INF/service/*</exclude>
<exclude>**/src/main/resources/META-INF/services/*</exclude>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you add prefix placeholder before the root directory /src?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it. When running mvn -B clean apache-rat:check and building RocketMQ Spring Boot, the rocketmq-spring-boot-samples project will also be checked. If I do not add the prefix placeholder, an unapproved license error will be reported. Like this: https://travis-ci.org/github/apache/rocketmq-spring/builds/748732956

import org.apache.rocketmq.client.consumer.DefaultLitePullConsumer;
import org.apache.rocketmq.remoting.RPCHook;

public class DefaultLitePullConsumerWithTopic extends DefaultLitePullConsumer {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DefaultLitePullConsumerWithTopic is not a rational name here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compared with DefaultLitePullConsumer, this class just has one more Topic attribute. So, I named this class DefaultLitePullConsumerWithTopic. Do you have any better suggestions for the naming of this class?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not make a new class here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where can this new class be maked? What's your suggestion?

* @param topic topic name
* @param count count of message
*/
void addProducerMessageCount(String topic, int count);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

int or long should be consistent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter type is always int, and long is not used.

import java.util.List;
import java.util.ServiceLoader;

public class MetricExtensionProvider {
Copy link
Member

@vongosling vongosling Dec 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not make class by yourself. I strongly recommend you use a similar service loader mode in RocketMQ[1]. It has many backoff strategies and verified in the production environment.

[1] https://github.com/apache/rocketmq/blob/master/broker/src/main/java/org/apache/rocketmq/broker/util/ServiceProvider.java

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ServiceProvider is in the broker module, RocketMQ-Spring has no dependency on the broker module, so it cannot be used directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could copy and make some improvements. Pls make a reference description when you do it .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ServiceLoader is a simple Service Provder Framework provided since JDK 1.6. I think the reference description is this: https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html


@Test
public void testAddProducerMessageCount() {
MetricExtensionProvider.addProducerMessageCount("topic1", 111);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be helpful for your code convincible if you could make more corner cases, such as have you simulated the count of failures?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unit test is mainly used to test whether the SPI of MetricExtensionProvider is running normally.The scenario where the counting fails you mentioned was considered by the developer when implementing the MetricExtension interface.

@heihaozi heihaozi requested a review from vongosling December 11, 2020 08:04
@vongosling
Copy link
Member

Do we consider the possibility of using aop to finish this feature? @RongtongJin Would you like to help to design here?

@RongtongJin RongtongJin added the discuss Discuss whether to support label Jan 2, 2021
@RongtongJin RongtongJin removed this from the 2.1.2 milestone Jan 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Discuss whether to support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support the expansion of metrics data statistics in RocketMQ-Spring.
3 participants