Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

客户端输出Instance changed event事件日志后,5-20s时间内还能请求到已下线的实例对应的endpoint #1232

Open
chengyouling opened this issue Mar 5, 2024 · 0 comments
Assignees

Comments

@chengyouling
Copy link
Collaborator

chengyouling commented Mar 5, 2024

场景:
部署3个服务,客户端consumer,服务端provider1、provider2,均设置无损下线waitTimeForShutDownInMillis时间为15000ms,provider2正常下线,一直压测请求客户端。

现象:
客户端consumer在waitTimeForShutDownInMillis时间后,本地实例更新打印Instance changed event事件日志,还能请求到已下线的provider2。

日志:
2024-03-05 10:42:18.637 INFO 63392 --- [-discovery-task] o.a.s.s.c.client.ServiceCenterDiscovery : Instance changed event, current: revision=2b863900874b21f5992b0787e5ea1fcb51caaa8c, instances=rest://10.37.110.162:8085|basic-provider|#; origin: revision=f16c617ab3a4052d3b385d572255f5fc7ae192f6, instances=rest://10.37.110.162:8088|basic-provider|rest://10.37.110.162:8085|basic-provider|#; appId=basic-application, serviceName=basic-provider

2024-03-05 10:42:43.772 ERROR 63392 --- [nio-8082-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.http.conn.ConnectTimeoutException: Connect to 10.37.110.162:8088 [/10.37.110.162] failed: Connect timed out] with root cause

java.net.SocketTimeoutException: Connect timed out
at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[na:na]
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) ~[na:na]
at java.base/java.net.Socket.connect(Socket.java:633) ~[na:na]
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.13.jar:4.5.13]
......
2024-03-05 10:43:18.500 INFO 63392 --- [nio-8082-exec-4] access_logger : 65e686c60a7de9e3|WebMVC receive request|0:0:0:0:0:0:0:1||0|0|/sayHello
2024-03-05 10:43:18.501 INFO 63392 --- [nio-8082-exec-4] access_logger : 65e686c60a7de9e3|RestTemplate start request||10.37.110.162:8085|0|0|/sayHello

问题定位:
根据异常项目错误日志,构建demo测试发现,服务发现使用的CachingServiceInstanceListSupplier获取实例信息,即开启了spring cloud缓存机制,反向定位发现,主要是依赖的spring-cloud-starter-loadbalancer组件中引入了evictor依赖,evictor中有ConcurrentMapWithTimedEviction的实现类,从而满足defaultLoadBalancerCacheManager初始化条件。

处理方法:
1、将spring cloud的负载缓存开关设置为false

spring:
  cloud:
    loadbalancer:
      cache:
        enabled: false

2、剔除evictor的依赖

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-loadbalancer</artifactId>
    <exclusions>
      <exclusion>
        <groupId>com.stoyanr</groupId>
        <artifactId>evictor</artifactId>
      </exclusion>
    </exclusions>
</dependency>

3、删除spring-cloud-starter-loadbalancer依赖

@chengyouling chengyouling self-assigned this Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant