Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Question]chunjun 启动yarn-session报错 #1868

Open
4 tasks done
heqxin opened this issue Jan 25, 2024 · 1 comment
Open
4 tasks done

[Question]chunjun 启动yarn-session报错 #1868

heqxin opened this issue Jan 25, 2024 · 1 comment
Labels
question Further information is requested

Comments

@heqxin
Copy link

heqxin commented Jan 25, 2024

Search before asking

  • I had searched in the issues and found no similar question.

  • I had googled my question but i didn't get any help.

  • I had read the documentation: ChunJun doc but it didn't help me.

Description

环境
Hadoop 3.3.0
flink 1.12.7
chunjun 1.12.8

flink lib:

flink-csv-1.12.7.jar
flink-dist_2.12-1.12.7.jar
flink-json-1.12.7.jar
flink-shaded-zookeeper-3.4.14.jar
flink-table_2.12-1.12.7.jar
flink-table-blink_2.12-1.12.7.jar
log4j-1.2-api-2.16.0.jar
log4j-api-2.16.0.jar
log4j-core-2.16.0.jar
log4j-slf4j-impl-2.16.0.jar

CHUNJUN_HOME：

chunjun-core.jar
connector
ddl-plugins
dirty-data-collector
docker-build
formats
metrics
restore-plugins

执行：

./yarn-session.sh -t $CHUNJUN_HOME -d

报错：

ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Fatal error occurred in the cluster entrypoint.
org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException: Could not start the ResourceManager akka.tcp://[email protected]:37027/user/rpc/resourcemanager_0
    at org.apache.flink.runtime.resourcemanager.ResourceManager.onStart(ResourceManager.java:235) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStart(RpcEndpoint.java:181) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StoppedState.start(AkkaRpcActor.java:605) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:180) ~[chunjun-ddl-mysql.jar:?]
    at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [chunjun-ddl-mysql.jar:?]
    at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [chunjun-ddl-mysql.jar:?]
    at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [chunjun-ddl-mysql.jar:?]
    at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [chunjun-ddl-mysql.jar:?]
    at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [chunjun-ddl-mysql.jar:?]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [chunjun-ddl-mysql.jar:?]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [chunjun-ddl-mysql.jar:?]
    at akka.actor.Actor.aroundReceive(Actor.scala:517) [chunjun-ddl-mysql.jar:?]
    at akka.actor.Actor.aroundReceive$(Actor.scala:515) [chunjun-ddl-mysql.jar:?]
    at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [chunjun-ddl-mysql.jar:?]
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [chunjun-ddl-mysql.jar:?]
    at akka.actor.ActorCell.invoke(ActorCell.scala:561) [chunjun-ddl-mysql.jar:?]
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [chunjun-ddl-mysql.jar:?]
    at akka.dispatch.Mailbox.run(Mailbox.scala:225) [chunjun-ddl-mysql.jar:?]
    at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [chunjun-ddl-mysql.jar:?]
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [chunjun-ddl-mysql.jar:?]
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [chunjun-ddl-mysql.jar:?]
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [chunjun-ddl-mysql.jar:?]
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [chunjun-ddl-mysql.jar:?]
Caused by: org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException: Cannot initialize resource provider.
    at org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager.initialize(ActiveResourceManager.java:124) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.startResourceManagerServices(ResourceManager.java:247) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.onStart(ResourceManager.java:231) ~[chunjun-ddl-mysql.jar:?]
    ... 22 more
Caused by: org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException: Could not start resource manager client.
    at org.apache.flink.yarn.YarnResourceManagerDriver.initializeInternal(YarnResourceManagerDriver.java:181) ~[chunjun-restore-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.AbstractResourceManagerDriver.initialize(AbstractResourceManagerDriver.java:81) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager.initialize(ActiveResourceManager.java:122) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.startResourceManagerServices(ResourceManager.java:247) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.onStart(ResourceManager.java:231) ~[chunjun-ddl-mysql.jar:?]
    ... 22 more
Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.reflect.InvocationTargetException
    at org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:81) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.java:49) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy$1.run(RMProxy.java:148) ~[hadoop-yarn-common-3.3.0.jar:?]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_333]
    at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_333]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1734) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:144) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider.init(DefaultNoHARMFailoverProxyProvider.java:65) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.createNonHaRMFailoverProxyProvider(RMProxy.java:171) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.newProxyInstance(RMProxy.java:131) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:102) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:73) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:193) ~[hadoop-yarn-client-3.3.0.jar:?]
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.serviceStart(AMRMClientAsyncImpl.java:131) ~[hadoop-yarn-client-3.3.0.jar:?]
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.flink.yarn.YarnResourceManagerDriver.initializeInternal(YarnResourceManagerDriver.java:169) ~[chunjun-restore-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.AbstractResourceManagerDriver.initialize(AbstractResourceManagerDriver.java:81) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager.initialize(ActiveResourceManager.java:122) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.startResourceManagerServices(ResourceManager.java:247) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.onStart(ResourceManager.java:231) ~[chunjun-ddl-mysql.jar:?]
    ... 22 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_333]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_333]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_333]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_333]
    at org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:78) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.java:49) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy$1.run(RMProxy.java:148) ~[hadoop-yarn-common-3.3.0.jar:?]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_333]
    at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_333]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1734) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:144) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider.init(DefaultNoHARMFailoverProxyProvider.java:65) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.createNonHaRMFailoverProxyProvider(RMProxy.java:171) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.newProxyInstance(RMProxy.java:131) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:102) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:73) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:193) ~[hadoop-yarn-client-3.3.0.jar:?]
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.serviceStart(AMRMClientAsyncImpl.java:131) ~[hadoop-yarn-client-3.3.0.jar:?]
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.flink.yarn.YarnResourceManagerDriver.initializeInternal(YarnResourceManagerDriver.java:169) ~[chunjun-restore-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.AbstractResourceManagerDriver.initialize(AbstractResourceManagerDriver.java:81) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager.initialize(ActiveResourceManager.java:122) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.startResourceManagerServices(ResourceManager.java:247) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.onStart(ResourceManager.java:231) ~[chunjun-ddl-mysql.jar:?]
    ... 22 more
Caused by: java.lang.AbstractMethodError: org.apache.hadoop.ipc.ProtobufRpcEngine2.getProxy(Ljava/lang/Class;JLjava/net/InetSocketAddress;Lorg/apache/hadoop/security/UserGroupInformation;Lorg/apache/hadoop/conf/Configuration;Ljavax/net/SocketFactory;ILorg/apache/hadoop/io/retry/RetryPolicy;Ljava/util/concurrent/atomic/AtomicBoolean;)Lorg/apache/hadoop/ipc/ProtocolProxy;
    at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:579) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:545) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:492) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:457) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:643) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:600) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.<init>(ApplicationMasterProtocolPBClientImpl.java:61) ~[hadoop-yarn-common-3.3.0.jar:?]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_333]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_333]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_333]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_333]
    at org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:78) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.java:49) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy$1.run(RMProxy.java:148) ~[hadoop-yarn-common-3.3.0.jar:?]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_333]
    at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_333]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1734) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.getProxy(RMProxy.java:144) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider.init(DefaultNoHARMFailoverProxyProvider.java:65) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.createNonHaRMFailoverProxyProvider(RMProxy.java:171) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.newProxyInstance(RMProxy.java:131) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:102) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:73) ~[hadoop-yarn-common-3.3.0.jar:?]
    at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:193) ~[hadoop-yarn-client-3.3.0.jar:?]
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.serviceStart(AMRMClientAsyncImpl.java:131) ~[hadoop-yarn-client-3.3.0.jar:?]
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) ~[chunjun-connector-iceberg.jar:?]
    at org.apache.flink.yarn.YarnResourceManagerDriver.initializeInternal(YarnResourceManagerDriver.java:169) ~[chunjun-restore-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.AbstractResourceManagerDriver.initialize(AbstractResourceManagerDriver.java:81) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager.initialize(ActiveResourceManager.java:122) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.startResourceManagerServices(ResourceManager.java:247) ~[chunjun-ddl-mysql.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.onStart(ResourceManager.java:231) ~[chunjun-ddl-mysql.jar:?]
    ... 22 more

直接执行./yarn-session.sh 没有问题

Code of Conduct
@heqxin heqxin added the question Further information is requested label Jan 25, 2024
@Icarus-Alpha
Copy link

@heqxin 我也是这个问题，您解决了吗？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@heqxin @Icarus-Alpha