? ? ? flink 运行任务突然报错,org.apache.flink.configuration.IllegalConfigurationException: TaskManager memory configuration failed: Derived JVM Overhead size (2.500gb (2684354560 bytes)) is not in configured JVM Overhead range [192.000mb (201326592 bytes), 1024.000mb (1073741824 bytes)]
org.apache.flink.configuration.IllegalConfigurationException: TaskManager memory configuration failed: Derived JVM Overhead size (2.500gb (2684354560 bytes)) is not in configured JVM Overhead range [192.000mb (201326592 bytes), 1024.000mb (1073741824 bytes)].
at org.apache.flink.runtime.clusterframework.TaskExecutorProcessUtils.processSpecFromConfig(TaskExecutorProcessUtils.java:163)
at org.apache.flink.client.deployment.AbstractContainerizedClusterClientFactory.getClusterSpecification(AbstractContainerizedClusterClientFactory.java:49)
at org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:65)
at org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:213)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1057)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1911)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: Derived JVM Overhead size (2.500gb (2684354560 bytes)) is not in configured JVM Overhead range [192.000mb (201326592 bytes), 1024.000mb (1073741824 bytes)].
at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.sanityCheckJvmOverhead(ProcessMemoryUtils.java:223)
at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.deriveJvmOverheadFromTotalFlinkMemoryAndOtherComponents(ProcessMemoryUtils.java:205)
at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.deriveJvmMetaspaceAndOverheadFromTotalFlinkMemory(ProcessMemoryUtils.java:172)
at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.deriveProcessSpecWithTotalFlinkMemory(ProcessMemoryUtils.java:105)
at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.memoryProcessSpecFromConfig(ProcessMemoryUtils.java:80)
at org.apache.flink.runtime.clusterframework.TaskExecutorProcessUtils.processSpecFromConfig(TaskExecutorProcessUtils.java:160)
... 10 more
? ?运行参数有配置-Dtaskmanager.memory.process.size 进程内存大小,报错不在范围内,排查参数发现在conf/flink-conf.xml 配置文件里面有 taskmanager.memory.flink.size这个参数
??配置项?? | ??TaskManager 配置参数?? | ??JobManager 配置参数?? |
---|
Flink 总内存 | taskmanager.memory.flink.size | jobmanager.memory.flink.size | 进程总内存 | taskmanager.memory.process.size | jobmanager.memory.process.size |
结论:进程总内存和 Flink 总内存冲突,去掉Flink 总内存,就正常运行起来(Flink(八):Flink 进程内存、总内存内存简介_zhengcongyi的博客-CSDN博客)
|