1. 程式人生 > >【Docker容器啟動問題】容器啟動時, exceptions.JedisConnectionException:Could not get a resource from the pool

【Docker容器啟動問題】容器啟動時, exceptions.JedisConnectionException:Could not get a resource from the pool

問題現場環境:

1、本地虛擬機器CentOS7 下的docker環境。

2、docker下的redis、mysql已正常啟動,且虛擬機器外可正常訪問。

3、啟動容器(SpringCloud 閘道器服務)需要使用 docker下的redis、MySql。

 


問題現象:

在啟動容器時,出現錯誤異常c.n.d.s.t.d.RedirectingEurekaHttpClient:Request execution error和nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool,如下所示:

2018-09-13 12:53:25.742 ERROR 1 --- [           main] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error

com.sun.jersey.api.client.ClientHandlerException: java.net.NoRouteToHostException: No route to host (Host unreachable)
	at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.4.12.jar!/:1.4.12]
	at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.getApplicationsInternal(AbstractJerseyEurekaHttpClient.java:194) ~[eureka-client-1.4.12.jar!/:1.4.12]

……

Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
2018-09-13 12:53:29.150 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application startup failed

org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.cloud.netflix.eureka.EurekaDiscoveryClientConfiguration'; nested exception is org.springframework.data
.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:50) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:348) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:151) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:114) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:879) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.finishRefresh(EmbeddedWebApplicationContext.java:144) ~[spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) ~[spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at com.wondersoft.gateway.GatewayApplication.main(GatewayApplication.java:31) [classes!/:1.5.2.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_111]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_111]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_111]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_111]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]

 


問題分析:

1、異常c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error

com.sun.jersey.api.client.ClientHandlerException: java.net.NoRouteToHostException: No route to host (Host unreachable),是處於閘道器在連線另外一個服務(註冊中心)時,丟擲的異常,屬於無法正常路由連線到註冊中心服務。

2、異常nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool,是處於正在連線redis時,丟擲的異常,屬於無法連線redis。

基於上述兩點,問題屬於容器間網路通訊問題

 


解決方法:

1、關閉防火牆。

使用root使用者,執行  systemctl stop firewalld.service 命令,關閉防火牆。

 

2、重新啟動 閘道器服務容器,正常啟動,則問題解決。

3、如果啟動時,出現錯誤

iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8111 -j DNAT,如下圖:

 

[[email protected] data]# docker start gateway
Error response from daemon: Cannot start container gateway: iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8111 -j DNAT --to-destination 172.17.0.5:8111 ! -i docker0: iptables: No chain/target/match by that name.
 (exit status 1)
Error: failed to start containers: [gateway]

則需要重建docker0網路恢復,按順序執行如下命令,即可。

pkill docker 
iptables -t nat -F 
ifconfig docker0 down 
brctl delbr docker0 
docker -d 
service docker restart

執行過程如下:

[[email protected] data]# pkill docker 
[[email protected] data]# iptables -t nat -F 
[[email protected] data]# ifconfig docker0 down 
[[email protected] data]# brctl delbr docker0 
[[email protected] data]# docker -d 
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock) 
INFO[0000] [graphdriver] using prior storage driver "devicemapper" 
WARN[0000] Running modprobe bridge nf_nat failed with message: , error: exit status 1 
INFO[0000] Loading containers: start.                   
..............
INFO[0000] Loading containers: done.                    
INFO[0000] Daemon has completed initialization          
INFO[0000] Docker daemon                                 commit=786b29d execdriver=native-0.2 graphdriver=devicemapper version=1.7.1
^CINFO[0061] Processing signal 'interrupt'                
[[email protected] data]# systemctl restart docker

4、啟動所需容器即可。