Tomcat集群session复制,httpd/nginx反代Tomcat集群

233次阅读

共计 23363 个字符，预计需要花费 59 分钟才能阅读完成。

一个大型站点都会涉及到动态应用，动态应用都需要做会话保持，常见的会话保持方式就三种，一是 session stick，二是session replication，三是session share，对于小型规模的tomcat 集群，大多者会采用 session replication 方式，但阅读官方文档也好，查询大牛博客也罢，发现均有不准确之处，所以亲测成功实现之后得出如下文档，还望高人指点。

实验环境：

操作系统：CentOS 7.2

tomcat版本：tomcat-7.0.54（yum安装方式）

httpd版本：httpd-2.4.6（yum安装方式）

nginx版本：nginx-1.10.2（yum安装方式）

iptables状态：disable

SELinux状态：disable

TomcatA配置过程

[root@TomcatA ~]# ifconfig

eno16777736:flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.20 netmask255.255.255.0 broadcast 192.168.0.255

inet6 fe80::20c:29ff:fefe:64f3 prefixlen 64 scopeid0x20<link>

ether 00:0c:29:fe:64:f3 txqueuelen 1000 (Ethernet)

RX packets 1147 bytes 141324(138.0 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 659 bytes 102242 (99.8KiB)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

lo:flags=73<UP,LOOPBACK,RUNNING> mtu65536

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10<host>

loop txqueuelen 0 (Local Loopback)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 0 bytes 0 (0.0 B)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

[root@TomcatA ~]# cat/etc/yum.repos.d/hehe.repo

[cdrom-repo]

name=cdrom-repo

baseurl=file:///cdrom

gpgcheck=0

[aliyun-epel]

name=aliyun-epel

baseurl=http://mirrors.aliyun.com/epel/7/x86_64/

gpgcheck=0

[root@TomcatA ~]#

[root@TomcatA ~]# yum install -y tomcat

[root@TomcatA ~]# mkdir -pv/usr/share/tomcat/webapps/ROOT/WEB-INF

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT’

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT/WEB-INF’

[root@TomcatA ~]# vim/usr/share/tomcat/webapps/ROOT/index.jsp

[root@TomcatA ~]# cat/usr/share/tomcat/webapps/ROOT/index.jsp

<%@ page language="Java" %> 
<html> 
         <head><title>TomcatA</title></head> 
         <body> 
                   <h1><fontcolor="red">TomcatA</font></h1> 
                   <table                border="1"> 
                            <tr> 
                                     <td>SessionID</td> 
                            <%session.setAttribute("test.com","test.com"); %> 
                                     <td><%=session.getId() %></td> 
                            </tr> 
                            <tr> 
                                     <td>Createdon</td> 
                                     <td><%=session.getCreationTime() %></td> 
                            </tr> 
                   </table> 
         </body> 
</html>

[root@TomcatA ~]# scp/usr/share/tomcat/webapps/ROOT/index.jsp 192.168.0.21:/usr/share/tomcat/webapps/ROOT/

The authenticity of host ‘192.168.0.21(192.168.0.21)’ can’t be established.

ECDSA key fingerprint isfc:7f:15:2b:2d:69:c9:c1:b0:79:7c:ce:ab:c0:0c:36.

Are you sure you want to continueconnecting (yes/no)? yes

Warning: Permanently added ‘192.168.0.21’(ECDSA) to the list of known hosts.

root@192.168.0.21’s password:

index.jsp 100% 410 0.4KB/s 00:00

[root@TomcatA ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 128 :::22 :::*

[root@TomcatA ~]# systemctl start tomcat

[root@TomcatA ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

[root@TomcatA ~]#

TomcatB环境配置

[root@TomcatB ~]# ifconfig

eno16777736:flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.21 netmask255.255.255.0 broadcast 192.168.0.255

inet6 fe80::20c:29ff:fed8:be3b prefixlen 64 scopeid0x20<link>

ether 00:0c:29:d8:be:3b txqueuelen 1000 (Ethernet)

RX packets 839 bytes 107014(104.5 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 430 bytes 68356 (66.7KiB)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10<host>

loop txqueuelen 0 (Local Loopback)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 0 bytes 0 (0.0 B)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

[root@TomcatB ~]# cat/etc/yum.repos.d/hehe.repo

[cdrom-repo]

name=cdrom-repo

baseurl=file:///cdrom

gpgcheck=0

[aliyun-epel]

name=aliyun-epel

baseurl=http://mirrors.aliyun.com/epel/7/x86_64/

gpgcheck=0

[root@TomcatB ~]#

[root@TomcatB ~]# yum install -y tomcat

[root@TomcatB ~]# mkdir -pv/usr/share/tomcat/webapps/ROOT/WEB-INF

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT’

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT/WEB-INF’

[root@TomcatB ~]# vim/usr/share/tomcat/webapps/ROOT/index.jsp

[root@TomcatB ~]# cat/usr/share/tomcat/webapps/ROOT/index.jsp

<%@ page language="java" %> 
<html> 
         <head><title>TomcatB</title></head> 
         <body> 
                   <h1><fontcolor="blue">TomcatB</font></h1> 
                   <table                border="1"> 
                            <tr> 
                                     <td>SessionID</td> 
                            <%session.setAttribute("test.com","test.com"); %> 
                                     <td><%=session.getId() %></td> 
                            </tr> 
                            <tr> 
                                     <td>Createdon</td> 
                                     <td><%=session.getCreationTime() %></td> 
                            </tr> 
                   </table> 
         </body> 
</html>

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 128 :::22 :::*

[root@TomcatB ~]# systemctl start tomcat

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

[root@TomcatB ~]#

测试两台 tomcat 服务器工作是否正常

使用浏览器访问结果如图所示

Tomcat 集群 session 复制,httpd/nginx 反代 Tomcat 集群

接下来就需要配置前端的反代服务器，以及 tomcat 集群 session 复制了

httpd反代 tomcat 集群配置

在这里我们为了节约资源就把 httpd 反代服务器配置到 TomcatA 服务器上

[root@TomcatA ~]# yum install -y httpd

[root@TomcatA ~]# vim/etc/httpd/conf.d/httpd_tomcat.conf

[root@TomcatA ~]# cat/etc/httpd/conf.d/httpd_tomcat.conf

#LoadModule proxy_modulemodules/mod_proxy.so 
#LoadModule proxy_ajp_modulemodules/mod_proxy_ajp.so 
#LoadModule proxy_balancer_modulemodules/mod_proxy_balancer.so 
   
<proxy balancer://tomcatcluster> 
         BalancerMemberajp://192.168.0.20:8009 route=TomcatA loadfactor=1 
         BalancerMemberajp://192.168.0.21:8009 route=TomcatB loadfactor=1 
         ProxySetlbmethod=byrequests 
         #ProxySetstickysession=JSESSIONID 
</Proxy> 
   
<VirtualHost *:80> 
         ServerNameproxy.test.com 
         ProxyViaOn 
         ProxyRequestsOff 
         ProxyPreserveHostOn 
         <Proxy*> 
                   Requireall granted 
         </Proxy> 
         ProxyPass/ balancer://tomcatcluster/
         ProxyPassReverse/ balancer://tomcatcluster/
         <Location/> 
                   Requireall granted 
         </Location> 
         <Location/balancer-manager> 
                   SetHandlerbalancer-manager 
                   ProxyPass! 
                   Requireall granted 
         </Location> 
</VirtualHost>

[root@TomcatA ~]# vim/etc/httpd/conf/httpd.conf

修改如下内容

ServerName www.example.com:80 
<IfModule dir_module> 
   DirectoryIndex index.jsp 
</IfModule>

[root@TomcatA ~]# httpd -t

Syntax OK

[root@TomcatA ~]# cp/etc/tomcat/server.xml{,.bak}

[root@TomcatA ~]# vim/etc/tomcat/server.xml

默认配置文件需要修改以下几个地方

Engine 标签下添加集群配置内容

   <ClusterclassName="org.apache.catalina.ha.tcp.SimpleTcpCluster"
           channelSendOptions="8"> 
   
       <ManagerclassName="org.apache.catalina.ha.session.DeltaManager"
           expireSessionsOnShutdown="false"
            notifyListenersOnReplication="true"/> 
   
       <ChannelclassName="org.apache.catalina.tribes.group.GroupChannel"> 
           <MembershipclassName="org.apache.catalina.tribes.membership.McastService"
                address="228.0.0.4"
               port="45564"
                frequency="500"
                dropTime="3000"/> 
           <ReceiverclassName="org.apache.catalina.tribes.transport.nio.NioReceiver"
                address="192.168.0.20"
                port="4000"
                autoBind="100"
               selectorTimeout="5000"
                maxThreads="6"/> 
   
           <SenderclassName="org.apache.catalina.tribes.transport.ReplicationTransmitter"> 
                <TransportclassName="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/> 
           </Sender> 
           <InterceptorclassName="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/> 
           <InterceptorclassName="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/> 
       </Channel> 
   
       <ValveclassName="org.apache.catalina.ha.tcp.ReplicationValve"
           filter=""/> 
       <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/> 
   
       <DeployerclassName="org.apache.catalina.ha.deploy.FarmWarDeployer"
           tempDir="/tmp/war-temp/"
           deployDir="/tmp/war-deploy/"
           watchDir="/tmp/war-listen/"
           watchEnabled="false"/> 
   
       <ClusterListenerclassName="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/> 
       <ClusterListenerclassName="org.apache.catalina.ha.session.ClusterSessionListener"/> 
</Cluster>

修改日志格式为 httpd combined 格式

<Valve className="org.apache.catalina.valves.AccessLogValve"directory="logs"
      prefix="localhost_access_log."suffix=".txt"
      pattern="%h %l %u %t&quot;%r&quot; %s %b &quot;%{Referer}i&quot; 
            &quot;%{User-Agent}i&quot;" />

[root@TomcatB ~]# mv/etc/tomcat/server.xml{,.bak}

[root@TomcatA ~]# scp/etc/tomcat/server.xml 192.168.0.21:/etc/tomcat/

root@192.168.0.21’s password:

server.xml 100% 8226 8.0KB/s 00:00

[root@TomcatA ~]#

对比 TomcatA 需要修改的地方

<Engine name="Catalina"defaultHost="localhost" jvmRoute="TomcatB"> 
           <ReceiverclassName="org.apache.catalina.tribes.transport.nio.NioReceiver"
               address="192.168.0.21"

添加应用程序自己的 web.xml 配置文件，需要有 <distributable/> 这个标签

[root@TomcatA ~]# vim/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml

[root@TomcatA ~]# cat/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml

<?xml version="1.0"encoding="ISO-8859-1"?> 
   
<web-appxmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
                     http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" 
 version="3.0"
 metadata-complete="true"> 
   
         <distributable/> 
   
</web-app>

[root@TomcatA ~]# scp/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml192.168.0.21:/usr/share/tomcat/webapps/ROOT/WEB-INF

root@192.168.0.21’s password:

web.xml 100% 356 0.4KB/s 00:00

[root@TomcatA ~]# systemctl restart tomcat

[root@TomcatA ~]# systemctl start httpd

[root@TomcatA ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 128 :::80 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

LISTEN 0 50 ::ffff:192.168.0.20:4000 :::*

LISTEN 0 1 ::ffff:127.0.0.1:8005 :::*

[root@TomcatA ~]#

TomcatA 重启成功日志如下

[root@TomcatA ~]# tail -f/var/log/tomcat/catalina.2017-09-26.log

Sep 26, 2017 2:57:34 AMorg.apache.catalina.core.StandardServer await

INFO: A valid shutdown command was receivedvia the shutdown port. Stopping the Server instance.

Sep 26, 2017 2:57:34 AMorg.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler[“http-bio-8080”]

Sep 26, 2017 2:57:35 AMorg.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Nativelibrary which allows optimal performance in production environments was notfound on the java.library.path:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

Sep 26, 2017 2:57:35 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler[“http-bio-8080”]

Sep 26, 2017 2:57:35 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler[“ajp-bio-8009”]

Sep 26, 2017 2:57:35 AMorg.apache.catalina.startup.Catalina load

INFO: Initialization processed in 379 ms

Sep 26, 2017 2:57:35 AMorg.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Sep 26, 2017 2:57:35 AMorg.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: ApacheTomcat/7.0.54

Sep 26, 2017 2:57:35 AM org.apache.catalina.ha.tcp.SimpleTcpClusterstartInternal

INFO: Cluster is about to start

Sep 26, 2017 2:57:35 AMorg.apache.catalina.tribes.transport.ReceiverBase bind

INFO: Receiver Server Socket boundto:/192.168.0.20:4000

Sep 26, 2017 2:57:35 AM org.apache.catalina.tribes.membership.McastServiceImplsetupSocket

INFO: Setting cluster mcast soTimeout to500

Sep 26, 2017 2:57:35 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:4

Sep 26, 2017 2:57:36 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Done sleeping, membershipestablished, start level:4

Sep 26, 2017 2:57:36 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:8

Sep 26, 2017 2:57:37 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Done sleeping, membershipestablished, start level:8

Sep 26, 2017 2:57:37 AMorg.apache.catalina.ha.deploy.FarmWarDeployer start

SEVERE: FarmWarDeployer can only work ashost cluster subelement!

Sep 26, 2017 2:57:37 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deploying web application directory/var/lib/tomcat/webapps/ROOT

Sep 26, 2017 2:58:13 AMorg.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance forsession ID generation using [SHA1PRNG] took [35,019] milliseconds.

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Register manager localhost# tocluster element Engine with name Catalina

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Starting clustering manager atlocalhost#

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.DeltaManager getAllClusterSessions

INFO: Manager [localhost#]: skipping statetransfer. No members active in cluster group.

Sep 26, 2017 2:58:13 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deployment of web applicationdirectory /var/lib/tomcat/webapps/ROOT has finished in 35,563 ms

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.JvmRouteBinderValve startInternal

INFO: JvmRouteBinderValve started

Sep 26, 2017 2:58:13 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler[“http-bio-8080”]

Sep 26, 2017 2:58:13 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler[“ajp-bio-8009”]

Sep 26, 2017 2:58:13 AM org.apache.catalina.startup.Catalinastart

INFO: Server startup in 37684 ms

TomcatB重启成功后 TomcatA 会产生如下日志

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.io.BufferPool getBufferPool

INFO: Created a buffer pool with maxsize:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl

Sep 26, 2017 3:01:06 AMorg.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded

INFO: Replication memberadded:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0,21}:4000,{192, 168, 0, 21},4000, alive=1014, securePort=-1, UDP Port=-1, id={5025 -81 7 -94 116 76 51 -100 -57 116 -83 10 93 -110 -108}, payload={},command={}, domain={},]

TomcatB 需做如下修改

[root@TomcatB ~]# cat/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml

<?xml version="1.0"encoding="ISO-8859-1"?> 
   
<web-appxmlns="http://java.sun.com/xml/ns/javaee"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
                     http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" 
 version="3.0"
  metadata-complete="true"> 
   
         <distributable/> 
   
</web-app>

[root@TomcatB ~]# systemctl restart tomcat

TomcatB 成功重启过程日志如下

[root@TomcatB ~]# tail -f/var/log/tomcat/catalina.2017-09-26.log

Sep 26, 2017 3:01:03 AMorg.apache.catalina.core.StandardServer await

INFO: A valid shutdown command was receivedvia the shutdown port. Stopping the Server instance.

Sep 26, 2017 3:01:03 AMorg.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler[“http-bio-8080”]

Sep 26, 2017 3:01:04 AMorg.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Nativelibrary which allows optimal performance in production environments was notfound on the java.library.path:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

Sep 26, 2017 3:01:04 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler[“http-bio-8080”]

Sep 26, 2017 3:01:04 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler[“ajp-bio-8009”]

Sep 26, 2017 3:01:04 AM org.apache.catalina.startup.Catalinaload

INFO: Initialization processed in 413 ms

Sep 26, 2017 3:01:04 AMorg.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Sep 26, 2017 3:01:04 AMorg.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: ApacheTomcat/7.0.54

Sep 26, 2017 3:01:04 AMorg.apache.catalina.ha.tcp.SimpleTcpCluster startInternal

INFO: Cluster is about to start

Sep 26, 2017 3:01:04 AMorg.apache.catalina.tribes.transport.ReceiverBase bind

INFO: Receiver Server Socket boundto:/192.168.0.21:4000

Sep 26, 2017 3:01:04 AMorg.apache.catalina.tribes.membership.McastServiceImpl setupSocket

INFO: Setting cluster mcast soTimeout to500

Sep 26, 2017 3:01:04 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:4

Sep 26, 2017 3:01:05 AMorg.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded

INFO: Replication memberadded:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0,20}:4000,{192, 168, 0, 20},4000, alive=209565, securePort=-1, UDP Port=-1,id={6 -86 -55 -67 102 127 77 -57 -67 -67 -105 -20 -37 -55 76 -22}, payload={},command={}, domain={},]

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Done sleeping, membershipestablished, start level:4

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:8

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.io.BufferPool getBufferPool

INFO: Created a buffer pool with maxsize:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl

Sep 26, 2017 3:01:06 AM org.apache.catalina.tribes.membership.McastServiceImplwaitForMembers

INFO: Done sleeping, membershipestablished, start level:8

Sep 26, 2017 3:01:06 AMorg.apache.catalina.ha.deploy.FarmWarDeployer start

SEVERE: FarmWarDeployer can only work ashost cluster subelement!

Sep 26, 2017 3:01:06 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deploying web application directory/var/lib/tomcat/webapps/ROOT

Sep 26, 2017 3:01:48 AMorg.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance forsession ID generation using [SHA1PRNG] took [41,566] milliseconds.

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Register manager localhost# tocluster element Engine with name Catalina

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Starting clustering manager atlocalhost#

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager getAllClusterSessions

INFO: Manager [localhost#], requestingsession state from org.apache.catalina.tribes.membership.MemberImpl[tcp://{192,168, 0, 20}:4000,{192, 168, 0, 20},4000, alive=253179, securePort=-1, UDPPort=-1, id={6 -86 -55 -67 102 127 77 -57 -67 -67 -105 -20 -37 -55 76 -22},payload={}, command={}, domain={},]. This operation will timeout if no sessionstate has been received within 60 seconds.

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager waitForSendAllSessions

INFO: Manager [localhost#]; session statesend at 9/26/17 3:01 AM received in 114 ms.

Sep 26, 2017 3:01:48 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deployment of web applicationdirectory /var/lib/tomcat/webapps/ROOT has finished in 42,286 ms

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.JvmRouteBinderValve startInternal

INFO: JvmRouteBinderValve started

Sep 26, 2017 3:01:48 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler[“http-bio-8080”]

Sep 26, 2017 3:01:48 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler[“ajp-bio-8009”]

Sep 26, 2017 3:01:48 AMorg.apache.catalina.startup.Catalina start

INFO: Server startup in 44401 ms

集群 session 配置成功，使用浏览器访问反代服务器的 80 端口检验效果

Tomcat 集群 session 复制,httpd/nginx 反代 Tomcat 集群

至此，httpd反代 tomcat 集群，并且 tomcat 集群 session 复制都成功实现了

Nginx反代 tomat 集群配置

在这里我们为了节约资源就把 nginx 反代服务器配置到 TomcatB 服务器上

[root@TomcatB ~]# yum install -y nginx

[root@TomcatB ~]# cp/etc/nginx/nginx.conf{,.bak}

[root@TomcatB ~]# vim /etc/nginx/nginx.conf

Nginx默认配置文件需要修改以下几处

定义后端 tomcat 集群

upstreamtomcatservers {
       server 192.168.0.20:8080; 
       server 192.168.0.21:8080; 
}

注释 IPV6 监听

   server {
       listen       80 default_server; 
       #listen       [::]:80default_server;

定义完全反代至后端 tomcat 集群

       location / {
           root  /usr/share/tomcat/wepapps/ROOT; 
            index  index.jsp; 
           proxy_pass http://tomcatservers; 
       }

定义让后端 tomcat 集群获取客户端真实IP

       proxy_set_header Host $host; 
       proxy_set_header X-Real-IP $remote_addr; 
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

[root@TomcatB ~]# nginx -t

nginx: the configuration file/etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conftest is successful

[root@TomcatB ~]#

继 httpd 反代 tomcat 集群配置之后需要修改的地方

[root@TomcatA ~]# vim/etc/tomcat/server.xml

       <ValveclassName="org.apache.catalina.valves.AccessLogValve"directory="logs"
              prefix="localhost_access_log." suffix=".txt"
              pattern="%{X-Forwarded-For}i %l %u %t &quot;%r&quot; %s %b&quot;%{Referer}i&quot; 
                      &quot;%{User-Agent}i&quot;"/> 
               <!--pattern="%h %l %u %t&quot;%r&quot; %s %b &quot;%{Referer}i&quot; 
                       &quot;%{User-Agent}i&quot;"/> -->

[root@TomcatA ~]# systemctl restart tomcat

[root@TomcatB ~]# vim/etc/tomcat/server.xml

       <ValveclassName="org.apache.catalina.valves.AccessLogValve"directory="logs"
              prefix="localhost_access_log." suffix=".txt"
              pattern="%{X-Forwarded-For}i %l %u %t &quot;%r&quot; %s %b&quot;%{Referer}i&quot; 
                      &quot;%{User-Agent}i&quot;"/> 
               <!--pattern="%h %l %u %t&quot;%r&quot; %s %b &quot;%{Referer}i&quot; 
                       &quot;%{User-Agent}i&quot;"/> -->

[root@TomcatB ~]# systemctl restart tomcat

Httpd 反代 tomcat 集群无需在 httpd 添加额外传递客户端 ip 的参数以及修改 tomcat 日志记录格式就可以让后端 tomcat 记录客户端的真实 IP，但nginx 反代 tomcat 集群时必须首先在 nginx 上添加向后端 tomat 服务器传递客户端真实 IP 的参数，其次还需要修改 tomcat 服务器的默认日志记录格式，才能成功记录客户端的真实IP

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port PeerAddress:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

LISTEN 0 50 ::ffff:192.168.0.21:4000 :::*

LISTEN 0 1 ::ffff:127.0.0.1:8005 :::*

[root@TomcatB ~]# systemctl start nginx

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q Local Address:Port Peer Address:Port

LISTEN 0 128 *:80 *:*

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

LISTEN 0 50 ::ffff:192.168.0.21:4000 :::*

LISTEN 0 1 ::ffff:127.0.0.1:8005 :::*

[root@TomcatB ~]#

Nginx反代 tomcat 集群 session 复制效果检验

Tomcat 集群 session 复制,httpd/nginx 反代 Tomcat 集群

这里主要是为了给大家展示 tomcat 集群 session 复制的效果，所以在 httpd 或者 nginx 反代 tomat 集群这方面并没有给大家展示更多情况，httpd反代 tomcat 集群可以使用 http 协议、ajp协议、mod_jk协议三种方式反代，httpd反代 tomcat 集群时可以在实现 session 复制的同时，实现 session 粘性，nginx反代 tomcat 集群时还可以配置权重、健康状态检查、sorry server，在这里不做全面展示，有需求者可查阅本博客相应专业板块。