Postgres-XL集群部署与管理指南

共计 21654 个字符，预计需要花费 55 分钟才能阅读完成。

Postgres-XL 是一个基于 PostgreSQL 数据库的横向扩展开源 SQL 数据库集群，具有足够的灵活性来处理不同的数据库工作负载, 架构如下图所示：

• Web 2.0
• 操作数据存储
• GIS 的地理空间
• 混合业务工作环境
• OLTP 写频繁的业务
• 多租户服务提供商托管环境
• 完全 ACID，保持事务一致性
• 包含 JSON 的 Key-value 存储
• 需要 MPP 并行性商业智能 / 大数据分析
  
  各个组件介绍如下：
• Global Transaction Monitor (GTM)
  全局事务管理器，确保群集范围内的事务一致性。GTM 负责发放事务 ID 和快照作为其多版本并发控制的一部分。集群也可以配置一个或多个备用 GTM，以改进可用性。此外，可以在协调器间配置 GTM 代理，可用于改善可扩展性，减少 GTM 的通信量。
• GTM Standby
  GTM 的备用节点，在 pgxc,pgxl 中，GTM 控制所有的全局事务分配，如果出现问题，就会导致整个集群不可用，为了增加可用性，增加该备用节点。当 GTM 出现问题时，GTM Standby 可以升级为 GTM，保证集群正常工作。
• GTM Proxy
  GTM 需要与所有的 Coordinators 通信，为了降低压力，可以在每个 Coordinator 机器上部署一个 GTM Proxy。
• Coordinator
  协调员管理用户会话，并与 GTM 和数据节点进行交互。协调员解析，并计划查询，并给语句中的每一个组件发送下一个序列化的全局性计划。为节省机器，通常此服务和数据节点部署在一起。
• Data Node
  数据节点是数据实际存储的地方。数据的分布可以由 DBA 来配置。为了提高可用性，可以配置数据节点的热备以便进行故障转移准备。
  总之，GTM 是负责 ACID 的，保证分布式数据库全局事务一致性。得益于此，就算数据节点是分布的，但是在主节点操作增删改查事务时，就如同只操作一个数据库一样简单。Coordinator 是调度的，将操作指令发送到各个数据节点。datanodes 是数据节点，分布式存储数据。
  

  1、安装 Postgres-XL

  1.1 集群规划

  四台机器规划如下图所示：
  

  1.2 操作系统配置

  禁用防火墙、selinux。各个节点通过 yum 安装以下软件包：

  [root@pg01 ~]# yum -y install bzip2 readline-devel flex make gcc rsync

  另外，各个节点创建 postgres 用户：

  [root@pg01 ~]# groupadd postgres;useradd -g postgres postgres;echo RedHat|passwd --stdin postgres

  用户创建完成后，还需要在各个节点配置 postgres 用户的免密码登录(略)。然后编辑 postgres 用户的环境变量，加入如下内容：

  [root@pg01 ~]# mkdir /u01;chown postgres:postgres /u01
  [root@pg01 ~]# su - postgres
  [postgres@pg01 ~]$ vi .bashrc
  export PGUSER=postgres
  export PGHOME=/usr/local/pgsql
  export PGXC_CTL_HOME=/u01/pgxl/pgxc_ctl
  export LD_LIBRARY_PATH=$PGHOME/lib
  export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
  export PATH=$PGHOME/bin:$PATH:$HOME/.local/bin:$HOME/bin

  1.3 Postgres-XL 安装

  在各个节点以 root 用户进行安装，默认的安装目录是 /usr/local/pgsql。

  [root@pg01 ~]# wget https://×××w.postgres-xl.org/downloads/postgres-xl-10r1.tar.gz;tar -xzf postgres-xl-10r1.tar.gz;cd postgres-xl-10r1;./configure&&make&&make install
  [root@pg01 postgres-xl-10r1]#  cd contrib;make && make install

  2、配置 Postgres-XL 集群

  2.1 生成 pgxc_ctl 配置文件

  在一个节点操作即可。

  [postgres@pg01 ~]$ pgxc_ctl
  PGXC prepare
  PGXC exit

  编辑生成的配置文件，内容如下：

  pgxcInstallDir=/u01
  pgxcOwner=$USER
  pgxcUser=$pgxcOwner
  tmpDir=/tmp
  localTmpDir=$tmpDir
  #--------GTM Master--------
  gtmName=gtm_master
  gtmMasterServer=pg01
  gtmMasterPort=20001
  gtmMasterDir=$pgxcInstallDir/pgxl/nodes/gtm
  gtmExtraConfig=none
  gtmMasterSpecificExtraConfig=none
  #-------GTM Slave---------
  gtmSlave=y
  gtmSlaveName=gtm_slave
  gtmSlaveServer=pg02 
  gtmSlavePort=20001
  gtmSlaveDir=$pgxcInstallDir/pgxl/nodes/gtm_slave
  gtmSlaveSpecificExtraConfig=none
  #-------GTM Proxy--------
  gtmProxyDir=$pgxcInstallDir/pgxl/nodes/gtm_pxy
  gtmProxy=y 
  gtmProxyNames=(gtm_pxy1 gtm_pxy2)
  gtmProxyServers=(pg01 pg02)
  gtmProxyPorts=(20002 20002)
  gtmProxyDirs=($gtmProxyDir $gtmProxyDir)
  gtmPxyExtraConfig=none
  gtmPxySpecificExtraConfig=(none none)
  #-----Coordinators Master----------
  coordMasterDir=$pgxcInstallDir/pgxl/nodes/coord
  coordSlaveDir=$pgxcInstallDir/pgxl/nodes/coord_slave
  coordArchLogDir=$pgxcInstallDir/pgxl/nodes/coord_archlog
  coordNames=(coord1 coord2)
  coordPorts=(5433 5433)
  poolerPorts=(5434 5434)
  coordPgHbaEntries=(0.0.0.0/0)
  coordMasterServers=(pg01 pg02)
  coordMasterDirs=($coordMasterDir $coordMasterDir)
  coordMaxWALsernder=5
  coordMaxWALSenders=($coordMaxWALsernder $coordMaxWALsernder)
  #-----Coordinators Slave----------
  coordSlave=y
  coordSlaveSync=y
  coordSlaveServers=(pg03 pg04)
  coordSlavePorts=(5433 5433)
  coordSlavePoolerPorts=(5434 5434)
  coordSlaveDirs=($coordSlaveDir $coordSlaveDir)
  coordArchLogDirs=($coordArchLogDir $coordArchLogDir)
  coordExtraConfig=coordExtraConfig
  cat > $coordExtraConfig <<EOF
  log_destination = 'stderr'
  logging_collector = on
  log_directory = 'pg_log'
  listen_addresses = '*'
  max_connections = 1000
  EOF
  coordSpecificExtraConfig=(none none)
  coordExtraPgHba=none
  coordSpecificExtraPgHba=(none none)
  #------Datanodes Master----------
  datanodeMasterDir=$pgxcInstallDir/pgxl/nodes/dnmaster
  datanodeSlaveDir=$pgxcInstallDir/pgxl/nodes/dnslave
  datanodeArchLogDir=$pgxcInstallDir/pgxl/nodes/dn_archlog
  primaryDatanode=pg03
  datanodeNames=(datanode1 datanode2)
  datanodePorts=(5436 5436)
  datanodePoolerPorts=(5437 5437)
  datanodePgHbaEntries=(0.0.0.0/0)
  datanodeMasterServers=(pg03 pg04)
  datanodeMasterDirs=($datanodeMasterDir $datanodeMasterDir)
  datanodeMaxWalSender=0 
  datanodeMaxWALSenders=($datanodeMaxWalSender $datanodeMaxWalSender)
  datanodeSlave=n

  2.2 Posgres-XL 集群初始化

  初始化完成后，会自动启动集群。

[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf init all
/bin/bash
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Reading configuration using /u01/pgxl/pgxc_ctl_bash --home /u01/pgxl --configuration /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************

Current directory: /u01/pgxl
Initialize GTM master
The files belonging to this GTM system will be owned by user "postgres".
This user must also own the server process.

fixing permissions on existing directory /u01/pgxl/nodes/gtm ... ok
creating configuration files ... ok
creating control file ... ok

Success.
Done.
Start GTM master
server starting
Initialize GTM slave
The files belonging to this GTM system will be owned by user "postgres".
This user must also own the server process.

fixing permissions on existing directory /u01/pgxl/nodes/gtm_slave ... ok
creating configuration files ... ok
creating control file ... ok

Success.
Done.
Start GTM slavepgxc_ctl(13032):1811220924_48 server starting
Done.
Initialize all the gtm proxies.
Initializing gtm proxy gtm_pxy1.
Initializing gtm proxy gtm_pxy2.
The files belonging to this GTM system will be owned by user "postgres".
This user must also own the server process.

fixing permissions on existing directory /u01/pgxl/nodes/gtm_pxy ... ok
creating configuration files ... ok

Success.
The files belonging to this GTM system will be owned by user "postgres".
This user must also own the server process.

fixing permissions on existing directory /u01/pgxl/nodes/gtm_pxy ... ok
creating configuration files ... ok

Success.
Done.
Starting all the gtm proxies.
Starting gtm proxy gtm_pxy1.
Starting gtm proxy gtm_pxy2.
server starting
server starting
Done.
Initialize all the coordinator masters.
Initialize coordinator master coord1.
Initialize coordinator master coord2.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /u01/pgxl/nodes/coord ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /u01/pgxl/nodes/coord ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
Done.
Starting coordinator master.
Starting coordinator master coord1
Starting coordinator master coord2
2018-11-22 09:24:59.252 CST [13950] LOG:  listening on IPv4 address "0.0.0.0", port 5433
2018-11-22 09:24:59.252 CST [13950] LOG:  listening on IPv6 address "::", port 5433
2018-11-22 09:24:59.254 CST [13950] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5433"
2018-11-22 09:24:59.302 CST [13950] LOG:  redirecting log output to logging collector process
2018-11-22 09:24:59.302 CST [13950] HINT:  Future log output will appear in directory "pg_log".
2018-11-22 09:24:59.060 CST [13633] LOG:  listening on IPv4 address "0.0.0.0", port 5433
2018-11-22 09:24:59.061 CST [13633] LOG:  listening on IPv6 address "::", port 5433
2018-11-22 09:24:59.062 CST [13633] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5433"
2018-11-22 09:24:59.109 CST [13633] LOG:  redirecting log output to logging collector process
2018-11-22 09:24:59.109 CST [13633] HINT:  Future log output will appear in directory "pg_log".
Done.
Initialize all the coordinator slaves.
Initialize the coordinator slave coord1.
Initialize the coordinator slave coord2.
Done.
Starting all the coordinator slaves.
Starting coordinator slave coord1.
Starting coordinator slave coord2.
2018-11-22 09:25:05.987 CST [13330] LOG:  listening on IPv4 address "0.0.0.0", port 5433
2018-11-22 09:25:05.987 CST [13330] LOG:  listening on IPv6 address "::", port 5433
2018-11-22 09:25:05.989 CST [13330] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5433"
2018-11-22 09:25:06.039 CST [13330] LOG:  redirecting log output to logging collector process
2018-11-22 09:25:06.039 CST [13330] HINT:  Future log output will appear in directory "pg_log".
2018-11-22 09:25:05.517 CST [13266] LOG:  listening on IPv4 address "0.0.0.0", port 5433
2018-11-22 09:25:05.517 CST [13266] LOG:  listening on IPv6 address "::", port 5433
2018-11-22 09:25:05.518 CST [13266] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5433"
2018-11-22 09:25:05.567 CST [13266] LOG:  redirecting log output to logging collector process
2018-11-22 09:25:05.567 CST [13266] HINT:  Future log output will appear in directory "pg_log".
Done
Initialize all the datanode masters.
Initialize the datanode master datanode1.
Initialize the datanode master datanode2.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /u01/pgxl/nodes/dnmaster ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /u01/pgxl/nodes/dnmaster ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
Done.
Starting all the datanode masters.
Starting datanode master datanode1.
Starting datanode master datanode2.
2018-11-22 09:25:15.129 CST [13604] LOG:  listening on IPv4 address "0.0.0.0", port 5436
2018-11-22 09:25:15.130 CST [13604] LOG:  listening on IPv6 address "::", port 5436
2018-11-22 09:25:15.131 CST [13604] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5436"
2018-11-22 09:25:15.148 CST [13604] LOG:  redirecting log output to logging collector process
2018-11-22 09:25:15.148 CST [13604] HINT:  Future log output will appear in directory "pg_log".
2018-11-22 09:25:14.657 CST [13539] LOG:  listening on IPv4 address "0.0.0.0", port 5436
2018-11-22 09:25:14.657 CST [13539] LOG:  listening on IPv6 address "::", port 5436
2018-11-22 09:25:14.659 CST [13539] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5436"
2018-11-22 09:25:14.676 CST [13539] LOG:  redirecting log output to logging collector process
2018-11-22 09:25:14.676 CST [13539] HINT:  Future log output will appear in directory "pg_log".
Done.
ALTER NODE coord1 WITH (HOST='pg01', PORT=5433);
ALTER NODE
CREATE NODE coord2 WITH (TYPE='coordinator', HOST='pg02', PORT=5433);
CREATE NODE
CREATE NODE datanode1 WITH (TYPE='datanode', HOST='pg03', PORT=5436);
CREATE NODE
CREATE NODE datanode2 WITH (TYPE='datanode', HOST='pg04', PORT=5436);
CREATE NODE
SELECT pgxc_pool_reload();
 pgxc_pool_reload 
------------------
 t
(1 row)

CREATE NODE coord1 WITH (TYPE='coordinator', HOST='pg01', PORT=5433);
CREATE NODE
ALTER NODE coord2 WITH (HOST='pg02', PORT=5433);
ALTER NODE
CREATE NODE datanode1 WITH (TYPE='datanode', HOST='pg03', PORT=5436);
CREATE NODE
CREATE NODE datanode2 WITH (TYPE='datanode', HOST='pg04', PORT=5436);
CREATE NODE
SELECT pgxc_pool_reload();
 pgxc_pool_reload 
------------------
 t
(1 row)

Done.
EXECUTE DIRECT ON (datanode1) 'CREATE NODE coord1 WITH (TYPE=''coordinator'', HOST=''pg01'', PORT=5433)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode1) 'CREATE NODE coord2 WITH (TYPE=''coordinator'', HOST=''pg02'', PORT=5433)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode1) 'ALTER NODE datanode1 WITH (TYPE=''datanode'', HOST=''pg03'', PORT=5436)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode1) 'CREATE NODE datanode2 WITH (TYPE=''datanode'', HOST=''pg04'', PORT=5436)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode1) 'SELECT pgxc_pool_reload()';
 pgxc_pool_reload 
------------------
 t
(1 row)

EXECUTE DIRECT ON (datanode2) 'CREATE NODE coord1 WITH (TYPE=''coordinator'', HOST=''pg01'', PORT=5433)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode2) 'CREATE NODE coord2 WITH (TYPE=''coordinator'', HOST=''pg02'', PORT=5433)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode2) 'CREATE NODE datanode1 WITH (TYPE=''datanode'', HOST=''pg03'', PORT=5436)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode2) 'ALTER NODE datanode2 WITH (TYPE=''datanode'', HOST=''pg04'', PORT=5436)';
EXECUTE DIRECT
EXECUTE DIRECT ON (datanode2) 'SELECT pgxc_pool_reload()';
 pgxc_pool_reload 
------------------
 t
(1 row)

Done.

通过查询 pgxc_node 表可以获取集群节点信息：

2.3 Posgres-XL 集群启动与关闭

[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf stop all
[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf start all

2.4 Posgres-XL 集群服务监控

[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf monitor all
/bin/bash
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Reading configuration using /u01/pgxl/pgxc_ctl_bash --home /u01/pgxl --configuration /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /u01/pgxl
Running: gtm master
Running: gtm slave
Running: gtm proxy gtm_pxy1
Running: gtm proxy gtm_pxy2
Running: coordinator master coord1
Running: coordinator slave coord1
Running: coordinator master coord2
Running: coordinator slave coord2
Running: datanode master datanode1
Running: datanode slave datanode1
Running: datanode master datanode2
Running: datanode slave datanode2

3、测试 Posgres-XL 集群

3.1 创建测试表并插入数据

创建一张 log 表，然后插入 500 条数据，如下：

[postgres@pg01 ~]$ psql -p5433
psql (PGXL 10r1, based on PG 10.5 (Postgres-XL 10r1))
Type "help" for help.
postgres=# CREATE TABLE log (id numeric NOT NULL,stamp timestamp with time zone,user_id numeric);
postgres=# copy log from '/u02/tmp/log.csv' with csv;
COPY 499

3.2 查看表数据分布信息

可以在协调器节点或者各个数据节点查看，这里使用下面的语句直接统计每个数据节点的表数据分布情况：

postgres=# SELECT xc_node_id, count(*) FROM log GROUP BY xc_node_id;

3.3 建表说明

• distribute 表
  默认情况下，系统会将插入的数据，按照拆分规则，分配到不同的 datanode 节点中存储，也就是 sharding 技术。每个 datanode 节点只保存了部分数据，通过 coordinate 节点可以查询完整的数据视图。上面创建的 log 表就是 distribute 表。建表语法如下：

  postgres=# CREATE TABLE log (id numeric NOT NULL,stamp timestamp with time zone,user_id numeric) distribute by hash(id);

  distribute 表数据分布如下图：
  

• replication 表
  各个 datanode 节点中，表的数据完全相同。也就是说，在插入数据时，系统会分别在每个 datanode 节点插入相同数据。读数据时，只需要读任意一个 datanode 节点上的数据即可。建表语法如下：

  postgres=# CREATE TABLE log2 (id numeric NOT NULL,stamp timestamp with time zone,user_id numeric) distribute by replication;
  postgres=# copy log2 from '/u02/tmp/log.csv' with csv;

  replication 表数据分布如下图：
  
  不论在哪个数据节点查询，显示的结果都一样的。

  4、Postgres-XL Slave 节点管理

  为了提高集群可用性，以下配置数据节点的热备以便进行故障转移切换。

  4.1 新增 Slave 数据节点

  在 gtm 节点操作，新增两个 slave 数据节点，如下：

[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
/bin/bash
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Reading configuration using /u01/pgxl/pgxc_ctl_bash --home /u01/pgxl --configuration /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /u01/pgxl
PGXC add datanode slave datanode1 pg05 5436 5437 /u01/pgxl/nodes/dnslave /u01/pgxl/nodes/dn_slave_war /u01/pgxl/nodes/dn_archlog
PGXC add datanode slave datanode2 pg06 5436 5437 /u01/pgxl/nodes/dnslave /u01/pgxl/nodes/dn_slave_war /u01/pgxl/nodes/dn_archlog

4.2 主备节点切换

检查所有服务运行正常:

[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf monitor all
/bin/bash
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Reading configuration using /u01/pgxl/pgxc_ctl_bash --home /u01/pgxl --configuration /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /u01/pgxl
Running: gtm master
Running: gtm slave
Running: gtm proxy gtm_pxy1
Running: gtm proxy gtm_pxy2
Running: coordinator master coord1
Running: coordinator slave coord1
Running: coordinator master coord2
Running: coordinator slave coord2
Running: datanode master datanode1
Running: datanode slave datanode1
Running: datanode master datanode2
Running: datanode slave datanode2

当前的主节点是 pg03 和 pg04，如下图：

切换主备节点：

[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf monitor all
/bin/bash
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Reading configuration using /u01/pgxl/pgxc_ctl_bash --home /u01/pgxl --configuration /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /u01/pgxl
Running: gtm master
Running: gtm slave
Running: gtm proxy gtm_pxy1
Running: gtm proxy gtm_pxy2
Running: coordinator master coord1
Running: coordinator slave coord1
Running: coordinator master coord2
Running: coordinator slave coord2
Running: datanode master datanode1
Running: datanode slave datanode1
Running: datanode master datanode2
Running: datanode slave datanode2
[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf failover datanode datanode1 datanode2
/bin/bash
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Reading configuration using /u01/pgxl/pgxc_ctl_bash --home /u01/pgxl --configuration /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /u01/pgxl
Failover specified datanodes.
Failover the datanode datanode1.
Failover datanode datanode1 using GTM itself
Actual Command: ssh postgres@pg05 "(pg_ctl promote -Z datanode -D /u01/pgxl/nodes/dnslave) > /tmp/pg01_STDOUT_4264_0 2>&1" < /dev/null > /dev/null 2>&1
Bring remote stdout: scp postgres@pg05:/tmp/pg01_STDOUT_4264_0 /tmp/STDOUT_4264_1 > /dev/null 2>&1
Actual Command: ssh postgres@pg05 "(pg_ctl restart -w -Z datanode -D /u01/pgxl/nodes/dnslave -o -i; sleep 1) > /tmp/pg01_STDOUT_4264_2 2>&1" < /dev/null > /dev/null 2>&1
Bring remote stdout: scp postgres@pg05:/tmp/pg01_STDOUT_4264_2 /tmp/STDOUT_4264_3 > /dev/null 2>&1
2018-11-23 14:52:34.655 CST [26323] LOG:  listening on IPv4 address "0.0.0.0", port 5436
2018-11-23 14:52:34.655 CST [26323] LOG:  listening on IPv6 address "::", port 5436
2018-11-23 14:52:34.657 CST [26323] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5436"
2018-11-23 14:52:34.675 CST [26323] LOG:  redirecting log output to logging collector process
2018-11-23 14:52:34.675 CST [26323] HINT:  Future log output will appear in directory "pg_log".
ALTER NODE
 pgxc_pool_reload 
------------------
 t
(1 row)

EXECUTE DIRECT
 pgxc_pool_reload 
------------------
 t
(1 row)

EXECUTE DIRECT
 pgxc_pool_reload 
------------------
 t
(1 row)

ALTER NODE
 pgxc_pool_reload 
------------------
 t
(1 row)

Failover the datanode datanode2.
Failover datanode datanode2 using GTM itself
Actual Command: ssh postgres@pg06 "(pg_ctl promote -Z datanode -D /u01/pgxl/nodes/dnslave) > /tmp/pg01_STDOUT_4264_4 2>&1" < /dev/null > /dev/null 2>&1
Bring remote stdout: scp postgres@pg06:/tmp/pg01_STDOUT_4264_4 /tmp/STDOUT_4264_5 > /dev/null 2>&1
Actual Command: ssh postgres@pg06 "(pg_ctl restart -w -Z datanode -D /u01/pgxl/nodes/dnslave -o -i; sleep 1) > /tmp/pg01_STDOUT_4264_6 2>&1" < /dev/null > /dev/null 2>&1
Bring remote stdout: scp postgres@pg06:/tmp/pg01_STDOUT_4264_6 /tmp/STDOUT_4264_7 > /dev/null 2>&1
2018-11-23 14:52:38.607 CST [26317] LOG:  listening on IPv4 address "0.0.0.0", port 5436
2018-11-23 14:52:38.607 CST [26317] LOG:  listening on IPv6 address "::", port 5436
2018-11-23 14:52:38.609 CST [26317] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5436"
2018-11-23 14:52:38.628 CST [26317] LOG:  redirecting log output to logging collector process
2018-11-23 14:52:38.628 CST [26317] HINT:  Future log output will appear in directory "pg_log".
ALTER NODE
 pgxc_pool_reload 
------------------
 t
(1 row)

EXECUTE DIRECT
 pgxc_pool_reload 
------------------
 t
(1 row)

EXECUTE DIRECT
 pgxc_pool_reload 
------------------
 t
(1 row)

ALTER NODE
 pgxc_pool_reload 
------------------
 t
(1 row)

Done.
[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf monitor all
/bin/bash
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /u01/pgxl/pgxc_ctl_bash.
Reading configuration using /u01/pgxl/pgxc_ctl_bash --home /u01/pgxl --configuration /u01/pgxl/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /u01/pgxl
Running: gtm master
Running: gtm slave
Running: gtm proxy gtm_pxy1
Running: gtm proxy gtm_pxy2
Running: coordinator master coord1
Running: coordinator slave coord1
Running: coordinator master coord2
Running: coordinator slave coord2
Running: datanode master datanode1
Running: datanode master datanode2

切换完成后，旧的 master 节点作废，而新建的两个 slave 节点就转变为 master 节点角色。

4.3 删除 Slave 节点

[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf remove datanode slave datanode1 clean
[postgres@pg01 ~]$ pgxc_ctl -c /u01/pgxl/pgxc_ctl/pgxc_ctl.conf remove datanode slave datanode2 clean