Bucardo 是基于表复制的系统
Bucardo 可以实现PostgreSQL数据库的双master/多master的方案
Bucardo的核心是一个Perl守护进程，它侦听通知请求并对其进行操作，方法是连接到远程数据库并来回复制数据。
守护进程需要的所有特定信息都存储在主bucardo数据库中，包括复制所涉及的所有数据库的列表以及如何到达这些数据库、要复制的所有表以及如何复制每个表。
运行Bucardo的第一步是向主Bucardo数据库添加两个或更多数据库。
完成此操作后，将添加关于要复制哪些表的信息以及表的任何分组。然后添加同步。
同步被称为复制操作，将一组特定的表从一台服务器复制到另一台服务器或一组服务器。
一旦设置好Bucardo，触发器就开始存储有关所有相关的表中哪些行被更改的信息。对于多主机。
过程是这样的：1、对表进行更改并记录在bucardo_delta表中。2、向主Bucardo守护进程发送通知，让它知道表已经更改。3、守护进程通知控制器进行同步，然后返回侦听。4、控制器创建一个“kid”来处理复制，或者通知已经存在的复制。5、孩子开始一个新的事务，禁用相关表上的触发器和规则。6、然后，它收集自上次复制以来哪些行发生了更改的列表，然后比较这两个行以确定应该做什么。7、如果存在冲突，则会运行标准冲突处理程序，或者为每个表设置的自定义处理程序来对内容进行排序。8、触发器和规则被重新启用，事务被提交。9、如果事务失败，则运行任何自定义异常处理程序。10、子程序向控制器发出它已经完成的信号。
|
|
Bucardo 软件依赖包有如下：
build, test, and install Perl 5 (at least 5.8.3)build, test, and install Postgres (at least 8.2)build, test, and install the DBI module (at least 1.51)build, test, and install the DBD::Pg module (at least 2.0.0)build, test, and install the DBIx::Safe module (at least 1.2.4)
本次系统linux版本是6.5 Perl是linux系统自带；默认版本是v5.10.1；可行
[root@node3 Bucardo]# perl -vThis is perl, v5.10.1 (*) built for x86_64-linux-thread-multi
在编译的过程需要带上 --with-perl;
安装DBI 依赖 Test-Simple；
# 安装 Test-SimpleTest-Simple下载地址 https://metacpan.org 搜索 Test-Simple# 解压Test-Simple源码包tar -zxvf Test-Simple-1.302183.tar.gz# 进入Test-Simple目录进行安装cd Test-Simple-1.302183perl Makefile.PLmakemake install# 接下来安装DBI# 下载地址：http://search.cpan.org/CPAN/authors/id/T/TI/TIMB/DBI-1.630.tar.gztar -zxvf DBI-1.630.tar.gzcd DBI-1.630perl Makefile.PLmakemake install
安装最新的DBD::Pg；需要依赖 Time::HiRes
# 安装 Time::HiRes下载地址： https://metacpan.org 搜索 Time::HiRestar -zxvf Time-HiRes-1.9764.tar.gz cd Time-HiRes-1.9764perl Makefile.PLmakemake install# 接下来安装DBD::Pg# 安装DBD::Pg；需要确定已经安装 PostgreSQL 环境；即pg_config输出# 加载 PostgreSQL 环境变量tar -zxvf DBD-Pg-3.14.2.tar.gzcd DBD-Pg-3.14.2perl Makefile.PLmakemake install
# 下载地址：https://bucardo.org/DBIx-Safe/tar -zxvf DBIx-Safe-1.2.5.tar.gzcd DBIx-Safe-1.2.5perl Makefile.PLmakemake install
|
# 在网站https://bucardo.org/Bucardo/下载tar -zxvf Bucardo-5.6.0.tar.gzcd Bucardo-5.6.0perl Makefile.PLmakemake install# 查看Bucardo版本[postgres@node3 ~]$ bucardo --versionbucardo version 5.6.0
|
元数据库存储bucardo复制的配置信息,使用 bucardo install 命令创建；
|
在系统用户home目录下创建；内容如下：
log_conflict_file = /home/postgres/bucardo/log/bucardo_conflict.logpiddir = /home/postgres/bucardo/runreason_file = /home/postgres/bucardo/log/bucardo.restart.reason.logwarning_file = /home/postgres/bucardo/log/bucardo.warning.logsyslog_facility = LOG_LOCAL1
|
mkdir -p /home/postgres/bucardo/logmkdir -p /home/postgres/bucardo/run
|
准备工作完成；开始执行 bucardo install 命令创建 元数据库;
# /usr/local/share/bucardo/bucardo.schema[postgres@node3 ~]$ bucardo installThis will install the bucardo database into an existing Postgres cluster.Postgres must have been compiled with Perl support,and you must connect as a superuserCurrent connection settings:1. Host: <none>2. Port: 60003. User: bucardo4. Database: bucardo5. PID directory: /home/postgres/bucardo/runEnter a number to change it, P to proceed, or Q to quit: 1-- 推荐使用unix套接字方式连接数据;可以使用dbuser:postgres;dbname:postgres去连接。Change the host to: /tmpChanged host to: /tmpCurrent connection settings:1. Host: /tmp2. Port: 60003. User: postgres4. Database: postgres5. PID directory: /home/postgres/bucardo/runEnter a number to change it, P to proceed, or Q to quit: PAttempting to create and populate the bucardo database and schemaDatabase creation is completeUpdated configuration setting "piddir"Installation is now complete.If you see errors or need help, please email bucardo-general@bucardo.orgYou may want to check over the configuration variables next, by running:bucardo show allChange any setting by using: bucardo set foo=bar
|
本次简单演示下；在同个数据库服务器下不同的实例；
用数据库master、slave来演示
[postgres@node3 ~]$ createdb master[postgres@node3 ~]$ createdb slave
|
执行bucardo add db命令；记录同步数据库的连接信息
[postgres@node3 ~]$ bucardo add db db1 host=/tmp dbname=master user=postgresAdded database "db1"[postgres@node3 ~]$ bucardo add db db2 host=/tmp dbname=slave user=postgresAdded database "db2"#这里推荐使用unix套接字；若元数据库跟需要复制的数据不是在同一台;则需要使用TCP/IP连接的方式。bucardo add db db1 host=IP port=PGPORT dbname=PGDB user=PGUSRR password=PGPASSWD
|
添加 dbgroup 数据库组； 在一套复制系统中可以有多个源数据库、目标数据库。指定哪个数据库是源数据库，哪个是目标数据库
[postgres@node3 ~]$ bucardo add dbgroup dbgrp01 db1:source db2:targetCreated dbgroup "dbgrp01"Added database "db1" to dbgroup "dbgrp01" as sourceAdded database "db2" to dbgroup "dbgrp01" as target
|
# 创建复制表tbl_lottu01[postgres@node3 ~]$ psql master postgres -c "create table tbl_lottu01(id int primary key, info text, reg_time timestamp)"CREATE TABLE[postgres@node3 ~]$ psql slave postgres -c "create table tbl_lottu01(id int primary key, info text)"CREATE TABLE# 添加 relgroup 复制集；即要复制的表、序列的集合[postgres@node3 ~]$ bucardo add relgroup relgrp01 tbl_lottu01Created relgroup "relgrp01"Added the following tables or sequences: public.tbl_lottu01 (DB: db1)The following tables or sequences are now part of the relgroup "relgrp01": public.tbl_lottu01
|
添加同步；在创建同步需要指定dbgroup 和relgroup
[postgres@node3 ~]$ bucardo add sync sync01 relgroup=relgrp01 dbgroup=dbgrp01 conflict_strategy=bucardo_latest2020-11-03 17:08:51.494 CST [6988] LOG: Starting validate_sync for sync012020-11-03 17:08:51.494 CST [6988] CONTEXT: PL/Perl function "validate_sync"SQL statement "SELECT validate_sync('sync01')"PL/Perl function "validate_sync"2020-11-03 17:08:51.494 CST [6988] STATEMENT: INSERT INTO bucardo.sync (herd,name,dbs,conflict_strategy) VALUES ($1,$2,$3,$4)2020-11-03 17:08:51.619 CST [6988] LOG: Ending validate_sync for sync012020-11-03 17:08:51.619 CST [6988] CONTEXT: PL/Perl function "validate_sync"SQL statement "SELECT validate_sync('sync01')"PL/Perl function "validate_sync"2020-11-03 17:08:51.619 CST [6988] STATEMENT: INSERT INTO bucardo.sync (herd,name,dbs,conflict_strategy) VALUES ($1,$2,$3,$4)Added sync "sync01"由于bucardo是解决PostgreSQL双主/多主的方案。可能会存在主键/唯一约束的冲突
"conflict_strategy" The conflict resolution strategy to use in the sync. Supported values: "bucardo_source" The rows on the "source" database always "win". In other words, in a conflict, Bucardo copies rows from source to target. "bucardo_target" The rows on the "target" database always win. "bucardo_skip" Any conflicting rows are simply not replicated. Not recommended for most cases. "bucardo_random" Each database has an equal chance of winning each time. This is the default. "bucardo_latest" The row that was most recently changed wins. "bucardo_abort" The sync is aborted on a conflict.
|
启动bucardo服务
[postgres@node3 ~]$ bucardo startChecking for existing processesStarting Bucardo[postgres@node3 ~]$ 2020-11-03 17:21:07.440 CST [7178] LOG: Starting validate_sync for sync012020-11-03 17:21:07.440 CST [7178] CONTEXT: PL/Perl function "validate_sync"2020-11-03 17:21:07.440 CST [7178] STATEMENT: SELECT validate_sync('sync01')2020-11-03 17:21:07.494 CST [7178] LOG: Ending validate_sync for sync012020-11-03 17:21:07.494 CST [7178] CONTEXT: PL/Perl function "validate_sync"2020-11-03 17:21:07.494 CST [7178] STATEMENT: SELECT validate_sync('sync01')[postgres@node3 ~]$ ps -ef | grep bucardopostgres 7178 6376 0 17:21 ? 00:00:00 postgres: bucardo bucardo [local] idlepostgres 7184 6376 0 17:21 ? 00:00:00 postgres: bucardo bucardo [local] idlepostgres 7187 6376 0 17:21 ? 00:00:00 postgres: bucardo bucardo [local] idlepostgres 7191 6376 0 17:21 ? 00:00:00 postgres: bucardo bucardo [local] idlepostgres 7196 7066 0 17:21 pts/1 00:00:00 grep bucardo在源库插入记录
[postgres@node3 ~]$ psql master postgrespsql (12.2)Type "help" for help.master=# insert into tbl_lottu01 values (1, 'lottu01');INSERT 0 1 master=# insert into tbl_lottu01 values (2, 'lottu02');INSERT 0 1master=# \q在目标库查看记录[postgres@node3 ~]$ psql slave postgrespsql (12.2)Type "help" for help.slave=# select * from tbl_lottu01; id | info ----+--------- 1 | lottu01 2 | lottu02(2 rows)
|
|
Bucardo 可以实现PostgreSQL数据库的双master/多master的方案；
其实逻辑复制可以设计双主架构；在原生逻辑流复制针对冲突检查需要手动处理；但是在pglogical有对冲突处理。bucardo也有冲突处理。
以上面为例
# 添加数据库组；指定db2是源数据库，db1是目标数据库bucardo add dbgroup dbgrp02 db2:source db1:target# 添加同步bucardo add sync sync02 relgroup=relgrp01 dbgroup=dbgrp02 conflict_strategy=bucardo_latest这样可以完成双主架构master=# insert into tbl_lottu01 values (3, 'lottu03');INSERT 0 1slave=# insert into tbl_lottu01 values (4, 'lottu04');INSERT 0 1查看master/slave结果是=# select * from tbl_lottu01; id | info ----+--------- 1 | lottu01 2 | lottu02 4 | lottu04 3 | lottu03(4 rows)
|
在生产环境数据库；都应有测试环境；有时需要在测试环境需要最新的数据验证新加的代码正确性。
# 关闭bucardo服务bucardo stop# 更新同步为增量同步；bucardo update sync sync01 onetimecopy=2 "onetimecopy" 0: 关闭 1: fullcopy；采用delete/copy的方式 2: 增量copy；# 启动bucardo服务bucardo start
联系客服
