[Bucardo-general] BUG?

Michelle Sullivan michelle at sorbs.net
Tue Nov 19 08:26:17 UTC 2013


Ok put this one in the IRC channel, but don't know if anyone saw it so
will repeat it here:

I have found what I consider to be a bug (happy to get feedback on
whether it is or not).

So my Setup:

4 database servers, 2 east coast USA, 2 west coast USA, 67(ish) tables,
800GB data (currently), 128GB RAM, 4Tbytes storage.. Multimastered using
Bucardo 4.99.10.

Sync setup to replicate them all, all working fine (mostly, occasional
errors that causes a restart, but all in sync)

9 other DB servers around the world, same DB as the 4 masters, however
only 5 tables replicated to them from the masters as they only have
4-16G RAM and 50-800GB Disk storage... Slaves only (they only get writes
from the masters, nothing writes to them locally.)

Now obviously as the 9 remote slaves are only hosting 5 of the tables a
separate sync is required, so we have 2 syncs:

sorbsmm (the multimaster) with master-at1:source, master-at2:source,
master-sc1:source, master-sc2:source
sorbsdns (the slaves) with master-at2:source, slave-se:target,
slave-de:target, slave-da:target slave-au:target, slave-uk:target,
slave-ie:target, slave-fi:target, slave-nl:target ...etc

All writes are currently made to master-a1, make-delta is on.

It doesn't work... nothing is replicated to the sorbsdns sync (this has
been running for 2 months now)

Manually adding master-at1:source master-sc1:source master-sc2 to the
DBmap table for the 'sorbsslaves' dbgroup and reloading the sync, and
suddenly there are 400,000+ rows to replicate

It doesn't work... This fails with continual serialisation errors (the
rows are being replicated twice on the 5 tables as they are effectively
masters in both sorbsmm and sorbsrbls) - fortunately for me sorbsmm
continues and the fail is the sorbsrbls sync.

Manually removing master-at2:source master-sc1:source master-sc2 to the
DBmap table for the 'sorbsslaves' dbgroup and reloading the sync, and
replication works.

Seems sorted?  Yes, but also no..  The point is to make redundancy and
eliminate single points of failure for the system... I can (in theory -
haven't managed to get it working properly/how I want yet) use pgpool to
write to any master and have the changes replicated to the remote
slaves.  I do not want to separate the tables  out from the master sync
as there are referral integrity checks and the slaves are slower than
the masters so the master replication would fail.

"HELP"..! ;-)

My thought is is there should be something other than 'source' and
'target' like a 'sourceandtarget' - for multimaster hosts (or
'masteronly' - for a host that gets writes but does not receive
replication updates) as the rows are already on master-at1, master-at2,
master-sc1 and master-sc2 so the sorbsrbls tables are not replicated
with 'source' but more seen as 'changes can happen on any of these
servers and need to be replicated to the targets, but other sources are
not targets so only write to the targets'

Thoughts?  Solutions? (if there is an answer that I have missed in how
to configure bucardo.)

Thanks,

-- 
Michelle Sullivan
http://www.mhix.org/



More information about the Bucardo-general mailing list