[Bucardo-general] Pushing bucardo's limits

[Ben Chobot]

Hi there. I'm struggling to use bucardo to replicate large, active tables. "Large" and "active" are relative terms, so think "450GB without indices" and "in the 12 hours it takes to for the initial copy, we have accumulated ~10 million deltas in bucardo's log." This makes for very long transactions, and very sad validation attempts that have yet to validate faster than new changes come in.

I'm wondering if anybody else has dealt with bucardo on this scale, and if so, if they have any hints on how to get things started? I know bucardo can keep up with the steady state demand. It's just that initial deluge of bytes that seem to be putting it into a death spiral.

I don't know if it makes a difference but I'm trying to use multiple syncs in parallel to try to speed up the entire load - but because of FK constraints, about 2/3 of them are all in a single sync. Most tables aren't as painful as these large ones, but those.... I haven't figured out a way around it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://bucardo.org/pipermail/bucardo-general/attachments/20260121/00d0f107/attachment.htm>