[Bucardo-general] Migrate Data Between RDS Postgres Clusters using Bucardo

[Su Yu]

Hi Bucardo community,

We are new to Bucardo and we are trying to migrate data from an old active Postgres AWS RDS cluster to a brand-new cluster using Bucardo. And I have some questions here.

Background about migration plan:
1. Take a snapshot of an active old cluster (source cluster).
2. Create a new cluster (target cluster) using the snapshot from step 1 (so that I don't need to copy schema individually).
3. Create a Bucardo server and set up sync between the old and new cluster with one time copy first then sync delta only.
4. After the new cluster is in sync with the old one, switch application point to the new cluster.

Question 1: How should I set up the onetimecopy property?
So please correct me if I'm wrong:
I think I should set it to onetimecopy=1 when I add the sync. It will automatically delete all data from the new cluster table first and do a full load from the source cluster, then automatically change to the delta mode, am I right?
And during the full load, all changes happened on source cluster will be buffered and will be applied to the target cluster after the completion of full load, am I right?

Question 2: Table has "auto increment" primary key.
We have some use cases using sequence nextval to generate incremental numbers as the primary keys for the table. And I have a concern about key collisions during migration. Because when we switch the application endpoint, we will have moments that application pods point to both old and new clusters. That might have a chance to create primary key collisions. Is this concern valid?

Question 3: What is the best practice of using a herd?
If I have 5 dbs in the source cluster, and each db has 10 tables, should I create 5 herds to sync 5 dbs concurrently? or create 1 herd to sync the 5 dbs sequentially? What is the best practice here?

Question 4: Any issue could be caused by Bucardo if the source cluster has a lot of traffic?
Is it possible that after I set the sync, and start bucardo, somehow it will crash the source DB? maybe somehow lock the table? Is that concern valid?

Please feel free to let us know if you need more details.


Thank you,

Su
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://bucardo.org/pipermail/bucardo-general/attachments/20220120/2ac2f475/attachment.htm>