[Bucardo-general] Bucardo-general Digest, Vol 133, Issue 1
Om Prakash Jaiswal
op12om at yahoo.co.in
Tue Feb 12 14:23:11 UTC 2019
Bucardo is creating load on database. I would suggest use other solutions to send/receive data between two nodes like Kafka or rabbitmq. It will not create load on db. It is running smoothly for me.
RegardsOm PrakashBangalore
Sent from Yahoo Mail on Android
On Tue, Feb 12, 2019 at 4:11, bucardo-general-request at bucardo.org<bucardo-general-request at bucardo.org> wrote: Send Bucardo-general mailing list submissions to
bucardo-general at bucardo.org
To subscribe or unsubscribe via the World Wide Web, visit
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
or, via email, send a message with subject or body 'help' to
bucardo-general-request at bucardo.org
You can reach the person managing the list at
bucardo-general-owner at bucardo.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Bucardo-general digest..."
Today's Topics:
1. bucardo VAC errors (Paul Theodoropoulos)
2. new Bucardo installation with a 1TB DB (Lucas Possamai)
3. Re: new Bucardo installation with a 1TB DB (David Christensen)
4. Re: new Bucardo installation with a 1TB DB (Lucas Possamai)
5. Re: new Bucardo installation with a 1TB DB (Lucas Possamai)
----------------------------------------------------------------------
Message: 1
Date: Mon, 4 Feb 2019 15:43:43 -0800
From: Paul Theodoropoulos <paul at anastrophe.com>
To: "bucardo-general at bucardo.org List" <bucardo-general at bucardo.org>
Subject: [Bucardo-general] bucardo VAC errors
Message-ID: <3e427bee-89fd-b023-0f34-702886110164 at anastrophe.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Was running Bucardo 5.4.1. Recently upgraded servers from Debian Jessie
to Debian Stretch. This upgraded postgresql from 9.4 to 9.6, and
upgraded perl from 5.20.1 to 5.24.1. I reinstalled all required perl
modules (including Bucardo).
Since then, I get errors such as these:
(27541) [Mon Feb? 4 15:37:53 2019] VAC Warning! VAC was killed at line
7868: DBD::Pg::st pg_result failed: ERROR:? relation
"bucardo.delta_pg_toast_pg_toast_21502_index" does not exist
LINE 1: DELETE FROM bucardo.delta_pg_toast_pg_toast_21502_index USIN...
??????????????????? ^
QUERY:? DELETE FROM bucardo.delta_pg_toast_pg_toast_21502_index USING
(SELECT txntime AS tt FROM bucardo.track_pg_toast_pg_toast_21502_index
GROUP BY 1 HAVING COUNT(*) = 1) AS foo WHERE txntime = tt AND txntime <
now() - interval '45 seconds'
CONTEXT:? PL/pgSQL function bucardo.bucardo_purge_delta_oid(text,oid)
line 49 at EXECUTE
SQL statement "SELECT bucardo.bucardo_purge_delta_oid($1, myrec.tablename)"
PL/pgSQL function bucardo_purge_delta(text) line 12 at SQL statement at
/usr/local/share/perl/5.24.1/Bucardo.pm line 7868.
Just to be thorough, I upgraded to Bucardo 5.5.0 (followed all upgrade
steps), but no joy, the errors remain.
Thoughts?
By the way, really like the improvements to 'bucardo status', with more
details of what bucardo is doing 'right now'.
--
Paul Theodoropoulos
www.anastrophe.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20190204/f546167b/attachment-0001.html>
------------------------------
Message: 2
Date: Wed, 6 Feb 2019 16:39:18 +1300
From: Lucas Possamai <drum.lucas at gmail.com>
To: bucardo-general at bucardo.org
Subject: [Bucardo-general] new Bucardo installation with a 1TB DB
Message-ID:
<CAE_gQfXNEV9_YDEznOFtf36GoPALb1PaANQh6=AOEGTruUYfQg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi.
We're moving our EC2 Postgres 1TB DB to RDS and decided to use Bucardo, as
it supports multiple PG versions.
We have a PG 9.2 cluster, and instead of upgrading it, we'll use Bucardo to
replicate to a PG 9.6, then use DMS or pg_dump to restore it in RDS.
Because of the size of the current 9.2 DB, I cannot stop the application so
it doesn't write to the DB while I do pg_dump and restore it in the new
bucardo slave.
So, I thought I would do something like this:
1. Install Bucardo and add large tables to a pushdelta sync
2. Copy the tables to the new server (e.g. with pg_dump)
3. Startup Bucardo and catch things up (e.g. copy all rows changes since
step 2)
Steps for the above would be pretty much like this article
<https://www.endpoint.com/blog/2009/09/16/migrating-postgres-with-bucardo-4>;
if I'm not mistaken.
My questions are:
1. Is it the right approach? do you guys have any other suggestions?
2. I'll have the following: pg-9.2 master --> bucardo instance (with the
bucardo DB only) --> pg-9.6 slave from bucardo
1. When doing the pg_dump, I only need to restore it on the "pg-9.6
slave from bucardo" instance, correct? the bucardo DB does not store the
data?
Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20190206/33ff1108/attachment-0001.html>
------------------------------
Message: 3
Date: Wed, 6 Feb 2019 08:09:48 -0600
From: David Christensen <david at endpoint.com>
To: Lucas Possamai <drum.lucas at gmail.com>
Cc: bucardo-general at bucardo.org
Subject: Re: [Bucardo-general] new Bucardo installation with a 1TB DB
Message-ID: <8A53628A-5187-41F4-848F-B1FA6CC91295 at endpoint.com>
Content-Type: text/plain; charset="utf-8"
> We're moving our EC2 Postgres 1TB DB to RDS and decided to use Bucardo, as it supports multiple PG versions.
> We have a PG 9.2 cluster, and instead of upgrading it, we'll use Bucardo to replicate to a PG 9.6, then use DMS or pg_dump to restore it in RDS.
>
> Because of the size of the current 9.2 DB, I cannot stop the application so it doesn't write to the DB while I do pg_dump and restore it in the new bucardo slave.
>
> So, I thought I would do something like this:
> ? Install Bucardo and add large tables to a pushdelta sync
> ? Copy the tables to the new server (e.g. with pg_dump)
> ? Startup Bucardo and catch things up (e.g. copy all rows changes since step 2)
> Steps for the above would be pretty much like this article; if I'm not mistaken.
>
> My questions are:
> ? Is it the right approach? do you guys have any other suggestions?
Yeah, as long as you?re doing a one-way sync (master -> target) it?s sufficient to start capturing the deltas (generally by creating a sync with auto-kick off), then pg_dump the data, then kick the sync/set autokick until you?re caught up then cutover the app as needed.
> ? I'll have the following: pg-9.2 master --> bucardo instance (with the bucardo DB only) --> pg-9.6 slave from bucardo
> ? When doing the pg_dump, I only need to restore it on the "pg-9.6 slave from bucardo" instance, correct? the bucardo DB does not store the data?
Right, ?bucardo? database holds meta-information about the syncs, sync history, etc, but no user data.
> Thanks!
Best,
David
--
David Christensen
End Point Corporation
david at endpoint.com
785-727-1171
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20190206/2acd2482/attachment-0001.sig>
------------------------------
Message: 4
Date: Thu, 7 Feb 2019 09:18:18 +1300
From: Lucas Possamai <drum.lucas at gmail.com>
Cc: bucardo-general at bucardo.org
Subject: Re: [Bucardo-general] new Bucardo installation with a 1TB DB
Message-ID:
<CAE_gQfX25YY38ZEw-3a210-=Viq5BwY+1SVQGdB=dQRGYOrxcA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
On Thu, Feb 7, 2019 at 3:09 AM David Christensen <david at endpoint.com> wrote:
> > We're moving our EC2 Postgres 1TB DB to RDS and decided to use Bucardo,
> as it supports multiple PG versions.
> > We have a PG 9.2 cluster, and instead of upgrading it, we'll use Bucardo
> to replicate to a PG 9.6, then use DMS or pg_dump to restore it in RDS.
> >
> > Because of the size of the current 9.2 DB, I cannot stop the application
> so it doesn't write to the DB while I do pg_dump and restore it in the new
> bucardo slave.
> >
> > So, I thought I would do something like this:
> > ? Install Bucardo and add large tables to a pushdelta sync
> > ? Copy the tables to the new server (e.g. with pg_dump)
> > ? Startup Bucardo and catch things up (e.g. copy all rows changes
> since step 2)
> > Steps for the above would be pretty much like this article; if I'm not
> mistaken.
> >
> > My questions are:
> > ? Is it the right approach? do you guys have any other suggestions?
>
> Yeah, as long as you?re doing a one-way sync (master -> target) it?s
> sufficient to start capturing the deltas (generally by creating a sync with
> auto-kick off), then pg_dump the data, then kick the sync/set autokick
> until you?re caught up then cutover the app as needed.
>
>
First of all, thanks for your reply!
That would be pushdelta <https://bucardo.org/pushdelta/>, correct?
> > ? I'll have the following: pg-9.2 master --> bucardo instance
> (with the bucardo DB only) --> pg-9.6 slave from bucardo
> > ? When doing the pg_dump, I only need to restore it on the
> "pg-9.6 slave from bucardo" instance, correct? the bucardo DB does not
> store the data?
>
> Right, ?bucardo? database holds meta-information about the syncs, sync
> history, etc, but no user data.
>
>
Perfect!
> > Thanks!
>
>
>
Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20190207/072595ee/attachment-0001.html>
------------------------------
Message: 5
Date: Tue, 12 Feb 2019 11:40:40 +1300
From: Lucas Possamai <drum.lucas at gmail.com>
Cc: bucardo-general at bucardo.org
Subject: Re: [Bucardo-general] new Bucardo installation with a 1TB DB
Message-ID:
<CAE_gQfWAg2Ok=X4dMEy3dvWJhS_aWdXhS3cJFvcE80oZL6g0CQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
On Thu, 7 Feb 2019 at 09:18, Lucas Possamai <drum.lucas at gmail.com> wrote:
> On Thu, Feb 7, 2019 at 3:09 AM David Christensen <david at endpoint.com>
> wrote:
>
>> > We're moving our EC2 Postgres 1TB DB to RDS and decided to use Bucardo,
>> as it supports multiple PG versions.
>> > We have a PG 9.2 cluster, and instead of upgrading it, we'll use
>> Bucardo to replicate to a PG 9.6, then use DMS or pg_dump to restore it in
>> RDS.
>> >
>> > Because of the size of the current 9.2 DB, I cannot stop the
>> application so it doesn't write to the DB while I do pg_dump and restore it
>> in the new bucardo slave.
>> >
>> > So, I thought I would do something like this:
>> > ? Install Bucardo and add large tables to a pushdelta sync
>> > ? Copy the tables to the new server (e.g. with pg_dump)
>> > ? Startup Bucardo and catch things up (e.g. copy all rows changes
>> since step 2)
>> > Steps for the above would be pretty much like this article; if I'm not
>> mistaken.
>> >
>> > My questions are:
>> > ? Is it the right approach? do you guys have any other
>> suggestions?
>>
>> Yeah, as long as you?re doing a one-way sync (master -> target) it?s
>> sufficient to start capturing the deltas (generally by creating a sync with
>> auto-kick off), then pg_dump the data, then kick the sync/set autokick
>> until you?re caught up then cutover the app as needed.
>>
>>
> First of all, thanks for your reply!
> That would be pushdelta <https://bucardo.org/pushdelta/>, correct?
>
Hi.
When using "type=pushdelta" on bucardo 5.4.1, I get the error: Unknown
option 'type'
What is the equivalent of "type=pushdelta" on Bucardo 5? Couldn't find that
on the documentation.
Cheers.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20190212/7e7205bc/attachment.html>
------------------------------
_______________________________________________
Bucardo-general mailing list
Bucardo-general at bucardo.org
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
End of Bucardo-general Digest, Vol 133, Issue 1
***********************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20190212/85399159/attachment-0003.html>
More information about the Bucardo-general
mailing list