[Bucardo-general] Fwd: Syncing smaller batches

[Ioana Danes]

Hi Greg,

On Wed, Nov 4, 2015 at 9:36 AM, Greg Sabino Mullane <greg at endpoint.com>
wrote:

> On Tue, Nov 03, 2015 at 03:43:07PM -0500, Ioana Danes wrote:
> > > I installed the master branch from git and the performance is really
> good.
> > I still have some issues where there is a query running and it looks like
> > it is stuck. If I stop the syncing loop process and cancel the backend
> then
> > it recovers but I need to run more tests and make sure I am doing the
> right
> > steps on failover.
>
> Stuck is not good. Can you report on what query is getting stuck?
>

Ok the query is not stuck it is just very slow. It could be networking
problem or client problem. The 2 servers are on the same msa though.

I can reproduce the problem quite easily if I stop the sync for few minutes
so I could collect around 400,000 records to be synced (~ 200,000 for each
of my 2 tables).

The query is :
SELECT  DISTINCT "testtableid"
FROM    bucardo.delta_abrazo_testtable d
                     WHERE   NOT EXISTS
(
                                SELECT
1
                                FROM   bucardo.track_abrazo_testtable t
                                WHERE  d.txntime = t.txntime
                                AND    (t.target = 'dbgroup
sync_all1'::text)

)

A regular sync operation at 780 clients takes 2-3 seconds
Then I start the sync script and the first few syncs are slower than normal
(< 75 secs) because they have a lot to sync but then the third or fourth
takes even longer ~ 270 seconds.

I did run the same query in a pgAdmin during this time and it was extremely
fast even if it returned 200,000 records.


>
> > Is there already a way to call the sync externally in a service or I have
> > to write my own script?
>
> The canonical way to kick a sync is through the bucardo script. Anyone
> can run it, as long as things are configured correctly. You can also
> trigger it by sending a special database NOTIFY command.
>
> I use the bucardo kick sync command in an infinite loop for now for
testing.

Thanks for your help,
Ioana
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20151104/a9fc337c/attachment.html>