[Bucardo-general] Fwd: Syncing smaller batches

[Greg Sabino Mullane]

On Wed, Nov 04, 2015 at 02:37:58PM -0500, Ioana Danes wrote:
> Ok the query is not stuck it is just very slow. It could be networking
> problem or client problem. The 2 servers are on the same msa though.
> 
> I can reproduce the problem quite easily if I stop the sync for few minutes
> so I could collect around 400,000 records to be synced (~ 200,000 for each
> of my 2 tables).
> 
> The query is :
> SELECT  DISTINCT "testtableid"
> FROM    bucardo.delta_abrazo_testtable d
>                      WHERE   NOT EXISTS
> (
>                                 SELECT
> 1
>                                 FROM   bucardo.track_abrazo_testtable t
>                                 WHERE  d.txntime = t.txntime
>                                 AND    (t.target = 'dbgroup
> sync_all1'::text)
> 
> )
> 
> A regular sync operation at 780 clients takes 2-3 seconds
> Then I start the sync script and the first few syncs are slower than normal
> (< 75 secs) because they have a lot to sync but then the third or fourth
> takes even longer ~ 270 seconds.

Thanks for your work on hunting this down. We need to determine next if 
it is the query itself that is slow or something else. The Bucardo logs 
should help (even better, running with log_level=debug). You could also 
set log_min_duration_statement = 100 in postgresql.conf and see if 
the above query shows up.

-- 
Greg Sabino Mullane greg at endpoint.com
End Point Corporation
PGP Key: 0x14964AC8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 163 bytes
Desc: not available
URL: <https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20151105/494196ea/attachment.sig>