[Bucardo-general] please help with this.

[Greg Sabino Mullane]

> I dont understand your comments about database consistency. If two
> records need replicating now and two more in two minutes then what is
> the problem if they are replicated
> in one batch of four records or two batches of two ? The key point is
> that all records need to be replicated - not necessarily all in one
> transaction.

The problem is that Bucardo also looks at the table as it currently is, 
not at a point in time in the past. If more than one table is involved, 
we can't know which records are more important than others, so we must 
consider all changes since the last sync. If only a single table with 
no dependencies or internal constraints, it would be possible in theory 
to chunk them, I suppose. Anyone else have thoughts on that? I need to 
think about this at the beginning of a day, not the end. :)

>    The controller has to do a replication and sees that there are 100,000
>    lines in a table to replicate. It breaks the records into chunks of
>    maximum X records (set by choice for each sync).

We can only break by timestamp, not number. But we can certainly find a 
timestamp containing closest to a desired number (although with a little 
added cost).

>      * Each record is an "identity" record for an email address. It has a
>        bigint primary key and a unique emailaddress field (as well as some
>        other fields).
...
>      * The records (and conflicts) are created so fast that by the time
>        the controller starts again - more conflicts have been created and
>        the sync will never succeed.

Yeah, that's tricky in a swap sync situation. One option is to somehow 
check the other side for an existing email before insert (e.g. plperlu function), 
or remove that constraint and have a special after insert trigger clean things 
up as needed. Perhaps a before-sync Bucardo code could check for unique violations 
and handle them first. Or we could get Bucardo to act the way you are requesting. :)

-- 
Greg Sabino Mullane greg at endpoint.com
End Point Corporation
PGP Key: 0x14964AC8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: not available
Url : https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20100202/32d81247/attachment.bin