[Bucardo-general] please help with this.

[Grant Maxwell]

Hi Greg

Following your thought line on doing something before-sync, If a before-sync custom-code could access a list of rows that are to be sent and update the actual rows before they are sent
then I could fix the problem by preempting the conflict. So the code would need access to the sourcedbh, targetdbh, sourcerowlist.

regards
grant
	
On 03/02/2010, at 8:57 AM, Greg Sabino Mullane wrote:

>> I dont understand your comments about database consistency. If two
>> records need replicating now and two more in two minutes then what is
>> the problem if they are replicated
>> in one batch of four records or two batches of two ? The key point is
>> that all records need to be replicated - not necessarily all in one
>> transaction.
> 
> The problem is that Bucardo also looks at the table as it currently is, 
> not at a point in time in the past. If more than one table is involved, 
> we can't know which records are more important than others, so we must 
> consider all changes since the last sync. If only a single table with 
> no dependencies or internal constraints, it would be possible in theory 
> to chunk them, I suppose. Anyone else have thoughts on that? I need to 
> think about this at the beginning of a day, not the end. :)
> 
>>   The controller has to do a replication and sees that there are 100,000
>>   lines in a table to replicate. It breaks the records into chunks of
>>   maximum X records (set by choice for each sync).
> 
> We can only break by timestamp, not number. But we can certainly find a 
> timestamp containing closest to a desired number (although with a little 
> added cost).
> 
>>     * Each record is an "identity" record for an email address. It has a
>>       bigint primary key and a unique emailaddress field (as well as some
>>       other fields).
> ...
>>     * The records (and conflicts) are created so fast that by the time
>>       the controller starts again - more conflicts have been created and
>>       the sync will never succeed.
> 
> Yeah, that's tricky in a swap sync situation. One option is to somehow 
> check the other side for an existing email before insert (e.g. plperlu function), 
> or remove that constraint and have a special after insert trigger clean things 
> up as needed. Perhaps a before-sync Bucardo code could check for unique violations 
> and handle them first. Or we could get Bucardo to act the way you are requesting. :)
> 
> -- 
> Greg Sabino Mullane greg at endpoint.com
> End Point Corporation
> PGP Key: 0x14964AC8