[Bucardo-general] customecode attempts: missing 'rows' content and other fun
Meg Green
mawg at lanl.gov
Mon Mar 9 16:48:38 UTC 2009
Many thanks for your details.
My src_code did get quite messy in working thru the issue; excuse that.
My main problem was in missing the 'rows' -- with the get_rows set,
the @_ has 'rows' => {}, instead of the rows to sync. BUT I see you
are using $_->{'rowinfo'} which I don't have (empty or populated) in
@_.
It appears you are setting your whenrun to conflict. I am using
'before_sync' -- because I would actually like to simply not sync
some content. I'll try switching to 'conflict' and see what happens
to my @_.
Again, thanks for the good detail. I'll clean up src_code for easier
discussion in the thread.
Regards,
Meg
On Mar 9, 2009, at 9:10 AM, Lev Tannen wrote:
> Hello Meg,
> I recently wrote a customcode script and it works fine for me. It
> produces a result like
> Source Database = epls201 ---- the name of the source database
> Target Database = epls200 ---- the name of the target
> database
> Source/Target Table = aliases ---- the name of the table
> BUCARDO_ID:104 == 104 --- the values of the primary key
> on both sides (they are equals by definition)
> alias_recid:104 == 104 --- the values of a column on
> both sides (they are equals in this example)
> description:conflict test 111 <> conflict test 011 --- the values
> of a column on both sides (they are different in this example,
> which was the reason for the conflict)
>
> Bellow is the script that produced that result.
> Hope this will help.
> Lev Tannen
>
> ******************************************************
> my ($result) = @_;
> open conflicts_log, "+>>/home/bucardo/conflict.log" or die $!;
> if (defined $result->{dummy})
> {
> return 0;
> }
>
> my $sourcename = $result->{sourcename};
> my $targetname = $result->{targetname};
> my $rowinfo = $result->{rowinfo};
> my $table = $rowinfo->{table};
> my $sourcerow = $rowinfo->{sourcerow};
> my $targetrow = $rowinfo->{targetrow};
> my $pkeyname = $rowinfo->{pkeyname};
> my $pkey = $rowinfo->{pkey};
>
> my @SourceCols = keys(%$sourcerow);
> my @TargetCols = keys(%$targetrow);
> my @SourceValues = values(%$sourcerow);
> my @TargetValues = values(%$targetrow);
>
> print conflicts_log "
> ";
> print conflicts_log "Source Database = $sourcename
> ";
> print conflicts_log "Target Database = $targetname
> ";
> print conflicts_log "Source/Target Table = $table \n";
>
> my $sourceUpdate = "update $table set ";
> my $targetUpdate = "update $table set ";
>
> my $firstUpdate = 1;
>
> my $len = @SourceCols;
> for (my $i= 0;$i < $len ;$i++)
> {
> print conflicts_log $SourceCols[$i] . ":" . $SourceValues[$i];
> if ($SourceValues[$i] eq $TargetValues[$i] ) {
> print conflicts_log " == ";
> }
> else {
> print conflicts_log " <> ";
>
> if ($firstUpdate == 0) {
> $sourceUpdate .= ", ";
> $targetUpdate .= ", ";
> }
> $sourceUpdate .= "$SourceCols[$i] = '$SourceValues[$i]'";
> $targetUpdate .= "$TargetCols[$i] = '$TargetValues[$i]'";
>
> $firstUpdate = 0;
> }
> print conflicts_log $TargetValues[$i] . "
> ";
> }
> print conflicts_log "
> ";
>
> $sourceUpdate .= " where $pkeyname = '$pkey'";
> $targetUpdate .= " where $pkeyname = '$pkey'";
>
> print conflicts_log $sourceUpdate . "
> ";
> print conflicts_log $targetUpdate . "
>
> ";
>
> close conflicts_log;
> return 0;
>
> **************
>
> On Thu, Mar 5, 2009 at 3:03 PM, Meg Green <mawg at lanl.gov> wrote:
> Hello,
>
> Longtime listener, first time caller.
>
> Thanks for any thoughts or suggestions.
> Regards,
> Meg
>
>
> -----------------------------------
> Here is the basic curiousity right now:
>
> Using customecode set on the sync, exec'ing before the sync, the
> passed hash does not contain the rows to be changed. There is
> likely something I am missing in the content here, but the 'rows'
> hashref is empty.
>
> The customecode is set to getrows:
>
> bucardo=# select * from bucardo.customcode;
> id | name |
> about | whenrun | getdbh | getrows
> |
>
> src_code
>
> | cdate
> ----+----------------
> +----------------------------------------------------------
> +-------------+--------+---------
> +---------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> --------------------------------------------------------
> +-------------------------------
> 2 | test_setlookup | test script eventually to deal with
> set_lookup conflicts | before_sync | t | t | my
> ($result) =@_; open F, '>/tmp/test_setlookup_out'; print F $result->
> {syncname}; use Data::Dumper; print F Dumper $result; close F;
> $result->{message} = 'BOB WUZ HERE'; $result->{warning} = 'TRYING
> TO USE BOB TO WARN'; return $result; return if $result->{dummy} |
> 2009-03-04 12:55:59.428957-07
> (1 row)
>
> my dump file (/tmp/test_setlookup_out) from this config is
> attached. Here is a snippet:
>
> <outfile snippet>
> $BUCARDO1 = {
> 'sourcename' => 'asiatic',
> 'sourcedbh' => bless( {}, 'DBIx::Safe' ),
> 'runagain' => 0,
> 'syncname' => 'isd_sync',
> 'rows' => {},
> 'message' => '',
> 'targetname' => '',
> 'synctype' => 'swap',
> 'error' => '',
> 'warning' => '',
> 'endsync' => '',
> 'goatlist' => [
> {
> 'schemaname' => 'public',
> 'db' => 'asiatic',
> 'does_makedelta' => '0',
> 'cols' => [
> 'num_code',
> 'name',
> </outfile snippet>
>
>
>
>
> Ideally I would like to only fire the customecode when the specific
> table has deltas, but attempting to set the customecode to the
> goat.id of the table never fired the code. I'll be breaking that
> table from the main sync next and working with it alone to see if I
> can get better results (but for deployment it seems best to not add
> the complexity of multiple syncs for the same project/dbs)
>
>
>
> -----------------------------------
> Here is more detail on our scenario:
>
> We have a "search results" table that has many inserts and deletes,
> and some rows that are untouched over very long periods of time
> (something like "saved searches").
>
> It appears we are generating many, many exceptions because of this
> table, and we are considering how best to deal with it.
>
> Solution One: use customecode and basically ignore attempts to add
> duplicate rows.
>
> Solution Two: use customecode to NOT replicate the "search
> results" (the saved searches do need to be replicated)
>
>
>
> _______________________________________________
> Bucardo-general mailing list
> Bucardo-general at bucardo.org
> https://mail.endcrypt.com/mailman/listinfo/bucardo-general
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20090309/c1ad8562/attachment.html
More information about the Bucardo-general
mailing list