[Bucardo-general] Bucardo gives up on nonresponsive remote host
Jonathan Brinkman
jb at blackskytech.com
Sat Nov 24 23:25:05 UTC 2012
It is ugly I agree, but what can I do? I created a bash script to check once
every 5 minutes to ensure log.bucardo has new data in it. It restarts
bucardo if not. It's been fairly active on some systems I have where remote
internet is spotty. The master server running bucardo is definitely giving
up on the remote after a few hours of disconnect. Or maybe it can't forgive
the disconnect even though the network connection has been restored.
Something gets hosed up, but restarting bucardo seems to fix the problem. It
restarts 2-4 times a day according to the bucardo.keepitup.restarts log.
[11/22/2012 06:30:05] Bucardo Restart Attempted!
[11/22/2012 06:40:05] Bucardo Restart Attempted!
[11/22/2012 08:20:05] Bucardo Restart Attempted!
[11/22/2012 17:35:07] Bucardo Restart Attempted!
[11/23/2012 12:35:07] Bucardo Restart Attempted!
[11/23/2012 15:05:06] Bucardo Restart Attempted!
[11/24/2012 01:00:01] Bucardo Restart Attempted!
[11/24/2012 01:05:01] Bucardo Restart Attempted!
[11/24/2012 06:20:05] Bucardo Restart Attempted!
[11/24/2012 16:05:05] Bucardo Restart Attempted!
#!/bin/sh
###########################################################################
# ENSURE BUCARDO STAYS RUNNING!
###########################################################################
#
#1. ensure using postgres user
#2. create/copy this file BST_bucardo_keepitup.sh
# on Cloud-DB:
# nano /home/postgres/BST_bucardo_keepitup.sh
# paste text of this doc into file
#3. chmod +x /home/postgres/BST_bucardo_keepitup.sh
# chmod 771 /home/postgres/BST_bucardo_keepitup.sh
#4. test file: ./BST_bucardo_keepitup.sh
#5. set up cron (every 5 minutes):
# crontab -e
# */5 * * * * root /home/postgres/BST_bucardo_keepitup.sh
#
###########################################################################
#
#check /home/postgres/log.bucardo
#if timestamp is older than 10 minutes old, run sudo bucardo_ctl restart
# Log file used to document the automatic bucardo restarts
LogFile="/home/postgres/bucardo.keepitup.restarts";
# Has the log file NOT been modified in the last 5 minutes?
if [ `find -P -O3 /home/postgres -maxdepth 1 -mmin -5 -name log.bucardo | wc
-l` -eq 0 ] ; then
# Restart Bucardo
bucardo_ctl restart
# Write this event to a separate log
echo $(date +"[%m/%d/%Y %H:%M:%S] ") "Bucardo Restart Attempted!" >>
$LogFile
fi;
-----Original Message-----
From: bucardo-general-bounces at bucardo.org
[mailto:bucardo-general-bounces at bucardo.org] On Behalf Of
bucardo-general-request at bucardo.org
Sent: Friday, November 09, 2012 10:20 AM
To: bucardo-general at bucardo.org
Subject: Bucardo-general Digest, Vol 62, Issue 12
Send Bucardo-general mailing list submissions to
bucardo-general at bucardo.org
To subscribe or unsubscribe via the World Wide Web, visit
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
or, via email, send a message with subject or body 'help' to
bucardo-general-request at bucardo.org
You can reach the person managing the list at
bucardo-general-owner at bucardo.org
When replying, please edit your Subject line so it is more specific than
"Re: Contents of Bucardo-general digest..."
Today's Topics:
1. Re: Bucardo-general Digest, Vol 62, Issue 10 (Jonathan Brinkman)
2. Re: Bucardo-general Digest, Vol 62, Issue 10 (Greg Sabino Mullane)
3. Re: Bucardo-general Digest, Vol 62, Issue 10 (Jonathan Brinkman)
4. Re: VAC Warning! VAC was killed at line 6507: DBD::Pg::st
pg_result failed: ERROR: relation "bucardo.delta_17032" does not
exist (Frank NJ.)
----------------------------------------------------------------------
Message: 1
Date: Fri, 9 Nov 2012 09:17:57 -0500
From: "Jonathan Brinkman" <jb at blackskytech.com>
To: <bucardo-general at bucardo.org>
Subject: Re: [Bucardo-general] Bucardo-general Digest, Vol 62, Issue
10
Message-ID: <003101cdbe85$04a5e080$0df1a180$@com>
Content-Type: text/plain; charset="us-ascii"
There is no question Bucardo is stopping and not restarting without manual
intervention. I'm thinking about creating a bash script that restarts
bucardo if the latest log record in log.bucardo is dated longer than, say,
10 minutes ago. I'd hoped not to have to do that.
Using 4.4.8
Yesterday at 5:21PM is the last entry in the bucardo.warning.log
[Thu Nov 8 15:00:19 2012] CTL Warning! Kid 1410 seems to have
died. Sync "vog_pushdelta_cms_replication"
[Thu Nov 8 16:20:15 2012] KID Kid exiting at cleanup_kid. Reason:
DBD::Pg::db do failed: ERROR: could not serialize access due to concurrent
update at /usr/local/share/perl/5.10.1/Buc$
main error: none source error: none target error: 7 States://40001
[Thu Nov 8 16:20:20 2012] CTL Warning! Kid 11959 seems to have
died. Sync "vog_pushdelta_cms_replication"
The log.bucardo file seems to indicate it gave up. Here are the last
entries, ending yesterday at 5:21PM (16 hrs ago)
[Thu Nov 8 17:12:13 2012] KID Final database backend PID is 17499
[Thu Nov 8 17:12:13 2012] KID Kid exiting at cleanup_kid. Reason:
DBI connect('dbname=vog_cms_gate;port=5432;host=10.0.1.38','bucardo',...)
failed: could not connect to server: No route to host
Is the server running on host "10.0.1.38" and accepting
TCP/IP connections on port 5432? at
/usr/local/share/perl/5.10.1/Bucardo.pm line 267
[Thu Nov 8 17:12:13 2012] KID Removed pid file
"/var/run/bucardo/bucardo.kid.sync.cmsvog_pushdelta_main_to_gate.cmsvog_gate
.pid"
[Thu Nov 8 17:12:22 2012] CTL Warning! Kid 17496 seems to have
died. Sync "cmsvog_pushdelta_main_to_gate"
[Thu Nov 8 17:16:16 2012] KID Final database backend PID is 17638
[Thu Nov 8 17:16:16 2012] KID Kid exiting at cleanup_kid. Reason:
Ping failed for source database cmsvog_replication
[Thu Nov 8 17:16:16 2012] KID Removed pid file
"/var/run/bucardo/bucardo.kid.sync.vog_pushdelta_cms_replication.cmsvog_main
.pid"
[Thu Nov 8 17:16:21 2012] CTL Warning! Kid 15697 seems to have
died. Sync "vog_pushdelta_cms_replication"
-----Original Message-----
From: bucardo-general-bounces at bucardo.org
[mailto:bucardo-general-bounces at bucardo.org] On Behalf Of
bucardo-general-request at bucardo.org
Sent: Wednesday, November 07, 2012 7:00 AM
To: bucardo-general at bucardo.org
Subject: Bucardo-general Digest, Vol 62, Issue 10
Send Bucardo-general mailing list submissions to
bucardo-general at bucardo.org
To subscribe or unsubscribe via the World Wide Web, visit
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
or, via email, send a message with subject or body 'help' to
bucardo-general-request at bucardo.org
You can reach the person managing the list at
bucardo-general-owner at bucardo.org
When replying, please edit your Subject line so it is more specific than
"Re: Contents of Bucardo-general digest..."
Today's Topics:
1. bucardo time-out setting? (Jonathan Brinkman)
2. Re: bucardo time-out setting? (Greg Sabino Mullane)
----------------------------------------------------------------------
Message: 1
Date: Tue, 6 Nov 2012 08:16:30 -0500
From: "Jonathan Brinkman" <jb at blackskytech.com>
To: <bucardo-general at bucardo.org>
Subject: [Bucardo-general] bucardo time-out setting?
Message-ID: <001501cdbc20$efcfc9c0$cf6f5d40$@com>
Content-Type: text/plain; charset="us-ascii"
I have Bucardo running at a client site where the connection between the
local server and cloud server is unreliable due to Comcast being lousy there
(it could be the POS router they provided.).
When an extended internet outage occurs, after a while Bucardo quits trying
to reconnect and waits (indefinitely) for a manual restart.
Is there a way to extend or remove the timeout so that the cloud server will
keep trying to reinitiate the syncs?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20121106/ed
fb8437/attachment-0001.html>
------------------------------
Message: 2
Date: Tue, 6 Nov 2012 09:36:13 -0500
From: Greg Sabino Mullane <greg at endpoint.com>
To: Jonathan Brinkman <jb at blackskytech.com>
Cc: bucardo-general at bucardo.org
Subject: Re: [Bucardo-general] bucardo time-out setting?
Message-ID: <20121106143613.GV3140 at tinybird.home>
Content-Type: text/plain; charset="us-ascii"
> Is there a way to extend or remove the timeout so that the cloud
> server will keep trying to reinitiate the syncs?
Which timeout are you seeing? And which version of Bucardo are they using?
In theory, things should retry indefinitely already.
--
Greg Sabino Mullane greg at endpoint.com
End Point Corporation
PGP Key: 0x14964AC8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: not available
URL:
<https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20121106/2f
a95436/attachment-0001.sig>
------------------------------
_______________________________________________
Bucardo-general mailing list
Bucardo-general at bucardo.org
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
End of Bucardo-general Digest, Vol 62, Issue 10
***********************************************
------------------------------
Message: 2
Date: Fri, 9 Nov 2012 09:29:36 -0500
From: Greg Sabino Mullane <greg at endpoint.com>
To: Jonathan Brinkman <jb at blackskytech.com>
Cc: bucardo-general at bucardo.org
Subject: Re: [Bucardo-general] Bucardo-general Digest, Vol 62, Issue
10
Message-ID: <20121109142936.GB3140 at tinybird.home>
Content-Type: text/plain; charset="us-ascii"
On Fri, Nov 09, 2012 at 09:17:57AM -0500, Jonathan Brinkman wrote:
> There is no question Bucardo is stopping and not restarting without
> manual intervention. I'm thinking about creating a bash script that
> restarts bucardo if the latest log record in log.bucardo is dated
> longer than, say,
> 10 minutes ago. I'd hoped not to have to do that.
Yeah, that's ugly.
> [Thu Nov 8 17:16:16 2012] KID Removed pid file
> "/var/run/bucardo/bucardo.kid.sync.vog_pushdelta_cms_replication.cmsvo
> g_main
> .pid"
> [Thu Nov 8 17:16:21 2012] CTL Warning! Kid 15697 seems to have
> died. Sync "vog_pushdelta_cms_replication"
Is there more than that? I would expect the MCP to log stuff as it leaves as
well. Does this sync have stayalive and kidsalive as true?
To be clear, it's the remote database(s) that are sometimes unreachable, not
the main Bucardo database, right?
--
Greg Sabino Mullane greg at endpoint.com
End Point Corporation
PGP Key: 0x14964AC8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: not available
URL:
<https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20121109/a4
a53bc5/attachment-0001.sig>
------------------------------
Message: 3
Date: Fri, 9 Nov 2012 10:12:53 -0500
From: "Jonathan Brinkman" <jb at blackskytech.com>
To: "'Greg Sabino Mullane'" <greg at endpoint.com>
Cc: bucardo-general at bucardo.org
Subject: Re: [Bucardo-general] Bucardo-general Digest, Vol 62, Issue
10
Message-ID: <003d01cdbe8c$b161d4d0$14257e70$@com>
Content-Type: text/plain; charset="us-ascii"
Yes it's the remote database that is sometimes unreachable.
Stayalive and kidsalive are true
Sync name: cmsvog_swap_main_and_gate
Current state: idle (PID = 21547)
Type: swap
Source herd/database: cmsvog_main_and_gate / cmsvog_main
Target database: cmsvog_gate
Tables in sync: 21
Last good: 36s (time to run: 2s)
Last good time: Nov 09, 2012 09:39:40 Target: cmsvog_gate
Ins/Upd/Del: 1 / 0 / 0
Last bad: 20m 40s (time to run: 16h 23m 49s)
Last bad time: Nov 09, 2012 09:19:36 Target: cmsvog_gate
Latest bad reason: MCP removing stale q entry PID file:
/var/run/bucardo/bucardo.ctl.sync.cmsvog_swap_main_and_gate.pid
PID file created: Fri Nov 9 09:21:29 2012
Status: active
Limitdbs: 0
Priority: 0
Checktime: none
Overdue time: 00:00:00
Expired time: 00:00:00
Stayalive: yes Kidsalive: yes
Rebuild index: 0 Do_listen: yes
Ping: yes Makedelta: no
Onetimecopy: 0
------------------------------
Here are the final lines from the log.bucardo before it died. Looks like the
last successful sync was at 16:56:01
[Thu Nov 8 16:56:01 2012] KID Got a notice for
cmsvog_pushdelta_main_to_gate: cmsvog_main -> cmsvog_gate [Thu Nov 8
17:11:55 2012] KID Final database backend PID is 17484 [Thu Nov 8 17:11:55
2012] KID Kid exiting at cleanup_kid. Reason:
DBD::Pg::db do failed: SSL SYSCALL error: No route to host at
/usr/local/share/perl/5.10.1/Bucardo.pm line 4314.
main error: none source error: none target error: 7 States://22000 [Thu Nov
8 17:11:55 2012] KID Removed pid file
"/var/run/bucardo/bucardo.kid.sync.cmsvog_pushdelta_main_to_gate.cmsvog_gate
.pid"
[Thu Nov 8 17:12:01 2012] CTL Rows updated child 2209 to aborted in q: 1
[Thu Nov 8 17:12:01 2012] CTL Warning! Kid 2209 seems to have died. Sync
"cmsvog_pushdelta_main_to_gate"
[Thu Nov 8 17:12:11 2012] CTL Cleaning up aborted sync from q table for
"cmsvog_gate". PID was 2209 [Thu Nov 8 17:12:11 2012] CTL Re-adding sync
to q table for database "cmsvog_gate"
[Thu Nov 8 17:12:12 2012] CTL Creating kid to handle resurrected q row
[Thu Nov 8 17:12:12 2012] CTL Created new kid 17496 for sync
"cmsvog_pushdelta_main_to_gate" to database "cmsvog_gate"
[Thu Nov 8 17:12:12 2012] KID New kid, syncs "cmsvog_main" to
"cmsvog_gate" for sync "cmsvog_pushdelta_main_to_gate" alive=1 Parent=2203
Type=pushdelta [Thu Nov 8 17:12:12 2012] KID PID: 17496 [Thu Nov 8
17:12:12 2012] KID Bucardo database backend PID is 17497 [Thu Nov 8
17:12:12 2012] KID Source database backend PID is 17498 [Thu Nov 8
17:12:13 2012] KID Final database backend PID is 17499 [Thu Nov 8 17:12:13
2012] KID Kid exiting at cleanup_kid. Reason: DBI
connect('dbname=vog_cms_gate;port=5432;host=10.0.1.38','bucardo',...)
failed: could not connect to server: No route to host
Is the server running on host "10.0.1.38" and accepting
TCP/IP connections on port 5432? at
/usr/local/share/perl/5.10.1/Bucardo.pm line 267 [Thu Nov 8 17:12:13 2012]
KID Removed pid file
"/var/run/bucardo/bucardo.kid.sync.cmsvog_pushdelta_main_to_gate.cmsvog_gate
.pid"
[Thu Nov 8 17:12:22 2012] CTL Warning! Kid 17496 seems to have died. Sync
"cmsvog_pushdelta_main_to_gate"
[Thu Nov 8 17:16:16 2012] KID Final database backend PID is 17638 [Thu Nov
8 17:16:16 2012] KID Kid exiting at cleanup_kid. Reason: Ping failed for
source database cmsvog_replication [Thu Nov 8 17:16:16 2012] KID Removed
pid file
"/var/run/bucardo/bucardo.kid.sync.vog_pushdelta_cms_replication.cmsvog_main
.pid"
[Thu Nov 8 17:16:21 2012] CTL Warning! Kid 15697 seems to have died. Sync
"vog_pushdelta_cms_replication"
[
-----Original Message-----
From: Greg Sabino Mullane [mailto:greg at endpoint.com]
Sent: Friday, November 09, 2012 9:30 AM
To: Jonathan Brinkman
Cc: bucardo-general at bucardo.org
Subject: Re: [Bucardo-general] Bucardo-general Digest, Vol 62, Issue 10
On Fri, Nov 09, 2012 at 09:17:57AM -0500, Jonathan Brinkman wrote:
> There is no question Bucardo is stopping and not restarting without
> manual intervention. I'm thinking about creating a bash script that
> restarts bucardo if the latest log record in log.bucardo is dated
> longer than, say,
> 10 minutes ago. I'd hoped not to have to do that.
Yeah, that's ugly.
> [Thu Nov 8 17:16:16 2012] KID Removed pid file
> "/var/run/bucardo/bucardo.kid.sync.vog_pushdelta_cms_replication.cmsvo
> g_main
> .pid"
> [Thu Nov 8 17:16:21 2012] CTL Warning! Kid 15697 seems to have
> died. Sync "vog_pushdelta_cms_replication"
Is there more than that? I would expect the MCP to log stuff as it leaves as
well. Does this sync have stayalive and kidsalive as true?
To be clear, it's the remote database(s) that are sometimes unreachable, not
the main Bucardo database, right?
--
Greg Sabino Mullane greg at endpoint.com
End Point Corporation
PGP Key: 0x14964AC8
------------------------------
Message: 4
Date: Fri, 9 Nov 2012 15:18:44 +0000
From: "Frank NJ." <njiaga at gmail.com>
To: "bucardo-general at bucardo.org" <bucardo-general at bucardo.org>
Subject: Re: [Bucardo-general] VAC Warning! VAC was killed at line
6507: DBD::Pg::st pg_result failed: ERROR: relation
"bucardo.delta_17032" does not exist
Message-ID:
<CAJxne4eS9aLwOEL3X3Dh7PLr7nqb0OSpMUjuEx_PFd+vj0f-Pg at mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Because of this bucardo has stopped for several hours, no restart without
manual intervention:
$ tail -f /var/log/bucardo/log.bucardo
PL/pgSQL function "bucardo_purge_delta" line 12 at SQL statement at
/usr/local/share/perl5/Bucardo.pm line 6507.
(3377) [Fri Nov 9 09:50:51 2012] MCP Warning: Killed (line 5038):
DBD::Pg::st execute failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request. at
/usr/local/share/perl5/Bucardo.pm line 5038.
DBD::Pg::db rollback failed: no connection to the server at
/usr/local/share/perl5/Bucardo.pm line 2175.
(3654) [Fri Nov 9 09:50:52 2012] CTL Controller for "londonhouse" was
killed at line 1250: MCP request
(3377) [Fri Nov 9 09:50:52 2012] MCP Process 25858 did not respond to a
kill 0
(3377) [Fri Nov 9 09:50:52 2012] MCP PID from
bucardo.kid.sync.londonhouse.pid is gone, removing file
(3377) [Fri Nov 9 09:50:52 2012] MCP End of cleanup_mcp. Sys time: Fri Nov
9 09:50:52 2012. Database time: 2012-11-09 09:50:52.866363+01
(3654) [Fri Nov 9 09:50:53 2012] CTL Controller 3654 exiting at
cleanup_controller. Reason: MCP request
On 8 November 2012 16:35, Frank NJ. <njiaga at gmail.com> wrote:
> SQL statement "SELECT bucardo.bucardo_purge_delta( $1 , $2 )"
> PL/pgSQL function "bucardo_purge_delta" line 12 at SQL statement at
> /usr/local/share/perl5/Bucardo.pm line 6507.
> (3377) [Thu Nov 8 17:27:11 2012] MCP Created VAC 3745
> (3745) [Thu Nov 8 17:27:11 2012] VAC New VAC daemon. PID=3745
> (3745) [Thu Nov 8 17:27:11 2012] VAC Connected to database "shop_db_cm"
> with backend PID of 3747
> (3745) [Thu Nov 8 17:27:14 2012] VAC Connected to database "shop_db_uk"
> with backend PID of 23742
> NOTICE: Rows deleted from "delta_sales_SalesPerson": 0 Rows deleted
> from
> "track_sales_SalesPerson": 0
> NOTICE: Rows deleted from "delta_person_BusinessEntity": 0 Rows
> deleted from "track_person_BusinessEntity": 0
> (3745) [Thu Nov 8 17:27:17 2012] VAC Warning! VAC was killed at line
> 6507: DBD::Pg::st pg_result failed: ERROR: relation "bucardo.delta_17032"
> does not exist
> LINE 1: DELETE FROM bucardo."delta_17032" USING (SELECT txntime AS t...
> ^
> QUERY: DELETE FROM bucardo."delta_17032" USING (SELECT txntime AS tt
> FROM bucardo."track_17032" GROUP BY 1 HAVING COUNT(*) = 1) AS foo
> WHERE txntime = tt AND txntime < now() - interval '45 seconds'
> CONTEXT: PL/pgSQL function "bucardo_purge_delta" line 44 at EXECUTE
> statement SQL statement "SELECT bucardo.bucardo_purge_delta( $1 , $2
> )"
> PL/pgSQL function "bucardo_purge_delta" line 12 at SQL statement at
> /usr/local/share/perl5/Bucardo.pm line 6507.
> (3377) [Thu Nov 8 17:28:11 2012] MCP Created VAC 3767
> (3767) [Thu Nov 8 17:28:11 2012] VAC New VAC daemon. PID=3767
> (3767) [Thu Nov 8 17:28:11 2012] VAC Connected to database "shop_db_cm"
> with backend PID of 3769
> (3767) [Thu Nov 8 17:28:16 2012] VAC Connected to database "shop_db_uk"
> with backend PID of 26100
> NOTICE: Rows deleted from "delta_sales_SalesPerson": 0 Rows deleted
> from
> "track_sales_SalesPerson": 0
> NOTICE: Rows deleted from "delta_person_BusinessEntity": 0 Rows
> deleted from "track_person_BusinessEntity": 0
> (3767) [Thu Nov 8 17:28:17 2012] *VAC Warning! VAC was killed at line
> 6507: DBD::Pg::st pg_result failed: ERROR: relation "bucardo.delta_17032"
> does not exist*
> LINE 1: DELETE FROM bucardo."delta_17032" USING (SELECT txntime AS t...
> ^
> QUERY: DELETE FROM bucardo."delta_17032" USING (SELECT txntime AS tt
> FROM bucardo."track_17032" GROUP BY 1 HAVING COUNT(*) = 1) AS foo
> WHERE txntime = tt AND txntime < now() - interval '45 seconds'
> CONTEXT: PL/pgSQL function "bucardo_purge_delta" line 44 at EXECUTE
> statement SQL statement "SELECT bucardo.bucardo_purge_delta( $1 , $2
> )"
> PL/pgSQL function "bucardo_purge_delta" line 12 at SQL statement at
> /usr/local/share/perl5/Bucardo.pm line 6507.
> (3377) [Thu Nov 8 17:29:11 2012] MCP Created VAC 3789
> (3789) [Thu Nov 8 17:29:11 2012] VAC New VAC daemon. PID=3789
> (3789) [Thu Nov 8 17:29:11 2012] VAC Connected to database "shop_db_cm"
> with backend PID of 3791
> (3789) [Thu Nov 8 17:29:15 2012] VAC Connected to database "shop_db_uk"
> with backend PID of 28502
> NOTICE: Rows deleted from "delta_sales_SalesPerson": 0 Rows deleted
> from
> "track_sales_SalesPerson": 0
> NOTICE: Rows deleted from "delta_person_BusinessEntity": 0 Rows
> deleted from "track_person_BusinessEntity": 0
> (3789) [Thu Nov 8 17:29:33 2012] VAC Warning! VAC was killed at line
> 6507: DBD::Pg::st pg_result failed: ERROR: relation "bucardo.delta_17032"
> does not exist
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://mail.endcrypt.com/pipermail/bucardo-general/attachments/20121109/93
b6d137/attachment.html>
------------------------------
_______________________________________________
Bucardo-general mailing list
Bucardo-general at bucardo.org
https://mail.endcrypt.com/mailman/listinfo/bucardo-general
End of Bucardo-general Digest, Vol 62, Issue 12
***********************************************
More information about the Bucardo-general
mailing list