[Bucardo-general] Newbie Questions

David E. Wheeler david at justatheory.com
Tue Sep 18 16:16:09 UTC 2012


On Sep 17, 2012, at 5:08 PM, Greg Sabino Mullane <greg at endpoint.com> wrote:

>> So it's the thing that's LISTENing?
> 
> Well, they all listen - it's how they talk to each other, but 
> the MCP is the one listening for the Bucardo "triggerkick" triggers, 
> as well as communication via the 'bucardo' program.

Yeah, that's what I meant.

>> Since reads the DB on startup, does that mean I have to 
>> kick it if I add new tables to be replicated?
> 
> No - all "ping" tables are auto-kicked on Bucardo startup, mostly 
> to cover the case that something happened while Bucardo was down 
> an unable to receive the triggerkick notices.

No, I mean if Bucardo has been running a while, and I need to change its configuration (add new dbs, goats, whatever), do I need to kick it in order for it to pick up the configuration changes?

>> Oh? Which is preferable, and why? I would assume that persistent 
>> would be preferable in a high transaction environment, less so in 
>> a low transaction environment. And by "transactions" I mean 
>> "replicable events".
> 
> Yes. For low activity, you do not want persistent, as it just ties up 
> resources waiting around for something to happen. It's not really 
> a lot of resources though, so all in all it's usually easier just to 
> leave it as persistent until a problem is seen.

That makes sense.

>> Hrm. Sounds like maybe it no longer needs to be a separate process, eh?
> 
> Not quite. As the kid can be in the middle of a long database activity, 
> it's nice to have something that can be easily signalled and can (for 
> example) reap the kid if needed. While that job could be moved onto the 
> MCP, it gets a little out of its design goal of being a simple 
> postmaster-like process that farms out the real work to child processes.

Gotcha.

> Also, we may go back to multiple kids someday. For true multi-master 
> situations it does not make sense, as a single kid has to access all the 
> databases, but there might be cases such as one-master, multiple-kids where it 
> still might be an overall win. Measurement and testing showed that to 
> be a pretty big "might", but I'd also like to keep it separate for any 
> future ideas that pop up.

Seems sensible. And like less work. :-)

This reminds me, though. I'm planning to build a system with two masters, each in its own data center. There will be several databases, all replicated between the two DCs. But some will depend on each other. For example, a "customers" table in a "main" database might need to be replicated to other databases in the same cluster, so that they can conveniently do JOINs and such. That would be a master/slave (god I hate these terms) configuration. I assume that this is not a big deal for Bucardo, that it can do a variety of different types of replications in a single instance, yes?

>> I guess I can have Bucardo replicate its own database, eh? 
>> That would keep things pretty simple.
> 
> It could, but I usually advise something simpler such as WAL-shipping 
> or an occasional pg_dump.

Well, if Bucardo is already there and running, why not just use it, too? With trigger-based NOTIFYs, it doesn't seem as if it would add much overhead in addition to what it's already doing for the system, would it?

> Right. It's all about the number of rows that have changed. Anything 
> under 100,000 rows is probably not going to be much of a problem.

As I figured, thanks.

>> How stable is 5.0 if I were to target it? Do you have a release date in 
>> mind? Seems like it has been close for a long time…
> 
> Sigh. Yeah, the release seems asymptotic. It is pretty darn stable for 
> simple use cases, it's some of the edge cases that we are trying to 
> iron out. One of the last big things is some changes to the conflict 
> resolution system. Right now, because we disable triggers, there is 
> potential for violation of foreign keys in certain situations. I have 
> ideas on a solution, but no time to implement yet. It may or may not 
> hold up a release of version 5.

Sounds important.

> There are some small bugs in the TODO 
> list that *are* going to hold up B5, so any help there is appreciated.

These?

* Fix quoting bug reported by Sean M. Pappalardo
* Fix need for password argument when using md5
* Get Drizzle tests verified as working
* Get bucardo-report working again
* Note 8.2 minimum requirement somewhere (probably from COPY (select))
* Get some solid tests for the makedelta system
* Fix bug with checking wrong tables in databases on startup reported to list

Where will I find context for these? They appear to be more notes to yourself. Which I can appreciate, but not be much help with. :-)

> There are some outstanding issues raised on the mailing list as well; 
> most should be in the TODO, but those should all be considered blockers 
> as well.

If you could put together a canonical list with context, I could likely put in a little time to fix some of them. Would be great to have a list of things to be done for 5.0, so that anyone could see where things stand and how things are progressing. Maybe use GitHub issues and a milestone? I find it pretty useful for a personal project I work on.

Best,

David




More information about the Bucardo-general mailing list