tech notes

This is something new I’ve just learned that only exists on OpenBSD. Up until today I thought that the only way to manually failover a carp setup was to down the carp interface on the master.

It looks like there is an easier way of doing it on OpenBSD. In fact OpenBSD uses this feature itself during the boot process. Just before setting up all interfaces it “demotes” all carp interfaces so they won’t become master interfaces for their ip addresses until all enabled system daemons, pf, ipsec etc have been configured and started. After that the whole carp group of interfaces is put back to the neutral state and they can become master interfaces (if there is no advskew set on them).

How is it done?

OpenBSD has this concept of groups of interfaces. It’s easy to spot it when you do ifconfig:

# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33208
groups: lo
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0×3
vic0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:0c:29:9c:5e:57
groups: egress
media: Ethernet autoselect
status: active
inet 172.21.33.5 netmask 0xffffff00 broadcast 172.21.33.255
inet6 fe80::20c:29ff:fe9c:5e57%vic0 prefixlen 64 scopeid 0×1
enc0: flags=0<> mtu 1536
carp0: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:00:00:00:00
groups: carp

Each interface has its own default group (or groups). The default group for all carp interfaces is… the carp group! You can create your own groups and add interfaces to them. An interface can belong to multiple groups. Here’s how to create a new group and add carp0 to it:

# ifconfig carp0 group mygroup
# ifconfig carp0
carp0: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:00:00:00:00
groups: carp mygroup

and here is how to remove it

# ifconfig carp0 -group mygroup
# ifconfig carp0
carp0: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:00:00:00:00
groups: carp

All groups have this additional property called the demote count which is used by carp during the master election process. Using this property you can demote a group of interfaces:

# ifconfig -g carp carpdemote 128

and promote it back:

# ifconfig -g carp -carpdemote 128

and you can see the current value:

# ifconfig -g carp
carp: carp demote count 0

So how is this better than downing all your carp interfaces by doing something like this:

for i in `ls /etc/hostname.carp*`; do echo $i | awk -F. ‘{print $2}’ | xargs -I% ifconfig % down; done

When you down your carp interface they no longer take part in the whole “carp process”. Basically since they are down they no longer advertise their presence and cannot be elected as masters. So if your backup server dies and all carp interfaces on your master are down you loose your connectivity.

Carp demote counter acts in a bit similar way to advskew but has higher precendence over it. So a carp interface with advskew set to 0 and demote counter set to 10 will be ranked lower (and become slave) than another carp interface with advskew 100 and demote counter set to 0.

Plus, by logically groupping carp interfaces you can failover only one group at a time, and when you have a lot of interfaces this is certainly easier then using ifconfig down.