tech notes

Dovecot MySQL stored procedures patch

admin — Fri, 07 May 2010 10:18:00 +0000

With this patch it’s possible to use stored procedure calls in dovecot’s configuration.

It implements the correct logic of handling mysql multiple statements as described in this MySQL document. I’m not sure it’s the correct place to implement it in (dovecot’s design is very clean and this is a rather quick hack), but it’s been working fine in production for a couple of months now.

Munin plugin for dovecot logins

admin — Fri, 07 May 2010 10:03:04 +0000

Graphs dovecot imap and pop3 logins based on entries in the log file. May require changing the location of the log file (hardcoded to /var/log/dovecot-info.log)

Munin plugin for Ironport

admin — Sun, 31 Jan 2010 00:28:10 +0000

A simple munin plugin for graphing Ironport statistics. All it graphs is CPU, memory, disk IO and queue utilisation. It requires SNMP to be enabled on the Ironport (and assumes you’re using SNMP v3) and snmpwalk to be installed on the munin server. The plugin supports the following environment variables:

* snmp_community – by default set to v3get (default Ironport setting)
* snmp_password – you have to set it yourself
* snmp_oidbase – set to 1.3.6.1.4.1.15497.1.1.1 which points at Ironport’s MIBS.

Save it as ipstatus_ and link to it via ipstatus_yourironporthostname

Portauditing jails

admin — Thu, 07 Jan 2010 11:41:08 +0000

I run most of the services in separate jails (that’s all easy to set up and maintain with ezjail) and quite often end up having 10+ jails even on simple mail/web server installations.

Installing portaudit, updating its database and running it every day in each jail seems to be a waste of space and resources. Instead I decided to check all jails from the host system during nightly security checks.

This should be saved as /usr/local/etc/periodic/security/420.jailportaudit (with chmod 555):

#!/bin/sh

RET_VAL=”"

get_jail_name()
{
jid=$1
RET_VAL=`jls | egrep “^ +$jid ” | awk ‘{print $3}’`
}

check_jail()
{
jid=$1

get_jail_name $jid
echo “==== checking jail :: ” $RET_VAL ” :: ====”
/usr/sbin/jexec $jid pkg_info | /usr/bin/awk ‘{print $1}’ | /usr/bin/xargs /usr/local/sbin/portaudit
echo
}

main()
{
param=$1

for i in `jls | tail +2 | awk ‘{print $1}’`
do
check_jail $i
done
}

main $@

Fixing corrupted Ironport Queue

admin — Tue, 05 Jan 2010 16:35:09 +0000

Looks like there’s a hidden command you can use to fix queue problems similar to these:

Critical: Queue: Your queue has been corrupted; UNABLE TO REPAIR: unable to
mount queue: ‘(\’qstore/gcq.py get_time_sorted_gens|919\’, “ \'exceptions.OSError\'>“, “[Errno 2] No such file or directory:
\’/var/db/godspeed/gen/gen063.chk\’”, \’[qstore/gcq.py mount|1387]
[qstore/gcq.py load|996] [qstore/gcq.py get_time_sorted_gens|919]\’)’

Critical: Error while sending alert: Unable to send System/Critical alert to xxx@xxx.com with subject “Critical ironport: Queue: Your queue has been corrupted; UNABLE TO REPAIR: unab…”.

What it means is basically that the workqueue is corrupted and the ironport is unable to accept/deliver emails. Rebooting doesn’t help (doesn’t really change anything). There’s however a way of recovering your ironport from this problem. The hidden command is:

resetqueue

It deletes the broken queue, creates a new one, removes all messages in the system quarantines and reboots the ironport. After this clean up operation your ironport should be as new.

What’s interesting is that by looking at various error messages thrown by Ironports from time to time (especially when something breaks more seriously – doesn’t happen too often Ironports are quite solid) you can actually see what’s running under the hood. Other than it running on something derived from FreeBSD (can’t be that far off it as they actually contribute some code back to the OS) it looks like it’s mostly run by python scripts. That’s interesting from the performance perspective as even the queue management seems to be written in python. Also the database used internally seems to be some version of PostgreSQL. A very nice choice of software…

Haven’t seen anything yet that would suggest what MTA (if it’s not something created by Ironport) is used there. Sure the fact it can write qmail compatible log files doesn’t mean anything )

how to create a new kvm image for eucalyptus UEC

admin — Sun, 15 Nov 2009 13:29:51 +0000

Eucalyptus is a new cloud management software available with the latest version of ubuntu (karmic koala). It’s compatible with Amazon’s EC2 (in terms of images and CLI tools) and it’s opensource so anyone can build a EC2 like service. That is… when it becomes a little bit more stable…

Eucalyptus comes with 2 images you can use to create vm instances on it – a 64 bit and 32 bit versions of ubuntu. These images are very well prepared and well integrated with EC2/Eucalyptus. But what if you want to run a different linux distro in your Eucalyptus based lab? Here’s a quick howto on creating a simple CentOS image on a ubuntu box with kvm installed:

Creating a new disk Image

This will be the main hdd in your virtual image, so make sure to give it as much space as you think you’ll need. Since we’re building a kvm image, we can use a qcow2 format for disk images. Qcow2 is an expandable image format, so it’ll only take as much storage space as it’s actually used withing the image.

kvm-img create -f qcow2 image.img 20G

OS Installation

Fetch an .iso of the distribution you want installed in the image.

wget http://www.mirrorservice.org/sites/mirror.centos.org/5.4/isos/x86_64/CentOS-5.4-x86_64-netinstall.iso

and start the installation process:

sudo kvm -m 256 -cdrom Centos-5.4-x86_64-netinstall.iso -drive file=image.img,if=scsi,index=0 -boot d -curses -net
nic,vlan=0,model=e1000,macaddr=00:16:3e:de:ad:01 -net tap

if your installation process requires more than 256MB of RAM change the -m option, and if you need more processors available, you can use the ‘-c’ option.

The command above will boot a new kvm instance, with the disk image you’ve created as the primary hdd and the iso as the first bootable device. Also the ‘-curses’ option will make the kvm display all console output to your ssh session. (I’m assuming here, you’re creating this image over a remote connection, if you’re not you can probably skip the -curses option and kvm should use sdl drivers instead)

After finishing the installation you can test the new virtual machine by running:

sudo kvm -m 256 -drive file=image.img,if=scsi,index=0,boot=on -boot c -curses -net nic,vlan=0,model=e1000,macaddr=00:16:e3:de:ad:01 -net tap

At this point you can add all the packages you want to have installed, all users, any settings that need to be present in your new UEC instances.

Now it’s also a good time to copy the kernel and the initrd image from your new vm image some place outside. They will be used later on to create and upload an complete virtual image to your UEC.

Before you shut your new shiny image down there’s one more step to be done:

Integration with UEC

Your new image needs to know what IP it has when started in UEC and also, it needs to know the public bit of the ssh key allowed to access it. The way it’s done in UEC (and EC2?) is via a restful interface provided by the cloud. The interface is available under this URL: http://169.254.169.254/latest/meta-data. You can use wget to see what information is provided:

wget -q -O – http://169.254.169.254/latest/meta-data/

What’s interesting for us here is the public key data which is available here:

http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key

As we need to automate the whole process a bit, let’s put it all into an init.d script:

#!/bin/bash
#
. /etc/rc.d/init.d/functions

RETVAL=0

start()
{
fetch_ssh_key
regenerate_ssh_keys
}

stop()
{
echo “nothing to stop…”
}

regenerate_ssh_keys()
{
rm -f /etc/ssh/ssh_host_key /etc/ssh/ssh_host_rsa_key /etc/ssh_ssh_host_dsa_key
[ -f /etc/ssh/ssh_host_key ] || (ssh-keygen -f /etc/ssh/ssh_host_key -t rsa1 -C ‘host’ -N ” | logger -s -t “ec2″)
[ -f /etc/ssh/ssh_host_rsa_key ] || (ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -C ‘host’ -N ” | logger -s -t “ec2″)
[ -f /etc/ssh/ssh_host_dsa_key ] || (ssh-keygen -f /etc/ssh/ssh_host_dsa_key -t dsa -C ‘host’ -N ” | logger -s -t “ec2″)

echo “—–BEGIN SSH HOST KEY FINGERPRINTS—–” |logger -s -t “ec2″
ssh-keygen -l -f /etc/ssh/ssh_host_key.pub |logger -s -t “ec2″
ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub |logger -s -t “ec2″
ssh-keygen -l -f /etc/ssh/ssh_host_dsa_key.pub |logger -s -t “ec2″
echo “—–END SSH HOST KEY FINGERPRINTS—–” |logger -s -t “ec2″
}

fetch_ssh_key()
{
if [ ! -d /root/.ssh ] ; then
mkdir -p /root/.ssh
chmod 700 /root/.ssh
fi

curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/ssh-key
if [ $? -eq 0 ] ; then
cat /tmp/ssh-key >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
rm /tmp/ssh-key
# disable password logging
sed -i.bkp ‘s/^PasswordAuthentication yes/PasswordAuthentication no/’ /etc/ssh/sshd_config
fi
}

case “$1″ in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
condrestart)
echo “not implemented”
;;
status)
echo “not implemented”
;;

*)
echo $”Usage: $0 {start|stop|restart|reload|condrestart|status}”
RETVAL=1
esac

exit $RETVAL

Enable this script in your boot process. The script will not only download and install this key on root account. It also regenerates your instance’ keys and displays their fingerprints on the console (so you can see them by doing euca-get-console-output i-instance_number)

This script is just a simple example, you can also take the python uec backend available in ubuntu and use it to do the same thing i a slightly better and cleaner way.

Uploading to UEC

The last step is uploading your image to UEC:

bundle and upload the previously copied kernel first:

euca-bundle-image -i kvm-kernel/vmlinuz-2.6.28-11-generic –kernel true
euca-upload-bundle -k mybucket -m /tmp/vmlinuz-2.6.28-11-generic.manifest.xml
euca-register mybucket/vmlinuz-2.6.28-11-generic.manifest.xml

save the k-* output produced by the last command above and proceed with initrd:

euca-bundle-image -i kvm-kernel/initrd.img-2.6.28-11-generic
euca-upload-bundle -b mybucket /tmp/initrd.img-2.6.28-11-generic.manifest.xml
euca-register mybucket/initrd.img-2.6.28-11-generic.manifest.xml

as above, save the i-* output and upload the image now:

euca-bundle-image -i image.img –kernel $PREVIOUSLY_SAVED_KERNELID –ramdisk $PREVIOUSLY_SAVED_IMAGERD
euca-upload-bundle -b mybucket -m /tmp/image.img.manifest.xml
euca-register mybucket/image.img.manifest.xml

All done, your new image should be visible after euca-describe-images -a.

ssh port knocking with pf

admin — Sat, 24 Oct 2009 13:39:41 +0000

The idea of port knocking is simple – a service, normally firewalled accepts connections from a given source IP if that IP address has connected to certain ports in some special sequence. This is a simplified implementation of this idea using pf to protect the ssh service.

In pf.conf file:

…
### pf tables
table persist
…
### pf rules
block in log all
…
pass in quick on $if proto tcp from to $me port 22 flags S/SA keep state
…
# there’s no service listening on 31337 so we need synproxy state to complete the handshake
pass in quick on $if proto tcp from any to $me port {31337} synproxy state (max-src-conn-rate 3/5, overload
…

This will open port 22 on the $me host if there are 3 attempts to connect to port 31337 within 5 seconds.

From this moment ssh access to $me is granted. This shouldn’t probably be allowed forever, so this crontab entry will clear all entries in the ssh_accept table not used within last 5 minutes:

*/5 * * * * root /sbin/pfctl -q -t ssh_accept -T expire 300

RPM build environment on CentOS

admin — Wed, 02 Sep 2009 21:42:23 +0000

Just a quick note on how to build RPMs

setting up build environment

RPMs should be built from a ”’standard user”’ account, ”’not root”’. This saves a lot of trouble when something goes wrong during package preparation/installation and keeps the build environment clean.

Here’s how to setup build environment in your home directory:

mkdir -p ~/build/{BUILD,RPMS,S{OURCES,PECS,RPMS}}

These directories are for:

BUILD – that’s the place where your source package will get untargzipped, patched and compiled
RPMS – will contain your RPMS when done
SOURCES – upload all source .tar.gz’s and your patches here
SPECS – .spec files with instructions on how to build your packages
SRPMS – source RPMS, a bit more about them below

Now you need to tell rpm-build where to look for these directories:

echo “%_topdir $HOME/build” > ~/.rpmmacros

and finally you need to install some software:

sudo yum install rpm-build redhat-rpm-config mock

(Keep in mind, that the list above is not complete as it doesn’t contain any compilers, libtools, automakes etc, that might be required to actually build your source package.)

and create a ‘special’ user for the mock package (this is to allow fakeroot builds):

sudo useradd -M -d / -s /sbin/nologin mockbuild

building from SRPMs (easy!)

SRPMS (source RPMS) contain the .spec file and all necessary source files and patches needed to build a binary package. This can be usefull if there’s no binary package available for your platform but there’s a source one available for a similar one (redhat.srpms to build packages for centos).

One way to build a binary package from a source one is to:

rpmbuild –rebuild package-1.0.src.rpm

This will do all the magic and (hopefully) put a binary package in build/RPMS/$arch/

Other way is to install (as user, it will put all sources and patches to build/SOURCES and the spec file to build/SPECS) the source package first and then use rpmbuild:

rpm -i package-1.0.src.rpm
rpmbuild -ba ~/build/SPECS/package-1.0.spec

According to the documentation you can build RPMs from .tar.gzs if they contain a .spec file. You would do it by using -ta option for rpmbuild instead of -ba.

building from source

This means creating your own package from scratch. There’s how to do it:

First see if the package builds from source without any tweaking. Check if it (or the resulting package you want built) requires any special options to be passed to either ./configure or make. Also at this point think of any modifications to the original source code you’d like to have (changes in default config files, additional modules compiled in or removed from the package etc etc).

If you do need some changes done, this is a good time to make all the patches.
Make a clean and untouched copy of the untargzipped package:

tar xvfz moosoft-1.0.tar.gz
cp -Rvp moosoft-1.0 moosoft-1.0.orig

now make all the required changes in the moosoft-1.0 directory and generate patch or a set of patches:

diff -uNr moosoft-1.0.orig moosoft-1.0 > moosoft-1.0.bigpatch.patch
# or:
diff -uNr moosoft-1.0.orig/etc/moo.conf moosoft-1.0/etc/moo.conf >
moosoft-1.0.config.patch
diff -uNr moosoft-1.0.orig/doc/ moosoft-1.0/doc >
moosoft-1.0.documentation.patch
# etc… to generate multiple patches

Copy these patches and the original moosoft-1.0.tar.gz file to ~/build/SOURCES.

The last thing to do is to prepare a recipe for rpmbuild how to build your package. Create moosoft-1.0.spec file in ~/build/SPECS:

#                  _____________________________
#         (__)    /                             \
#         (oo)   ( This is an example .spec file )
#  /-------\/  --'\_____________________________/
# / |     ||
#*  ||----||
#   ^^    ^^
####
# package preamble
# one line description of the package
Summary: Very important Unix tool
Name: moo
Version: 1.0
# this marks the internal release version: moo-1.0-1.666.x86_64.rpm
Release: 1.666
License: BSD
# group you want your package in, mostly for GUI package browsers
# some example groups used by vendors:
# http://www.rpmfind.net/linux/RPM/Groups.html
Group: Networking/Daemons
# your name for example
Packager:
#
Source: http://full.url.to.the/package/%{name}-%{version}.tar.gz
Source1: moo.init
# list all your patches here:
Patch0: moo-1.0.etc.patch
Patch1: moo-1.0.documentation.patch
# list all packages required to build this package
BuildRequires: openssl-devel
Provides:
# list all packages that conflict with this one:
Conflicts: bse
BuildRoot: %{_tmppath}/%{name}-%{version}-build

####
# full length description
%description

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc eros elit,
pretium eu vestibulum vel, blandit vel nisl. Fusce mattis volutpat
pellentesque. Etiam sit amet magna eget dui faucibus rutrum. Aliquam a ligula
erat. Proin metus tortor, sollicitudin et accumsan quis, hendrerit non leo.
Curabitur egestas neque sed nulla vulputate vel bibendum sapien tristique.
Vivamus malesuada, magna et semper iaculis, magna lorem adipiscing erat, vel
euismod enim nulla vel tortor. Vestibulum accumsan placerat sagittis. Sed
commodo pretium lectus et dignissim. Ut nec orci tellus.

#####
# this prepares a fresh build directory in ~/build/BUILD, useful macros here
# are:
# %setup - cleans any previous builds and untargzips the source
# %patch - applies patches
# any other commands here are executed as standard sh commands
%prep

%setup
%patch
%patch1

./configure --enable-something --with-something-else

#####
# this tells rpmbuild how to build your package, rpmbuild runs it as a sh
# script
%build
make

#####
# all the steps necessary to install your package into $RPM_BUILD_ROOT
# first step is to clear $RPM_BUILD_ROOT
%install
[ "$RPM_BUILD_ROOT" != "/" ] && rm -rf $RPM_BUILD_ROOT
#install all files under RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT

# now you can remove uneeded stuff
rm -f $RPM_BUILD_ROOT{_prefix}/sbin/rc.moo

#####
# NOTE: this section is optional
# commands run just before the package is installed
%pre
/usr/sbin/useradd -c "moo user" -r -s /bin/false -u 666 -d / moo 2> /dev/null
|| :

#####
# NOTE: this section is optional
# commands run before uninstalling the software
%preun
/sbin/service moo stop > /dev/null 2>&1
/sbin/chkconfig --del moo

#####
# NOTE: this section is optional
# commands run after installing the package
%post
/sbin/chkconfig -add moo
touch /var/log/moo

#####
# NOTE: this section is optional
# commands run after unistalling the package
%postun
/sbin/service moo stop
/usr/sbin/userdel moo

#####
# list all the files that are part of the package. If a file is not in the
# list rpmbuild won't put it in the package
# see below on how to automate the process of creating this list.
# some useful macros here:
# %doc filename - installs filename into /usr/share/doc/moo-1.0/
# %doc /path/to/filename - installs filename into /path/to/filename and marks
# it as being documentation
# %config /etc/config_file - similar for configuration files
# %attr(mode, user, group) file - lets you specify file attributes applied
# during installation, use - if you want to use defaults
%files
/usr/bin/moo
/usr/sbin/moomoo
# this will package the dir and all directories inside it
/example/of/a/dir
# this will package only the 'dir' directory
%dir /example/of/a/dir

#####
# document changes between package releases
%changelog
* Wed Jul 8 2009 Your Name Here
- another version of the package

* Mon Jul 6 2009 Your Name Here
- initial version of the package

To get the list of all installed files do

rpmbuild -bi ~/build/SPECS/moo-1.0.spec

this will go through prep, build and install steps of the spec file and install all files under $RPM_BUILD_ROOT, which in this case should be /var/tmp/moo-1.0-build.

ls -R1 /var/tmp/moo-1.0-build

and use the output to populate the %files section

Now the package can be build by running:

rpmbuild -ba ~/build/SPECS/moo-1.0.spec

building with modules

Sometimes it’s helpful to split one package into multiple modules. Here’s an example .spec file for a library, it will produce two packages: libmoo (with the shared objects provided by the library) and libmoo-devel (with all headers and static libraries)

#                  _____________________________
#         (__)    /                             \
#         (oo)   ( This is an example .spec file )
#  /-------\/  --'\_____________________________/
# / |     ||
#*  ||----||
#   ^^    ^^
####
# package preamble
# one line description of the package
Summary: An Example library
Name: libmoo
Version: 1.0
# this marks the internal release version: libmoo-1.0-1.666.x86_64.rpm
Release: 1.666
License: BSD
# group you want your package in, mostly for GUI package browsers
# some example groups used by vendors:
# http://www.rpmfind.net/linux/RPM/Groups.html
Group: Development/Libraries
# your name for example
Packager:
#
Source: http://full.url.to.the/package/%{name}-%{version}.tar.gz
# list all your patches here:
Patch0: moo-1.0.etc.patch
Patch1: moo-1.0.documentation.patch
# list all packages required to build this package
BuildRequires: openssl-devel
Provides:
# list all packages that conflict with this one:
Conflicts: bse
BuildRoot: %{_tmppath}/%{name}-%{version}-build

####
# changes in the preamble for libmoo-devel package
%package devel
Summary: libmoo development files and headers
Group: Development/Libraries
# -devel requires the main package
Requires: %name = %version

####
# full length description
%description

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc eros elit,
pretium eu vestibulum vel, blandit vel nisl. Fusce mattis volutpat
pellentesque. Etiam sit amet magna eget dui faucibus rutrum. Aliquam a ligula
erat. Proin metus tortor, sollicitudin et accumsan quis, hendrerit non leo.
Curabitur egestas neque sed nulla vulputate vel bibendum sapien tristique.
Vivamus malesuada, magna et semper iaculis, magna lorem adipiscing erat, vel
euismod enim nulla vel tortor. Vestibulum accumsan placerat sagittis. Sed
commodo pretium lectus et dignissim. Ut nec orci tellus.

####
# full description of the -devel package
%description devel

static libraries and headers for libmoo 

#####
# this prepares a fresh build directory in ~/build/BUILD, useful macros here
# are:
# %setup - cleans any previous builds and untargzips the source
# %patch - applies patches
# any other commands here are executed as standard sh commands
%prep

%setup
%patch
%patch1

./configure --enable-something --with-something-else

#####
# this tells rpmbuild how to build your package, rpmbuild runs it as a sh
# script
%build
make

#####
# all the steps necessary to install your package into $RPM_BUILD_ROOT
# first step is to clear $RPM_BUILD_ROOT
%install
[ "$RPM_BUILD_ROOT" != "/" ] && rm -rf $RPM_BUILD_ROOT
#install all files under RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT

# now you can remove uneeded stuff
rm -f $RPM_BUILD_ROOT{_prefix}/sbin/rc.moo

#####
# NOTE: this section is optional
# commands run just before the package is installed
%pre
/usr/sbin/useradd -c "moo user" -r -s /bin/false -u 666 -d / moo 2> /dev/null
|| :

#####
# NOTE: this section is optional
# commands run before uninstalling the software
%preun
/sbin/service moo stop > /dev/null 2>&1
/sbin/chkconfig --del moo

#####
# NOTE: this section is optional
# commands run after installing the package
%post
/sbin/ldconfi

#####
# NOTE: this section is optional
# commands run after unistalling the package
%postun
/sbin/ldconfig

#####
# list all the files that are part of the package. If a file is not in the
# list rpmbuild won't put it in the package
# see below on how to automate the process of creating this list.
# some useful macros here:
# %doc filename - installs filename into /usr/share/doc/moo-1.0/
# %doc /path/to/filename - installs filename into /path/to/filename and marks
# it as being documentation
# %config /etc/config_file - similar for configuration files
# %attr(mode, user, group) file - lets you specify file attributes applied
# during installation, use - if you want to use defaults
%files
%defattr(-,root,root,0755)
%attr(755,root,root) %_prefix/lib/lib*.so.*
%doc INSTALL ChangeLog

#####
# separate list of files for the -devel modules
%files devel
%defattr(-,root,root,0755)
%attr(755,root,root) %_prefix/lib/lib*.so
%attr(644,root,root) %_prefix/lib/*.a
#exclude *.la files
%exclude %_prefix/lib/*.la

#####
# document changes between package releases
%changelog
* Wed Jul 8 2009 Your Name
- another version of the package

* Mon Jul 6 2009 Your Name
- initial version of the package

building the package with rpmbuild -ba libmoo-1.0.spec will produce two files:

libmoo-1.0-1.666.arch.rpm and libmoo-devel-1.0-1.666.arch.rpm

Graphing Alteon configuration

admin — Mon, 31 Aug 2009 22:15:19 +0000

If you happen to use Nortel’s Alteon load balancers and need something to help you visualise your configuration here’s a script that produces graphs of connections and dependencies between groups, virtual IPs, real IPs and services in the Alteon configuration. It’s not very useful, the code is ugly, but the graphs are cool to look at Especially for huge config files with lots of services and VIPs. The script uses two great python libraries – pygraph and graphviz.

Getting all prefixes advertised by an AS

admin — Wed, 22 Jul 2009 21:33:07 +0000

This simple script displays all prefixes advertised by an AS. It uses RIPE’s looking glass to get their local BGP table and parses it for the given AS number.

For example, to get all the other networks advertised by the AS dischaos.com is in, first I need to get Layered tech’s AS number:

whois -h www.ris.ripe.net 216.32.74.94

% This is RIPE NCC’s Routing Information Service
% whois gateway to collected BGP Routing Tables
% IPv4 or IPv6 address to origin prefix match
%
% For more information visit http://www.ripe.net/ris/riswhois.html

route: 216.0.0.0/8
origin: AS3303
descr: SWISSCOM Swisscom (Switzerland) Ltd
lastupd-frst: 2009-06-08 11:31Z 192.65.185.243@rrc04
lastupd-last: 2009-06-08 11:31Z 192.65.185.243@rrc04
seen-at: rrc04
num-rispeers: 1
source: RISWHOIS

route: 216.32.0.0/14
origin: AS3561
descr: SAVVIS – Savvis
lastupd-frst: 2009-06-08 12:56Z 198.32.160.22@rrc11
lastupd-last: 2009-07-22 12:06Z 195.69.145.49@rrc03
seen-at: rrc00,rrc01,rrc03,rrc04,rrc06,rrc07,rrc10,rrc11,rrc12,rrc13,rrc15,rrc16
num-rispeers: 86
source: RISWHOIS

route: 216.32.64.0/19
origin: AS22576
descr: LAYER3-ASN – Layered Technologies, Inc.
lastupd-frst: 2009-06-08 12:56Z 198.32.160.22@rrc11
lastupd-last: 2009-07-22 14:55Z 193.232.244.147@rrc13
seen-at: rrc00,rrc01,rrc03,rrc04,rrc06,rrc07,rrc10,rrc11,rrc12,rrc13,rrc15,rrc16
num-rispeers: 85
source: RISWHOIS

Their AS number is 22576. Now lets see what other networks they advertise:

$ ./getprefixes.py 22576
networks advertised by AS22576:
64.92.160.0/20
72.21.32.0/19
72.36.128.0/17
72.232.0.0/17
72.232.128.0/19
72.232.160.0/21
72.232.168.0/22
72.232.172.0/23
72.232.174.0/23
72.232.176.0/20
72.232.192.0/18
72.233.0.0/19
72.233.28.0/22
72.233.32.0/19
72.233.64.0/18
72.233.64.0/20
72.233.80.0/23
72.233.82.0/23
72.233.84.0/23
72.233.86.0/23
72.233.88.0/21
72.233.96.0/22
72.233.100.0/22
72.233.104.0/21
72.233.112.0/21
72.233.120.0/21
72.233.127.0/24
208.95.152.0/22
209.67.208.0/20
216.32.64.0/19