Getting Started with Bro Intrusion Detection System (IDS)

If you have a computer network then you need to ensure an intrusion detection system (IDS) is a part of your cybersecurity strategy. The value of monitoring the traffic on your network far outweighs the cost of a breach. Although most IDS systems are commercial, there are a few open-source IDS solutions.

Snort and Suricata are popular open-source firewall/IDS solutions, but come with a few drawbacks. For a small operation they may work well, but for medium or larger networks they can bring more work and less value. Their key drawback at this time is that Snort/Suricata-capable devices do not communicate with other capable devices on the network, nor are they centrally managed. With cyberattacks becoming more sophisticated, a security-conscious organization needs a better solution.

There is a third major player in the open-source IDS game. The Bro Network Security Monitor, developed originally by higher education, provides both a network protocol analyzer and a security tool. It’s strength is the ability to correlate traffic across multiple Bro devices on a network, and add additional and customizable plugins. In other words, instead of having multiple independent IDS boxes on your network, you could have a single clustered system that correlates information across the network.

The drawback to Bro is advanced and can take some patience to setup for the beginner. This introduction guide is meant to help you by taking out some of the pain and outlining the steps you can take to build your own Bro cluster. Keep in mind that some of your steps may be different depending on your operating system and setup.

System Specs:

OS: Red Hat Linux 7

Manager Server: 8 CPU, 32 GB RAM, 1 TB hard drive with a large log partition.

Workers: 12 physical hyper-threaded CPUs, 96GB RAM, 120 GB SSD, 10 Gbps Intel X710 NIC

Network Information: The network I’ll be testing this out on has over 1Gbps total traffic (Tx and Rx), hence the 10 Gbps NIC in the workers. It is a typical network that is heavy on web browsing, but uses a lot of other apps as well. As seen in typical networks, it also has microbursts as well.

Big Picture

First I’ll tell you how I installed the prerequisites for Bro and prepped the operating system. Next, I’ll compile Bro and install it along with some needed Bro plugins. Third, I’ll walk through troubleshooting, tuning, and adjustments. Last, I talk about integration into Elasticsearch. The logical and physical diagrams show what the ending architecture looks like at the end of this series.

Logical Diagram

Physical Diagram

Bro Prerequisites

First, I recommend having a separate partition dedicated to the logs and files Bro produces. A sysadmin can partition a disk accordingly, or give you some attached storage. You don’t have to have this, but just know that if your logs fill up your root partition then Bro will crash and the operating system becomes unstable.

In order to install Bro, one needs to install all the prerequisites. This included the following:

• C++ Actor Framework
• LibGeoIP Database
• Gperftools
• Ipsumdump
• PF_RING (needed for network throughput over 1 Gbps)
• And a few others

The Few Others:

There are a few packages that can be installed via yum. Just run the following command from the CLI:

sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python-devel swig zlib-devel

Check for errors in the output. If yum can’t find a package, you may need to enable an additional repository: yum-config-manager –enable rhel-7-server-optional-rpms

C++ Actor Framework

This is also straightforward if you know where to look.  Download the source from https://build.opensuse.org/package/show/devel:libraries:caf/caf.

make
make install

LibGeoIP

Bro uses LibGeoIP for locating geographically IP addresses. If you’re going to send your logs to Elasticsearch or Splunk later, you could probably skip this step and let those take care of the GeoIP features. If you want Bro to do it, keep in mind that this must be installed consistently across all workers and manager in your cluster, and updated regularly.

To install, run

sudo yum install GeoIP-devel

To provide more precise locations such as by city, run these commands to download and install the databases.

wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
gunzip GeoLiteCity.dat.gz
#Rename the file and move to /var/lib/GeoIP:
mv GeoLiteCity.dat /var/lib/GeoIPCity.dat
#*note* - alternative path.
mv GeoLiteCity.dat /usr/share/GeoIP/GeoIPCity.dat
#Now do the same for IPV6 addresses.
wget http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz
gunzip GeoIPv6.dat.gz
mv GeoIPv6.dat /var/lib/GeoIPv6.dat
#*note* - alternative path.
mv GeoIPv6.dat /usr/share/GeoIP/GeoIPCityv6.dat

Ipsumdump

IPsumdump can be “summarizes TCP/IP dump files into a self-describing ASCII format easily readable by humans and programs.”

Download the latest version from http://read.seas.harvard.edu/~kohler/ipsumdump/

gunzip ipsumdump-1.86.tar.gz
tar -xvf ipsumdump-1.86.tar
cd ipsumdump-1.86/
./configure
make
sudo make install

gperftools

Note: gperftools-devel doesn’t necessarily come with the standard repositories. You have to find another rpm for it. Just make sure your OS version and the repository version matches up.

yum install gperftools-lib

The previous command installs the library, but you still need the development package.

wget http://mirror.centos.org/centos/7/os/x86_64/Packages/gperftools-devel-2.4-8.el7.x86_64.rpm
rpm -ivh gperftools-devel-2.4-8.el7.x86_64.rpm

PF_RING

The default Linux drivers are a poor choice when it comes to network throughput over 1 Gbps. To take full advantage of a 10 Gbps NIC, a third-party driver is required. If your workers are going to be processing less than 1Gbps, you can probably skip down to the section “Compiling Bro” later in this guide.

Ntop provides a free driver called PF_RING Vanilla, and a paid version called PF_RING-ZC. ZC supposedly provides better performance as it doesn’t copy packets as the default Kernel or PF_RING Vanilla does. Other alternatives include AF_PACKET, NetMap, and Myricom. The latter works with Myricom NICs while the rest work mostly with Intel NICs. A comparison of each one is not available at this time. For the time being, I’ll focus on PF_RING Vanilla as it is somewhat popular.

Just like the other prerequisites, PF_Ring libraries need to be installed in a directory consistent across all workers and manager. If PF_Ring libraries are already installed in the default installation location (usually /usr/local), remove it before proceeding. When compiling Bro, the compiler might find the default instance instead of the one you specify in the configure command.

The PF_Ring version may be different, and you might opt to choose a more generic name. Also, although the directory for the driver has “ZC” in its name, you won’t actually be using the ZC features unless you have a license and set it up properly.

I like to include the version number of PF_Ring in the name of the directory. Although I started with version 6.5, version 6.6 was released before publishing this article.

I am using the i40e driver, which has less versatility than other drivers. I hear the ixgbe, although possibly older, allows for greater flexibility and adjustment on the settings such as the ring slots.

#Install git and download latest PF_RING driver
yum install git
git clone https://github.com/ntop/PF_RING

#Compile and install libraries
cd ./PF_RING/userland/lib
./configure prefix=/opt/pf_ring-6.5.0
make
make install

#Compile and install pcap libraries
cd ../libpcap
./configure prefix=/opt/pf_ring-6.5.0
make
make install

#Compile and install kernel module
cd ../../../PF_RING/kernel
make
make install

#Compile driver
cd ../../PF_RING/drivers/intel/i40e/i40e-1.5.18-zc/src
make

The script that loads the driver needs to be modified to optimize the NIC and OS settings.

vim load_driver.sh

Since this NIC will only be used for receiving traffic and not transmitting packets, I modified the line insmod ../../../../../kernel/pf_ring.ko to

insmod ../../../../../kernel/pf_ring.ko enable_tx_capture=0

Since Red Hat doesn’t have a “killall” command, I replaced the “killall irqbalance” line with

pkill irqbalance

“NIC queues” specifies how many CPUs you want the NIC to send interrupts to. In the case of the i40e driver, this is specified in the ethtool -L $IF combined <number> setting. Since my machine has 24 logical cores, and I like to keep one core free for other management processes, I modify this line to say

Ethtool -L $IF combined 23

Other settings I modify:

ethtool -K $IF rxvlan off txvlan off

This next line to modify is extremely important for Bro’s performance. Bro will calculate hashes, and if it receives hashes from the NIC it goes a little crazy. We turn these settings off of the NIC by the line:

ethtool -K $IF rxhash off tx off rx off

The last line I modified is the HUGEPAGES_NUM variable from 512 to 1024. It seems no matter the number I chose, the OS will only allocate half of the requested pages.

After saving the script and exiting VIM, I load the driver by running

./load_driver.sh

Verify your NIC settings by running the following ethtool commands:

ethtool -k em1
ethtool -c em1
ethtool -G em1

PF_RING comes with some basic tools to use with the driver. Although these are not necessary for Bro, they can be helpful when troubleshooting. Compile PF_RING/userland/examples by running make. An easy testing tool is pfcount, which gives you stats about the NIC including throughput and how many packets were dropped on the NIC before being read.

./pfcount -I em1

If you see no dropped packets and throughput, you should be good to go. Optimizing these settings for your particular OS, NIC, and driver can take time and patience, and some trial-and-error.

Operating System Preparation

I need to prepare the operating system to communicate with the other workers in the cluster. This includes adjusting iptables (or firewall settings for your system), and distributing SSH keys. I won’t go into specifics on these settings as they are highly specific to your system setup, and there are good guides already. I will note on the IP tables that it might be hard to anticipate which ports the Bro manager and workers need open to each other. It may be easier at first to open up all ports to each other, then later when you have things working to lock down which ports are actually open. For example, this iptable rule opens up a port range on the typically used by Bro on the manager.

iptables -A IN_public_allow -s 10.1.0.10 -p tcp --dport 47760:47771 -j ACCEPT

As a note, the bro manager process uses port 22 by default to initiate connections and manage a worker in general. The workers use higher port numbers to connect back over SSH to the manager, so the manager needs to have higher port numbers open to the workers. Otherwise you will only get one-way communication.

Adjusting OS Buffers

Linux allocates some memory (aka buffers) to store packets as they wait to be read by a process. The default settings of most Linux distributions is not adequate. Typically what happens is the network traffic can be have large bursts. Another reason to have larger buffers is because the machine for short instances might at time not be able to keep up with the traffic coming in. This could be due to some types of analysis requiring more time. I like to have enough memory to receive the short bursts and give me some wiggle room on processing.

Following the Method #2 on http://dak1n1.com/blog/7-performance-tuning-intel-10gbe/ is a great start to adjust these settings. The intro addresses irqbalance which we already address when loading the PF_Ring driver. It also covers cpuspeed, but I haven’t seen any difference with it. Method #1 is for transmitting data, which I’m not doing so I don’t need to worry about. Method #3 is for squeezing the last bit of juice out of this, but again I haven’t seen amazing results from it.

This completes the prerequisites needed before installing Bro IDS. This is part 1 of 4 in the guide. Continue on to Part 2, Compiling and Installing Bro.