Introduction

"NSCA is a Linux/Unix daemon allows you to integrate passive alerts and checks from remote machines and applications with Nagios. Useful for processing security alerts, as well as redundant and distributed Nagios setups." (Source: http://exchange.nagios.org/directory/Addons/Passive-Checks/NSCA--2D-Nagios-Service-Check-Acceptor/details)

Active checks are initiated by the Nagios server.

  • Local checks are executed on the Nagios server:
    • to check the localhost (i.e. the Nagios server).
    • to use local processes to check remote servers and services (e.g. through HTTP).
  • Remote checks are initiated from the Nagios server through NRPE:
    • NRPE sends a check request to the client,
    • the client executes the check command and sends the results to the Nagios server through NRPE,
    • the server receives and processes the results.

Passive checks are initiated by the client; the Nagios server waits for the result passively. It can be configured to warn if it doesn't receive any results in a timely manner; if, for example, you expect a message from the client every 24 hours, it can raise an alert if no message was sent after 28 hours.

We use NSCA for a number or purposes:

  • Check if Nagios works properly on our customers servers behind their firewalls.
  • Check results of jobs (e.g. backups).
  • Integrate our own log management.

This transcript covers the following activities:

  • NSCA installation on the Nagios server.
  • NSCA installation on a client (in this case a Nagios server behind a company firewall).
  • Set up the client server to send "alive messages" to the Nagios server (in this case to prove that the remote Nagios instance is running - a sort of "Dead man's switch").

References

http://nagios.sourceforge.net/docs/3_0/passivechecks.html

http://nagios.sourceforge.net/docs/3_0/extcommands.html

http://nagios.sourceforge.net/docs/3_0/freshness.html

http://exchange.nagios.org/directory/Addons/Passive-Checks/NSCA--2D-Nagios-Service-Check-Acceptor

http://terminalinflection.com/nagios-alert-forwarding-nsca/

http://repoforge.org

Prerequisites

Nagios and Nagios Client must be installed.

The following actions have to be run as root both on the client and on the server.

Download and Install the RepoForge Repository

In the previous transcripts described here, I installed all modules from the sources. For NCSA, however, I used the RepoForge repository to install the packages using "yum".

The RepoForge is based on a different folder layout, but that can be coped with easily.

Please check http://pkgs.repoforge.org/rpmforge-release for the latest version of the package and modify the command below as required.

rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

Include the RepoForge Repository in the yum Path

It must be made sure that the additional repository are searched after the standard CentOS repositories. This is accomplished by setting an appropriate priority for all repositories where the "enabled" flag is missing or set to "1":

vi /etc/yum.repos.d/CentOS-Base.repo
### locate the [base] section and add: ###
priority=1
### repeat the same for the [updates] and [extras] sections. ###
### and for all sections where "enabled = 1". ###

The result should look like:

...
[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
priority=1
#released updates
[updates]
name=CentOS-$releasever - Updates
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
priority=1
#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
priority=1
...

Now set the priority for the new repository to 5:

vi /etc/yum.repos.d/rpmforge.repo
### locate the [rpmforge] section and add: ###
priority=5
### repeat the same for all sections where "enabled = 1". ###

The result should look like this:

...
[rpmforge]
name = RHEL $releasever - RPMforge.net - dag
baseurl = http://apt.sw.be/redhat/el6/en/$basearch/rpmforge
mirrorlist = http://mirrorlist.repoforge.org/el6/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1
priority=5
...

As a last step you should delete the cache for the Yum's Fastestmirror plugin:

rm -f /var/cache/yum/timedhosts.txt

It will be renewed automatically next time you use "yum install".

Install and Configure NSCA Server

Installation

yum install nagios-nsca nagios-nsca-client
chown nagios:nagios /etc/nagios/send_nsca.cfg /etc/nagios/nsca.cfg
chmod 600 /etc/nagios/send_nsca.cfg /etc/nagios/nsca.cfg

Base Configuration

Before you start the configuration, you must choose a password and an encryption method. They have to be set identically on the server and all clients.

The server configuration file /etc/nagios/nsca.cfg references "/var/nagios/rw". If you followed the installation steps in this How-To section, this path must be changed to "/usr/local/nagios/var/rw/".

vi /etc/nagios/nsca.cfg
### change the paths ###
:%s_/var/nagios/rw_/usr/local/nagios/var/rw_g
### modify the following entries: ###
password=some_password
decryption_method=2

Now we have to set the password and the encryption method in the client configuration file /etc/nagios/send_nsca.cfg.

vi /etc/nagios/send_nsca.cfg 
### modify the following entries: ###
password=some_password
encryption_method=2

Automatic Startup of NSCA Server

chkconfig nsca on
service nsca start

Modify /etc/services

vi /etc/services
### insert the following lines before description of port 5671 ###
nrpe            5666/tcp                # NRPE
nsca            5667/tcp                # NSCA

Firewall Configuration

You must open the firewall for port 5671. I recommend to use one entry for each NSCA client:

vi /etc/sysconfig/iptables
### insert: ###
-A INPUT -m state --state NEW -m tcp -p tcp -s my-client.my-domain.tld --dport 5667 -j ACCEPT

Now restart the firewall:

service iptables restart

Nagios Configuration

Base Configuration (nagios.dfg)

I suggest to modify nagios.cfg in two steps:

  • At a minimum, "check_host_freshness" must be anabled.
  • As soon as the system is running as expected, you should disable logging for passive checks and external commands; otherwise the Nagios log will be cluttered with more or less meaningless output.
Enable check_host_freshness
vi /usr/local/nagios/etc/nagios.cfg
### locate "check_host_freshness" and set it to "1": ###
check_host_freshness=1
Disable excessive logging
vi /usr/local/nagios/etc/nagios.cfg
### locate and set the following entries: ###
log_external_commands=0
log_passive_checks=0

Now Nagios must be restarted:

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
### if ok: ###
service nagios restart

Add a Command to Handle NSCA Timeouts

If external commands are not sent from the client in a timely fashion, and if "check_host_freshness" has been enabled, the service can be set up to run an active check. This requires a command to be defined - in our case in "commands.cfg".

vi /usr/local/nagios/etc/objects/commands.cfg
### insert ###
define command {
        command_name    nsca-timeout
        command_line    $USER1$/check_dummy 2 "No reply from NSCA client"
        }

Create an NSCA Service Template

A service must be set up to handle incoming external commands with

  • active_checks_enabled set to 0.
  • passive_checks_enabled set to 1.
  • check_freshness set to 1.
  • freshness_threshold set to the timeout you need.
  • check_command set to the new command "nsca-timeout".

For this purpose I added the following entry to templates.cfg:

vi /usr/local/nagios/etc/objects/templates.cfg
define service {
        name                            gutzmann-service-nagios-alive
        use                             gutzmann-service-critical
        service_description             Nagios Alive
        active_checks_enabled           0
        passive_checks_enabled          1
        check_freshness                 1
        freshness_threshold             60
        check_command                   nsca-timeout
        register                        0
        }

Create/Modify Host and Service Entries

We create one file per Nagios client in /usr/local/nagios/etc/objects/hosts/ which contains the host and service definitions. Here you see an example for a remote Nagios host with "passive_checks_enabled" set to "1":

vi /usr/local/nagios/etc/objects/hosts/ks-monitor-1.cfg 
define host {
        use                     gutzmann-host-critical
        host_name               ks-monitor-1
        alias                   monitor-1@k+s
        hostgroups              gutzmann-servers-critical
        address                 xxx.xxx.xxx.xxx
        passive_checks_enabled  1
}
define service {
        use                     gutzmann-service-nagios-alive
        host_name               ks-monitor-1
        }

Restart Nagios

Now Nagios must be restarted again. As the remote client has not been set up yet, you will receive an alert after the timeout period has been reached (in our case after about three minutes).

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
### if ok: ###
service nagios restart

Install and Configure NSCA Client

The first step is the base installation and configuration of NSCA. In a second step you must create either create a job to send messages to the NSCA server (in our case it checks if Nagios is running), or add the message handling to an existing job (e.g. a backup job).

Installation

See "Prerequisites" for the installation of the RepoForge repository.

yum install nagios-nsca-client
chown nagios:nagios /etc/nagios/send_nsca.cfg
chmod 600 /etc/nagios/send_nsca.cfg

Base Configuration

Now we have to set the password and the encryption method in the client configuration file /etc/nagios/send_nsca.cfg as on the server:

vi /etc/nagios/send_nsca.cfg 
### modify the following entries: ###
password=some_password
encryption_method=2

Initial Test

You can manually send a message to the server. In our case this would look like this:

echo "ks-monitor-1;Nagios Alive;0;Running" | send_nsca monitor-a.gutzmann.com -d ";" -c /etc/nagios/send_nsca.cfg

It refers to "host_name" and "service_description" as defined on the Nagios server (monitor-a.gutzmann.com). "0" stands for the numeric status code (0 = "OK"). The message "Running" will be displayed as status information.

If you refresh the Nagios Services display, the service status should change from "CRITICAL" to "OK". If logging is still enabled, you can trace the activities in the Nagios log on the server:

tail -f /usr/local/nagios/var/nagios.log

Set Up a Job to Send NSCA Messages

Now we have to create a job to check if Nagios is up. This can be done with the Nagios plugin "check_procs". In our case, this is a simple Perl script:

check-nagios-alive.pl
#!/usr/bin/perl
        $vHostName = "ks-monitor-1";
        $vServiceDescription = "Nagios Alive";
        $vNagiosServer = "monitor-a.gutzmann.com";
        $vResult = `/usr/local/nagios/libexec/check_procs -C nagios -c 1:`;
        $vStatus = $?;
        if ($vStatus == 0) {
                $vCode = "0";
        } else {
                $vCode = "2";
        }
        $vMessage = "$vHostName\t$vServiceDescription\t$vCode\t$vResult\n";
        open(SEND,"|/usr/sbin/send_nsca $vNagiosServer -c /etc/nagios/send_nsca.cfg") || die "Could not run $send_nsca: $!\n";
        print SEND $vMessage;
        close SEND;
        printf $vMessage;

This script must be made executable:

chmod 700 check-nagios-alive.pl

In the last step, a cron job has to be created:

crontab -e
* * * * * /root/scripts/check-nagios-alive.pl > /tmp/check-nagios-alive.log 2>&1
  • No labels