Upgrading Nagios

For those of who have to look after multiple servers then you’ll understand the benefits of monitoring systems. In fact anything that will tell you of a problem before it becomes a major one has got to be a good thing!

Now whilst there’s many offerings out there in the market with solutions to fit every budget and requirements I’ve gone for Nagios. Nagios is an open source offering which really differs from others in the fact that it’s just the framework for the monitoring and you have to build or bolt on the checks on top. For me, this modular design is excellent as I know I’m starting with a blank slate and then adding only the bits I need.

Having have a Nagios box in place for well over a year now, I felt it was time to update the configuration into something a bit more manageable (covered in another post) but before I could do that I took the time to upgrade the application itself. It was an interesting process and one with a few tweaks from standard which may be applicable to others.

The upgrade of a vanilla install is pretty simple and covered here, however I’ll throw down a quick summary:

ssh onto the box and run as the local nagios account
su -l nagios

Remove the old HTML files that were being used by the gui frontend as they’re going to be upgraded.
rm /usr/local/nagios/share/{main,side,index}.html

Change into your downloads directory and download the latest source code of nagios
cd /downloads
wgetttp://osdn.dl.sourceforge.net/sourceforge/nagios/nagios-3.x.tar.gz

Extract the tarball
tar xzf nagios-3.x.tar.gz
cd nagios-3.x

Run the confguration script with the name of the group used to control the external command file permissions
./configure –with-command-group=nagcmd

Compile the source code, install the updated binaries and web interface.
make all
make install

Verify the configuration files all work with this handy command:
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Restart nagios service like thus:
/sbin/service nagios restart

Note: I found some of these stages more reliable to run as root rather than the nagios user.

At this point you’re pretty much done with the upgrade of a vanilla system. However, I use the Nuvola style to make Nagios that much easier on the eyes.

The Nuvola style sheet is pretty easy to install and tweak but does override some of the default web front end. Unfortunately when you upgrade Nagios it tramples all over these files so you have to rerun the install almost from scratch.

How to repoint Veritas Replication Exec console

Sometimes you’ve just got to move an application to another server. It’s just the way it goes. Unfortunately quite often that means you’ve got to remove / reinstall all the clients …. which can be a pain.

We recently had to migrate the “master server” (otherwise known as the RMS server) for replication exec and change the Replication Neighborhood at the same time. Personally, I really didn’t want to reinstall the console on numerous PC’s and servers so I found the registry keys that do the pointing.

To change the RMS server, alter: HKLM\SOFTWARE\VERITAS\ENL\Network.TCPIP\gateway to the new server name or IP address. You’ll have to restart the Veritas service for this to kick in but it’s a 2 second job.

To change the Replication Neighborhood go to KLM\SOFTWARE\VERITAS\ENL\System\netgroup and alter that to the new name. No service restart required for this one.

Job done.

Data Replication Script

If, like me, you’re concerned about your data and like to have it multiple places then I’m sure you’ll have looked into methods of copying, replicating and synchronising this data about the shop. Now there are many methods to get this data a’movin and as many different cost levels as you can think of.

At my place of work we use an application by Veritas called Replication Exec and whilst it does the job, it does have some limitations IMO….. one of which I ran into today.

[fade out]

Admiral: Ok, son. We’ve got a major problem and we’re fresh out of ideas! WE need to replicate the critical data from the Eastern domain to the Central domain and we need it to work now!!

Capt Graycat: No problem, sir. Nothing I can’t accomplish with enough firepower.

Admiral: I admire your guts, Captain, but all our normal methods have failed. Replication Exec is useless and dead in the water. Surely you don’t think you can beat a powerhouse like that at it’s own game?!

Capt Graycat: Let me at it, sir! And don’t call me Shirley.

Capt Graycat ripples of a heroic salute and steps out into the night to take on the challenge. Over his shoulder he calls “smoke me a kipper, I’ll be back for breakfast” and disappears into the glum

Admiral’s sexy PA: What a guy! *swoon*

[fade back into real life]

Get all of that? Reread it if needed as there might be questions later 🙂

So what to do? Primary objective, sorry I meant the main thing is to get the data copied and that’ll mean a differential copy. Problems we’re going to run into are two separate domains that don’t trust each other properly (don’t ask), a lot of data to copy and get some value added features out of it because we’re that good.

My plan is to get the script to go like this:

1. Make a working directory for the logs
2. replicate the data using robocopy and various settings and log it into the “working” directory
3. move the log to a dated folder and rename it to today’s date
4. email it to various people
5. clean up afterwards

Easy, huh?

ok, I’ll not discuss the making of folders or moving files about bit as that’s been done to death elsewhere.

Oh sod it, here’s the code –


REM   DR_Data_Replication.bat
REM
REM   Mirrors Server1 share contents from SOURCE_SERVER to TARGET_SERVER.
REM   Creates monthly directory and daily log then e-mails to address(es)
REM
REM   Written by - Graycat
REM

REM Creating Log Folder
md c:\#Scripts\Logs\Working\

REM Starting Replication …..

REM Replicating “Server1” into “Data_Share” on TARGET_SERVER
Robocopy e:\Server1 \\TARGET_SERVER\Data_Share /MIR /XN /XF AUTORUN.INF /LOG+:c:\#Scripts\Logs\Working\Working.txt /R:3 /rh:2200-0400 /pf

REM Move and rename log
move c:\#Scripts\Logs\Working\Working.txt c:\#Scripts\Logs\%date:~-4,4%\%date:~-7,2%\%date:~-4,4%%date:~-7,2%%date:~0,2%.txt

REM E-mailing Log file
echo DR data replication from SOURCE_SERVER to TARGET_SERVER logs attached. > body.txt
c:\#scripts\mpack.exe -s “DR Replication” -d body.txt -c application/exe -o body.msg c:\#Scripts\Logs\%date:~-4,4%\%date:~-7,2%\%date:~-4,4%%date:~-7,2%%date:~0,2%.txt
c:\#Scripts\bmail.exe -s 192.1.1.245 -f Robocopy@mydomain.com -t me@mydomain.com -h -m body.msg

REM Clean up
del c:\#Scripts\body.msg
del c:\#Scripts\body.txt

Exit

hopefully most of that follows on from what I was after in the script flow. Robocopy is in the Windows resource kit I believe but easily downloaded from MS. The mpack is for encoding MIME e-mail messages so that they can be sent with an attachment. It’s a cracking little free application and can be found over here along with bmail which is what was used to actually send the e-mail.

Ok, the robocopy switches might be of interest so here goes –

• MIR – mirror directory
• /XN – exclude newer files
• /XF AUTORUN.INF – exclude files called autorun.inf
• /LOG+:c:\#Scripts\Logs\Working\Working.txt – log to the end of working.txt log file
• /R:3 – retry three times if it fails
• /rh:2200-0400 – allow to run between 2200 and 0400
• /pf – check the allowed run time for each file as it’s done. this’ll stop it running after 0400

on test runs this worked great …. apart from robocopy would pause rather than terminate when it hit the end of its allowed run time.
To get round this, I added another script to run at 0400 that used Taskkill /IM robocopy /f to kill any process called robocopy. This would then allow the rest of the script to run through nicely.

Right, that’s me for today. Time to go home. I’ll report back in a few days as to how it went.

Where’s my MSCONFIG gone?!

[dodgey sci-fi voice over]

There exists and application.

So powerful that it keeps the load-up in line.

It has existed since the dawn of XP.

But now it has …… gone!

[/dodgey voice over]

Ok, it’s nothing quite that amazing or worthy of it’s own show with a special effects budget but it is very useful.

MSCONFIG is an application in Windows XP that is used to troubleshoot the start-up process of a machine. With it you can see what is starting up when, what applications are launching in the background, what services are set to start or not and a few other things.

The most common use I’ve seen for it is to simply speed up the boot process by cutting down what is launched in the background.

When you launch this from the command line (type msonfig and hit go) you’ll be presented with a number of tabs. These are:

• General – gives you options on what mode to start Windows in (safe mode etc)
• System.ini – is where old versions of Windows lists system files to be loaded, such as drivers and fonts
• Win.ini – is pretty much redundant nowadays but pertains to old versions of Windows
• Boot.ini – provides some extra boot options
• Services – controls the services
• Startup – deals with what applications are launched at start-up.

There’s also a “Tools” tab but that’s only in Vista so I’m not going to cover it here. Oh, whilst msconfig is native in XP, you can copy it over to a 2000 Pro machine and it’ll run fine from there. handy if you’ve got a mixed environment.

Now more often than not this little app is just there and saves a lot of time and effort. However, on the odd occasion simply typing msconfig in the run box will produce the dreaded “command not recognised” error message.

But worry not! It is still there, it’s just not been registered. To quickly find it you need to go to “c:\windows\pchealth\helpctr\binaries” and low, you will find what you seek.

Personally, if a machine slides sufficiently that I need to use msconfig and its not there then it instantly jumps up the queue for being rebuilt ala the ghosting process. However, if that’s not possible and you want to fix it so you can just type msconfig at the root command line you can correct the issue in the registry. The path declared under HKLM\sotware\microsoft\windows\currentversion\app paths\msconfig\

Deploying Visio 2007

……. or how to force Visio 2007 to do deploy kind of like 2003.

Now if you’re new to the Office 2007 side of things you may not be aware that the deployment side of things has completely changed from the 2003 release ……. and it’s not for the better in my opinion.

With the previous version you would make an admin install point on a share and then create the GPO and apply a msp to customise the settings and / or parts you want deploying. Using this method you could have one install point and use msp’s to deploy different versions. For example, you can make an admin install point using Office 2003 Professional and then use msp’s to cut the installed applications down to just those that are needed for each of the other versions such as Basic, Small Business, Standard etc.

This was the approved and recommended method for deploying the Office suite of applications but no more. Microsoft are now pushing people towards using their System Management Server (SMS) or
System Center Configuration Manager 2007 as it is now called to roll out software and as such have seriously crippled it by removing the ability to use msp’s and customise your install.

That’s the bad side of things but if you think about it, how much do you actually customise your Office installs?

One improvement with 2007 is that the admin install point is created by just copying the contents of the CD on to a share. Unfortunately when you combine this with the lack of support for the msp transforms, it means you’re going to have to have a copy of the CD for each version you’re going to deploy. Annoying but not overwhelmingly difficult.

The only way to customise the install is using the config.xml file located in the same directory as the msi files. About the only useful things you can put in there are install location, software key, licensed user and company. All useful things but that’s as far as it goes really.

My basic process for rolling out the 2007 office applications is this:

1. Copy the CD up on to the deployment share (for us this is a DFS site that is replicated round the company)
2. Configure the config.xml file with the required info
3. Create a GPO with the msi
4. Apply as required and filter as needed

Pretty simple, huh? I’ll let you know how it goes with our roll out 😉

Generating an event log with a batch script

It’s not something I’ve ever really had to do before but as part of some troubleshooting today I needed to prove a script was not only running but connecting to a remote executable. My plan was to add a local entry into the event log and then echo the username, date and time to a centralised log file alongside the remote executable.

The first part was remarkably easy to achieve! Unbeknownst to me there is an executable under %windir%\system32 called eventcreate.exe …. which does exactly what is says on the tin, namely create an event in the event log. A few switches are all you need to specify event type (error, info, warning), ID number (anything from 1 – 1000) and the information included.

Here’s what I went with:

%windir%\system32\eventcreate.exe /T INFORMATION /ID 666 /D “SCRIPT RAN LOCALLY”

Told you it was easy!

Slapping the username, date and time into the remote log file was just a case of using the following:

echo %username% %date% %time% >> \\remote\log\file.txt

And now hopefully there’s going to be a local event and a remote log file to help me sort this darn issue out!

Only time will tell but I’ll keep in touch.