syslog

RHEL6: All Up in Your Face with Auditd

/ Christopher Paquin

Strongbad2kqAuditd is the userland piece of the RHEL audit tool suite. When its up and running, audit messages sent by the kenel will be send to log files that you have configured. By default, only a small and limited number of messages will be picked up by Auditd; these are mostly messages related to authentication and authorization.

Auditd has three main config files as shown below

  • /etc/sysconfig/auditd – basic configuration options
  • /etc/audit/auditd.conf – main config file
  • /etc/audit/audit.rules – auditing rules

Auditd and Syslog:

Its possible to send audit messages to a syslog. By setting active=yes in /etc/audisp/plugins.d/syslog.conf you can send all your audit messages to syslog. If your system is setup to log to a remote syslog server, then your audit messages will go along for the ride as well. Note that you can also send audit messages to a remote logging server via native audit protocol over TCP. I am not going to go into this option, but I want to make sure that we are aware that it exists.

 

Looking for Audit Events in All the Wrong Places:

Auditd includes a handy-dandy tool for searching audit logs. Ausearch. You can check out all your current audit log messages using the command below.

[root@ip-172-31-21-28 ~]# ausearch -l

Viewing audit logs in their raw format can be accomplished with the command below

[root@ip-172-31-21-28 ~]# ausearch –raw

The -a option allows you to search by audit event ids

[root@ip-172-31-21-28 ~]# ausearch -a 282

Auditd also includes ausearch, which allows you to get a quick summary of audit events, rather than trying to view massive audit logs. Usage and output shown below.

root@ip-172-31-21-28 ~]# aureport

Summary Report
======================
Range of time in logs: 07/17/2014 10:21:36.438 – 07/17/2014 19:52:49.556
Selected time for report: 07/17/2014 10:21:36 – 07/17/2014 19:52:49.556
Number of changes in configuration: 4
Number of changes to accounts, groups, or roles: 24
Number of logins: 20
Number of failed logins: 4
Number of authentications: 75
Number of failed authentications: 3
Number of users: 3
Number of terminals: 18
Number of host names: 19
Number of executables: 14
Number of files: 0
Number of AVC's: 10
Number of MAC events: 20
Number of failed syscalls: 10
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 244
Number of keys: 0
Number of process IDs: 203
Number of events: 1132

You can also use aureport and ausearch together. Simliar to the powerfull partnership between Batman and Robin, these two tools complement each other in ways that you can only imagine. Check out my sexy bits below.

[root@ip-172-31-21-28 ~]# ausearch –start today –raw | aureport

Summary Report
======================
Range of time in logs: 07/17/2014 10:21:36.438 – 07/17/2014 20:01:01.911
Selected time for report: 07/17/2014 10:21:36 – 07/17/2014 20:01:01.911
Number of changes in configuration: 4
Number of changes to accounts, groups, or roles: 24
Number of logins: 20
Number of failed logins: 4
Number of authentications: 75
Number of failed authentications: 3
Number of users: 3
Number of terminals: 18
Number of host names: 19
Number of executables: 14
Number of files: 0
Number of AVC's: 10
Number of MAC events: 20
Number of failed syscalls: 10
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 244
Number of keys: 0
Number of process IDs: 205
Number of events: 1144

Want to know another cool tool that is part of auditd? I know, its a lot to take in at one time, but I am sure that you can handle it. Using autrace you can trace and investigate system calls made by a process. 

Want to see everything that nslookup is doing? Then run the command below.

[root@ip-172-31-21-28 ~]# autrace /usr/bin/nslookup google.com

This will output a pid for you to trace with ausearch.

Trace complete. You can locate the records with 'ausearch -i -p 3359'

 

Related articles

RHEL6- Getting Up Close and Personal With Rsyslog
Linux audit files to see who made changes to a file

RHEL6: Using Advanced Log File Filtering in Rsyslog

/ Christopher Paquin

100-8591So by default when you forward logs to a syslog/rsyslog server all the logs end up in the same file (ususally configured to go to the messages file). Sometimes one may prefer to forward logs from a particular server to a separate logfile. I know for a fact that my sometimes friends in our info-sec group prefers this setup.

While managing such a setup for more than a handful of hosts would probably be a nightmare, its not actaually hard to setup on the most basic level.

First create a custom filters.conf (or whatever you want to name it) in your /etc/rsyslog.d directory. Below is the file that I created.

[root@ip-172-31-21-28 rsyslog.d]# cat filters.conf
:fromhost, isequal, "ip-172-31-25-104.ec2.internal" /var/log/ip-172-31-25-104.ec2.internal/messages
:fromhost, isequal, "ip-172-31-25-104.ec2.internal" ~

Note that you will need to include two lines in your file for each host, one with the specific file/location that you want to send your filtered log messages, and a second line that directs rsyslog to discard any messages from the specified hosts once they have been logged to the specified locations. This keeps these messages from ending up in the messages file as well as the file defined in your filter file.

 

Related articles

RHEL6 – Introduction to IPtables, Part II
HomeLab: Basic Syslog Configuration on Cisco Catalyst Devices
RHEL6- Getting Up Close and Personal With Rsyslog

Sudosh Make Me a Sandwich: Install and Configure Sudosh in Solaris 9/10

/ Christopher Paquin

SandwichWelcome to the way-back machine again. This time we are going to take a look at installing and configuring sudosh on Solaris with the purpose of using sudosh as a replacement for sudo (at least sudo to root). Using sudosh in this manner allows us to still allow users to become root, but also allows us to track each and every command that they run once they become root.

Note that there are a couple of ways to install sudosh: from source or from a package. I chose installing from a native solaris package for a couple of reasons.

  1. I found a package
  2. I would need a C compliler otherwise to compile it
  3. I had many systems on which to install sudosh

So the first think that I did was to install the package. Below I have dropped the package in /tmp and have gunzipped it.

# pkgadd -d ./sudosh2-1.0.2-sol10-sparc-local

Now, lets not be stupid, lets back up the sudoers file first.

# cd /usr/local/etc/

# cp sudoers sudoers.20140114

Now we can run visudo and modify the current sudoers file. First you should add the command alias below as it keeps things neat and simple.

Cmnd_Alias      SUDOSH=/usr/local/bin/sudosh

Now lets referece the alias that we created above in the User Privilege Section. Rinse and repeat for any user or group that is allowed to become root.

systems ALL=(ALL)       SUDOSH

Now we are going to make a modification to /usr/local/etc/sudosh.conf. Note that this step is not required. I am changing the default syslog facility used so that sudosh will log using the same facility as regular auth events.

syslog.priority         = LOG_INFO
syslog.facility         = LOG_AUTH

Now I am modifying my syslog.conf to ensure that all log messages of DEBUG level or higher are forwarded to my remote syslog hosts. This includes my sudosh events which have a priority of INFO. Note that spacing is very important in the solaris syslog.conf

#loghost
*.debug @loghost1
*.debug @loghost2

Since this is Solaris 10, we can restart syslog via svcadm

# svcadm restart svc:/system/system-log:default

 

FYI – Relevant xkcd here

 

Related articles

Sudo
Preparing a SUN/ORACLE Solaris server for SAN access – SAN step 4
Sun First Ever UltraSPARC-based Laptop

HomeLab: Basic Syslog Configuration on Cisco Catalyst Devices

/ Christopher Paquin / Leave a comment

FrontiervilleblueoxIn my homelab setup I am dumping syslog on all my devices to my Linux desktop. Have not figure out what I am going to do with it yet, but I see myself either setting up Splunk or Greylog in the near future. Note, a while back I wrote a post on how to configure rsyslog on RHEL 6 – s0 if you are interested you can find that post here.

So lets get down to brass tacks and configure some freaking syslog.

In this instance we are configuring syslog redirection on a Cisco 3548xl switch. Note we are in configure terminal mode.

First we must tell our device to insert timestamps on

s-3550-1(config)#service timestamps log datetime

Now we tell the device where to send the syslog messages

s-3550-1(config)#logging 192.168.0.195

Now we tell the device which log levels to send to the syslog server. In this instance I am sending warning level messages and above. This is pretty verbose, but its a home lab so I am not worried about a slew of log messages pounding my syslog server.

s-3550-1(config)#logging trap warning

For reference I am including the logging levels below.

Emergency: 0

Alert: 1

Critical: 2

Error: 3

Warning: 4

Notice: 5

Informational: 6

Debug: 7

Now lets review what we have done with the show logging command

s-3550-1#show logging
Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns)
    Console logging: level debugging, 13 messages logged
    Monitor logging: level debugging, 0 messages logged
    Buffer logging: level debugging, 13 messages logged
    File logging: disabled
    Trap logging: level warnings, 13 message lines logged
        Logging to 192.168.0.195, 0 message lines logged

 

Note that this procedure is exactly the same on my Cisco 2621 switch.

 

Related articles

HomeLab: The Cisco 3560G
HomeLab: Simple SSH Setup on a Cisco Router
HomeLab: Cisco 3550 Switch Software Configuration Guide
Using Good Old Syslog When Troubleshooting (by Tony Fortunato)
HomeLab: Configuring the NTP Client on a Cisco Catalyst Switches
[PATCH 1/9] syslog_ns: add syslog_namespace and put/get_syslog_ns

RHEL6 – Getting Up Close and Personal With Rsyslog

/ Christopher Paquin / Leave a comment

LogRsyslog has replaced Syslog as the default logging daemon in RHEL6. Rsyslog was designed to complete with syslog-ng and has several enhancements over plain old syslog. This includes but is not limited to more granularity with timestamps, direct database logging,   TCP support, and  relay server names in host fields which makes it easier to track the path a message has taken.

Below we are going to take a look at a few simple rsyslog configuration items.

Configure Rsyslog to Accept Remote Logs.

Within /etc/rsyslog.conf, comment out either the TCP or UDP syslog reception lines below. TCP is more reliable, however UDP is more widely supported.

# Provides UDP syslog reception
#$ModLoad imudp.so
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp.so
#$InputTCPServerRun 514

Configure a Server to Send Logs to a Remote Host.

To send all messages of info priority or higher to a remote host via udp, use the following format. Note that 10.1.224.34 is the remote server that I want to send logs to.

*.info    @10.1.224.34

To send the same priorities to the remote host via TCP, use two "@@"

*.info    @@10.1.224.34

Note that you can specify the port number on which to send by using IP:PORT. When no port is specified the default port of 514 is used.

Note that depending on your configuration you may need to alter your IPtables configuration on your sending and/or receiving server. In my case I needed to allow UDP on port 514 on my remote syslog server. To accomplish this I used system-config-firewall-tui which added the following line to /etc/sysconfig/iptables.

-A INPUT -m state –state NEW -m udp -p udp –dport 514 -j ACCEPT

Which shows up as what you see below in the output of 'iptables -L'

ACCEPT     udp  –  anywhere             anywhere            state NEW udp dpt:syslog

Testing Your Configuration

Ok lets send a test to our remote syslog server. Note that rsyslog has been restarted on both hosts.

# logger "testing to remote rsyslog server"

Checking the messages file on the remote host we can see that the test message has arrived.

Aug 13 14:55:26 vfatmin02 root: testing to remote rsyslog server

 

Configuring Remote Syslog in ESXi 4.1 via the Linux RCLI

/ Christopher Paquin / Leave a comment
Groundhog

First thing you need to know about ESXi is that it rolls its log after a reboot, meaning if your ESXi server crashes there will be no logs to review and no way of knowing what when to hell and where.

For this reason it is imperative that you setup a remote syslog server and send your logs there. Vmware support will tell you this and shame you if you have not setup remote syslogging.

Fortunately the first hit on google for “ESX syslog how to” will take you directly to the VMware KB article. You can find it here. However this page does not contain directions on how to do this via the Linux rcli, it only contains directions on how to do this via the VMA (management appliance) and from Windows PowerCLI. Really Vmware?

Follow the directions below if you run a real operating system and are not a Windows Admin.

First configure your remote ESXi host to forward syslog to your syslog server.

esxcfg-syslog –server esx04 –setserver vsyslog02 –setport 514

Then verify your settings

esxcfg-syslog –server esx04 –show

Output will appear as follows if everything worked right.

Current remote syslog server settings:
Server name : (vsyslog02)
Server port : 514