So by default when you forward logs to a syslog/rsyslog server all the logs end up in the same file (ususally configured to go to the messages file). Sometimes one may prefer to forward logs from a particular server to a separate logfile. I know for a fact that my sometimes friends in our info-sec group prefers this setup.
While managing such a setup for more than a handful of hosts would probably be a nightmare, its not actaually hard to setup on the most basic level.
First create a custom filters.conf (or whatever you want to name it) in your /etc/rsyslog.d directory. Below is the file that I created.
Note that you will need to include two lines in your file for each host, one with the specific file/location that you want to send your filtered log messages, and a second line that directs rsyslog to discard any messages from the specified hosts once they have been logged to the specified locations. This keeps these messages from ending up in the messages file as well as the file defined in your filter file.
Rsyslog has the ability to forward/recieve encrypted logs using certificates. I am going to go over a very quick and dirty install and configuration. Since I am again sitting in Starbucks and do not have access to my homelab and its abundant resources, I am once again going to use Amazon EC2 instances of RHEL 6.5 for testing. The two instances that I will use we will call Server1 and Server2. Our goal here is not to build out an entire environment, but rather we are just looking to get this working on its most basic level.
Lets start with the log reciever server as we need to install a couple of packages. The first provides the certtool utility and the second provides the tls library for rsyslog.
Also note that we are going to be looking at some documentation on the remote hosts to make this install and config a bit easier. You are probably going to want to install lynx so you can view the html documentation files. On my server1 instance, the rsyslog docs are in the following directory – /usr/share/doc/rsyslog-5.8.10.
This is the file that you want to take a look at. Below I am using lynx to view it.
lynx tls_cert_server.html
Now lets head on over to the sending server and install our packages again.
Grab these bits from the file and modify to match your system. Then drop it in a file in /etc/rsyslog.d on your rsyslog receiving server. I am calling my logging.conf
$ModLoad imuxsock # local messages $ModLoad imtcp # TCP listener
# make gtls driver the default $DefaultNetstreamDriver gtls
$InputTCPServerStreamDriverAuthMode x509/name $InputTCPServerStreamDriverPermittedPeer *.example.net $InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode $InputTCPServerRun 6514 # start up listener at port 10514
Now, from the same file as you used above, grab the following giggly-bits and stick them out on the rsyslog sending server. Modify the file to match your needs and drop in /etc/rsyslog.d. To keep things simple I am going to call this file logging.conf as well
# certificate files – just CA for a client $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem # set up the action $DefaultNetstreamDriver gtls # use gtls netstream driver $ActionSendStreamDriverMode 1 # require TLS for the connection $ActionSendStreamDriverAuthMode anon # server is NOT authenticated *.* @@(o)server.example.net:10514 # send (all) messages
Now, lets review where we are. Our config files are in place on our sending server and our recieving server. Plus we have installed all required packages. Now lets get on to the keys. This is the confusing part, so I will go through my process one step at a time. Once again you can use the built in documentation to help you get this part right.
Jump on the recieving server and check out the file below via lynx
lynx tls_cert_ca.html
Jump into your newly created keys directory.
[root@ip-172-31-21-28 ~]# cd /etc/rsyslog-keys/
And run the command below to generate the private key for your CA (Certficate Authority)
[root@ip-172-31-21-28 rsyslog-keys]# certtool –generate-privkey –outfile ca-key.pem Generating a 2048 bit RSA private key…
Now we are going to generate a self signed cert from our private key.
This one also requires you to answer a bunch of questions. These are the two that have "yes" for the answer.
Is this a TLS web client certificate? (y/N): y Is this also a TLS web server certificate? (y/N): y
Now at this point you should be able to start rsyslog on each server. Using the logger command on your sending server, you should be able to send a message to your local syslog and see it forward to the sender.
Grub, is the standard boot loader used by each and every Linux type operating system that I can think of. RHEL 6 uses what I guess we are now calling grub 1.o, since grub 2.0 has been released and in use by Fedora for the last few releases. You will also find that grub 2.0 has replaced grub 1.0 in RHEL 7. At some point I plan to explore grub 2 at lenght, but today is not that day (unless something strange happens before I go to bed tonight — you never know).
Anyway – I digress. Below is an excerpt from the current grub.conf that is in use by my EC2 RHEL 6.5 instance. I've made it really tiny to save space.
# grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You do not have a /boot partition. This means that # all kernel and initrd paths are relative to /, eg. # root (hd0) # kernel /boot/vmlinuz-version console=ttyS0 ro root=/dev/vda1 # initrd /boot/initrd-[generic-]version.img #boot=/dev/vda default=0 timeout=1
splashimage=(hd0)/boot/grub/splash.xpm.gz
hiddenmenu title Red Hat Enterprise Linux Server (2.6.32-431.17.1.el6.x86_64) root (hd0) kernel /boot/vmlinuz-2.6.32-431.17.1.el6.x86_64 console=ttyS0 ro root=UUID=05e7b919-2577-40a7-91fb-1ccdede87fc4 rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 xen_blkfront.sda_is_xvda=1 console=ttyS0,115200n8 console=tty0 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM rd_NO_DM rhgb quiet initrd /boot/initramfs-2.6.32-431.17.1.el6.x86_64.img
To configure a password to use for grub you need to use the grub-crypt command. as you can see in the example below, grub-crypt prompts you for your password, although you can also pass the password to grub-crypt in the inital command. Grub-crypt spits out your hashed and salted password.
You have a couple of options when inserting the password into the grub.conf. You can either stick this entry near the top of the file, which protects each and every boot image that you might be configured to use, or you can stick the entry directly into a particular stanza to protect a specific boot image. For example, if you are building your servers with the option to pxe-boot from grub, or wipe the disk from grub, you might want to protect these options with a password.
Also note that adding a password to grub does not keep an unauthorized user from being able to boot or reboot a server, rather what it does is protect the grub menu from being edited manually durring a reboot. This keeps nefarious users from being able to break the boot sequence and boot into single-user mode where they can compromise your system.
Plugable Authentication Modules, or PAM, is the standard mechanism that most Unix and Linux Operatng Systems use for user credential authentication. By design, PAM is broken out into a number of files, each with a specific purpose. Before you can get started with PAM you need to understand a bit about how PAM configuration files are formatted. So lets get into that first before we try to bite off anything more.
PAM Config File Standards:
PAM config files follow a standard format as shown below.
Rule Types Control Module [module arguments]
There are 4 Rule Types
auth – used to authenticate users/passwords
account – set properties for a user's account
password – controls password changes
session – sets and controls environmental variables
And there are 5 Control Types
required – a module that a user is required to pass
sufficient – a user is not required to pass a sufficient module
optional – these modules do not have to be passed sucessfully
include – these rules reference other PAM config files
requisite – similar to required, but if failed, no further rules are checked
Configuring pam_cracklib:
Pam_cracklib is used to define password complexity. It has several module arguments that can be used to define password complexity and lenght. Its most common arguments are show below
ucredit – when used in the following format (ucredit=-n) requires the defined number of uppercase characters in a password
dcredit – when used in the following format (dcredit=-n) requires the defined number of digits in a password
ocredit – when used in the following format (ocredit=-n) requires the defined number of other (think symbols) type charaters in a password
lcredit – when used in the following format (lcredit=-n) requires the defined number of lower case letters in a password
minclass – defines the minimum number of different character classes that must be present in a password.
minlen – defines the minimum required lenght of a password.
Here's a usage example of the cracklib module from a /etc/pam.d/system-auth file. In this example try_first_pass tells pam to try to use any cached credentials, while retry allows a user to try their password 3 times before the fail this module.
Pam_tally2 can be used to lock users after a defined number of failed login attempts. The example below, taken from the system-auth file will lock a user after 3 failed login attempts, will automatically unlock the user after 300 seconds, and will do so quietly, without any notification to the user.
The command pam_tally2 can be used to list users with failed logins and can also be used to reset a user's failed login count. See reset example below
pam_tally2 –reset –u testuser
Note that pam_tally2 deprecates the faillock module.
Supplementary PAM Configuration Options:
Want to limit a user to a particular number of concurrent ssh sessions? You can set this up in /etc/security/limits.conf if you are calling the pam_limits.so in your pam configs. Limits.conf provides the example below. Just copy the format and you are off to the races. Remember to remove the #.
Ok this one is going to be quick and dirty. I have been sitting in Starbucks for about 3 hours now in what at this point is the hardest and most uncomfortable chair in the world. At this point its almost torture.
Anyway, enough about my bottom. Ever had the need to search your /etc/password file for users with duplicate UIDS or duplicate usernames? Want to make sure that no other user, other than root, has a uid of 0? Well then today is your lucky day.
For this test I have added a couple of test users so that we have some results.
The command below is searching for any two users with the same UID, and will return a UID number of any UID that it finds twice.
As you can see from the example above, the command found two users with the UID of 0. As we all know this is not good. Time to see who as been up to some funny business. Guess what, its was me. I added a user called testuser and manually changes his UID to 0.
Interesting item of note, you cannot just remove this user, as the system thinks that you are trying to remove the root user. First I need to edit /etc/passwd and manually change the user’s uid.
[root@localhost ~]# userdel testuser
userdel: user testuser is currently used by process 1
[root@localhost ~]# vi /etc/passwd
[root@localhost ~]# userdel testuser
Now lets search for any users with duplicated user names
And again, you just cannot delete these users, you must clean them up manually.
[root@localhost ~]# userdel testuser1
Multiple entries named ‘testuser1’ in /etc/passwd. Please fix this with pwck or grpck.
userdel: cannot remove entry ‘testuser1’ from /etc/passwd
Ever heard of AIDE, neither had I. Apparently its a simple intrusion detection application that can be used to monitor file changes. It can be confired to monitor permission, ownership, timestamp, or content changes.
Lets install it. Its in the stock Redhat repos, so its a piece of cake to install via yum.
[root@localhost ~]# yum -y install aide
Once installed, you can tweak the config file (/etc/aide.conf) to your liking. The stock config is pretty robust, so I am going to trim it down a bit and just monitor /etc for permission changes, and /bin for what are defined as normal changes. Normal looks at file hashes to see if the files have been modified.
/bin NORMAL /etc PERMS
Now lets start aide
[root@localhost ~]# aide –init
AIDE, version 0.15.1
### AIDE database at /var/lib/aide/aide.db.new.gz initialized.
Now this part is silly, we need to rename the database created above to the name that aide is configured to use.
Hey no changes. Now lets monkey with something and see if aide catches it. In this example we are creating a new file in /etc. As seen below aide catches the new file and reports on it.
[root@localhost ~]# touch /etc/aide.test.change [root@localhost ~]# aide –check AIDE 0.15.1 found differences between database and filesystem!! Start timestamp: 2014-07-15 19:51:14
Summary: Total number of files: 5054 Added files: 1 Removed files: 0 Changed files: 0
————————————————— Added files: —————————————————
added: /etc/aide.test.change
So now lets re-initialize the database, which is pretty much a snapshot.
[root@localhost ~]# aide –init
AIDE, version 0.15.1
### AIDE database at /var/lib/aide/aide.db.new.gz initialized.
Now lets change the permissions on our test file and see if aide catches the change. I'll spare you the suspense and let you know that aide did its job. See below.
[root@localhost ~]# chmod 777 /etc/aide.test.change [root@localhost ~]# aide –check AIDE 0.15.1 found differences between database and filesystem!! Start timestamp: 2014-07-15 19:54:09
Summary: Total number of files: 5054 Added files: 0 Removed files: 0 Changed files: 2
Now aide on its own is just a simple tool, but run via cron with a bit of tuning and a bit more logic behind it and I can see it being a very useful tool. Looking forward to playing with it more.
In my previous post I went over standard filesystem attibutes in Linux, and how to set and view those attibutes with lsattr and chattr. You can view that post here if you are interested.
In this post we are going to go over extended filesystem attibutes. Now there is not much to this as you are probably not going to ever have to use these settings. That being said, its not a bad thing to be aware of.
Attribute names are strings that can be set and configured at will using the setfatr command. They can be viewed with the getfattr command. There are 4 namespaces of attibutes, security, system, trusted, and user.
When using getfattr ( which I pronounce as getfatter) the -d option dumps only user namespace attibutes. The rest of the namespace attributes can be viewed by using the -m option along with the namespace name. In the example below you can see that there are no user namespace attibutes set on my anaconda-ks.cfg file, however there are attibutes set in the security namespace.
However this could be cool if you got an md5 sum on a file and dumped it into a file attibute. You could, in theory, use this process to see if someone has messed with one of your config files.
So by default when you forward logs to a syslog/rsyslog server all the logs end up in the same file (ususally configured to go to the messages file). Sometimes one may prefer to forward logs from a particular server to a separate logfile. I know for a fact that my sometimes friends in our info-sec group prefers this setup.
