In this post, I am going to walk you through the process of installing and configuring two- factor SSH authentication via Google Authenticator. My base system is running a fresh install of RHEL 7.2
Installation Steps
The first step on my system was to install autoreconf, automake, and libtool. These packages are required by the bootstrap.sh script that we will need to in a couple more steps.
# yum -y install autoconf automake libtool
Now, we are going to install Git.
#yum -y install git
One more dependency to knock out. Install pam-devel as shown below.
# yum -y install pam-devel
Next, we clone the google-authenticator Git repo. In this example, I am cloning to /root
Here is my small, crude, little Kickstart configuration and post install script that I have up and running in my lab at home. Don’t expect to find anything too fancy here, as this Kickstart was purposefully built to be small and to the point. Here, the point was to spin up a VM, run through a basic install of CentOS/Redhat Linux, and install VMware Tools along with a Puppet agent.
Note that this post assumes that you have a working Kickstart server.
First lets take a look at our kickstart file, CentOS-6.6-x86_64-minimal.ks
The section directly below kicks off our kickstart ks file. Here we set our root password (no that’s not my hash) and setup our network interface for DHCP. We do a tiny bit of disk partitioning, and setup very simple LVM. Then we choose our packages. As you can see my package list is not at all fancy, I just want to make sure that I have pretty much every package that might need for a lab VM.
[code language=”bash”]
# Kickstart file for RHEL 6 Minimal
# Small Disk
text
install
url –url=http://10.1.0.106/ks/loop/CentOS-6.6-x86_64-bin-DVD1
lang en_US.UTF-8
keyboard us
network –onboot yes –device eth0 –bootproto dhcp –noipv6
rootpw –iscrypted $6$X/4YYZPN$4Sv.khxXms8N8vRssR/Vl35w/m80FF5P6p7aX0D7EFfD9p734F6tU4kXdcSCoOjPiXLrVxqfKxxxxxxxxxxxq5551
firewall –disabled
authconfig –enableshadow –passalgo=sha512
selinux –permissive
timezone America/New_York
Now let me pause to point out the section below. This is the %pre script that I am using to prompt me for the VM hostname before the install begins. The hostname needs to be set before you install puppet on the VM, otherwise you are going to have to recreate your puppet certificates after you set properly set your hostname post install and reboot.
[code language=”bash”]
%pre –log=/root/ks_pre.log
#change to tty6 to get input
chvt 6
exec </dev/tty6 > /dev/tty6
#Prompt for hostname
echo "What is my hostname?"
read NAME
echo "NETWORKING=yes
HOSTNAME=${NAME}" > network
chvt 1
[/code]
Now we run a simple post install, along with a custom post install script. It is this script that will install Vmware tools and Puppet. Myself, I prefer keeping most of my code out of the actual Kickstart ks file, however you can always jam all your code into it if you like. You will just need to validate your syntax first, as I have not tested my config this way.
[code language=”bash”]
%post –nochroot
# bring in hostname collected from %pre, then source it
cp network /mnt/sysimage/etc/sysconfig/network
. /mnt/sysimage/etc/sysconfig/network
# force hostname change
/mnt/sysimage/bin/hostname $HOSTNAME
#Post Install
%post –log=/root/ks-post.log
cd /root
echo "Getting the post install script – if this takes a long time check network or path"
wget http://10.1.0.106/ks/scripts/centos-6-postinstall.bash
echo "Running the post install script"
/bin/bash centos-6-postinstall.bash
[/code]
Ok, so below is the post install script that I am calling in the section above. After a quick modification of my hosts file, I pull down the Puppet installer from my local Puppet server. Next we install the open source VMware tools packages, after creating the required yum repofile.
#Switch to the 6th console and redirect all i/o
exec < /dev/tty6 > /dev/tty6 2> /dev/tty6
chvt 6
# Lets make sure we know who the puppet server is before we get too far
echo "Adding hosts entry for puppet master"
echo "10.1.0.115 puppet puppet.lab.localdomain" >> /etc/hosts
#Minor grub.conf modifications
sed -i ‘s/rhgb quiet//’ /boot/grub/grub.conf
sed -i ‘s/hiddenmenu//’ /boot/grub/grub.conf
sed -i ‘s/timeout=5/timeout=10/’ /boot/grub/grub.conf
#Kick off first puppet run, for some reason I think you might need to do this twice.
sleep 5
echo "Running Puppet for the first time"
puppet agent –test
puppet agent –test
#Tell us we have reached the end
echo "We have reached the end of the post-install script"
[/code]
A couple of additional details to note about the post install script above. I like to modify the grub.conf so that I unhide the menu and increase the time out. I also like to make sure that we disable the Redhat graphical boot screen… I want to make sure its easy to catch any errors or miss-configurations in my kickstarts.
So first off let me start by saying that I know that there is a ton of information out there on how to get started with Git. Heck, when you create your repo in GitLab it spits these instructions right out in front of your nose. However, what I have found is that most instructions tell you what to do to get started with git, however they do not tell you exactly what you are doing. You end up running a few command and then sit back and try to figure out what you actually just did.
That being said, getting started with Git has been the hardest part of the process, as most of us traditional grey-beard sysadmins are not that familiar with code management. When I was first getting started in technology, there were developers and there were sysadmins, and those two worlds were extremely separate. Now, we as we enter the age of DEVOPS and automation these two worlds are once again colliding (like they did in the beginning, but more on that another day).
In my lab I decided to build a stand alone vm for Git and Puppet. After doing some research – and asking others– on how and what to do to get Git up and running with a nice front end web interface, I decided to get started by installing the GitLab Omnibus package for Centos 6. This process was quick, easy, and painless.
Once I had a working webUI up and running it was time to create my first repo. Instead of trying to accomplish this from my Fedora workstaion, I just clicked on the "New Project" button on the GitLab dashboard and created an empty repo called "General_Scripts".
Back on my Fedora workstation I created a new directory in my home dir called git, and inside that directory I created a directory called "General_Scripts" as I had done in the webUI.
Now it was time to use Git.
First off you need to configure a few global Git options. This only needs to be done once.
Once these global configs are set you can then you can move on to seeding your repo. Here you see my change directories to the Global_Scripts directory and tell Git to initialize this directory as a repo.
#cd Global_Scripts
#git init
Now lets create a file and add it to the repo. Here were are going to create a simple README containing a description of my new repo. The instructions do not tell you that you have to put anything in this initial file, however what good is an empty README anyway
#vi README.md
Now tell git that this file needs to be added to the "General_Scripts" repo that we created a few steps ago.
#git add README.md
Now lets commit that file with a nice little comment. Commit comments should describe what the file we added or what we changed in an existing file
#git commit -m "Added README.md"
Now we need to tell our local git command what remote repo we are going to sync to. Note my git repo url is puppet.lab.localdomain, fatmin is my user's namespace, and General_Scripts is my repo.
This is a neat and very useful trick that I learned today. Lets say that you want to be able to monitor and log all keystrokes that are typed as root. This is particularly useful as normally you can only log when a user uses sudo to run a command. If the user has the abilty to become root however, then they have effectively eluded yourattempts to track their activity. Like Thomas Magnum shaking a tail, they are free to scoot around your island with the top down.
So how do you stop this from occuring? How to you log all activity and keystrokes made by root without implementing a bloated 3rd party software that will probably cost and arm and a leg? You use PAM you dingbat.
The secret sauce in this security burrito is the pam_tty_audit.so module. Here is how to use it,
Below is my stock /etc/pam.d/system-auth file
#%PAM-1.0 # This file is auto-generated. # User changes will be destroyed the next time authconfig is run. auth required pam_env.so auth sufficient pam_fprintd.so auth sufficient pam_unix.so nullok try_first_pass auth requisite pam_succeed_if.so uid >= 500 quiet auth required pam_deny.so
session optional pam_keyinit.so revoke session required pam_limits.so session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid session required pam_unix.so
Now look above and then look below at my modified system-auth file. Note the additonal session entry for pam_tty_audit.so.
[root@ip-172-31-21-28 pam.d]# cat system-auth #%PAM-1.0 # This file is auto-generated. # User changes will be destroyed the next time authconfig is run. auth required pam_env.so auth sufficient pam_fprintd.so auth sufficient pam_unix.so nullok try_first_pass auth requisite pam_succeed_if.so uid >= 500 quiet auth required pam_deny.so
Note that you will need to add the pam_tty_audit.so call to your /etc/pam.d/password-auth as well to ensure that you capture all of root's keystrokes, no matter how they log in.
Auditd gives you the ability to write your own custom audit rules. This functionality allows an administrator to keep a close eye on system calls, file access, and user behavior. This added functionality is especially useful in environments that are requred to adhear to compliance standards that are above and beyond normal standards. Think PCI.
Once of the simplest rules to add is a watch rule which can be set on files and directories. In the example below we are watching the /etc/passwd file for permission changes (writes and attibute changes specifically). We are creating a custom key to use for organizational purposes.
[root@ip-172-31-21-28 ~]# auditctl -w /etc/passwd -p wa -k edit_watch
Here is a cool one – lets audit all binary executions under /usr/bin.
[root@ip-172-31-21-28 ~]# auditctl -w /usr/bin -p x
Using the -l option you can list your current audit rules, and using the -s option you can see the current status of the auditd subsystem
Auditd is the userland piece of the RHEL audit tool suite. When its up and running, audit messages sent by the kenel will be send to log files that you have configured. By default, only a small and limited number of messages will be picked up by Auditd; these are mostly messages related to authentication and authorization.
Its possible to send audit messages to a syslog. By setting active=yes in /etc/audisp/plugins.d/syslog.conf you can send all your audit messages to syslog. If your system is setup to log to a remote syslog server, then your audit messages will go along for the ride as well. Note that you can also send audit messages to a remote logging server via native audit protocol over TCP. I am not going to go into this option, but I want to make sure that we are aware that it exists.
Looking for Audit Events in All the Wrong Places:
Auditd includes a handy-dandy tool for searching audit logs. Ausearch. You can check out all your current audit log messages using the command below.
[root@ip-172-31-21-28 ~]# ausearch -l
Viewing audit logs in their raw format can be accomplished with the command below
[root@ip-172-31-21-28 ~]# ausearch –raw
The -a option allows you to search by audit event ids
[root@ip-172-31-21-28 ~]# ausearch -a 282
Auditd also includes ausearch, which allows you to get a quick summary of audit events, rather than trying to view massive audit logs. Usage and output shown below.
root@ip-172-31-21-28 ~]# aureport
Summary Report ====================== Range of time in logs: 07/17/2014 10:21:36.438 – 07/17/2014 19:52:49.556 Selected time for report: 07/17/2014 10:21:36 – 07/17/2014 19:52:49.556 Number of changes in configuration: 4 Number of changes to accounts, groups, or roles: 24 Number of logins: 20 Number of failed logins: 4 Number of authentications: 75 Number of failed authentications: 3 Number of users: 3 Number of terminals: 18 Number of host names: 19 Number of executables: 14 Number of files: 0 Number of AVC's: 10 Number of MAC events: 20 Number of failed syscalls: 10 Number of anomaly events: 0 Number of responses to anomaly events: 0 Number of crypto events: 244 Number of keys: 0 Number of process IDs: 203 Number of events: 1132
You can also use aureport and ausearch together. Simliar to the powerfull partnership between Batman and Robin, these two tools complement each other in ways that you can only imagine. Check out my sexy bits below.
Summary Report ====================== Range of time in logs: 07/17/2014 10:21:36.438 – 07/17/2014 20:01:01.911 Selected time for report: 07/17/2014 10:21:36 – 07/17/2014 20:01:01.911 Number of changes in configuration: 4 Number of changes to accounts, groups, or roles: 24 Number of logins: 20 Number of failed logins: 4 Number of authentications: 75 Number of failed authentications: 3 Number of users: 3 Number of terminals: 18 Number of host names: 19 Number of executables: 14 Number of files: 0 Number of AVC's: 10 Number of MAC events: 20 Number of failed syscalls: 10 Number of anomaly events: 0 Number of responses to anomaly events: 0 Number of crypto events: 244 Number of keys: 0 Number of process IDs: 205 Number of events: 1144
Want to know another cool tool that is part of auditd? I know, its a lot to take in at one time, but I am sure that you can handle it. Using autrace you can trace and investigate system calls made by a process.
Want to see everything that nslookup is doing? Then run the command below.
Here is my small, crude, little Kickstart configuration and post install script that I have up and running in my lab at home. Don’t expect to find anything too fancy here, as this Kickstart was purposefully built to be small and to the point. Here, the point was to spin up a VM, run through a basic install of CentOS/Redhat Linux, and install VMware Tools along with a Puppet agent.
