Red Hat OpenStack 8: Making your Undercloud Immutable

chain-and-padlock-macro-wallpaper-background

Introduction

This article will show you how to block the overcloud from being deleted.

Blocking Users from Deleting the Overcloud Stack

First make a backup copy of /etc/heat/policy.json

$sudo cp /etc/heat/policy.json /etc/heat/policy.json.orig

Run the command below to see the default stacks:delete policy.

$ sudo grep -m1 stacks:delete /etc/heat/policy.json
“stacks:delete”: “rule:deny_stack_user”,

Then, make it so that we deny anyone and everyone from removing the stack, even if you’re an admin.

Note, that this means that the policy would have to be reverted back to the original configuration to delete the stack in the future. See sed command below.

$ sudo sed -i /stacks:delete/{s/rule:.*/’rule:deny_everybody”,’/}
/etc/heat/policy.json

Verify your changes.

$ sudo grep -m1 stacks:delete /etc/heat/policy.json
“stacks:delete”: “rule:deny_everybody”,

Blocking Users from Deleting Nova Instances

In addition to blocking users from accidentally deleting your overcloud from heat, you should also block the accidental deletion of the overcloud nodes from nova.

First, run the command below to make a backup of /etc/nova/policy.json.

$ sudo cp /etc/nova/policy.json /etc/nova/policy.json.orig

Run the command below to see the default compute:delete policy.

$ sudo grep compute:delete /etc/nova/policy.json
“compute:delete”: “rule:admin_or_owner”,

Now let’s change the policy so that it blocks anyone and everyone from deleting a compute node.

$ sudo sed -i /compute:delete/{s/rule:.*/’rule:deny_everybody”,’/}
/etc/nova/policy.json

Now we can verify our changes.

$ sudo grep compute:delete /etc/nova/policy.json
“compute:delete”: “rule:deny_everybody”,

Advertisements

RHEL 7 Two-Factor SSH Via Google Authenticator

650x300xgoogle-authenticator-header.png.pagespeed.gp+jp+jw+pj+js+rj+rp+rw+ri+cp+md.ic.194erIegOZ.jpg

In this post,  I am going to walk you through the process of installing and configuring two- factor SSH authentication via Google Authenticator. My base system is running a fresh install of RHEL 7.2

Installation Steps

The first step on my system was to install autoreconf, automake, and libtool. These packages are required by the bootstrap.sh script that we will need to in a couple more steps.

# yum -y install autoconf automake libtool

Now, we are going to install Git.

#yum -y install git

One more dependency to knock out. Install pam-devel as shown below.

# yum -y install pam-devel

Next, we clone the google-authenticator Git repo. In this example, I am cloning to /root

# git clone https://github.com/google/google-authenticator.git
Cloning into ‘google-authenticator’…
remote: Counting objects: 1435, done.
remote: Total 1435 (delta 0), reused 0 (delta 0), pack-reused 1435
Receiving objects: 100% (1435/1435), 2.32 MiB | 0 bytes/s, done.
Resolving deltas: 100% (758/758), done.

Now change directory as shown below and run bootstrap.sh.

# cd /root/google-authenticator/libpam

# ./bootstrap.sh

Now run the following commands to finalize the module installs.

# ./configure

#make

#make install

Assuming that you do not run into any errors, the following modules will be installed.

  • /usr/local/lib/security/pam_google_authenticator.so
  • /usr/local/lib/security/pam_google_authenticator.la

Continue reading

Redhat Satellite 5: How to Clone Security Errata to a Software Channel

space_dogFirst check to see if the errata is available to your local satellite server. To accomplish this log into your organizations satellite server and click on the “Errata” tab. Then on the left side of the page click on “Advanced Search”.

In the search box enter the RHSA number (Redhat Security Advisory Number) for the errata that you want to clone/update. In this example I am searching for RHSA-2014:1924, which is a Thunderbird security update.

If your search does not return any results, you will need to manually sync your local Satellite Server with Redhat.To accomplish this you need to ssh into your local satellite server and run the command shown below. Note that this does not update any packages/errata. This does update the list of availbile packages/errata.

/usr/bin/satellite-sync
[root@myserver ~]# satellite-sync –email
10:08:09 Red Hat Satellite – live synchronization
10:08:09 url: https://satellite.rhn.redhat.com
10:08:09 debug/output level: 1
….truncated….

Once you are able to locate the specific fix in via “Erratum Search” you may proceed to the next step. In this example, as I stated above, I am searching for RHSA-2014:1924.

clone_erratta

Now that our local Satellite server is aware of our specific errata, click on “Clone Errata” on the left side of the page. Search the page “Errata Management” for the specific fix that you want to apply. Note that the “Errata Management” page does have built in search functionality — don’t ask me why — so you must search using your browser’s own page search function.

clone_thunderbird

Once you have located the correct Security Advisory, put a check in the box and spend about 5 minutes scrolling down to the bottom of the page. Stop when your arm is tired, or once you locate the “Clone Errata” button. Obviously you want to click this.

Note that your newly added and updated errata/package may not become immediatley availible to install. You nay need to run the following commands to refresh/reload your repos.

#yum clean all

Then check for updates with the command below.

#yum check-update

RHEL6: All Up in Your Face with Auditd

Strongbad2kqAuditd is the userland piece of the RHEL audit tool suite. When its up and running, audit messages sent by the kenel will be send to log files that you have configured. By default, only a small and limited number of messages will be picked up by Auditd; these are mostly messages related to authentication and authorization.

Auditd has three main config files as shown below

  • /etc/sysconfig/auditd – basic configuration options
  • /etc/audit/auditd.conf – main config file
  • /etc/audit/audit.rules – auditing rules

Auditd and Syslog:

Its possible to send audit messages to a syslog. By setting active=yes in /etc/audisp/plugins.d/syslog.conf you can send all your audit messages to syslog. If your system is setup to log to a remote syslog server, then your audit messages will go along for the ride as well. Note that you can also send audit messages to a remote logging server via native audit protocol over TCP. I am not going to go into this option, but I want to make sure that we are aware that it exists.

 

Looking for Audit Events in All the Wrong Places:

Auditd includes a handy-dandy tool for searching audit logs. Ausearch. You can check out all your current audit log messages using the command below.

[root@ip-172-31-21-28 ~]# ausearch -l

Viewing audit logs in their raw format can be accomplished with the command below

[root@ip-172-31-21-28 ~]# ausearch –raw

The -a option allows you to search by audit event ids

[root@ip-172-31-21-28 ~]# ausearch -a 282

Auditd also includes ausearch, which allows you to get a quick summary of audit events, rather than trying to view massive audit logs. Usage and output shown below.

root@ip-172-31-21-28 ~]# aureport

Summary Report
======================
Range of time in logs: 07/17/2014 10:21:36.438 – 07/17/2014 19:52:49.556
Selected time for report: 07/17/2014 10:21:36 – 07/17/2014 19:52:49.556
Number of changes in configuration: 4
Number of changes to accounts, groups, or roles: 24
Number of logins: 20
Number of failed logins: 4
Number of authentications: 75
Number of failed authentications: 3
Number of users: 3
Number of terminals: 18
Number of host names: 19
Number of executables: 14
Number of files: 0
Number of AVC's: 10
Number of MAC events: 20
Number of failed syscalls: 10
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 244
Number of keys: 0
Number of process IDs: 203
Number of events: 1132

You can also use aureport and ausearch together. Simliar to the powerfull partnership between Batman and Robin, these two tools complement each other in ways that you can only imagine. Check out my sexy bits below.

[root@ip-172-31-21-28 ~]# ausearch –start today –raw | aureport

Summary Report
======================
Range of time in logs: 07/17/2014 10:21:36.438 – 07/17/2014 20:01:01.911
Selected time for report: 07/17/2014 10:21:36 – 07/17/2014 20:01:01.911
Number of changes in configuration: 4
Number of changes to accounts, groups, or roles: 24
Number of logins: 20
Number of failed logins: 4
Number of authentications: 75
Number of failed authentications: 3
Number of users: 3
Number of terminals: 18
Number of host names: 19
Number of executables: 14
Number of files: 0
Number of AVC's: 10
Number of MAC events: 20
Number of failed syscalls: 10
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 244
Number of keys: 0
Number of process IDs: 205
Number of events: 1144

Want to know another cool tool that is part of auditd? I know, its a lot to take in at one time, but I am sure that you can handle it. Using autrace you can trace and investigate system calls made by a process. 

Want to see everything that nslookup is doing? Then run the command below.

[root@ip-172-31-21-28 ~]# autrace /usr/bin/nslookup google.com

This will output a pid for you to trace with ausearch.

Trace complete. You can locate the records with 'ausearch -i -p 3359'

 

Related articles

RHEL6- Getting Up Close and Personal With Rsyslog
Linux audit files to see who made changes to a file

Password Protecting Grub in RHEL 6

19552780-illustration-of-cartoon-wormGrub, is the standard boot loader used by each and every Linux type operating system that I can think of. RHEL 6 uses what I guess we are now calling grub 1.o, since grub 2.0 has been released and in use by Fedora for the last few releases. You will also find that grub 2.0 has replaced grub 1.0 in RHEL 7. At some point I plan to explore grub 2 at lenght, but today is not that day (unless something strange happens before I go to bed tonight — you never know).

 

Anyway – I digress. Below is an excerpt from the current grub.conf that is in use by my EC2 RHEL 6.5 instance. I've made it really tiny to save space.

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You do not have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /, eg.
#          root (hd0)
#          kernel /boot/vmlinuz-version console=ttyS0 ro root=/dev/vda1
#          initrd /boot/initrd-[generic-]version.img
#boot=/dev/vda
default=0
timeout=1

splashimage=(hd0)/boot/grub/splash.xpm.gz

hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-431.17.1.el6.x86_64)
        root (hd0)
        kernel /boot/vmlinuz-2.6.32-431.17.1.el6.x86_64 console=ttyS0 ro root=UUID=05e7b919-2577-40a7-91fb-1ccdede87fc4 rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 xen_blkfront.sda_is_xvda=1 console=ttyS0,115200n8 console=tty0 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM rd_NO_DM rhgb quiet
        initrd /boot/initramfs-2.6.32-431.17.1.el6.x86_64.img

To configure a password to use for grub you need to use the grub-crypt command. as you can see in the example below, grub-crypt prompts you for your password, although you can also pass the password to grub-crypt in the inital command. Grub-crypt spits out your hashed and salted password.

[root@ip-172-31-22-45 ~]# grub-crypt
Password:
Retype password:
$6$V0qkcD4b/CouCY/L$.7vrHd.X6MZWzzDxvXdQn3avPwv2rUX80XKuS.Hvh1YqR11ZyINZszvJ/e0qH6VcsFMAVWAPWXtugHHYTzYj90

Now stick this password in your grub.conf using the format below.

password –encrypted $6$V0qkcD4b/CouCY/L$.7vrHd.X6MZWzzDxvXdQn3avPwv2rUX80XKuS.Hvh1YqR11ZyINZszvJ/e0qH6VcsFMAVWAPWXtugHHYTzYj90

You have a couple of options when inserting the password into the grub.conf. You can either stick this entry near the top of the file, which protects each and every boot image that you might be configured to use, or you can stick the entry directly into a particular stanza to protect a specific boot image. For example, if you are building your servers with the option to pxe-boot from grub, or wipe the disk from grub, you might want to protect these options with a password.

Also note that adding a password to grub does not keep an unauthorized user from being able to boot or reboot a server, rather what it does is protect the grub menu from being edited manually durring a reboot. This keeps nefarious users from being able to break the boot sequence and boot into single-user mode where they can compromise your system.

 

Related articles

RHEL6 – Restore Grub on MBR
Grub
RHEL6 – Mount: Could Not Find Any Free Loop Device

Introduction to AIDE – Advanced Intrusion Detection Environment

Wally_Gator_PhotoEver heard of AIDE, neither had I. Apparently its a simple intrusion detection application that can be used to monitor file changes.  It can be confired to monitor permission, ownership, timestamp, or content changes.

Lets install it. Its in the stock Redhat repos, so its a piece of cake to install via yum.

 

[root@localhost ~]# yum -y install aide

Once installed, you can tweak the config file (/etc/aide.conf) to your liking. The stock config is pretty robust, so I am going to trim it down a bit and just monitor /etc for permission changes, and /bin for what are defined as normal changes. Normal looks at file hashes to see if the files have been modified.

/bin    NORMAL
/etc    PERMS
 

Now lets start aide

[root@localhost ~]# aide –init

AIDE, version 0.15.1

### AIDE database at /var/lib/aide/aide.db.new.gz initialized.

 

Now this part is silly, we need to rename the database created above to the name that aide is configured to use.

[root@localhost ~]# cp /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz

 

Now lets check for changes.

[root@localhost ~]# aide –check

AIDE, version 0.15.1

### All files match AIDE database. Looks okay!

Hey no changes. Now lets monkey with something and see if aide catches it. In this example we are creating a new file in /etc. As seen below aide catches the new file and reports on it.

 

[root@localhost ~]# touch /etc/aide.test.change
[root@localhost ~]# aide –check
AIDE 0.15.1 found differences between database and filesystem!!
Start timestamp: 2014-07-15 19:51:14

Summary:
  Total number of files:        5054
  Added files:                  1
  Removed files:                0
  Changed files:                0

—————————————————
Added files:
—————————————————

added: /etc/aide.test.change

 

So now lets re-initialize the database, which is pretty much a snapshot.

[root@localhost ~]# aide –init

AIDE, version 0.15.1

### AIDE database at /var/lib/aide/aide.db.new.gz initialized.

 

Don't forget to overwrite the old database.

[root@localhost ~]# cp /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz
cp: overwrite ‘/var/lib/aide/aide.db.gz’? yes

Now lets change the permissions on our test file and see if aide catches the change.  I'll spare you the suspense and let you know that aide did its job. See below.

 

[root@localhost ~]# chmod 777 /etc/aide.test.change
[root@localhost ~]# aide –check                    
AIDE 0.15.1 found differences between database and filesystem!!
Start timestamp: 2014-07-15 19:54:09

Summary:
  Total number of files:        5054
  Added files:                  0
  Removed files:                0
  Changed files:                2

—————————————————
Changed files:
—————————————————

changed: /etc/aide.test.change
changed: /root/.mozilla/firefox/8u03e3hs.default/sessionstore.js

—————————————————
Detailed information about changes:
—————————————————

File: /etc/aide.test.change
 Perm     : -rw-r–r–                       , -rwxrwxrwx
 ACL      : old = A:
—-
user::rw-
group::r–
other::r–
—-
                  D: <NONE>
            new = A:
—-
user::rwx
group::rwx
other::rwx
—-
                  D: <NONE>

Now aide on its own is just a simple tool, but run via cron with a bit of tuning and a bit more logic behind it and I can see it being a very useful tool. Looking forward to playing with it more.

Related articles

How To Install Aide on a DigitalOcean VPS
RHEL6 – How to Setup an Anonymous Download Only FTP Server

Yum: Under the Covers with GPG Keys

Skeleton_key_1_keychain_sculpture_photo_cut_outs-r558fef9b275d4609b522ac07970a6af6_x7sa6_8byvr_324Hey look at this spooky key. Don't be frightened little one. Nothing scary is going to happen to you here. This is a safe place. As a matter of fact, if you stick around you might just learn a thing or two. A thing or two about GPG!

First off do any of us really know what GPG stands for? Well yes we do! It stands for GNU Privacy Guard. RPM Package creators use GPG to apply a digital signature to their packages. If a package was tampered with, then its GPG signature will no longer match what was placed in the original package.

First off to check what keys you have installed on your Linux server you can run the following rpm command as show in the example below.

[root@ip-172-31-22-45 ~]# rpm -qa gpg-pubkey
gpg-pubkey-0608b895-4bd22942
gpg-pubkey-fd431d51-4ae0493b
gpg-pubkey-2fa658e0-45700c69

 

Neet I have three keys installed. But lets say want to install another one. Well I can do so with the command below. In this example I have navigated to /etc/pki/rpm-gpg and am going to install the redhat beta key on my server.

[root@ip-172-31-22-45 rpm-gpg]# rpm –import RPM-GPG-KEY-redhat-beta

Hey that was fun. Now lets get our hands a bit dirtier.

Want to get more information on a specific key. Then this command is your huckleberry. Here you can see that this is the pubkey for the EPEL repo.

[root@ip-172-31-22-45 rpm-gpg]# rpm -qi gpg-pubkey-0608b895-4bd22942
Name        : gpg-pubkey                   Relocations: (not relocatable)
Version     : 0608b895                          Vendor: (none)
Release     : 4bd22942                      Build Date: Sat 14 Jun 2014 09:13:58 AM EDT
Install Date: Sat 14 Jun 2014 09:13:58 AM EDT      Build Host: localhost
Group       : Public Keys                   Source RPM: (none)
Size        : 0                                License: pubkey
Signature   : (none)
Summary     : gpg(EPEL (6) <epel@fedoraproject.org>)
Description :

 

To verify signature of a downloaded package, use the rpm command as shown below. In this example I have highlighted the key that was used to sign this package.

# rpm -vK nautilus-dropbox-1.6.0-1.fedora.i386.rpm
nautilus-dropbox-1.6.0-1.fedora.i386.rpm:
    Header V3 RSA/SHA1 Signature, key ID 5044912e: OK
    Header SHA1 digest: OK (a4d51906633f92913db075ba33946f50999c245e)
    V3 RSA/SHA1 Signature, key ID 5044912e: OK
    MD5 digest: OK (1b8ff7abc18f68bf274e24fc57fd3a87)

 

Using the bolded information in the example above, I can then use this information to track down the exact key that was used to sign the package.

[root@localhost Downloads]# rpm -qa | grep 5044912e
gpg-pubkey-5044912e-4b7489b1

 

Is this awesome, well not really, but you never know when you might need to use this information. Like on a test. Wink Wink.

Related articles

Using Yum Update to Apply Security Patches Only
How to Enable EPEL Repository on CentOs for Yum
Signing rpm packages with GPG