Today we’re going to use the example of maniplulating the configuration files for Ganglia using cli commands that you can include in a bash script.

Changing a single line entry

In this example, we just need to find a single line in the configuration file and replace it:

From:  data_source “my cluster” localhost

To:  data_source “Envision Cluster” 60 localhost

1. Backup the configuration file first:

           cp /etc/ganglia/gmetad.conf /etc/ganglia/gmetad.conf.orig
           

2. Perform the sed replace

           sed -e "s/^data_source \"my cluster\" localhost/data_source \"Envision Cluster\" 60 localhost/g" /etc/ganglia/gmetad.conf > /etc/ganglia/gmetad.conf.postreplace
           

   sed -e: tell sed to execute the command string
   s/: means search
   ^: means that the search string must be found at the very start of the line
   /g: means replace every occurrence of the found string
   /etc/ganglia/gmetad.conf > /etc/ganglia/gmetad.conf.postreplace: look for the search string in /etc/ganglia/gmetad.conf but save the replaced version to /etc/ganglia/gmetad.conf.postreplace. We do this because sed struggles to find and replace in the same file.

3. Replace the original file with the modified one

           if [ -e /etc/ganglia/gmetad.conf ]; then
           	rm -f /etc/ganglia/gmetad.conf
           fi
           mv -f /etc/ganglia/gmetad.conf.postreplace /etc/ganglia/gmetad.conf.cnf
           

Replacing content between two defined strings

In this example, a single line find and replace is not practical for 2 reasons. Firstly, its too much work to replace each line individually. Secondly, there are multiple lines in the configuration file containing ‘name = “unspecified”‘ and whilst we could use sed to replace the first occurrence of the search string, it’s less reliable over time. So we’re going to tell sed to search for the string ‘cluster {‘ and delete it and everything after it until it hits the string ‘}’. Once we’ve deleted this configuration block, we can just add a new one at the bottom of the file easily.

From:
cluster {
    name = “unspecified”
    owner = “unspecified”
    latlong = “unspecified”
    url = “unspecified”
}

To:
cluster {
    name = “Envision Cluster”
    owner = “Envision Systems”
    latlong = “unspecified”
    url = “http://www.envision-systems.com.au”
}

1. Backup the configuration file first:

           cp /etc/ganglia/gmond.conf /etc/ganglia/gmond.conf.orig
           

2. Use sed to delete the old configuration block

           sed '/cluster {/,/}/d' /etc/ganglia/gmond.conf > /etc/ganglia/gmond.conf.postreplace
           

   /d: tells sed to delete
   /etc/ganglia/gmond.conf > /etc/ganglia/gmond.conf.postreplace: look for the search string in /etc/ganglia/gmond.conf but save the replaced version to /etc/ganglia/gmond.conf.postreplace. We do this because sed struggles to find and replace in the same file.

3. Add the new configuration block

           echo "" >> /etc/ganglia/gmond.conf
           echo "/* CUSTOM CONFIGURATION SECTION */" >> /etc/ganglia/gmond.conf
           echo "cluster {" >> /etc/ganglia/gmond.conf
           echo "  name = \"Envision Cluster\"" >> /etc/ganglia/gmond.conf
           echo "  owner = \"Envision Systems\"" >> /etc/ganglia/gmond.conf
           echo "  latlong = \"unspecified\"" >> /etc/ganglia/gmond.conf
           echo "  url = \"http://www.envision-systems.com.au/\"" >> /etc/ganglia/gmond.conf
           echo "}" >> /etc/ganglia/gmond.conf
           echo "" >> /etc/ganglia/gmond.conf
           

1. Create a new linux user account for your sftp user

           #useradd sftpuser
           

2. Set the password for your new sftp account

           #passwd sftp
           

3. Change the login shell
   Edit /etc/passwd and change the sftp user’s login shell to /usr/libexec/openssh/sftp-server

           sftpuser:x:1234:3234:SFTP User:/home/sftpuser:/usr/libexec/openssh/sftp-server
           

4. Test
   Confirm that you can SFTP files to and from the sftp user’s home directory (/home/sftpuser/) but not ssh or sftp to the server using the sftp user account

This tutorial will step you through how to do it:

1. Log into your guest operating system as root

2. Install VMWare tools
   See my instructions here

3. Ask your ESX Infrastructure provider to increase your virtual disk size

4. List devices and identify device number (part of /dev/sda by default)

           fdisk -l
           

5. Create the new primary partition
   • Run fdisk (the default device is usually /dev/sda)

                             fdisk /dev/sda
                             

   • Press p to print the partition table and identify the number of partitions (there are 2 by default)

   • Press n to create a new primary partition

   • Press p for primary

   • Press Enter twice

   • Press w to write the changes to the partition table

6. Restart the guest operating system

           reboot
           

7. Verify that the new partition was created:

           fdisk -l
           

8. Convert the new partition to a physical volume:

           pvcreate /dev/sda3
           

   where /dev/sda3 is your new partition

9. Extend physical volume:

           vgextend  VolGroup /dev/sda3
           

   where /dev/sda3 is your new partition

10. Verify that there is new space available:

            vgdisplay VolGroup | grep "Free"
            

11. Extend the logical volume to take up 100% of the free space:
    In this case we’re extending the root logical volume

            lvextend -l +100%FREE /dev/mapper/VolGroup-lv_root
            

12. Expand the ext3 filesystem online:
    In this case we’re extending the root logical volume

            resize2fs /dev/mapper/VolGroup-lv_root
            

13. Verify that your disk space has increased:

            df -h
            

Based on VMWare instructions found here.

Here’s how to do it if your hosting provider is using ESX v4.1 and you are using a RedHat v6 variant:

1. Log into the guest operating system as root.

2. Add VMware GPG keys

               rpm --import http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-DSA-KEY.pub
               rpm --import http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-RSA-KEY.pub
           

3. Add the VMWare yum Repository

           echo -e "[vmware-tools]" > /etc/yum.repos.d/vmware-tools.repo
           echo -e "name=VMware Tools" >> /etc/yum.repos.d/vmware-tools.repo
           echo -e "baseurl=http://packages.vmware.com/tools/esx/4.1/rhel6/x86_64" >> /etc/yum.repos.d/vmware-tools.repo
           echo -e "enabled=1" >> /etc/yum.repos.d/vmware-tools.repo
           echo -e "gpgcheck=1" >> /etc/yum.repos.d/vmware-tools.repo
           

4. Clean your yum cache

           yum clean all
           

5. Install VMTools:

           yum install vmware-open-vm-tools-nox vmware-tools-nox vmware-open-vm-tools vmware-tools
           

For other versions of VMWare ESX look at Other VMWare ESX YUM Repositories, then select the most appropriate guest operating system. Then replace the baseurl setting in your /etc/yum.repos.d/vmware-tools.repo file to match your new chosen repository.

This tutorial will step you through how to mount a remote directory onto your Scientific Linux / CentOS /RedHat 6 machine.

For a mount on demand solution

1. Login to your linux server as root

2. Install fuse-sshfs

           #yum install fuse-sshfs
           

3. Create a mount directory

           #mkdir /mnt/sshfsMount
           

4. Mount the remote directory

           #sshfs user@192.168.1.1:/mySharePath /mnt/sshfsMount/
           

   SSHFS will then ask you to authenticate with the password for the user account you supplied

5. To unmount the drive:

           #umount /sshfs/sshfsMount/
           

To automate the process so that the server automatically mounts

1. Setup SSH keys so that the linux machine can automatically login to the remote machine via SSH
   To set this up you can see our tutorial on setting up automatic SSH logins without using a password

2. Setup SSH keys so that the linux machine can automatically login to the remote machine via SSH
   To set this up you can see our tutorial on setting up automatic SSH logins without using a password

   Now you should be able to use the following command without being asked for a password

                   #sshfs user@192.168.1.1:/mySharePath /mnt/sshfsMount/
                   

3. Install autofs

                   #yum install autofs
                   

4. Get the user and group id of the user you wish to control the share.
   In this example I’m going to use the root account
   

                   #cat /etc/passwd | grep root
                   root:x:0:0:root:/root:/bin/bash
                   

   The user id is the first number, and the coup id is the second number (0 and 0)

5. Edit the /etc/auto.master file and add the following line under the line for /misc, substituting your chosen user and group id’s
   This allows us to mount to any single directory of our choosing under root ( / ) as the local mounting point without having to use sub directories.

                   /-              /etc/auto.sshfs  uid=0,gid=0,--timeout=30,--ghost
                   

   Also comment out the following line by placing a # in front of it to avoid nsswitch errors

                   +auto.master
                   

6. Now create the file /etc/auto.sshfs and add the following line to it

                   /mnt/sshfsMount/ -fstype=fuse,rw,nodev,nonempty,noatime,allow_other,max_read=65536  :sshfs\#user@192.168.1.1\:/mySharePath
                   

   Now this remote directory will mount into the folder /mnt/sshfsMount every time we access that folder. If the folder is not being used for more than 30 seconds, it will automatically be unmounted.

7. Restart autos:

                   #service autofs restart
                   

8. test that the directory is mounted by listing its contents:

                   #ls /mnt/sshfsMount
                   

9. To unmount the drive:

                   #umount /mnt/sshfsMount/
                   

This tutorial will step you through how to install VMWare tools on a Scientific Linux / CentOS /RedHat 6 virtual machine with no GUI.

1. Start your virtual machine instance and log in as root using the actual VMWare window rather than a remote terminal session.

2. Go to the /tmp directory

           #cd /tmp
           

3. On the parent host system, select the menu option: Virtual Machine > Install VMWare Tools

   VMWare will then download the latest version of VMWare Tools and ask you to authenticate using a valid admin account on the parent host system.
   A dialog box will appear asking you to confirm the installation, click ‘Install’.

4. Mount the virtual CD-ROM and copy the files to /tmp

           #mount -o ro /dev/cdrom /mnt
           #cp /mnt/* .
           

5. Extract the files from the archive

           #tar xvfz VMwareTools*
           #cd vmware-tools-distrib
           

6. Run the installation script

           #perl vmware-install.pl
           

   Press Enter at all of the prompts to accept the script default settings

7. Reboot the virtual machine

           #reboot
           

   Then re-login as root when it comes back up

8. Check that the installation was successful

           #vmware-checkvm
           VMware software version 6 (good)
           

You will need to reinstall VMWare tools each time you upgrade/change the kernel on the virtual machine

Simply running the following command won’t upgrade your system from 6.0 to 6.1 as it will only draw updated packages from the repository associated with it’s current release (6.0):

#yum update

Solution

First let’s check the version we are currently using:

#cat /etc/redhat-release
Scientific Linux release 6.0 (Carbon)

All we need to do is tell yum the version number you’d like to source updates from as follows:

#yum --releasever=6.1 update

Now lets check that the update worked:

#cat /etc/redhat-release
Scientific Linux release 6.1 (Carbon)

Simple as that!

Here’s how to set it up using ModSSL, Apache 2, and Scientific, CentOS, or RedHat Linux…

Configure the Server

1. Install mod_ssl if you haven’t already:

   yum install mod_ssl openssl openssl-devel
   

2. To generate a self signed certificate, login to the server as root and generate the private key, replacing www.mysite.com with the domain you require:

   cd ~/
   openssl genrsa -out www.mysite.com.key 1024
   

3. Now generate the certificate signing request (CSR) and answer the questions is asks you as guided:

   openssl req -new -key www.mysite.com.key -out www.mysite.com.csr
   
   ---------------------------------------------------------------------------------------
   Country Name (2 letter code) [GB]: <Your 2 Character Country Code>
   State or Province Name (full name) [Berkshire]: <Your State>
   Locality Name (eg, city) [Newbury]: <Your City>
   Organization Name (eg, company) [My Company Ltd]: <Your Company Name>
   Organizational Unit Name (eg, section) []: <Leave Empty Unless Required>
   Common Name (eg, your name or your server's hostname) []:<Your Full Domain Name To Use With SSL>
   Email Address []:<Generic Organisation Email Address.  eg. info@mysite.com>
   
   Please enter the following 'extra' attributes
   to be sent with your certificate request
   A challenge password []: <Leave Empty Unless Required>
   An optional company name []: <Leave Empty Unless Required>
   ---------------------------------------------------------------------------------------
   

4. Now use the CSR and key to create a self-signed certificate:

   openssl x509 -req -in www.mysite.com.csr -signkey www.mysite.com.key -out www.mysite.com.crt
   

5. Create the apache SSL certificate and key directories, copy the required files and set permissions:

   mkdir -p /etc/httpd/conf/ssl.crt
   mkdir -p /etc/httpd/conf/ssl.key
   cat ~/www.envision-systems.com.au.crt > /etc/httpd/conf/ssl.crt/www.envision-systems.com.au.crt
   cat ~/www.envision-systems.com.au.key > /etc/httpd/conf/ssl.key/www.envision-systems.com.au.key
   chmod -R 600 /etc/httpd/conf/ssl.key
   chmod -R 600 /etc/httpd/conf/ssl.crt
   

6. Configure apache SSL:

   vim /etc/httpd/conf.d/ssl.conf
   

   <VirtualHost _default_:443>
           DocumentRoot /var/www/html/www.mysite.com
           DirectoryIndex index.php
           ServerName etc/httpd
   
           SSLEngine on
           SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
   
           SSLCertificateFile /etc/httpd/conf/ssl.crt/www.mysite.com.crt
           SSLCertificateKeyFile /etc/httpd/conf/ssl.key/www.mysite.com.key
   
           <Files ~ "\.(cgi|shtml|phtml|php3?)$">
               SSLOptions +StdEnvVars
           </Files>
           <Directory "/usr/local/apache/cgi-bin">
               SSLOptions +StdEnvVars
           </Directory>
   
           SetEnvIf User-Agent ".*MSIE.*" \
                    nokeepalive ssl-unclean-shutdown \
                    downgrade-1.0 force-response-1.0
           CustomLog /usr/local/apache/logs/ssl_request_log \
                     "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
   
   </VirtualHost>
   

   Ensure that your vim /etc/httpd/conf/httpd.conf contains a NameVirtualHost entry for port 443:

   NameVirtualHost *:443
   

7. Test apache configuration and restart if ok:

   apachectl configtest
   apachectl restart
   

Configure the Client

1. Login to your client machine as root, and copy the www.mysite.com.crt file to your client machine then set permissions:

   cd ~/
   chmod 600 www.envision-systems.com.au.crt
   

2. Install wget if you haven’t already:

   yum install wget
   

   Test the certificate using wget:

   wget --ca-certificate=www.mysite.com.crt https://www.mysite.com/
   

   If all went well you should see something like this:

   --2011-07-25 10:36:56--  https://www.mysite.com/
   Resolving www.mysite.com... 29.141.129.5
   Connecting to www.mysite.com|29.141.129.5|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: unspecified 1
   Saving to: `index.html.1'
       [ <=> ] 13,355      --.-K/s   in 0.001s
   2011-07-25 10:36:56 (17.1 MB/s) - `index.html.1' saved [13355]
   

Sometimes you need to see if a specific IP address is trying to access your server. Here’s a handy little bash command that will list all of the IP addresses contained in the current apache log and rank them by the number of times they appear (# of requests they have made).

cat /var/log/http/access_log | awk '{print $1}' | sort | uniq -c | sort -n

For the sake of this tutorial, my username on the local machine will be localuser@localserver and my username on the remote machine will be remoteuser@remoteserver. If you are running these automated taks in your crontab remember to specify the localuser as the user account running the script in your /etc/crontab file:

# Running backup for server conf files every night at 1.00 am
00  1   *   *   *   localuser   /backup/scripts/backupToRemoteServer.sh

WARNING: Allowing a local server to automatically log into a remote server means that if the local server gets compromised, then the intruders will also have automatic access to the remote server, compromising it also.

For this reason I always ensure that the remoteuser account has its privileges reduced to the point where it can’t do anything except what I require it to do. Setting up user privileges is outside the scope of this tutorial and depends greatly on the specific task you need to perform on the remote server.

1. Login to the local machine as localuser

#ssh localuser@localserver

2. Create an .ssh directory on the local machine in the localuser’s home directory

#mkdir ~/.ssh
#chmod 700 ~/.ssh

3. Generate the ssh keys on the local machine

#ssh-keygen -t dsa -C '<enter a description about your local server here>'

Output:

Generating public/private dsa key pair.
Enter file in which to save the key (/home/localuser/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):  <Leave Empty>
Enter same passphrase again:  <Leave Empty>
Your identification has been saved in /home/localuser/.ssh/id_dsa.
Your public key has been saved in /home/localuser/.ssh/id_dsa.pub.
The key fingerprint is:
64:be:2f:ec:d4:30:61:28:d4:8b:58:e9:bd:ea:f4:65 <local machine description as entered in above>

4. Set permissions for the SSH key on the local server

#chmod 600 ~/.ssh/id_dsa.pub

5. Open the ~/.ssh/id_dsa.pub file and copy its contents to your clipboard

#cat ~/.ssh/id_dsa.pub

6. Login to the remote server

#ssh remoteuser@remoteserver

7. Create an .ssh directory on the remote machine in the remoteuser’s home directory

#mkdir ~/.ssh
#chmod 700 ~/.ssh

8. Paste your clipboard contents (the local servers id_dsa.pub file) at the bottom of the authorized_keys file on the remote server

#vim ~/.ssh/authorized_keys

9. Set permissions on the remote servers authorized_keys file

#chmod 600 ~/.ssh/authorized_keys

10. Go back to the local machine as localuser and then try to log into the remote machine

Login to the local server:

#ssh localuser@localserver

Then try to access the remote server from the local server:

#ssh remoteuser@remoteserver

It should now login automatically.