This tutorial will step you through how to mount a remote directory onto your Scientific Linux / CentOS /RedHat 6 machine.

For a mount on demand solution

1. Login to your linux server as root

2. Install fuse-sshfs

           #yum install fuse-sshfs
           

3. Create a mount directory

           #mkdir /mnt/sshfsMount
           

4. Mount the remote directory

           #sshfs user@192.168.1.1:/mySharePath /mnt/sshfsMount/
           

   SSHFS will then ask you to authenticate with the password for the user account you supplied

5. To unmount the drive:

           #umount /sshfs/sshfsMount/
           

To automate the process so that the server automatically mounts

1. Setup SSH keys so that the linux machine can automatically login to the remote machine via SSH
   To set this up you can see our tutorial on setting up automatic SSH logins without using a password

2. Setup SSH keys so that the linux machine can automatically login to the remote machine via SSH
   To set this up you can see our tutorial on setting up automatic SSH logins without using a password

   Now you should be able to use the following command without being asked for a password

                   #sshfs user@192.168.1.1:/mySharePath /mnt/sshfsMount/
                   

3. Install autofs

                   #yum install autofs
                   

4. Get the user and group id of the user you wish to control the share.
   In this example I’m going to use the root account
   

                   #cat /etc/passwd | grep root
                   root:x:0:0:root:/root:/bin/bash
                   

   The user id is the first number, and the coup id is the second number (0 and 0)

5. Edit the /etc/auto.master file and add the following line under the line for /misc, substituting your chosen user and group id’s
   This allows us to mount to any single directory of our choosing under root ( / ) as the local mounting point without having to use sub directories.

                   /-              /etc/auto.sshfs  uid=0,gid=0,--timeout=30,--ghost
                   

   Also comment out the following line by placing a # in front of it to avoid nsswitch errors

                   +auto.master
                   

6. Now create the file /etc/auto.sshfs and add the following line to it

                   /mnt/sshfsMount/ -fstype=fuse,rw,nodev,nonempty,noatime,allow_other,max_read=65536  :sshfs\#user@192.168.1.1\:/mySharePath
                   

   Now this remote directory will mount into the folder /mnt/sshfsMount every time we access that folder. If the folder is not being used for more than 30 seconds, it will automatically be unmounted.

7. Restart autos:

                   #service autofs restart
                   

8. test that the directory is mounted by listing its contents:

                   #ls /mnt/sshfsMount
                   

9. To unmount the drive:

                   #umount /mnt/sshfsMount/
                   

This tutorial will step you through how to install VMWare tools on a Scientific Linux / CentOS /RedHat 6 virtual machine with no GUI.

1. Start your virtual machine instance and log in as root using the actual VMWare window rather than a remote terminal session.

2. Go to the /tmp directory

           #cd /tmp
           

3. On the parent host system, select the menu option: Virtual Machine > Install VMWare Tools

   VMWare will then download the latest version of VMWare Tools and ask you to authenticate using a valid admin account on the parent host system.
   A dialog box will appear asking you to confirm the installation, click ‘Install’.

4. Mount the virtual CD-ROM and copy the files to /tmp

           #mount -o ro /dev/cdrom /mnt
           #cp /mnt/* .
           

5. Extract the files from the archive

           #tar xvfz VMwareTools*
           #cd vmware-tools-distrib
           

6. Run the installation script

           #perl vmware-install.pl
           

   Press Enter at all of the prompts to accept the script default settings

7. Reboot the virtual machine

           #reboot
           

   Then re-login as root when it comes back up

8. Check that the installation was successful

           #vmware-checkvm
           VMware software version 6 (good)
           

You will need to reinstall VMWare tools each time you upgrade/change the kernel on the virtual machine

Simply running the following command won’t upgrade your system from 6.0 to 6.1 as it will only draw updated packages from the repository associated with it’s current release (6.0):

#yum update

Solution

First let’s check the version we are currently using:

#cat /etc/redhat-release
Scientific Linux release 6.0 (Carbon)

All we need to do is tell yum the version number you’d like to source updates from as follows:

#yum --releasever=6.1 update

Now lets check that the update worked:

#cat /etc/redhat-release
Scientific Linux release 6.1 (Carbon)

Simple as that!

Here’s how to set it up using ModSSL, Apache 2, and Scientific, CentOS, or RedHat Linux…

Configure the Server

1. Install mod_ssl if you haven’t already:

   yum install mod_ssl openssl openssl-devel
   

2. To generate a self signed certificate, login to the server as root and generate the private key, replacing www.mysite.com with the domain you require:

   cd ~/
   openssl genrsa -out www.mysite.com.key 1024
   

3. Now generate the certificate signing request (CSR) and answer the questions is asks you as guided:

   openssl req -new -key www.mysite.com.key -out www.mysite.com.csr
   
   ---------------------------------------------------------------------------------------
   Country Name (2 letter code) [GB]: <Your 2 Character Country Code>
   State or Province Name (full name) [Berkshire]: <Your State>
   Locality Name (eg, city) [Newbury]: <Your City>
   Organization Name (eg, company) [My Company Ltd]: <Your Company Name>
   Organizational Unit Name (eg, section) []: <Leave Empty Unless Required>
   Common Name (eg, your name or your server's hostname) []:<Your Full Domain Name To Use With SSL>
   Email Address []:<Generic Organisation Email Address.  eg. info@mysite.com>
   
   Please enter the following 'extra' attributes
   to be sent with your certificate request
   A challenge password []: <Leave Empty Unless Required>
   An optional company name []: <Leave Empty Unless Required>
   ---------------------------------------------------------------------------------------
   

4. Now use the CSR and key to create a self-signed certificate:

   openssl x509 -req -in www.mysite.com.csr -signkey www.mysite.com.key -out www.mysite.com.crt
   

5. Create the apache SSL certificate and key directories, copy the required files and set permissions:

   mkdir -p /etc/httpd/conf/ssl.crt
   mkdir -p /etc/httpd/conf/ssl.key
   cat ~/www.envision-systems.com.au.crt > /etc/httpd/conf/ssl.crt/www.envision-systems.com.au.crt
   cat ~/www.envision-systems.com.au.key > /etc/httpd/conf/ssl.key/www.envision-systems.com.au.key
   chmod -R 600 /etc/httpd/conf/ssl.key
   chmod -R 600 /etc/httpd/conf/ssl.crt
   

6. Configure apache SSL:

   vim /etc/httpd/conf.d/ssl.conf
   

   <VirtualHost _default_:443>
           DocumentRoot /var/www/html/www.mysite.com
           DirectoryIndex index.php
           ServerName etc/httpd
   
           SSLEngine on
           SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
   
           SSLCertificateFile /etc/httpd/conf/ssl.crt/www.mysite.com.crt
           SSLCertificateKeyFile /etc/httpd/conf/ssl.key/www.mysite.com.key
   
           <Files ~ "\.(cgi|shtml|phtml|php3?)$">
               SSLOptions +StdEnvVars
           </Files>
           <Directory "/usr/local/apache/cgi-bin">
               SSLOptions +StdEnvVars
           </Directory>
   
           SetEnvIf User-Agent ".*MSIE.*" \
                    nokeepalive ssl-unclean-shutdown \
                    downgrade-1.0 force-response-1.0
           CustomLog /usr/local/apache/logs/ssl_request_log \
                     "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
   
   </VirtualHost>
   

   Ensure that your vim /etc/httpd/conf/httpd.conf contains a NameVirtualHost entry for port 443:

   NameVirtualHost *:443
   

7. Test apache configuration and restart if ok:

   apachectl configtest
   apachectl restart
   

Configure the Client

1. Login to your client machine as root, and copy the www.mysite.com.crt file to your client machine then set permissions:

   cd ~/
   chmod 600 www.envision-systems.com.au.crt
   

2. Install wget if you haven’t already:

   yum install wget
   

   Test the certificate using wget:

   wget --ca-certificate=www.mysite.com.crt https://www.mysite.com/
   

   If all went well you should see something like this:

   --2011-07-25 10:36:56--  https://www.mysite.com/
   Resolving www.mysite.com... 29.141.129.5
   Connecting to www.mysite.com|29.141.129.5|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: unspecified 1
   Saving to: `index.html.1'
       [ <=> ] 13,355      --.-K/s   in 0.001s
   2011-07-25 10:36:56 (17.1 MB/s) - `index.html.1' saved [13355]
   

Sometimes you need to see if a specific IP address is trying to access your server. Here’s a handy little bash command that will list all of the IP addresses contained in the current apache log and rank them by the number of times they appear (# of requests they have made).

cat /var/log/http/access_log | awk '{print $1}' | sort | uniq -c | sort -n

For the sake of this tutorial, my username on the local machine will be localuser@localserver and my username on the remote machine will be remoteuser@remoteserver. If you are running these automated taks in your crontab remember to specify the localuser as the user account running the script in your /etc/crontab file:

# Running backup for server conf files every night at 1.00 am
00  1   *   *   *   localuser   /backup/scripts/backupToRemoteServer.sh

WARNING: Allowing a local server to automatically log into a remote server means that if the local server gets compromised, then the intruders will also have automatic access to the remote server, compromising it also.

For this reason I always ensure that the remoteuser account has its privileges reduced to the point where it can’t do anything except what I require it to do. Setting up user privileges is outside the scope of this tutorial and depends greatly on the specific task you need to perform on the remote server.

1. Login to the local machine as localuser

#ssh localuser@localserver

2. Create an .ssh directory on the local machine in the localuser’s home directory

#mkdir ~/.ssh
#chmod 700 ~/.ssh

3. Generate the ssh keys on the local machine

#ssh-keygen -t dsa -C '<enter a description about your local server here>'

Output:

Generating public/private dsa key pair.
Enter file in which to save the key (/home/localuser/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):  <Leave Empty>
Enter same passphrase again:  <Leave Empty>
Your identification has been saved in /home/localuser/.ssh/id_dsa.
Your public key has been saved in /home/localuser/.ssh/id_dsa.pub.
The key fingerprint is:
64:be:2f:ec:d4:30:61:28:d4:8b:58:e9:bd:ea:f4:65 <local machine description as entered in above>

4. Set permissions for the SSH key on the local server

#chmod 600 ~/.ssh/id_dsa.pub

5. Open the ~/.ssh/id_dsa.pub file and copy its contents to your clipboard

#cat ~/.ssh/id_dsa.pub

6. Login to the remote server

#ssh remoteuser@remoteserver

7. Create an .ssh directory on the remote machine in the remoteuser’s home directory

#mkdir ~/.ssh
#chmod 700 ~/.ssh

8. Paste your clipboard contents (the local servers id_dsa.pub file) at the bottom of the authorized_keys file on the remote server

#vim ~/.ssh/authorized_keys

9. Set permissions on the remote servers authorized_keys file

#chmod 600 ~/.ssh/authorized_keys

10. Go back to the local machine as localuser and then try to log into the remote machine

Login to the local server:

#ssh localuser@localserver

Then try to access the remote server from the local server:

#ssh remoteuser@remoteserver

It should now login automatically.

mail($to, $subject, $message, $headers);

This method sends a copy of the email, directly to the recipients mail server from the web server. However generally speaking, the web server is not the real mail server for the domain you’re trying to send from which can cause major issues.

For example, lets say I have a website at www.myweb.com and when someone registers it sends them an email from the address website@myweb.com. The recipients mail servers spam filtering facility may do an MX lookup on the myweb.com domain and notice that the real mail server for the myweb.com has a different IP address to the address where the email is originating from (the web servers IP address). For this reason web server emails are often flagged as spam and thus never received by the intended recipient.

To avoid this, we can relay the mail through the domains real mail server using SMTP. Using this method the mail will come to the recipients mail server from the real mail server for the senders domain as seen in its MX records. This then avoids the problematic spam filter rule.

To achieve this I use the Mail.php class as found in the PEAR::Mail library located at http://pear.php.net/package/Mail. If you’re using CentOS like me, you can easily install this package using the following command

#yum install php-pear-Mail

In this example I’m connecting to a gmail server which requires the connection to be over SSL:

<?php
require_once('Mail.php');
$host = 'ssl://smtp.gmail.com';
$port = '465';
$username = '<your email address>';
$password = '<yourPassword>';
$subject = 'my subject';
$to = '<to email address>';

$from = $username;
$message = 'test';
$headers = array ('From' => $from,
   'To' => $to,
   'Subject' => $subject);
$smtp = Mail::factory('smtp',
   array ('host' => $host,
     'port' =>; $port,
     'auth' => true,
     'username' => $username,
     'password' => $password));
$mail = $smtp->send($to, $headers, $message);
if (PEAR::isError($mail)) {
        echo('<p>'.$mail->getMessage().'</p>');
}
?>

If SSL wasn’t required I’d just need to remove the ‘ssl://’ from in front of the host declaration and change the port to 25 like this:

$host = 'mail.myweb.com';
$port = '25';

Obviously you’ll need to ensure that your firewall allows your web server to make connections to your smtp host over port 25 or 465 for SSL.

The load tests were conducted using the HP performance center; a technology that HP obtained as part of its acquisition of Mercury for approximately USD$4.5 billion in 2006.

To find out more about the load testing software visit http://en.wikipedia.org/wiki/HP_LoadRunner

Goal:
Handle 4,250 concurrent users generating approximately 70 full page loads per second.

1 full page load consisted of:
- 1 dynamically generated PHP file using MySQL
- 4 JavaScript files
- 7 CSS files
- 8 image files

Original starting environment:
- ServerModel: Dell R300
- RAM: 2GB (2 x 1GB chips)
- Operating System: CentOS release 5.5 (Final)
- Apache: v2.2.3 (running in prefork mode)
- MySQL: v5.0.77
- PHP: v5.1.6 (as an apache module)
- eAccelerator: v0.9.5.3
- 120Mbits of bandwidth

Round 1: Initial Test

Round 1: Configuration

At the start of the process we were pretty much using the default configurations for the entire lamp stack. Linux was running iptables and ip6tables in its default configuration. eAccelerator was operating with 32MB of memory with optimization and caching enabled.

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html

[mysqld]
max_connections = 100
max_user_connections = 0
max_connect_errors = 10
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 2M
read_buffer_size = 131072
read_rnd_buffer_size = 262144
myisam_sort_buffer_size = 8M
thread_cache_size = 0
query_cache_size= 0
thread_concurrency = 10

Round 1: Results

With these settings we got up to 30 page loads per second which was 42% of our target. Interestingly, we were only operating at about 8% CPU and about 50% of our memory capacity when we hit this limit.

Round 1: Review

Looking at the apache error logs we were getting a large number of MySQL errors:

mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Too many connections in xxx.php on line 15

So the MySQL configuration seemed to be our bottleneck:

Round 2

Round 2: Configuration

We did our first major review of the Apache and MySQL performance settings and adjusted them accordingly. We doubled the Apache settings and used the ‘huge’ configuration as supplied with mysql (/usr/share/doc/mysql-server-5.0.77/my-huge.cnf).

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

<IfModule prefork.c>
StartServers       16
MinSpareServers    10
MaxSpareServers   40
ServerLimit      512
MaxClients       512
MaxRequestsPerChild  8000
</IfModule>

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html

[mysqld]
# Memory usage
skip-locking
max_connections = 500
max_user_connections = 500
max_connect_errors = 999999
key_buffer = 384M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 32M
# Try number of CPU's*2 for thread_concurrency (eHound has 4 CPU's)
thread_concurrency = 8

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

As an extra precaution we locked the network card in the server to use 1Gbit:

#ethtool -s eth0 speed 1000 duplex full

Edit the configuration for the network card:

#vim /etc/sysconfig/network-scripts/ifcfg-eth0

Add the following line:

ETHTOOL_OPTS='autoneg on speed 1000 duplex full'

Restart the network:

#service network restart

Round 2: Results

With these settings we got up to 58 full page loads per second which was 59% of our target. Interestingly, we were still only operating at about 10% CPU capacity when we hit this limit but we were using approximately 70-80% of our memory.

Our MySQL errors had disappeared and there were no more errors in the Apache logs.

Round 2: Review

We were concerned that the system was starting to use swap memory which was slowing the server to a halt.

Round 3

Round 3: Configuration

We added an additional 2GB of RAM to the server so it now contained 4 x 1GB chips.

Round 3: Results

With the new RAM we still only got up to 58 full page loads per second which was 59% of our target. We were still only operating at about 10% CPU capacity but now we were only using about 40% of our memory.

Round 3: Review

Still no errors in the Apache logs and the load test farm was not receiving Apache errors. In fact it was reporting that it could not even connect to the server. This led us to believe that it was either a lack of bandwidth or a NIC/network/firewall configuration issue. After checking with our datacenter, we found that there were no inhibiting factors that would cause the problem described.

We increased the Apache & MySQL Limits and ran a different style of test.

Round 4

Round 4: Configuration

In this test we only loaded the dynamic components of the page as generated by PHP and MySQL and served by Apache. This meant that we told the load test farm not to download static content such as images, CSS or JavaScript files.

Also we increased the MySQL and Apache limits as follows:

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

<IfModule prefork.c>
StartServers     280
MinSpareServers   100
MaxSpareServers   300
ServerLimit      1536
MaxClients       1536
MaxRequestsPerChild  32000
</IfModule>

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html

[mysqld]
# Memory usage
skip-locking
max_connections = 764
max_user_connections = 764
max_connect_errors = 999999
key_buffer = 256M
max_allowed_packet = 1M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
# Try number of CPU's*2 for thread_concurrency (eHound has 4 CPU's)
thread_concurrency = 8

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

Round 4: Results

The results of this test were very interesting. We got up to 263 page loads without any issue. This consumed a lot more bandwidth than test 3 so we knew that bandwidth was not the issue. However the number of connections that both tests started to fail at were very similar.

Round 4: Review

So we knew we had a connection limit issue.

We also knew that the eAccelerator optcode cache was not dying at these high volumes, nor was MySQL, PHP or Apache.

We reviewing the kernel messages and found thousands of the following messages that were logged at the time of testing:

#cat /var/log/messages* | grep 'Aug 15'
...
Aug 15 01:04:27 localhost kernel: printk: 1395 messages suppressed.
Aug 15 01:04:27 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:32 localhost kernel: printk: 1561 messages suppressed.
Aug 15 01:04:32 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:37 localhost kernel: printk: 1274 messages suppressed.
Aug 15 01:04:37 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:42 localhost kernel: printk: 1412 messages suppressed.
...

Further investigation revealed that the iptables/ip6tables was activated and limiting the number of connections to the box because its table was full. Ordinarily when I set up a linux server I turn iptables off because I place hardware firewalls in front of the servers. However I didn’t have the opportunity to setup this box initially, so they were still activated. I however didn’t need them, so I deactivated them.

If you still need to keep iptables running you can simply adjust the following settings:
Check the current connections limit (only works if iptables is running):

#sysctl net.ipv4.netfilter.ip_conntrack_max
65536

Change the connections limit:

#vim /etc/sysctl.conf

Add the following lines:

# conntrack limits
#inet.ipv4.netfilter.ip_conntrack_max = 65536
net.ipv4.netfilter.ip_conntrack_max = 196608

Reload the config file:

#sysctl -p

Check the new connections limit:

#sysctl net.ipv4.netfilter.ip_conntrack_max
196608

Check the current buckets limit (only works if iptables is running):

#cat /proc/sys/net/ipv4/netfilter/ip_conntrack_buckets
8192

To change the buckets limit:

#vim /etc/modprobe.conf

Add the following lines:

options ip_conntrack hashsize=32768

Reboot the server:

#shutdown -r now

Check the new buckets limit:

#cat /proc/sys/net/ipv4/netfilter/ip_conntrack_buckets
24576

–
Alternatively if you don’t need iptables like me, you can just disable them:

#service iptables stop
#service ip6tables stop
#chkconfig iptables off
#chkconfig ip6tables off

Round 5

Round 5: Configuration

This test used exactly the same configuration with iptables disabled.

Round 5: Results

Success!!! We got to 4,250 concurrent users which is about 70 pages per second (loading all additional image, CSS and JavaScript files also) with zero errors and a 0.7 second average response time. This used about 120Mbits worth of bandwidth pipe. The datacenter ended up running out of pipe before the server had any issues.

At this rate we were running at about:
- 15% CPU utilisation
- 30% Memory usage (with 4GB RAM installed)
- 400 apache threads
- 100% Bandwidth

Round 5: Review

Key findings:
- Increase your Apache and MySQL limits
- Turn off iptables
- Ensure that you have enough RAM
- Ensure that you are checking logs from MySQL, Apache, and the kernel to pick up any errors and give you clues as to how to best solve them

Round 6

Round 6: Configuration

This test used exactly the same configuration as round 5 with 250Mbit pipe instead of a 120Mbit pipe.

Round 6: Results

Success!!! We got to 140 full page loads per second (including additional images, CSS and JavaScript files also) with zero errors and still a stable 0.7 second average response time. This used the full 250Mbits worth of bandwidth pipe. The datacenter ended up running out of pipe again before the server had any issues.

At this rate we were running at about:
- 30% CPU utilisation
- 40% Memory usage (with 4GB RAM installed)
- 800 apache threads
- 100% Bandwidth

Round 6: Review

Key findings:
- Even with 250Mbits of pipe, bandwidth is still the bottleneck in this configuration.

Round 7

Round 7: Configuration

Even though our server was performing fine, we were given another server to experiment on with much higher specs.

It was a Dell R710 with 48GB of RAM and 8 2.53MHz Xeon processors running in hyper-threading mode (essentially making 16 processors).

We also had this box connected to a dedicated 4Gbit optical internet feed to give it as much bandwidth as it needed.

Everything on the box was configured the same except for Apache and MySQL (which we took the last settings and multipled them by 4) and sysctl.

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

<IfModule prefork.c>
StartServers     1120
MinSpareServers   400
MaxSpareServers   1200
ServerLimit      6144
MaxClients       6144
MaxRequestsPerChild  128000
</IfModule>

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html

[mysqld]
# Memory usage
skip-locking
max_connections = 3056
max_user_connections = 3056
max_connect_errors = 999999
key_buffer = 1024M
max_allowed_packet = 4M
table_cache = 1024
sort_buffer_size = 4M
read_buffer_size = 4M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 256M
thread_cache_size = 32
query_cache_size= 64M
# Try number of CPU's*2 for thread_concurrency (eHound has 4 CPU's)
thread_concurrency = 32

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 64M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[myisamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[mysqlhotcopy]
interactive-timeout

We also added the following lines to sysctl:
ip_conntrack_max = 196608
net.ipv4.ip_local_port_range = 1025 65535   
net.ipv4.tcp_max_tw_buckets = 1000000
net.core.somaxconn = 10000
net.ipv4.tcp_max_syn_backlog = 2000
net.ipv4.tcp_fin_timeout = 30

Round 7: Results

We got to 200 full page loads per second (including additional images, CSS and JavaScript files also) with zero errors and still a stable 0.8 second average response time. This test used 330Mbits or about 8% worth of the bandwidth available. We stopped the test simply because we didn’t need to go any higher, but potentially could have gone much higher.

At this rate we were running at about:
- 16% CPU utilisation
- 6% Memory usage (with 48GB RAM installed)
- 1227 apache threads
- 8% Bandwidth

Round 7: Review

Key findings:
- Bandwidth seem to be a much bigger bottleneck than server capability.

Summary

A Dell R300 with good specs can be acquired for about AUD$4,000-$4,500 and should handle over 2800 hits per second on a 20/1 dynamic/static file ratio or 140 full page loads per second in our case if it is configured correctly.

You don’t need a $25,000 server to get good performance, you just need to take some time to make sure that you are getting the most out of your hardware and ensure that your available bandwidth can handle the load you require.

To install sar:

#yum install sysstat

To run sar and save the binary output to a file:

#sar -A -o /var/log/performanceTests/sarLog 5 0

sar parameters:
-A: records ALL server stats
-o: elects the binary output file Path
5 0: [interval count] in seconds: take stats every five seconds indefinitely

You can then use the sar tool to read and display the stats you require. I’ll be writing another article on how to do this soon.

#cd /var/www/html/myVirtualHost

Use the following command to count the lines of code in php, inc, js and css files:

#find . -type f -name '*.js' -o -name '*.css' -o -name '*.inc' -o -name '*.php'   | xargs wc -l | tail -n1

31065 total