Some Apache RewriteRules for Improved Security

A set of Apache RewriteRules, including curl commands to test them. Always test the rules, using a browser if possible, and curl. The curl output has been edited to make it easier to read.

Remember to write the rules carefully so you don’t deny access for valid requests, and use an appropriate 403 page, so real (good) people that arrive there have an opportunity to understand what happened and what they can do to request access. This may mean the rules blocked a valid request.

Route admin access through HTTPS

With the prevalence of laptops and WiFi, HTTPS is important for site security. This rule assumes the site is administered through a subdomain (admin.domain.com), and routes any request where the server name begins with admin through HTTPS.

RewriteCond %{SERVER_NAME} ^admin
RewriteCond %{HTTPS} =off [NC]
RewriteRule .* https://admin.domain.com [L]

Test

[user@localhost Backup]$ curl -i http://admin.domain.com
HTTP/1.1 302 Found
Location: https://admin.domain.com

Block Probing Requests, XSS Injection, and Unwelcome Referrers

Even if your server doesn’t have these scripts or URLs, it is good to block the requests. People or servers that are requesting them are not visiting your site, they’re attacking it.

This includes referrers that simply don’t make sense, or are seen in the logs or stats requesting content they shouldn’t.


RewriteCond %{REQUEST_URI} (\.aspx?|\.php)$ [NC,OR]
RewriteCond %{REQUEST_URI} (ldap|php\-?myadmin|scripts|mysql|wp\-login) [NC,OR]
RewriteCond %{QUERY_STRING} mouseover [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (sleuth|morfeus|wget|python|curl|perl|scanner|apache\-httpclient) [NC,OR]
RewriteCond %{HTTP_REFERER} \.(ws|in|ru|ua|tv)/?$ [NC,OR]
RewriteCond %{HTTP_ACCEPT_LANGUAGE} en-us [NC,OR]
RewriteRule ^.*  - [F]

Tests

RewriteCond %{REQUEST_URI} (\.aspx?|\.php)$ [NC,OR]

This site is a PHP site which routes requests through RewriteRules, there should not be any direct requests for .php. Since it’s a PHP application, requests for .asp and .aspx should never be received either.

[user@localhost ~]$ curl -I http://domain.com/login.aspx
HTTP/1.1 403 Forbidden

RewriteCond %{HTTP_REFERER} \.(ru|ua|tv)$ [NC]

After too many requests from referrers ending in .ru, .ua, and .tv, I decided to block them.

[user@localhost ~]$ curl -I -e ‘http://some.ru’ http://domain.com
HTTP/1.1 403 Forbidden

RewriteCond %{QUERY_STRING} mouseover [NC,OR]

This rule is in response to some sort of XSS injection attack which included onmouseover.

[user@localhost ~]$ curl -I http://domain.com/?onmouseover
HTTP/1.1 403 Forbidden

RewriteCond %{REQUEST_URI} php\-?myadmin|scripts|mysql|wp\-login [NC,OR]

These are all common requests for variations of phpmyadmin, utility scripts, access to mysql and the login for WordPress. They should not be received by this server.

[user@localhost ~]$ curl -I http://domain.com/scripts
HTTP/1.1 403 Forbidden

RewriteCond %{HTTP_ACCEPT_LANGUAGE} en-us [NC,OR]

The site I’m protecting is delivered in US English. If US English isn’t one of the languages accepted by the client, the request will be denied.

Checking the latest visitors log verified the rules are working properly.

Host: 113.53.253.77

/index.php HTTP Response: 403 Date: Feb 24 07:46:56 Bytes: 629
/admin/index.php HTTP Response: 403 Date: Feb 24 07:46:57 Bytes: 633
/admin/pma/index.php HTTP Response: 403 Date: Feb 24 07:46:58 Bytes: 636
/admin/phpmyadmin/index.php HTTP Response: 403 Date: Feb 24 07:46:59 Bytes: 639
/db/index.php HTTP Response: 403 Date: Feb 24 07:47:01 Bytes: 631
/dbadmin/index.php HTTP Response: 403 Date: Feb 24 07:47:02 Bytes: 634
/myadmin/index.php HTTP Response: 403 Date: Feb 24 07:47:03 Bytes: 634
/mysql/index.php HTTP Response: 403 Date: Feb 24 07:47:04 Bytes: 634
/mysqladmin/index.php HTTP Response: 403 Date: Feb 24 07:47:05 Bytes: 637
/typo3/phpmyadmin/index.php HTTP Response: 403 Date: Feb 24 07:47:06 Bytes: 640
/phpadmin/index.php HTTP Response: 403 Date: Feb 24 07:47:07 Bytes: 635
/phpmyadmin1/index.php HTTP Response: 403 Date: Feb 24 07:47:10 Bytes: 637
/phpmyadmin2/index.php HTTP Response: 403 Date: Feb 24 07:47:11 Bytes: 637
/pma/index.php HTTP Response: 403 Date: Feb 24 07:47:12 Bytes: 632
/web/phpMyAdmin/index.php HTTP Response: 403 Date: Feb 24 07:47:13 Bytes: 640
/xampp/phpmyadmin/index.php HTTP Response: 403 Date: Feb 24 07:47:14 Bytes: 641
/web/index.php HTTP Response: 403 Date: Feb 24 07:47:15 Bytes: 632
/websql/index.php HTTP Response: 403 Date: Feb 24 07:47:17 Bytes: 634
/phpmyadmin/index.php HTTP Response: 403 Date: Feb 24 07:47:18 Bytes: 636
/phpMyAdmin/index.php HTTP Response: 403 Date: Feb 24 07:47:20 Bytes: 637
/phpMyAdmin-2/index.php HTTP Response: 403 Date: Feb 24 07:47:21 Bytes: 639
/php-my-admin/index.php HTTP Response: 403 Date: Feb 24 07:47:22 Bytes: 638
/phpMyAdmin-2.2.3/index.php HTTP Response: 403 Date: Feb 24 07:47:23 Bytes: 642
/phpMyAdmin-2.2.6/index.php HTTP Response: 403 Date: Feb 24 07:47:24 Bytes: 643
/phpMyAdmin-2.5.1/index.php HTTP Response: 403 Date: Feb 24 07:47:24 Bytes: 642
/phpMyAdmin-2.5.4/index.php HTTP Response: 403 Date: Feb 24 07:47:25 Bytes: 643
/phpMyAdmin-2.5.5-rc1/index.php HTTP Response: 403 Date: Feb 24 07:47:31 Bytes: 646
/phpMyAdmin-2.5.5-rc2/index.php HTTP Response: 403 Date: Feb 24 07:47:32 Bytes: 646
/phpMyAdmin-2.5.5/index.php HTTP Response: 403 Date: Feb 24 07:47:32 Bytes: 643
/phpMyAdmin-2.5.5-pl1/index.php HTTP Response: 403 Date: Feb 24 07:47:33 Bytes: 646
/phpMyAdmin-2.5.6-rc1/index.php HTTP Response: 403 Date: Feb 24 07:47:34 Bytes: 646
/phpMyAdmin-2.5.6-rc2/index.php HTTP Response: 403 Date: Feb 24 07:47:35 Bytes: 646
/phpMyAdmin-2.5.6/index.php HTTP Response: 403 Date: Feb 24 07:47:36 Bytes: 643
/phpMyAdmin-2.5.7/index.php HTTP Response: 403 Date: Feb 24 07:47:37 Bytes: 643
/phpMyAdmin-2.5.7-pl1/index.php HTTP Response: 403 Date: Feb 24 07:47:38 Bytes: 646

* One last note. If you block curl, testing with curl must set the user agent to something other than curl, or every test request will be blocked by the user agent rule.

dojo 1.7.1 AMD with Zend Framework 1.11

I have been working on a new web application using Zend Framework 1.11 and Dojo. In order to use the AMD loader of dojo 1.7.1, I modified the Zend_Dojo_View_Helper_Dojo to include a flag indicating whether to use the AMD loader or not, and updated Zend_Dojo_View_Helper_Dojo_Container to render the code for the AMD loader, if it has been selected.

Code

Zend_Dojo_View_Helper_Dojo: http://web-notes.wirehopper.com/AMD_Dojo.txt
Zend_Dojo_View_Helper_Dijit: http://web-notes.wirehopper.com/AMD_Dijit.txt
Zend_Dojo_View_Helper_ComboBox: http://web-notes.wirehopper.com/AMD_ComboBox.txt
Zend_Dojo_View_Helper_Dojo_Container: http://web-notes.wirehopper.com/AMD_Container.txt

Default application layout: http://web-notes.wirehopper.com/AMD_default.txt

This is very new code, and it has not been tested with a built version of the javascript. Be sure to make a backup of Zend before applying these changes.

The code is updated frequently, you may want to check back in a few days to see if new code has been posted. Be sure to reload if you’ve visited the pages, since .txt files tend to be cached.

The updates are running well with a non-built version of code. I’m deferring build development.

dojo 1.7.1 AMD Page Example

This is a very simple example of how you can implement a dojo page with AMD. The intended architecture is that the HTML would be used for all pages and the page content and behavior would be modified by the javascript. With that in mind, the ‘page’ javascript is in js/page/main.js. The javascript file name can be set by passing the name of the page into the script.

Be ready to think differently, because this is a new way to build pages.

There’s a significant reduction in the number of requests required to render the page, only those files which are required are loaded. To see the difference, comment out the async:true and use the FireBug Net tab to see the number of files loaded.

Excellent References

It’s well worth reading these references carefully.

http://dojotoolkit.org/blog/learn-more-about-amd
http://www.slideshare.net/jthomas/moving-to-dojo-17-and-the-path-to-20
http://dojotoolkit.org/documentation/tutorials/1.7/recipes/app_controller
http://dojotoolkit.org/documentation/tutorials/1.7/declare
http://dojotoolkit.org/documentation/tutorials/1.7/modules/

HTML

The HTML code presents a button, and the javascript adds an onClick handler to it. Use FireBug’s Console to view the output.

<!DOCTYPE html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<META http-equiv="Content-Style-Type" content="text/css">
<title>dojo AMD</title>
<style type="text/css">
<!--
button
{
border:1px solid #ccc;
background-color:#eee;
font-size:0.7em;
padding:3px;
}
-->
</style>
</head>
<body>
<h1>dojo AMD page</h1>
<button id="myButton">my.Button!</button>
<script type="text/javascript"> 
//<!--
var djConfig = {
    parseOnLoad:true,
        isDebug:true,
        locale:"en_US",
        async:true,
    baseUrl: "js",
    tlmSiblingOfDojo: false,
    packages: [
        { name: "dojo", location: "../dojo/dojo" },
        { name: "dijit", location: "../dojo/dijit" },
        { name: "dojox", location: "../dojo/dojox" },
        { name: "app", location: "page", main: "main" },
    ]
};
//-->
</script>
<script src="http://ajax.googleapis.com/ajax/libs/dojo/1.7.1/dojo/dojo.js" type="text/javascript"></script> 
<script type="text/javascript"> 
//<!--
require(["app","dojo/domReady!"],function(app){app.init();});
//-->
</script>
</body>
</html>

js/page/main.js

The define is defining an anonymous module that requires dojo/dom and executes after dojo/domReady. It creates a module with one function - init. app.init() is called from the HTML. Note that the name app.init is file independent, so all the pages can have an init modules and the HTML can call app.init() regardless of which page is being initialized.

define(["dojo/on", "dojo/domReady!"],
        function(on){

                var filename = "js/page/main.js";

                on(document.getElementById("myButton"),"click",function(evt) {
                                console.log(filename+" click on myButton");
                };


                return {
                        init: function() {
                                console.log(filename+" init");
                        }
                };
});

This is just the tip of the iceberg. Explore. Enjoy.

For production code, be sure to use a build.

Zend_Rest - Example

I built a REST API and calling code using Zend Framework. This code also uses Doctrine (http://www.doctrine-project.org/projects/orm).

Prior to coding, I reviewed several resources on the ‘net in an attempt to follow best practices. I visited the following, as well as others:

routes.ini

These settings ensure the request is properly organized when received by the REST controller. Be sure the ACL is configured to allow access as appropriate as well.

# REST routes
resources.router.routes.api.type = "Zend_Rest_Route"
resources.router.routes.api.route = "/api/:id"
resources.router.routes.api.defaults.module = "api"
resources.router.routes.api.defaults.controller = "index"
resources.router.routes.api.api = "admin, vendor, client"

Calling Code

The calling code is a subset, only the relevant code is included.

Authentication is handled by JSON encoding, then encrypting the username and password in the Authorization header.

The process has two phases, a GET to test whether the element exists and a POST, PUT or DELETE depending on existence, error, or requested action. Since the GET is a test, no data is returned. POST and PUT pass the data in the body text, JSON encoded.

	
            $client = new Zend_Http_Client();
            $iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_256, MCRYPT_MODE_ECB);
            $iv = mcrypt_create_iv($iv_size, MCRYPT_RAND);
            $authorization = base64_encode(mcrypt_encrypt(MCRYPT_RIJNDAEL_256, $v['url'],
                Zend_Json::encode(array('username'=>$this->username,'password'=>$this->password)),
                MCRYPT_MODE_ECB, $iv));
            $client->setHeaders(array('keepalive' => true,'Authorization' => $authorization));
            $client->setUri($api['protocol'].'://'.$url.'/'.$api['path'].'/admin/'.$admin->uuid);
			$client->setMethod(Zend_Http_Client::GET);
			$response = $client->request();
			$status = $response->getStatus();
			$body = $response->getBody();
			switch ($status)
			{
				case 200:	/* Record exists */
					$next_method = ($type == 'allow') ? Zend_Http_Client::PUT : Zend_Http_Client::DELETE;
					break;
				case 404:	/* Record does not exist */
					if ($type == 'allow')
					{
						$client->setUri($api['protocol'].'://'.$url.'/'.$api['path'].'/admin');
						$next_method = Zend_Http_Client::POST;
						break;
					}
				default:	/* Error */
					return false;
			}
	
			if (($next_method == Zend_Http_Client::POST) ||
				($next_method == Zend_Http_Client::PUT))
				{
					$values = array('uuid','password','email','first_name','middle_name','last_name','status','last_modified_by');
					$admin_data = $admin->getData();
					foreach ($values as $k => $v)
						$data[$v] = $admin_data[$v];
					$jsonData = Zend_Json::encode($data);
					$client->setRawData($jsonData,'text/json');
				}
		
			$client->setMethod($next_method);
			$response = $client->request();
			$status = $response->getStatus();
			$client->getAdapter()->close();
			$return = in_array($status,array(200,201,204));

modules/api/controllers/AdminController.php

The REST controller uses a preDispatch override to ensure the submitted id, username and password are valid. Invalid requests are discarded with an HTTP/400. Unauthorized requests receive an HTTP/403.

GET requests use the submitted UUID to test for the existence of the data. Using a UUID ensures the data will be identified the same across different systems. The data has local identifiers as well, and some translation is applied, although it has been removed from the posted code.

The JSON data is read from the request body with a file_get_contents on php://input

<?php
class Api_AdminController extends Zend_Rest_Controller
{
	private $model = null;
	private $uuid = null;
	private $data = null;
	private $response = null;

	public function init()
	{
		$this->apiBaseUrl = $_SERVER['SERVER_NAME'].'/api/admin';
		$this->_helper->layout->disableLayout();
		$this->getHelper('viewRenderer')->setNoRender(true);
	}

	public function preDispatch()
	{
		// This ensures the required data is present and valid prior to processing any requests
		$hostname = new Zend_Validate_Hostname(array('allow'=>Zend_Validate_Hostname::ALLOW_DNS));
		$validators = array(
			'uuid'=>array(array('Regex','pattern' => '/^[\da-f]{8}-([\da-f]{4}-){3}[\da-f]{12}$/','allowEmpty'=>false,'presence'=>'required')),
			'username'=>array(array('EmailAddress','allowEmpty'=>false,'presence'=>'required','allow' => $hostname,'mx' => true)),
			'password'=>array(array('StringLength','min'=>8,'max'=>32)));
		$input = new Zend_Filter_Input(array(),$validators);
		$request = $this->getRequest();

        $this->data = Zend_Json::decode(file_get_contents('php://input'));
        if ($request->isPost())
            $this->uuid = $this->data['uuid'];
        else
        {
            $this->uuid = $request->getParam('id');
            $this->data['uuid'] = $this->uuid;
        }

        $iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_256, MCRYPT_MODE_ECB);
        $iv = mcrypt_create_iv($iv_size, MCRYPT_RAND);
        $authorization = Zend_Json::decode(trim(mcrypt_decrypt(MCRYPT_RIJNDAEL_256, $_SERVER['SERVER_NAME'],
            base64_decode($request->getHeader('Authorization')),
            MCRYPT_MODE_ECB, $iv)));
        $input->setData(array(
            'uuid'=>$this->uuid,
            'username'=>$authorization['username'],
            'password'=>$authorization['password']));

		if (!$input->isValid())
			$this->error(400);
		else
		{
			// Data is valid, authenticate based on username and password
		        $session = new Zend_Session_Namespace('global');
        		$adapter = new Application_Auth_Adapter_Doctrine($input->username,$input->password);
			$auth = Zend_Auth::getInstance();
			$result = $auth->authenticate($adapter);
	        	if (!$result->isValid()) 
				$this->error(403);
			else
			{
				// Prepare the response object
				$api = Zend_Registry::get('bootstrap')->getOption('rest');
				$this->response = array(
					'link'=>$api['protocol'].'://'.$this->apiBaseUrl.'/'.$input->id,
					'title'=>'Admin',
					'charset'=>'utf-8');
			}
		}

		$session=new Zend_Session_Namespace('global');
		switch ($session->type)
		{
			case 'client': 
				$this->model = 'Application_Model_Admin'; 
				break;
			case 'vendor': 
				$this->model = 'Application_Model_VendorAdmin'; 
				break;
			case 'admin': 
				$this->model = 'Application_Model_Admin'; 
				break;
			default: 
				$this->error(400);
				break;
		}
	}
	
	public function indexAction()
	{
		// Handles GET requests
	}
	
	public function getAction()
	{
		// Handles GET requests
		$q = Doctrine_Query::create()
			->from ($this->model)
			->where('uuid = ?',$this->uuid);
		$result = $q->fetchArray();
		$q->free();
		switch (count($result))
		{
			case 1:
				// The data is NOT sent back to the client.  The response serves only to acknowledge the existence of the record
				// In this application, the GET request is really querying for presence, not requesting data
				$this->response['updated'] =$result[0]['updated_at'];
				$this->_helper->json($this->response);
				break;
			case 0:
				$this->error(404);
				break;
			default:
				// Database should only have one instance of UUID
				$this->error(409,'Multiple matches - contact support');
				break;
		}
	}

	private function process()
	{
		$form = new Application_Access_Admin_View_Form();
		$form->populate($this->data);
		$values = array('uuid','password','email','first_name','middle_name','last_name','status','last_modified_by');

		$data = array();
		foreach ($elements as $k => $v)
		{
			$name = $v->getName();
			if (!in_array($name,$values))
				$form->removeElement($name);
			else
				$data[$name] = $v->getValue();
		}

		if ($form->isValidPartial($data))
		{
			$request = $this->getRequest();
			if (!$request->isPost())
			{
				$q = Doctrine_Query::create()
					->from ($this->model)
					->where('uuid = ?',$this->uuid);
				$admin = $q->fetchOne();
				$q->free();
			}
			else
			{
				$admin = new $this->model;
				$admin->assignDefaultValues();
			}
			$admin->fromArray($form->getValues());
			$response = $this->getResponse();
			try
			{
				$admin->save();
				if ($request->isPost())
					$response->setHttpResponseCode(201);
				else
					$response->setHttpResponseCode(204);
			}
			catch(Doctrine_Exception $e)
			{
   				$this->error(400,var_export($admin->getErrorMessages(),true));
			}
			catch (Exception $e)
			{
   				$this->error(400,var_export($e->getMessage(),true));
			}
		}
		else
   			$this->error(400,var_export($form->getMessages(),true));
	}

	public function postAction()
	{
		$this->process();
	}

	public function putAction()
	{
		$this->process();
	}
	
	public function deleteAction()
	{
		$q = Doctrine_Query::create()
			->from ($this->model)
			->where('uuid = ?',$this->uuid);
		$result = $q->fetchOne();
		$q->free();
		$result->delete();
		$response = $this->getResponse();
		$response->setHttpResponseCode(204);
	}

	private function error($code,$message=null)
	{
		// If a message is returned, it is text in the body
		$response = $this->getResponse();
		$response->setHttpResponseCode($code);
		if ($message != null)
			$response->setBody($message.PHP_EOL.PHP_EOL);
		$response->sendResponse();
		exit;
	}
}

Use curl to test the interface, or PHP.

Filtering Spam - Tips & Ideas

SSH in to the server and find the affected inbox.

Quick Look at the inbox

This command shows the From, X-Spam-Bar, and Subject, without the file name. If you want to see the file names, remove the -h. You may have to use a different header for the spam score.

grep -h “From: \|X-Spam-Bar: \|Subject: ” * | more

The output should show give you an overview of what has been delivered, where it came from, and the subject.

Spam Score

The spam score is an appealing tool because it adapts to the current spam environ, and includes many parameters such as the sending IP address, domain, email address, SPF and reverse DNS information.

This is a very simple script that gives an overview of the spam bar values. It can be used as a first pass t set the level for filtering.

echo ‘Items in Inbox’
ls -1 | wc -l
echo ‘No Spam Bar (probably not spam)’
grep -L “X-Spam-Bar:” * | wc -l
echo ‘X-Spam-Bar Counts’
echo ‘+’
grep -m1 “X-Spam-Bar: +$” * | wc -l
echo ‘++’
grep -m1 “X-Spam-Bar: ++$” * | wc -l
echo ‘+++’
grep -m1 “X-Spam-Bar: +++$” * | wc -l
echo ‘++++’
grep -m1 “X-Spam-Bar: ++++$” * | wc -l
echo ‘+++++’
grep -m1 “X-Spam-Bar: +++++$” * | wc -l
echo ‘++++++’
grep -m1 “X-Spam-Bar: ++++++$” * | wc -l

It’s good to check the spam bar for valid emails as well. Many times, ‘real’ email is given a spam score.

From addresses

Look for patterns in the from addresses. Common patterns include ‘info@somedomain.info’ - the info, either as the sending email user or TLD is frequently used.

grep -h -m1 “From: ” * | sort

Subject

The subjects should be checked for the patterns in a similar manner as the From header.

Body

Using the results of the From and Subject checks, review a few of the message bodies (read the emails). Look for common text that would not be used by people. For example, Dear email@domain.com. People wouldn’t use an email address in a salutation, neither would real newsletter senders or other respectable sources.

Set the Filters

Read the directions. All the systems work differently.

Test the filters with some valid emails and some spam to be sure they behave as intended.

I created three filters.

  • Discard - The discard filter checks for glaring spam signs, such as a server that sent many spams messages, a from address pattern, distinct text patterns that simply wouldn’t come from people or other valid sources. These are discarded without warning to the sender.
  • Fail with message - Fail with message warnings were issued for anything that looked like spam, but might still be a valid email. Since the only ones that will read the fail with message text are people, it’s worth sending a friendly message, with a proposed solution. A good proposed solution is to suggest they use the site’s contact form. This will usually bypass the spam filters. It’s probably not a good idea to put a URL in the message, since a creative spammer may use it.