web notes - a blog

To deny access to a site based on the referer URL, you can use Apache rewrite rules.

In the sample below, any referral that ends with .ru, .ua, or .tv will be rejected and redirected to the forbidden page (a 403).

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} \.(ru|ua|tv)$ [NC]
RewriteRule ^.*  - [F]

To test the rule, you can use curl, like so:

curl -e ‘http://test.tv’ domain.com

This post describes one way to ensure the javascript and CSS are requested by the browser when a new release is distributed.

The first approach is to prefix the file names with a version-release string, and create symlinks to the files during installation or the first execution. Many systems have a version identification mechanism.

To manage the symlinks, the following could be used:

#!/bin/bash

fclearold()
{
        echo 'fclearold'
        for g in $(find *.$1 -maxdepth 1 -type l);do
                echo $g
                rm -f $g
        done
}

fcreatenew()
{
        echo 'fcreatenew'
        for g in $(ls *.$1); do
                ln -s $g $2.$g
        done

}

version=`cat "$BASE/config/version`;
for f in 'js' 'css'; do
        echo $f
        pushd $f > /dev/null
        fclearold $f
        echo $version;
        fcreatenew $f $version
        popd > /dev/null
done

fclearold removes the old symlinks, fcreatenew makes new ones. It is assumed the javascript is in the js directory and all javascript files have a .js extension, and the CSS is in the css directory and all CSS files have a .css extension.

httpd.conf (or equivalent)

# Cache js and CSS files for 6 months - with a timestamp
<FilesMatch "\.(js|css)$">
  ExpiresActive On
  ExpiresDefault "access plus 6 months"
  Header set Cache-Control "max-age=15552000"
  RewriteEngine On
  RewriteRule (.*)/[0-9]+\.(.*)$ $1/$2
  FileETag MTime
</FilesMatch>

Timestamp Management Code (PHP)

        function sTimestamp()
        {
                $sTimestampFile = 'cache/timestamp';
                if (is_file($sTimestampFile))
                        $sRetVal = file_get_contents($sTimestampFile);
                else
                {
                        $sRetVal = time();
                        file_put_contents($sTimestampFile,$sRetVal);
                }
                return $sRetVal;
        }

Once the timestamp is set, it is cached in the cache/timestamp file. In this system, the cache is cleared when new releases are installed, so the absence of the timestamp file ensures an update and a new set of requested files.

The timestamp can applied to .js and .css file requests like so:

<script type="text/javascript" src="js/<?php echo $timestamp ?>.code.js"></script>

The file input allows an accept attribute to indicate what type of file may be submitted. The type is the client’s MIME type, which may vary by operating system, installed applications, and end user configuration.

A sample set of MIME types used for an accept attribute is:

accept="application/csv,text/csv,text/text,application/vnd.ms-excel,text/plain,application/octet-stream”

The browser usually doesn’t enforce the accept attribute.

The MIME type sent from the client is unreliable, since many clients use the file extension to indicate the MIME type for the browser, and that MIME type is sent to the server.

An alternative is to ignore the MIME type, but use the Linux file command to test the file, and use it for validation.

In the example below, there are three identical files of raw audio, with the extension of pdf, raw, and txt. Linux uses the file content to determine the type, rather than the extensions.

[tmp]$ file -i audio.*
audio.pdf: application/octet-stream
audio.raw: application/octet-stream
audio.txt: application/octet-stream

CoderZone.org launched recently.

It’s great new resource for web people, from ‘n00bs’ to ‘w00ts’. What makes it special:

• A great team of moderators. These guys are experienced and know the web.
• A library of code snippets, little bits of code that will save you a tremendous amount of time. You can contribute code, too.
• XHTML/HTML & CSS sandboxes so you can test out ideas quickly.
• An SQL sandbox for testing queries.
• It’s free.
• A very cool design.
• No ads, the forum is there to help people, not distract you with ads you aren’t going to click on anyway.

First, you’ll need to add the time required to deliver the request into the access_log. In this case, a custom_log is created. Note the bolded ^%D at the end, which will deliver the time required to serve the request in microseconds ( http://httpd.apache.org/docs/2.2/mod/mod_log_config.html#formats ).

CustomLog logs/custom_log “%h %l %u %t \"%r\” %>s %b \"%{Referer}i\” \"%{User-agent}i\” ^%D“

Add this into your Apache .conf file, as appropriate.

Next, restart Apache with apachectl restart. You can use apachectl configtest to check your editing.

The log file entry will look like this:

127.0.0.1 - - [14/Jul/2010:15:31:19 -0400] “GET /test/php/index.php HTTP/1.1″ 200 473 “-” “Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.4) Gecko/20100624 CentOS/3.6-8.el5.centos Firefox/3.6.4″ ^4097

The ^4097 indicates the number of microseconds to serve the request.

You’ll need the num_utils RPM, see the link above.

Get it and install it, like so:

wget http://suso.suso.org/programs/num-utils/downloads/rpm/num-utils-0.5-1.noarch.rpm
rpm -i num-utils-0.5-1.noarch.rpm

Use the following commands to extract the times from the log:

grep index.php custom_log | cut -d ‘^’ -f 2

And then, you can average them, like so:

cd /etc/httpd/logs
grep index.php custom_log | cut -d ‘^’ -f 2 | average
2611.33333333333

This post courtesy of Lyrix, Inc. ( http://lyrix.com / http://mobiso.com)