Mozilla/4.0 (compatible;)
This user agent was in the middle of many page requests in my Apache logs, requesting content referenced by link tags in the head section.
After a bit of research on one of the link tag URLs, I ran this script:
IPS=`grep Author access_log | cut -f 1 -d ' ' | sort | uniq`
for IP in $IPS
do
echo Testing "$IP"
host "$IP"
done
In almost every case, the requests came from large organizations - corporations, government agencies, and the military.
These institutions often use proxy servers, and Mozilla/4.0 (compatible;) must be a common user agent setting for the proxy server requests.
In the one case where it wasn’t a large organization, it was a blacklisted IP, and the user agent was Java.
The sample set was limited, but the pattern was clear.
Print article | This entry was posted by elvis on 05/10/11 at 07:21:00 pm . Follow any responses to this post through RSS 2.0. |