So, I came up with this module. I’ve been meaning to release it for ages, but as I need to clear down the server it was running on it’s made me take the five minutes to write this post. Still needs a bit of work, but does what it says on the tin.

It identifies non-local images in content, downloads them and serves them from a local cache.

Download the module here: img_proxy.tar.gz

If you use Drupal, you should already know the drill – extract the file to your modules directory and enable it in the modules page. The module is implemented as an input filter, so you will then have to add it to the appropriate input formats at http://your.site.com/admin/settings/filters

Notes

• Licenced under the GPL version 2.
• *** There is nothing in the code that verifies the file being proxied is an image!! *** (that’ll be in the next version!)
• May not work with sites that check the referer of the request for images.
• I’ve been meaning to release this for ages, but there is sill a lot of debug code thats just commented out.

Hopefully I’m not the only webmaster that ever had this issue and somebody will find this handy.

Maybe one day I’ll tidy up the code and submit it to the Drupal Module repository!

Enjoy.

The problem lies with mapping domain names (example.org) onto user accounts (Alice? Bob?). This is where Apache and mod_rewrite come to the rescue again. On Wintermute I’ve installed Apache 2, but this recipe will work just as well with Apache 1.3.

First, we create a file with domain names to username mappings:
/etc/apache2/map.hosted

Coming up with that file was the easy bit. The next bit was the major pain in the hole. I searched for aaaages to try and work out how to do this in php ( my php is significantly better than my Perl), but all the examples I could find were written in Perl. So, here are some keywords for search engines: how to write a mod_rewrite map in php not perl. php apache rewritemap.

The only improvement I may make to that is to get the users home directory info as opposed to assuming its under /home.

Now we just need put that script somewhere, make it executable by the Apache user and tell Apache to use it. Thats done like so:

I hope this is of use to people. Especially those folk looking for a php based rewrite script for Apache. Theres a lot of scope to expand the script. For example, the file data could be retrieved from a database and then stored in something like memcached if you were performing this sort of thing on a larger scale. Much better than having a large webserver config file that means the webserver has to be reloaded for each change.

The only problem I have with this, and I know its not insurmountable, is that the webserver user has to be able to read the files that users are storing. To allow this I’ve made the www-data user a member of the group all the users are members of. This also means that users can read each others files. So if somebody was to hardcode a password in a php file (for example) then it would be open to being read by any other user.

I also want to mention suPHP, it really does merit an article to itself, but as I’m not intending to cover it in depth (you can read all about it at the website) I’ll just say that I’ve enabled it on Wintermute to prevent users being able to do things like kill the web server processes via a php script. To do that, I installed the suphp-common Debian package. The default configs are all you need. For the non-Debian user, they look like this when taken together:

As usual comments on the above are most welcome. Especially quick links about solving my security issue!

Wist is installed with Debian Sarge using a custom kernel including the vserver patches from Debian. This sort of thing is detailed elsewhere so I won’t go over it again here.

The core server software (all from the Debian archive) will include: Apache 1.3 with PHP4, Apache-SSL, Bind, MySQL 4.1, Exim4, ClamAV and Dovecot (I’ll be detailing the email setup seperately)

The installation of the Bind9 DNS server is very straight forward and only hosts a few domains, most of which are served from either Wist or Wintermute using a handfull of A/CNAME records and wildcard dns.

The Apache-SSL server is only going to host https for www.elizium.net so the only changes from the stock install for that is to install the appropriate certificate. I tend to buy certificates from EV1Servers which I find very affordable.

The magic for this article happens in the config file for Apache 1.3 hosting plain http traffic.
To allow me to easily add arbitrary domains and subdomains without extra config of the DNS server, I’ve used wildcard DNS. This means that all DNS requests for any subdomain of a domain (eg. www.example.org and sub.example.org) will be directed to the same server unless there is a specific DNS record for that particular subdomain. Clear? Probably not but thats the best I can do just now.

I wanted to be able to create a directory structure which would let me map out the web sites. Lets say I’ve created a root folder at /var/http and domains under that will be at /var/http/example.org then the subdomains will reside at /var/http/example.org/www /var/http/example.org/sub following me? Good.

On Wist, the first part of the domain name is stripped to become the subdomain. This means that if a request for http://example.org is recieved it will go to /var/http/org/example. I tend to create redirects in each of those folders to www.example.org etc as the vanity www has been overly popular for the last million years.

How did I manage to do this? Using mod_rewrite! Much as I hate regex (well, to be honest, I don’t use it enough to easily remember its quirks) this is clearly the way forward in this case.

Concentrate, here comes the science bit:

Good, eh? I have to add my thanks to the guys in #regex when I was struggling to come up with this.

Apache handles all the rest – default directory indexes and the variety of options itself.

The one last addition to the whole bundle is logging. First I started with this:

Which is the same as a standard Apache log file line but with the hostname (eg. www.example.org) as the first field.

Good, but we can do better.

I’ve installed the vlogger package. This spits out a directory per hostname with dated YYYY-MM-DD-access.log files and a symlink called access.log to the current days logs. Priceless if you want to do per host webstats withouth pre-precessing a big monolithic log file for specific domains.

Hopefully some folk will find that useful. After I’ve detailed the webhosting setup on Wintermute – more regex, some the same, mostly different – I’ll tell you how I automatically generate my webstats.

Since then my bandwidth has gone over that fairly regularly. So I went shopping about. After a lot of time spend trying to convert € and $ into £ I eventually settled on the basic root server offering from 1&1 Internet (wist.arricc.net). The benefits? unlimited bandwidth and serial console access to the server – not to mention the spec is about double the memory and Mhz of Xev!

I’ve made a number of changes to the setup of Wist over Xev, with (hopefully!) no loss of services. Not least is the implementation of virtual servers. This has allowed me to let users run PHP scripts in a sandbox which won’t affect my own projects.

I’m going to attempt to detail the setup in stages as there are a number of things to cover – vservers, webservers & virtual hosting, database server, host access(ssh, ftp, etc.) – so hopefully I’ll be able to break them down in meaningful stages!