VWebhosting

Posted by on July 7, 2006
ApacheDebianHostingTechWist

So, to reiterate – the main server (Wist) will hold all my own stuff, then I’ll be creating a user Virtual Server (Wintermute) on top of that for users to play with and/or break. All the user stuff is backended in a Xoops database using Xoops groups.

Wist is installed with Debian Sarge using a custom kernel including the vserver patches from Debian. This sort of thing is detailed elsewhere so I won’t go over it again here.

The core server software (all from the Debian archive) will include: Apache 1.3 with PHP4, Apache-SSL, Bind, MySQL 4.1, Exim4, ClamAV and Dovecot (I’ll be detailing the email setup seperately)

The installation of the Bind9 DNS server is very straight forward and only hosts a few domains, most of which are served from either Wist or Wintermute using a handfull of A/CNAME records and wildcard dns.

The Apache-SSL server is only going to host https for www.elizium.net so the only changes from the stock install for that is to install the appropriate certificate. I tend to buy certificates from EV1Servers which I find very affordable.

The magic for this article happens in the config file for Apache 1.3 hosting plain http traffic.
To allow me to easily add arbitrary domains and subdomains without extra config of the DNS server, I’ve used wildcard DNS. This means that all DNS requests for any subdomain of a domain (eg. www.example.org and sub.example.org) will be directed to the same server unless there is a specific DNS record for that particular subdomain. Clear? Probably not but thats the best I can do just now.

I wanted to be able to create a directory structure which would let me map out the web sites. Lets say I’ve created a root folder at /var/http and domains under that will be at /var/http/example.org then the subdomains will reside at /var/http/example.org/www /var/http/example.org/sub following me? Good.

On Wist, the first part of the domain name is stripped to become the subdomain. This means that if a request for http://example.org is recieved it will go to /var/http/org/example. I tend to create redirects in each of those folders to www.example.org etc as the vanity www has been overly popular for the last million years.

How did I manage to do this? Using mod_rewrite! Much as I hate regex (well, to be honest, I don’t use it enough to easily remember its quirks) this is clearly the way forward in this case.

Concentrate, here comes the science bit:

RewriteEngine on
#Ignore requests for server wide directories
RewriteCond %{REQUEST_URI} !^/mailman/.*
RewriteCond %{REQUEST_URI} !^/pipermail/.*
RewriteCond %{REQUEST_URI} !^/cgi-bin/.*
RewriteCond %{REQUEST_URI} !^/icons/.*
#Rewrite the full request to include the host instead of just the path
#eg. /dir/index.html -> www.example.org/dir/index.html
RewriteRule ^(.+) %{HTTP_HOST}$1 [C]
#Rewrite the url to a file in a directory
#www.example.org/dir/index.html -> /var/http/example.org/www/dir/index.html
RewriteRule ^([^.]+)\.([^/]+)\/(.*)$ /var/http/$2/$1/$3 [L]

Good, eh? I have to add my thanks to the guys in #regex when I was struggling to come up with this.

Apache handles all the rest – default directory indexes and the variety of options itself.

The one last addition to the whole bundle is logging. First I started with this:

LogFormat "%{Host}i %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vfull
CustomLog /var/log/apache/access.log vfull

Which is the same as a standard Apache log file line but with the hostname (eg. www.example.org) as the first field.

Good, but we can do better.

CustomLog "|/sbin/vlogger -s access.log -u www-data -g www-data -t %Y%m%d-access.log /var/log/apache" vfull

I’ve installed the vlogger package. This spits out a directory per hostname with dated YYYY-MM-DD-access.log files and a symlink called access.log to the current days logs. Priceless if you want to do per host webstats withouth pre-precessing a big monolithic log file for specific domains.

Hopefully some folk will find that useful. After I’ve detailed the webhosting setup on Wintermute – more regex, some the same, mostly different – I’ll tell you how I automatically generate my webstats.

Tags: , , , ,

Comments (3)

  • Cool!

    Why Apache 1.3?

    Dovecot 1 should be out soon too!

    Nice ideas by the way. Hope to test out this setup too.

    Gavin.

    http://www.suretecsystems.com

  • I started with Apache 1.3 as that is the version I used to use on Xev and I wanted to be able to migrate my conf files across with minimal fuss.

    The Email stuff may be quite interesting as its all authenticating against MySQL – used to use Courier but there was something that wouldn’t work with the authentication against the db. Was very impressed when I swapped to Dovecot that I could get it to do what I wanted in a few minutes! But thats another article 🙂

  • I use cronolog to write out “a big monolithic” file every day, and split this into per-domain YYYY-MM-DD-access.log files after midnight using a custom Perl script.
    The script is run from cron, just before the stats (webalizer) and just after the reverse DNS lookups (webazolver) which run quicker from the one big file.
    Before I wrote the script, running out of file handles was a risk on a server with a large amount of (small or quiet) virtual hosts. I haven’t checked this recently (or with vlogger), but it probably consumes more memory to have many files being open for writing. If you have lots of (as in more than 50 on one server) virtual hosts getting more traffic from search engine spiders than from anything else, it makes more sense to have one server-wide access.log file.

Do you have anything to say?

Powered by Wordpress and Stripes Theme Entries (RSS) | Comments (RSS)