Google Answers Logo
View Question
 
Q: Apache mod_rewrite RewriteRule question ( Answered 5 out of 5 stars,   0 Comments )
Question  
Subject: Apache mod_rewrite RewriteRule question
Category: Computers
Asked by: bmulvaney-ga
List Price: $50.00
Posted: 10 Feb 2005 17:03 PST
Expires: 12 Mar 2005 17:03 PST
Question ID: 472608
Apache RewriteRule to redirect Get requests to another server based on file suffix.

I would like to set up a quick and dirty content load sharing scheme
using mod_rewrite on Apache 1.3.  The objective is to have large media
files (.pdf, .avi,
.mpeg, etc.) transparently served from a secondary server via URL
rewriting.  The goal is to preserve performance and conserve bandwidth
on the primary web server during heavy download periods.

I believe the httpd.conf syntax would look something like this:

RewriteEngine on
RewriteRule source_url  redirect_url  [R,L]

Where:

source_url has the form:  http://www.example.com/content/bigfile.mpeg
redirect_url has the form:  http://secondary.example.com/content/bigfile.mpeg

If so, I need help with the regular expression for the RewriteRule
that maps the 1st URL to the 2nd URL.  I'm looking for a regular
expression that, for a defined list of file types, redirects requests
for those files from http://www.example.com/* to
http://secondary.example.com/*

Related questions:

1) Is this a sound approach to splitting load across multiple servers?

2) Does it matter whether the context for the RewriteRule is at the
VirtualHost or the Directory level?

3) Can this approach be extended to cheap and cheerul load balancing
via mod_rewrite where rather than redirecting traffic based on a rule
to a single host, the redirection can happen to one of a list of hosts
on a random or round robin basis?

TIA!

Brian
Answer  
Subject: Re: Apache mod_rewrite RewriteRule question
Answered By: wildeeo-ga on 12 Feb 2005 12:16 PST
Rated:5 out of 5 stars
 
Hi, Brian. Thanks for your question.

You can achieve what you want using the following rule:

RewriteEngine on
RewriteRule ^(.*)/([^/]*\.)(avi|pdf|mpeg)$
http://secondary.example.com/$1/$2$3 [R,L]

(The RewriteRule line should not have any line breaks, if Google has inserted any).

This will redirect any requests that end in '.avi', '.pdf', or '.mpeg'
to the matching address on http://secondary.example.com/. If you need
to add more file types, add a bar and the type after '|mpeg'.
secondary.example.com can be replaced with your server's hostname. As
an example, if I wanted to also redirect files ending in '.bla' and
'.pie' to a server 'files.test.com', the line would become:

RewriteRule ^(.*)/([^/]*\.)(avi|pdf|mpeg|bla|pie)$
http://files.test.com/$1/$2$3 [R,L]

To answer your questions:

1. This is a fairly good way to serve files of specific types from a
different server, especially if bandwidth is the problem. However, too
many requests for large media files will just move the problem of load
onto the second server. If this is acceptable (for example, if you
wish to ensure your primary web server remains accessable), then this
method should work well.

It is also worth noting that enabling mod_rewrite will slow down
server performance a little, especially if it is applied site-wide.
The effects should be negligible, though.

2. Functionally, it does not matter whether this is included in the
VirtualHost directive, or in a .htaccess file. However, bear in mind
that if it is included in such a way as to apply to every request,
server performance may suffer unnecessarily. A better approach may be
to use it in a .htaccess file only in the directory with media -
'/content/', in your example.

Note that if you did this, you would probably have to change
http://secondary.example.com/$1/$2$3 in the example above to
http://secondary.example.com/content/$1/$2$3.

3. Yes, mod_rewrite is flexable enough to allow this.

To do this, you would need to create a text file containing a list of
possibile servers, for example:

mediaservers     files1|files2|files3|...

You would then need to add the following directive before the RewriteRule:

RewriteMap servers rnd:/path/to/your/server/list.txt

And finally, modify the rule to something similar to:

RewriteRule ^(.*)/([^/]*\.)(avi|pdf|mpeg)$
http://${servers:mediaservers}.example.com/$1/$2$3 [R,L]

This would select one of the servers listed in your text file at
random, effectively spreading the load between them. This isn't the
best solution, since it would not take into account, for example, a
server less able to cope with load than others.

For more information, see:

http://apache.active-venture.com/mod/mod_rewrite2.html#RewriteMap

This isn't really what mod_rewrite was designed for though, so the
performance probably won't be as good as that of a dedicated load
balancer, or software designed to balance loads. A better software
solution may be pound (http://www.apsis.ch/pound/), Pure Load Balancer
(http://plb.sunsite.dk/) or Zeus Load Balancer
(http://www.zeus.com/products/zlb/), as a few examples.

The apache module mod_backhand (http://www.backhand.org/mod_backhand/)
may also be useful if the problem is CPU usage. It does not help with
bandwidth utilisation, though.

I hope this helps. If you have any questions, please do not hesitate
to request a clarification.


The following links may also be of use to you:

Apache URL Rewriting Guide:
  http://httpd.apache.org/docs/misc/rewriteguide.html
Load balancing using mod_rewrite and mod_proxy:
  http://wiki.apache.org/cocoon/LoadBalancingWithModProxy
The pound load balancer:
  http://www.apsis.ch/pound/
Regular Expressions Reference:
  http://www.zvon.org/other/reReference/Output/

The following searches may also be useful:

'mod_rewrite load balancing':
  ://www.google.com/search?q=mod_rewrite+load+balancing

--wildeeo

Request for Answer Clarification by bmulvaney-ga on 14 Feb 2005 12:28 PST
The Rewrite rule works great, even if I don't really understand it. 
(Regex is a mystery to me.)  Only problem I'm having is that .wmv
(yes, I know, we shouldn't be using Windows Media Files) display as
text in FireFox when served from the redirected server versus as a
file to download or open in a helper application when served from the
primary server with the RewriteEngine off.  Internet Explorer handles
these files just fine on the primary as well as on the redirect to the
secondary.

My thought is that this is a content type handling difference between
the primary server (Apache 1.3) and the secondary (Apache 2.0).  But a
quick glance with the IEWatch plugin to look at the response headers
shows that on both servers, the content type in the HTTP response
header for .wmv files is text/plain.  Is it possible that something is
going on with the redirect that causes FireFox to pull down the WMV
file differently from one server to the other?

Thanks,

Brian

Clarification of Answer by wildeeo-ga on 14 Feb 2005 16:27 PST
I'll quickly try and explain "^(.*)/([^/]*\.)(avi|pdf|mpeg)$", the
pattern it matches against:

'(.*)' - There can be any characters at the start. Assign them to the variable $1.

'/' - Next, there must then be a '/' character. Don't assign it to a variable.

'([^/]*\.)' - Next, there is a sequence of any characters except the
'/' character, followed by a dot. This corresponds to 'file.' in an
example address http://bla.com/directory/file.avi. It will assign this
to $2.

'(avi|pdf|mpeg)$' - Lastly, it will match avi OR pdf OR mpeg, and
assign it to $3. The ending '$' character means that the URL must end
after this.

It them substitutes the values it matched in place of $1, $2 and $3 in
the second address.

Secondly, the WMV problem. I would agree that it's a content type
problem. I say this because with this kind of redirect, the server
sends a 302 Found HTTP message to the client with the new address. As
far as I know, the client then follows this address, discarding the
headers it received from the first request, so I'm pretty sure it's
not a problem caused by the redirect, especially if both servers
return the same MIME type for the file.

If you request the file directly from the second server and it appears
as plain text, this would confirm it's not a problem with the
redirect. You could try adding

AddType .wmv video/x-ms-wmv binary

in Apache's config file to change the MIME type of .wmv files to
'x-ms-wmv', hopefully forcing FireFox to download it.

Let me know if you have any other questions.

--wildeeo
bmulvaney-ga rated this answer:5 out of 5 stars and gave an additional tip of: $20.00
Excellent response with a thorough follow up that included a tutorial
on the solution as well as suggested fix for a related problem outside
the scope of the original question.

Comments  
There are no comments at this time.

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy