Post/read comments made about this page
NB! The file system can be different, depending on who you're hosting with. These worked on my old webhost, but not all of them work on the new one.
If you have a webhost that uses cpanel, there are some built in solutions for handling raw logs. But you're restricted to those, and they may not be adequate for you. Some hosts clean out their logs often, others do it seldom. While it's generally no problem with a small website with little traffic no matter what the host does, the really big sites with logs that grow exponentially might have a problem with cpanel's handling of logs - especially if there are security issues on your site. So I came up with a solution that works for me, and might work for you.
Daily backup of yesterday's log entries, gzipped. This file is run via cron:
<?
$yesterday = mktime(0,0,0,date("m"),date("d")-1,date("Y"));
$yesterday = strftime("%d/%b",$yesterday);
$thatday = mktime(0,0,0,date("m"),date("d")-1,date("Y"));
$thatday = strftime("%d%b%Y",$thatday);
exec("grep '$yesterday' /usr/local/apache/domlogs/domain.com | gzip -9 >> /home/username/logs/acronymofsite$thatday.gz");
?>
For those whose webhosts clean out the logs weekly, figure out which day they do it, and then run the script below that day right after they've cleaned it out. The script above will return an empty file that day, but it's less resource intensive, so I use this one for daily backup
Hopefully this one will work on your server. It does work here:
<?
$yesterday = mktime(0,0,0,date("m"),date("d")-1,date("Y"));
$yesterday = strftime("%d/%b",$yesterday);
$thatday = mktime(0,0,0,date("m"),date("d")-1,date("Y"));
$thatday = strftime("%Y%b%d",$thatday);
exec("gzip -dc /home/domlogs/domain.com.gz | grep '$yesterday' | gzip -9 >> /home/username/logs/acronymofsiteendweek$thatday.gz");
?>
Place the files in your root directory, or in a php or logs directory. Don't forget to create a logs directory if it's not there already. And no, I'm not talking about www root, but your home directory.
And now for the cron job. It should be entered something like this:
/usr/local/bin/php /home/username/public_html/php/dailylogs.php
Cpanel has a simple cron job installer, that will enable you to specify it running each day and exactly when. Here's an example of the Unix type interface:
6 3 * * * /usr/local/bin/php /home/username/public_html/php/endweeklogs.php
It's also possible to copy the entire previous week's worth of logs via cron if that's what you want:
cp -f /home/domlogs/domain.com.gz /home/username/access_log_previous.gz
Of course, you could make a php file for this too, to name it after the day you save it. Hmm, good idea, think I'll do that... So, I did:
<?
$today = date("Ymd");
exec("cp -f /home/domlogs/domain.com.gz /home/username/logs/$today-access_log.gz");
?>
Be aware that some hosts are terrible at log handling. They clean out the logs instead of installing enough hard disk space, sometimes even at irregular times. Others are committed to letting the customers have their logs. You never know... ALWAYS pay for a trial month, or maybe even more before you committ to hosting. Don't go for the deals where you pay for a year up front before you've even tested the host. You could end up being very sorry.
Disclaimer: I'm not a programmer, I just managed to cobble together a script from known ingredients, with some modifications and troubleshooting.
Credits: I had some help developing the initial script. Robert at TCH (great host BTW) gave me the starting point. I found the starting point for the date manipulation at Linux Guruz.The man pages for mktime, date and strftime were a great help. And of course, some of this is straight Linux commands, most of which I already knew.
This page was created by Ann Elisabeth Nordbo
and has its home at http://www.annelisabeth.com/
Updated 10.23.2005
Premiere issue April 8 2004