Friday, September 24, 2010

Deleting old log files based on size and time

There are times when the log files take up too much space in the system and bring the system to run out of space.  Fortunately there are few steps you can take to keep the log files under control for development servers.  Here I have listed a simple strategy:
1) Create a dedicated partition for the log files so if the log files grow out of space, that still does not crash the system by making no space left for your data and configuration files.
2) Keep all log files for all applications under a common directory dedicated for logs.  Replace the location of the log files within your application with symbolic links to the common logs folder or change the log configuration files to directly point them to the log directory.
3) Create a simple script to delete files that are older than certain number of days.
4) For files that does not get rolled over by the application, have a script that trims the head of the file by certain percentage of their size when the file size grows over a certain size limit.

If you follow these simple steps, your log files and directories will stay tidy and you still have logs left when you need them to debug issues.   Here is a sample script that you can use to cleanup old files and trim larger files.
logdir="/tmp/test"
trimAmt=50
tempFile=/tmp/trimedfile.$$
sizeLimit=+1G
fileAge= +30

cd $logdir

find . -mtime $fileAge -exec rm -f {} \;

for FILENAME in $(find . -size $sizeLimit -print )
do
    filesize=$(wc -l $FILENAME |  awk '{print $1}')
    trimsize=$(( $filesize -($filesize / $trimAmt)))
    echo " $FILENAME $filesize $trimsize "
    tail -$trimsize $FILENAME > /tmp/trimedfile.$$
    cat $tempFile > $FILENAME
    rm  $tempFile
done


2 comments:

Anonymous said...

sbobet Sign Up Betting Guide | Sportsbook | Free Bet Bonus
Sign up and bet on sports online, casino sbobet ทางเข้า games, and eSports. 188bet Free Bet No Deposit Bonus Code. matchpoint Bet $20 Get $100 in Free Bets · Bet $100 Get $200 in

mateo said...

The practice of deleting old log files based on both size and time is a pragmatic and efficient approach that reflects a thoughtful consideration for system optimization and resource management. By implementing this strategy, organizations can ensure that their log files, essential for system monitoring and diagnostics, do not accumulate indefinitely, preventing potential storage issues and improving overall system performance. The dual criteria of size and time add a layer of sophistication, allowing for a nuanced management of log files based on their relevance and impact on system resources.
a dispute over a contract between
contract dispute meaning