du-diff.py

du-diff is a simple tool which compares the output of two du commands, usually run on the same filesystem on different dates. It shows all the differences between the du results.

Simple example

For instance, take the following example outputs. Running du -x /var/log:

12897	log/sa
1914	log/cups
0	log/news
4	log/mysql
13085	log/samba
0	log/sandbox
0	log/iptraf
537	log/vsftpd
352	log/apache2
51605	log

And the output of the same command a week later:

12897	log/sa
1918	log/cups
0	log/news
4	log/mysql
13085	log/samba
0	log/sandbox
0	log/iptraf
537	log/vsftpd
352	log/apache2
51789	log

du-diff will show:

+       4       1918    log/cups
+       184     51789   log

This means that log/cups increased (+) in size for 4 du units, and the final size is 1918. Analogously (and partly in consequence) the log directory increased in 184.

Output of du-diff

The meaning of the columns is:

1. Change:
   • + - increased in size
   • - - decreased in size
   • N+ - new directory, increased in size
   • D- - deleted directory, decreased in size
2. Increment or decrement in size
3. Final size of the directory
4. Name of the directory

Applications

This may be simple, but it's very useful for managing where the space is lost in a server. You can have this simple root crontab:

* 21 * * 1 du -x / |gzip >/hdtmp/du/du-diff-root-`date +%Y%m%d`.txt.gz
* 22 * * 1 du -x /home |gzip >/hdtmp/du/du-diff-home-`date +%Y%m%d`.txt.gz

For each partition, I have a du dump each week. Those dumps may have more than 15000 lines. Even though, comparing two weeks, the du-diff output may be:

-       8       188     /tmp/mailgraph
+       4272    77382   /tmp
+       104     130557  /var/lib
+       888     21067   /var/log
+       441     264941  /var/www
+       4       185495  /var/cache
+       1437    668360  /var
+       773     699000  /usr/src
+       772     2060539 /usr
-       108     173624  /home/lant
+       48      698623  /home/viric
+       16      37227   /home/cercledeldrac
+       235     22185   /home/health
-       98106   1402163 /home
+       4       11168   /root
-       91620   4272196 /

The last output was got by a filter, filtering the du-diff output to 2 directory levels. The egrep filter can be easily changed to fit your needs:

• 1 level - | egrep '^[^/]*(/[^/]*){1,1}$'
• 2 levels - | egrep '^[^/]*(/[^/]*){1,2}$'
• 3 levels - | egrep '^[^/]*(/[^/]*){1,3}$'
• ...

Download

Download the latest version (GPL licensed): du-diff.py (It requires python)

usage: du-diff.py file1 file2

Future improvements

Write it in another language.