Menu Home

Mass-diffing files

As part of my day to day tasks, I’m auditing configuration files and tidying them up. Slowly, we’re moving towards standardised files and configuration management.

The first requirement of this is gathering the existing config files into one place. You could do this with any of a number of tools, I’ve been forced to write my own, and that’s both not my IP to share and it’s outside the scope of this post.

Once you’ve got your files gathered together, simply run diff (Note: the GNU version of diff) like this:

$ diff -qrs --from-file server1_etc_krb5.conf *_etc_krb5.conf | head -10
Files server1_etc_krb5.conf and server1_etc_krb5.conf are identical
Files server1_etc_krb5.conf and backup0001_etc_krb5.conf are identical
Files server1_etc_krb5.conf and clusterfs01_etc_krb5.conf differ
Files server1_etc_krb5.conf and clusterfs02_etc_krb5.conf differ
Files server1_etc_krb5.conf and clusterfs03_etc_krb5.conf differ
Files server1_etc_krb5.conf and clmgmt01_etc_krb5.conf are identical
Files server1_etc_krb5.conf and database01_etc_krb5.conf are identical
Files server1_etc_krb5.conf and database02_etc_krb5.conf differ
Files server1_etc_krb5.conf and database03_etc_krb5.conf are identical
Files server1_etc_krb5.conf and database04_etc_krb5.conf are identical

This is taken from a real world example, but I’ve obviously changed the hostnames

So, using this approach, we can quickly generate lists of hosts that match. It may be the case that there are two or three subtly different versions that are otherwise largely the same, and only a handful of outliers.

The next step would be to run the same command but using clusterfs01’s as the reference point

$ diff -qrs --from-file clusterfs01_etc_krb5.conf *_etc_krb5.conf

And to also directly diff server1 and clusterfs01’s files.

Once the differences are understood, merging can occur and cleaning up applied until a standard file rises from the mess.

Oh, and one more tip… Add these lines to your .bashrc file, again these are Linux specific:

# Enable wide diff, handy for side-by-side i.e. diff -y or sdiff
# Linux only, as -W/-w options aren't available in non-GNU
alias diff='diff -W $(( $(tput cols) - 2 ))'
alias sdiff='sdiff -w $(( $(tput cols) - 2 ))'

These enable (s)diff to use the full width of your terminal window.

Categories: Geeking Out Lunix Lunacy