bash : search multiple file patterns using single find command

One reader asked how find can be used to find various file patterns. For example in a directory which could be littered with various logs and other files, how do I use a single find command to find all shell scripts, perl scripts and say php scripts. Simple answer is to use multiple -name arguments combined with -o (or ORed) and if needed with -a (or ANDed). Other find conditions (like -mtime, -type etc) can be combined as well. Continue reading

Howto rollover a file when size exceeds using unix find

Here is a one liner to rollover a file to file.old when it exceeds the size using find command. Lets say we have a script in cron that runs and prints messages in a log file. Overtime the log file will grow and we would want to rollover the log file to log.old. Many solutions exist by finding the size and comparing it. Here is one elegant solution in one liner. Thanks to my colleague Vlad who gave the idea for using find and exec, and I added the automatic substitution or brace expansion from my knowledge-base. Happy sharing of knowledge.

find /var/log/ -name myapp.log -size +1M -exec mv {}{,.old} ;

Continue reading

Unix : find affected by current working directory

On may Unix variants, find first looks for current working directory before proceeding with what it was asked to find. Ubuntu 10.10 and Debian Squezze not affected and I did not check older versions, but debian 5.0.6 or Lenny is affected and list includes Solaris 10 and Solaris 11 Express. It is very easy to fall in this pitfall if you have some automated package installation which may invoke some scripts for starting applications at the end of installation while cleaning up the temporary directory the package was running from. I wasted couple of hours in going over all my scripts to understand what was going on. The ls command was working but find was not able to get me the list of files to process from unrelated directories. So I ended up redirecting find’s error out to standard out and viola, solution presents itself. That redirection should have been on top of my list. It tells you that “find : cannot get the current working directory”. Why it needs that? I don’t know. Linux has this fixed for some time now, but for some reason SunOS is still using the old find variant including Solaris 11 Express which is the latest version out. Maybe some historical reasons. If anyone know, please share.

So the solution to the problem was that before invoking the command that will continue to run and may need to call find, start it in a directory that will persist after package installation is complete, e.g. / or /tmp.
Continue reading

find broken symlinks

If you have a big project with multiple shared libraries, there is a chance to run into soft links that are no longer pointing anywhere. Here are couple of ways to find those dangling soft links that no longer point to any real file using the Unix find command.

These are in preferred order.

find /path/to/search -type l | (while read FN ; do /usr/bin/test -e "$FN" || echo "$FN"; done)
find /path/to/search -type l ! -exec /usr/bin/test -r {} ; -print
find -L /path/to/search -type l

So here is how these can be used to clean out those dangling soft links (again in preferred order). We use rm to ensure that any aliases don’t kick in. First one is the best way which is portable to most Unix platforms.

find /path/to/search -type l | (while read FN ; do /usr/bin/test -e "$FN" || rm -f "$FN"; done)
find -L /path/to/search -type l -exec rm -f {} ;
find -L /path/to/search -type l -delete

Note: When using in script, make sure to escape the properly.

Interoperability Issues between SunOS and Linux

-delete is not supported on SunOS find. It is available on GNU findutils. So using -exec to invoke rm command would be portable. As well as the test for existence /usr/bin/test -e is portable. Make sure to use /usr/bin/test instead of shell built in test because on sh the flag -e is not available.

Update (2011-08-12) : /usr/bin/test should be used for interoperability.

bash : run command on find results

Unix find command allows user to execute a command on the results by using -exec command. For example see below. {} get replaced by filename, Make sure to escape {} and ;.

find . -type f -exec ls -l {} ;

This triggers ls -l on all the files that were found. I used this with -maxdepth (how deep directory structure to examine) and -mtime (last modified time) to cleanup old files as below.

find /var/log -type f -mtime +30 -maxdepth 0 -exec rm -f {} ;

Continue reading