Craft Powerful Bash Pipelines with the Unix Xargs Command

An introduction to the xargs command with practical examples

Photo by Andras Vas on Unsplash

xargs is a Unix command used to build and execute commands from the standard input. You can combine it with other powerful Unix commands, such as grep, awk, etc., by using pipes. In simple terms, it passes the output of a command as the input of another.

It works on most Unix-like operating systems. But be aware that this command works slightly differently on macOS and Linux. The examples in this tutorial have been prepared and tested on Ubuntu distribution.

In this article, you’ll gain insight into xargs’s practical use cases through simple examples.

Let’s get started!

Common Uses

First, let’s see how the xargs command works by itself.

If you try to run it without any arguments, it will execute the echo command by default. To get an output, just type something and press Ctrl+D on your keyboard:

$ xargs
hello

The result:

hello

Handle multiple files at once

Usually, xargs is used in combination with other commands. To walk you through the typical use cases, I’ll need some test files. Let’s create our dummy data by using the xargs command straight away.

To create multiple files at once, combine xargs with the touch command:

$ echo test1.log test2.log test3.log 1.txt 2.csv | xargs touch$ ls
1.txt  2.csv test1.log  test2.log  test3.log

Note that the echo command’s output was passed as an input of the touch command.

Find files and perform various operations

One of the most common combinations of xargs is with the find command. It can be useful when you want to perform operations like renaming or deleting multiple files.

Suppose that you want to find and delete all log files containing test in their names. Let’s play with our dummy data:

$ find . -type f -name "*test*.log" | xargs ls
./test1.log  ./test2.log  ./test3.log$ find . -type f -name "*test*.log" | xargs rm -f$ ls
1.txt  2.csv  3.test

As you can see, at first, I executed the ls command to check which filenames I get. Then I ran the rm -f command to delete those files.

However, bear in mind that some files might have white spaces in their names. Consider the following filename: something with white space.log.

Let’s try to delete the file with the previous command:

$ ls
1.txt   2.csv   3.test  'something with white space.log'$ find . -type f -name "*.log" | xargs rm -f$ ls
1.txt   2.csv   3.test  'something with white space.log'

Unfortunately, we didn’t get the expected result. So, how do we deal with white spaces? Use the find -print0 option and xargs -0 option:

$ find . -type f -name "*.log" -print0 | xargs -0 rm -rf$ ls
1.txt  2.csv  3.test

This way, -print0 separates the filenames with a null character so that names containing spaces or newlines can be interpreted correctly by xargs.

Combine it with the grep command to find patterns in multiple files

Another typical usage is of xargs is with the grep command. For example, let’s try to find the string hello by scanning multiple files:

$ find . -type f | xargs grep 'hello'./1.txt:hello there
./3.test:hello here

Read input from argument files

So far, we’ve seen that the xargs command accepts input from the standard input. It can also read items from a file. To achieve this, add the -a option, which stands for — arg-file.

I’ve created an input file called files.txt:

$ cat files.txt 
3.test
1.txt$ xargs -a files.txt ls -lart
-rw-rw-r-- 1 kirshi kirshi 12 apr    2 21:27 1.txt
-rw-rw-r-- 1 kirshi kirshi 19 apr    2 21:27 3.test

We’ve just executed the ls -lart command on all files listed in the argument file.

Prompt the user for confirmation

Sometimes when we execute destructive operations like rm -f, we might want to ask the user for confirmation. The -p option can do this for us. It stands for — interactive and prompts the user for yes/no confirmation.

Let’s check out this example:

$ xargs -p -a files.txt rm -f
rm -f 3.test 1.txt ?...y$ ls
2.csv  files.txt

It’s pretty handy that it shows what operation is going to be performed. As you can see, the files have been deleted because I typed y for “Yes” when the program asked.

Execute commands line by line

The operation above deleted all files. What if we wanted to check and confirm them one by one? Here is how the -n option (— max-args) comes into play.

It requires a number of arguments you want xargs to pass per command line.

Let’s run the deletion command again — this time, with -n 1:

$ xargs -p -n1 -a files.txt rm -f
rm -f 3.test ?...y
rm -f 1.txt ?...n$ ls
1.txt  2.csv  files.txt

The 1.txt file was preserved because I typed n in the terminal.

Summary

We’ve seen how to use the xargs command combined with other commands to perform operations like:

• Creating/manipulating multiple files at once

• Finding files based on a pattern and performing operations on each file

• Reading input from argument files

• Prompting the user for confirmation before executing a command

These are some of the most frequently used cases, but you can do even more with this command. To explore other features, head over to the documentation.