John Borwick

Perl one liners for daily use

Almost every day, I write simple programs to save me time. For example, maybe a web site has a table with eight columns and I want to copy & paste the third column into an email. Maybe I need every other line of a list. Maybe a paragraph has a bunch of numbers in it and I want to sum the numbers.

A long time ago I was an expert Perl programmer. I taught a one-credit Perl class at NC State, "CSC255," I wrote several large projects in Perl for work such as an LDAP+DNS+DHCP configuration tool, and I made minor contributions to Perl and Perl modules. I eventually switched to Python depsite the Python vs. Perl rivalry of the late 1990s/early 2000s; I switched in no small part because Red Hat pre-loaded Python but not Perl in Red Hat Linux Advanced Server 2.1.

Although it might take you a long time to write arbitrary Perl programs, it wouldn't necessarily take much time to learn just enough to use Perl for basic tasks. Below I describe a few common Perl commands I use.

The pattern: copy + script + paste

For the programs I'm going to describe, the pattern is:

  1. Copy whatever text you want to analyze into your clipboard
  2. "Paste" the content into a Perl program and then copy the result back into the clipboard
  3. Paste the program output into wherever you need it

OS X

For OS X, to do step #2 you open the Terminal. The key OS X commands you need are pbpaste (pastes clipboard) and pbcopy (copies into clipboard). Here is an example program in OS X: 

pbpaste | perl -pe 's|\bJohn\b|Jeff|g' | pbcopy

This sends whatever you have in the clipboard via pbpaste to a Perl program. The Perl program, in this case, replaces "John" with "Jeff." The output of the Perl program is then sent to pbcopy, which updates the clipboard. The | character means "pipe," or send the output of the thing on the left to the input of the thing on the right.

Windows

Windows has several ways to run Perl, including via PowerShell, cmd, and/or the Windows Subsystem for Linux. I don't know the best option; what I use is cygwin. In cygwin the commands are getclip and putclip. Here is an example of this pattern: 

getclip | perl -pe 's|\bJohn\b|Jeff|g' | putclip

This sends whatever you have in the clipboard via getclip to a Perl program. The Perl program, in this case, replaces "John" with "Jeff." The output of the Perl program is then sent to putclip, which updates the clipboard.

Perl super-basics

Perl is a programming language that, in comparison to most other languages, is extra good at processing text. Unfortunately, Perl can be an extremely complicated ways because of its slogan, "TIMTOWTDI" (pronounced / tihm-toe'-dee /), which stands for, "There Is More Than One Way To Do It." It's really hard to learn all the ways to do something in Perl.

That said, Perl has a few command line options that make it very easy to do a few things with Perl.

-e option: Execute code

perl -e 'print 1+1'

This prints 2.

-n option: Loop over each line of input

This option will put the following code around your program for you: 

while (<>) {
  # your code goes here
}

This means "run this code for each line of input." Each line is put into a special variable called $_. For example: 

perl -ne 'print uc $_'

Runs the uc command over each line of input and prints it out. uc is Perl's uppercase function, so this code uppercases everything. Starting to put things together: 

pbpaste | perl -ne 'print uc $_' | pbcopy

will upper-case your clipboard in OS X (by sending the input into Perl, running this tiny program, and then sending that output back into the clipboard).

(As an example of how Perl can become really complicated: many functions take $_ as the default argument, so print uc actually does the same thing as print uc $_.)

-p ("print") option: Loop over each line of input and then print the output

This does the same thing as -n except it prints out $_ after each line. This option will put the following code around your program for you: 

while (<>) {
  # your code goes here
  print $_;
}

This is best used when you want to substitute things for other things. For example, if you wanted to replace vowels with asterisks: 

pbpaste | perl -pe 's/[aeiou]/*/g' | pbcopy

Now the program is starting to look like gibberish. The s means "substitute": substitute the thing on the left, [aeiou] (which means "one of these vowels"), with the thing on the right, the character *, globally (/g). This is a type of regular expression.

-l option: linebreaks

If you are missing returns (linebreaks) in your output, add -l. This adds returns each time you print.

-a and -F options: "awk" mode, good for delimiters e.g. tabs

When combined with -n or -p above, this becomes extremely useful. -a will split each line into a special array @F that you can reference. This option puts essentially the following code around your program: 

while (<>) {
  @F = split $VALUE_OF_F_COMMAND_LINE_OPTION, $_;
}

So for example if my clipboard has a table in it with three columns separated by tabs, and I want the middle column, I can run 

pbpaste | perl -F'\t' -alne 'print $F[1]' | pbcopy

The -a splits the clipboard on the -F delimiter (a tab, represented by \t). The three columns values are put into the array @F. The code then prints out the second column (counting from zero) and that's sent to the clipboard.

Example utility commands

Here are a few commands I run all the time.

Word count

This doesn't even use Perl. 

pbpaste | wc -w

This sends the clipboard to the wc command, which is UNIX's "word count" program. -w means show word count only.

Pull a column from a whitespace-separated table

pbpaste | perl -F'/\s+/' -alne 'print $F[2]'

Print each line that comes after a string

For example, if the clipboard looks something like: 

Test1
Description
Here's the description

Test2
Description
Here's another description

and I want to pull the lines after "Description," I can do this: 

pbpaste | perl -ne 'if ($DESC==1) { print; $DESC=0 }
  elsif (/^Description$/) { $DESC=1 }' | pbcopy

Add up any numbers in the clipboard

If I have a clipboard with a lot of numbers and I want to know the sum of the numbers: 

pbpaste | perl -ne '$X+=$1 while (/(\d+)/g); END { print $X }'

Again, this is starting to show how Perl can get more and more complicated. I made up the variable $X on the fly. The /(\d+)/g is another regular expression that means "find digits"; the parentheses mean "capture the digits you find and put them into $1". The END block is run only once, at the end of the program.

It's that simple! :-P

Getting started and learning more

If you do try out Perl, keep in mind that it's OK for your code to be ugly as long as it works. When you write one-line Perl programs, you're not trying to code for the ages. Also, don't get disheartened if you find out later that there was an easier way to write your program. That's just part of Perl; there's always an easier way to write your program.

Also, keep in mind Perl's three virtues: laziness, impatience, and hubris. If you get better at writing these short programs, you can potentially save yourself hours of work.

If you do want to learn more, you'll get the most value out of learning more about regular expressions. I enjoy regex crossword as a way of learning more about them.

There are also great Perl man pages, which I'm guessing have fallen out of vogue. perlrun(1), for example, describes Perl's command line options in excruciating detail. perlfunc(1) describes Perl's built-in functions.

If you have access to it, Programming Perl has historically been the best source of learning Perl. It's a bit weighty, though; the fourth edition is over 1100 pages.

Updated: