Regular Expressions

Regular Expressions Regular Expressions

By Karl Voelker

A Crucial Note

Many variations on regular expression syntax exist.
Most of the syntax you see here is commonly supported, but I make no guarantees.

What is a Regular Expression?

A pattern that:

Matches strings
Extracts parts of strings
Alters strings

A concept that:

Is old, within Computer Science
Completely changes how you deal with - and think about - text
Is often abbreviated to "regex" or "regexp"

Regular Expression Basics

Each character in a regex either:

Matches something, usually zero or more characters
Modifies the way in which the previous character works

A Simple Match

Regex: A

This matches the first letter "A" found in the input
We only find out one fact: whether or not the input matched the regex
Later, we learn why simply matching is useful

The Star

Regex: B*

This matches zero or more consecutive "B"s
Do these example inputs match? If so, how many characters do they match?
- "B"
- "BBBBBBB"
- "ABBBCDE"
- "ABCB"
- "Q"
- ""

More Quantifiers

These are applied like the star.

`+`	One or more
`?`	Zero or one
`{x}`	X
`{x,}`	X or more
`{,x}`	Zero through X
`{x,y}`	X through Y

Character Classes

These expressions each match any of a set of characters:

`.`	Anything but newlines
`\d`	Digit
`\s`	Whitespace (`\n`, `\r`, `\t`, space)
`\w`	Word (alphabetics, digits, underscore)
`\D`, `\S`, `\W`	Opposite of `\d`, `\s`, and `\w`, respectively

Note that when applied to Unicode input, these will act in an internationalized fashion.

More Character Classes

Some of the most powerful characters classes are the ones you create for yourself. Surround the characters to match with square braces:

`[abcd]`	Match "a", "b", "c", or "d"
`[\d\s]`	Match any digit or whitespace

You can also use character ranges, or invert the entire character class:

`[A-Zaeiou]`	Match any English vowel or uppercase letter
`[^_]`	Match anything but an underscore
`[^0-9A-Fa-f]`	Match anything but valid hexadecimal digits

Pop Quiz

Have you been paying attention? What will these match?

\w+ \w+
a.c.e*
dir?e
\d{1,3},\d{3},\d{3}
a?\s+a?
[A-Z][a-z]+

Boundary Matches

These special matches don't match characters:

`^`	Match the beginning of the input
`$`	Match the end of the input
`\b`	Match a "word boundary" - between a word character and a non-word character
`\B`	Match a "non-word boundary" - between two word or two non-word characters

Options

These options are hacks that alter the regular expression language:

Option	Description
`m`	`^` and `$` match the beginning and end of any line, not the entire string
`s`	`.` does not exclude newlines

Extracting Sub-Patterns

The part of the input that matched a sub-pattern can be extracted
Usually, sub-patterns come out as an array

Example:

Regex: href="([^"])">([^>]+)<
Input: <a href="http://karlv.net/">Click Here</a>
Sub-pattern matches: "http://karlv.net/", "Click Here"

Escaping Metacharacters

Metacharacters are those that do something special
Escape them with \
To escape many characters, surround with \Q and \E

For example:

\. matches a period
\+ matches a plus sign
\Q129.21.60.1\E matches "129.21.60.1"

Greed

By default, all matching is greedy: the match which covers the most characters is chosen.
Put ? after the quantifiers to make a match non-greedy.

For example: suppose you want to extract the first paragraph in an HTML document.

<p>(.*)</p> won't work if there are multiple paragraphs
<p>(.*?)</p> will always work

The greedy match covers all the characters from the opening tag of the first paragraph to the closing tag of the last paragraph.

Alternation and Grouping

Use the alternation operator to separate two patterns, either of which may match:

^(a+b+c+|d{4,6})$: Matches "abc" and "dddd" but not "abcdddd"
^(cat|dog)$: Matches "cat" or "dog"

Use parentheses to surround sub-patterns. This also affects the reach of the alternation operator:

^(a|b)(c|d)$: Matches "ac", "ad", "bc", or "bd"
^(cat|dog)s$: Matches "cats" or "dogs"

Validation

Whenever your program accepts input:

You should know what inputs are valid.
A regex can easily check inputs for validity.
Your validation regex must:
1. Match any valid input
2. Not match any invalid input
Sometimes, it is easier to match all invalid inputs instead

Do not risk the security of your software on my regular expressions! I take no responsibility.

Validation Example: Integers

Valid inputs	Regex
Non-negative integers	`^[0-9]+$`
Positive integers	`^[1-9]+[0-9]*$`
Integers	`^-?[0-9]+$`
Integers (better)	`^(-?[1-9]+[0-9]*\|0)$`

Why is the last example better?
What do these examples have in common?

Substitution

Substitution replaces the matching part of the input
When giving the replacement, you can refer to sub-pattern matches with \n

For example:

Regex: \b(cat|dog|bird|lunatic)\b
Replacement: \1s
"cat" becomes "cats"
"lunatic" becomes "lunatics"

Another example:

Regex: \b(\w+)\s+(\w+)\b
Replacement: \2, \1
"Karl Voelker" becomes "Voelker, Karl"

In Perl


my $input = "some data from somewhere";
my @extracts = $input =~ /regex (with) (parens)/;

How to learn more:

Run perldoc perlreref
Read this page

In Ruby


input = "some data from somewhere";
extracts = /regex (with) (parens)/.match(input);

Learn more at this page

In Python


input = "some data from somewhere"
extracts = re.compile("regex (with) (parens)").search(input)

Learn more at this page

In C++

You need to have the Boost.Regex library installed to use this example.


char *input = "some input from somewhere";
cmatch m;
regex r("regex (with) (parens)");
regex_match(input, m, r);

Learn more at this page

In Java


String input = "some data from somewhere";
Pattern p = Pattern.compile("regex (with) (parens)");
Matcher m = p.matcher(input);

Learn more at .

In C#


String input = "some data from somewhere";
Match extracts = Regex.Match("regex (with) (parens)", input);

Learn more at this page

In JavaScript


var input = "some data from somewhere";
var extracts = /regex (with) (parens)/.match(input);

Learn more at this page

In UNIX


echo "some data from somewhere" | egrep 'regex (with) (parens)'

To learn more, run man grep.

In PHP :(


$input = "some data from somewhere";
preg_match("/regex (with) (parens)/", $input, $extracts);

Learn more at this page

The End

Please check around your seats for brain matter before leaving.