Skip to content

Chapter 1 of 8

Character Matching: Metacharacters, Shorthands, and Classes

A regex engine begins by deciding which characters in the input stream a token can consume. The dot metacharacter . is the most permissive token: by default it matches any single character except the newline character \n. To make it match newlines as well, you enable the dotall mode using the s flag in JavaScript and PCRE, or the re.DOTALL constant in Python. The dot is convenient but dangerous in untrusted input because it can sweep up unwanted characters; always consider a more specific class when the surrounding context is known.

For the most common character categories, regex provides three shorthand classes. \d matches any decimal digit and is equivalent to [0-9] in ASCII mode, but with Unicode awareness it also includes digits from scripts such as Arabic-Indic. \w matches any word character, defined in ASCII as [A-Za-z0-9_] and extended to letters and digits from other scripts in Unicode mode; note that the underscore is always included. \s matches whitespace, including the space, tab \t, newline \n, carriage return \r, form feed \f, and vertical tab \v, with rare forms like the non-breaking space U+00A0 added in Unicode mode. Each of these shorthands has a negation: \D, \W, and \S respectively, which are useful when you need to assert that a position is not of a given type.

When the built-in shorthands are too broad or too narrow, you build your own class with square brackets. A class like [aeiou] matches any one of the listed characters, and you can negate it with a leading caret, so [^aeiou] matches any non-vowel. Ranges use a hyphen between endpoints, as in [a-z] or [0-9A-F] for hexadecimal digits; the hyphen is literal when it appears at the start, end, or right after a backslash inside the class. POSIX-style engines additionally accept named classes enclosed in [:...:], such as [[:alpha:]] or [[:space:]], which are useful when you want locale-aware letter or punctuation matching. To match a literal special character like a dot, parenthesis, or pipe, escape it with a backslash, since most punctuation marks carry metacharacter meaning.

All chapters
  1. 1Character Matching: Metacharacters, Shorthands, and Classes
  2. 2Quantifiers: Greedy, Lazy, Possessive, and Exact
  3. 3Anchors and Boundaries
  4. 4Groups, Captures, and Backreferences
  5. 5Lookaround Assertions
  6. 6Flags, Modes, and Engine Features
  7. 7Practical Recipes for Validation and Extraction
  8. 8Performance and Engine Internals

Drill it

Reading is not remembering. These come from the Regex Patterns Every Developer Should Know deck:

Q

What does the regex metacharacter <code>.</code> match by default?

A single character of any kind except the newline character (\n). To make it also match newlines, use the s (dotall / single-line) flag, e.g. /foo.bar/s in Java...

Q

Which character class matches any single digit in most regex flavors?

[0-9] or the shorthand \d. \d is equivalent to [0-9] in ASCII; with Unicode (u flag in JS, default in Python 3 str) it matches any Unicode decimal digit, e.g. A...

Q

What does the shorthand <code>\w</code> match?

Any word character. In ASCII mode this is [A-Za-z0-9_]; in Unicode mode it also matches letters and digits from other scripts, and the underscore.

Q

What does the shorthand <code>\s</code> match?

Any whitespace character: space, tab (\t), newline (\n), carriage return (\r), form feed (\f), and vertical tab (\v). In Unicode mode it can also match rare whi...