<othercredit role="translator"><firstname>Malcolm</firstname><surname>Hunter</surname><affiliation><address><email>malcolm.hunter@gmx.co.uk</email></address></affiliation><contrib>Conversion to British English</contrib></othercredit>
<para><emphasis>Regular Expressions</emphasis> provides us with a way to describe some possible contents of a text string in a way understood by a small piece of software, so that it can investigate if a text matches, and also in the case of advanced applications with the means of saving pieces or the matching text.</para>
<para>An example: Say you want to search a text for paragraphs that starts with either of the names <quote>Henrik</quote> or <quote>Pernille</quote> followed by some form of the verb <quote>say</quote>.</para>
<para>With a normal search, you would start out searching for the first name, <quote>Henrik</quote> maybe followed by <quote>sa</quote> like this: <userinput>Henrik sa</userinput>, and while looking for matches, you would have to discard those not being the beginning of a paragraph, as well as those in which the word starting with the letters <quote>sa</quote> was not either <quote>says</quote>, <quote>said</quote> or so. And then of cause repeat all of that with the next name...</para>
<para>With Regular Expressions, that task could be accomplished with a single search, and with a larger degree of preciseness.</para>
<para>To achieve this, Regular Expressions defines rules for expressing in details a generalisation of a string to match. Our example, which we might literally express like this: <quote>A line starting with either <quote>Henrik</quote> or <quote>Pernille</quote> (possibly following up to 4 blanks or tab characters) followed by a whitespace followed by <quote>sa</quote> and then either <quote>ys</quote> or <quote>id</quote></quote> could be expressed with the following regular expression:</para> <para><userinput>^[ \t]{0,4}(Henrik|Pernille) sa(ys|id)</userinput></para>
<para>The above example demonstrates all four major concepts of modern Regular Expressions, namely:</para>
<para>The caret (<literal>^</literal>) starting the expression is an assertion, being true only if the following matching string is at the start of a line.</para>
<para>The stings <literal>[ \t]</literal> and <literal>(Henrik|Pernille) sa(ys|id)</literal> are patterns. The first one is a <emphasis>character class</emphasis> that matches either a blank or a (horizontal) tab character; the other pattern contains first a subpattern matching either <literal>Henrik</literal> <emphasis>or</emphasis> <literal>Pernille</literal>, then a piece matching the exact string <literal> sa</literal> and finally a subpattern matching either <literal>ys</literal> <emphasis>or</emphasis> <literal>id</literal></para>
<para>The string <literal>{0,4}</literal> is a quantifier saying <quote>anywhere from 0 up to 4 of the previous</quote>.</para>
<para>Because regular expression software supporting the concept of <emphasis>back references</emphasis> saves the entire matching part of the string as well as sub-patterns enclosed in parentheses, given some means of access to those references, we could get our hands on either the whole match (when searching a text document in an editor with a regular expression, that is often marked as selected) or either the name found, or the last part of the verb.</para>
<para>All together, the expression will match where we wanted it to, and only there.</para>
<para>The following sections will describe in details how to construct and use patterns, character classes, assertions, quantifiers and back references, and the final section will give a few useful examples.</para>
<para>Patterns consists of literal strings and character classes. Patterns may contain sub-patterns, which are patterns enclosed in parentheses.</para>
<para>In patterns as well as in character classes, some characters have a special meaning. To literally match any of those characters, they must be marked or <emphasis>escaped</emphasis> to let the regular expression software know that it should interpret such characters in their literal meaning.</para>
<para>The regular expression software will silently ignore escaping a character that does not have any special meaning in the context, so escaping for example a <quote>j</quote> (<userinput>\j</userinput>) is safe. If you are in doubt whether a character could have a special meaning, you can therefore escape it safely.</para>
<title>Character Classes and abbreviations</title>
<para>A <emphasis>character class</emphasis> is an expression that matches one of a defined set of characters. In Regular Expressions, character classes are defined by putting the legal characters for the class in square brackets, <literal>[]</literal>, or by using one of the abbreviated classes described below.</para>
<para>Simple character classes just contains one or more literal characters, for example <userinput>[abc]</userinput> (matching either of the letters <quote>a</quote>, <quote>b</quote> or <quote>c</quote>) or <userinput>[0123456789]</userinput> (matching any digit).</para>
<para>Because letters and digits have a logical order, you can abbreviate those by specifying ranges of them: <userinput>[a-c]</userinput> is equal to <userinput>[abc]</userinput> and <userinput>[0-9]</userinput> is equal to <userinput>[0123456789]</userinput>. Combining these constructs, for example <userinput>[a-fynot1-38]</userinput> is completely legal (the last one would match, of cause, either of <quote>a</quote>,<quote>b</quote>,<quote>c</quote>,<quote>d</quote>, <quote>e</quote>,<quote>f</quote>,<quote>y</quote>,<quote>n</quote>,<quote>o</quote>,<quote>t</quote>, <quote>1</quote>,<quote>2</quote>,<quote>3</quote> or <quote>8</quote>).</para>
<para>As capital letters are different characters from their non-capital equivalents, to create a caseless character class matching <quote>a</quote> or <quote>b</quote>, in any case, you need to write it <userinput>[aAbB]</userinput>.</para>
<para>It is of cause possible to create a <quote>negative</quote> class matching as <quote>anything but</quote> To do so put a caret (<literal>^</literal>) at the beginning of the class: </para>
<para><userinput>[^abc]</userinput> will match any character <emphasis>but</emphasis> <quote>a</quote>, <quote>b</quote> or <quote>c</quote>.</para>
<para>In addition to literal characters, some abbreviations are defined, making life still a bit easier: <variablelist>
<listitem><para>This matches the Unicode character corresponding to the hexadecimal number hhhh (between 0x0000 and 0xFFFF). \0ooo (&ie;, \zero ooo) matches the <acronym>ASCII</acronym>/Latin-1 character corresponding to the octal number ooo (between 0 and 0377).</para></listitem>
<listitem><para>This matches a non-whitespace. Practically equal to <literal>[^ \t\r\n]</literal>, and equal to <literal>[^\s]</literal></para></listitem>
<listitem><para>Matches any <quote>word character</quote> - in this case any letter or digit. Note that underscore (<literal>_</literal>) is not matched, as is the case with perl regular expressions. Equal to <literal>[a-zA-Z0-9]</literal></para></listitem>
<listitem><para>Matches any non-word character - anything but letters or numbers. Equal to <literal>[^a-zA-Z0-9]</literal> or <literal>[^\w]</literal></para></listitem>
<para>The abbreviated classes can be put inside a custom class, for example to match a word character, a blank or a dot, you could write <userinput>[\w \.]</userinput></para>
<para>The following characters has a special meaning inside the <quote>[]</quote> character class construct, and must be escaped to be literally included in a class:</para>
<listitem><para>Ends the character class. Must be escaped unless it is the very first character in the class (may follow an unescaped caret)</para></listitem>
<listitem><para>Denotes a negative class, if it is the first character. Must be escaped to match literally if it is the first character in the class.</para></listitem>
<para>You may use a sub pattern to group a set of alternatives within a larger pattern. The alternatives are separated by the character <quote>|</quote> (vertical bar).</para>
<para>For example to match either of the words <quote>int</quote>, <quote>float</quote> or <quote>double</quote>, you could use the pattern <userinput>int|float|double</userinput>. If you only want to find one if it is followed by some whitespace and then some letters, put the alternatives inside a subpattern: <userinput>(int|float|double)\s+\w+</userinput>.</para>
<title>Capturing matching text (back references)</title>
<para>If you want to use a back reference, use a sub pattern to have the desired part of the pattern remembered.</para>
<para>For example, it you want to find two occurrences of the same word separated by a comma and possibly some whitespace, you could write <userinput>(\w+),\s*\1</userinput>. The sub pattern <literal>\w+</literal> would find a chunk of word characters, and the entire expression would match if those were followed by a comma, 0 or more whitespace and then an equal chunk of word characters. (The string <literal>\1</literal> references <emphasis>the first sub pattern enclosed in parentheses</emphasis>)</para>
<!-- <para>See also <link linkend="backreferences">Back references</link>.</para> -->
<para>A lookahead assertion is a sub pattern, starting with either <literal>?=</literal> or <literal>?!</literal>.</para>
<para>For example to match the literal string <quote>Bill</quote> but only if not followed by <quote> Gates</quote>, you could use this expression: <userinput>Bill(?! Gates)</userinput>. (This would find <quote>Bill Clinton</quote> as well as <quote>Billy the kid</quote>, but silently ignore the other matches.)</para>
<para>Sub patterns used for assertions are not captured.</para>
<para>See also <link linkend="assertions">Assertions</link></para>
<para><emphasis>Quantifiers</emphasis> allows a regular expression to match a specified number or range of numbers of either a character, character class or sub pattern.</para>
<para>Quantifiers are enclosed in curly brackets (<literal>{</literal> and <literal>}</literal>) and have the general form <literal>{[minimum-occurrences][,[maximum-occurrences]]}</literal> </para>
<para>The usage is best explained by example: <variablelist>
<para>When using quantifiers with no maximum, regular expressions defaults to match as much of the searched string as possible, commonly known as <emphasis>greedy</emphasis> behaviour.</para>
<para>Modern regular expression software provides the means of <quote>turning off greediness</quote>, though in a graphical environment it is up to the interface to provide you with access to this feature. For example a search dialogue providing a regular expression search could have a check box labelled <quote>Minimal matching</quote> as well as it ought to indicate if greediness is the default behaviour.</para>
<listitem><para>Matches the digits in <quote>1234 go</quote> and <quote>12345 now</quote>, but neither in <quote>567 eleven</quote> nor in <quote>223459 somewhere</quote></para></listitem>
<listitem><para>Matches all of <quote>blablabla</quote> and the <quote>bla</quote> in <quote>blackbird</quote> or <quote>tabla</quote></para></listitem>
<listitem><para>Matches <quote>/></quote> in <quote><closeditem/></quote> as well as <quote>></quote> in <quote><openitem></quote>.</para></listitem>
<para><emphasis>Assertions</emphasis> allows a regular expression to match only under certain controlled conditions.</para>
<para>An assertion does not need a character to match, it rather investigates the surroundings of a possible match before acknowledging it. For example the <emphasis>word boundary</emphasis> assertion does not try to find a non word character opposite a word one at its position, instead it makes sure that there is not a word character. This means that the assertion can match where there is no character, &ie; at the ends of a searched string.</para>
<para>Some assertions actually does have a pattern to match, but the part of the string matching that will not be a part of the result of the match of the full expression.</para>
<para>Regular Expressions as documented here supports the following assertions: <variablelist>
<varlistentry>
<term><userinput>^</userinput> (caret: beginning of string)</term>
<listitem><para>Matches the beginning of the searched string.</para> <para>The expression <userinput>^Peter</userinput> will match at <quote>Peter</quote> in the string <quote>Peter, hey!</quote> but not in <quote>Hey, Peter!</quote> </para> </listitem>
<term><userinput>$</userinput> (end of string)</term>
<listitem><para>Matches the end of the searched string.</para>
<para>The expression <userinput>you\?$</userinput> will match at the last you in the string <quote>You didn't do that, did you?</quote> but nowhere in <quote>You didn't do that, right?</quote></para>
<listitem><para>Matches if there is a word character at one side and not a word character at the other.</para>
<para>This is useful to find word ends, for example both ends to find a whole word. The expression <userinput>\bin\b</userinput> will match at the separate <quote>in</quote> in the string <quote>He came in through the window</quote>, but not at the <quote>in</quote> in <quote>window</quote>.</para></listitem>
<term><userinput>\B</userinput> (non word boundary)</term>
<listitem><para>Matches wherever <quote>\b</quote> does not.</para>
<para>That means that it will match for example within words: The expression <userinput>\Bin\B</userinput> will match at in <quote>window</quote> but not in <quote>integer</quote> or <quote>I'm in love</quote>.</para>
<listitem><para>A lookahead assertion looks at the part of the string following a possible match. The positive lookahead will prevent the string from matching if the text following the possible match does not match the <emphasis>PATTERN</emphasis> of the assertion, but the text matched by that will not be included in the result.</para>
<para>The expression <userinput>handy(?=\w)</userinput> will match at <quote>handy</quote> in <quote>handyman</quote> but not in <quote>That came in handy!</quote></para>
<listitem><para>The negative lookahead prevents a possible match to be acknowledged if the following part of the searched string does match its <emphasis>PATTERN</emphasis>.</para>
<para>The expression <userinput>const \w+\b(?!\s*&)</userinput> will match at <quote>const char</quote> in the string <quote>const char* foo</quote> while it can not match <quote>const TQString</quote> in <quote>const TQString& bar</quote> because the <quote>&</quote> matches the negative lookahead assertion pattern.</para>
