Chapter 3

Figure 3-13

Matching Optional Characters

Matching literal characters is straightforward, particularly when you are aiming to match exactly one literal character for each corresponding literal character that you include in a regular expression pattern. The next step up from that basic situation is where a single literal character may occur zero times or one time. In other words, a character is optional. Most regular expression dialects use the question mark (?) character to indicate that the preceding chunk is optional. I am using the term “chunk” loosely here to mean the thing that precedes the question mark. That chunk can be a single character or various, more complex regular expression constructs. For the moment, we will deal with the case of the single, optional character. More complex regular expression constructs, such as groups, are described in Chapter 7.

For example, suppose you are dealing with a group of documents that contain both U.S. English and British English.

You may find that words such as color (in U.S. English) appear as colour (British English) in some documents. You can express a pattern to match both words like this:

colou?r

You may want to standardize the documents so that all the spellings are U.S. English spellings.

Try It Out

Matching an Optional Character

Try this out using the Komodo Regular Expression Toolkit:

1.Open the Komodo Regular Expression Toolkit ,and clear any regular expression pattern or text that may have been retained.

2.Insert the text colour into the area for the text to be matched.

3.Enter the regular expression pattern colou?r into the area for the regular expression pattern. The text colour is matched, as shown in Figure 3-14.

56

Simple Regular Expressions

Figure 3-14

Try this regular expression pattern with text such as that shown in the sample file Colors.txt:

Red is a color.

His collar is too tight or too colouuuurful.

These are bright colours.

These are bright colors.

Calorific is a scientific term.

“Your life is very colorful,” she said.

How It Works

The word color in the line Red is a color. will match the pattern colou?r.

When the regular expression engine reaches a position just before the c of color, it attempts to match a lowercase c. This match succeeds. It next attempts to match a lowercase o. That too matches. It next attempts to match a lowercase l and a lowercase o. They match as well. It then attempts to match the pattern u?, which means zero or one lowercase u characters. Because there are exactly zero lowercase u characters following the lowercase o, there is a match. The pattern u? matches zero characters. Finally, it attempts to match the final character in the pattern — that is, the lowercase r. Because the next character in the string color does match a lowercase r, the whole pattern is matched.

There is no match in the line His collar is too tight or too colouuuurful. The only possible match might be in the sequence of characters colouuuurful. The failure to match occurs when the regular expression engine attempts to match the pattern u?. Because the pattern u? means “match zero or one lowercase u characters,” there is a match on the first u of colouuuurful. After that successful match, the regular expression engine attempts to match the final character of the pattern colou?r against the second lowercase u in colouuuurful. That attempt to match fails, so the attempt to match the whole pattern colou?r against the sequence of characters colouuuurful also fails.

57

Chapter 3

What happens when the regular expression engine attempts to find a match in the line These are bright colours.?

When the regular expression engine reaches a position just before the c of colours, it attempts to match a lowercase c. That match succeeds. It next attempts to match a lowercase o, a lowercase l, and another lowercase o. These also match. It next attempts to match the pattern u?, which means zero or one lowercase u characters. Because exactly one lowercase u character follows the lowercase o in colours, there is a match. Finally, the regular expression engine attempts to match the final character in the pattern, the lowercase r. Because the next character in the string colours does match a lowercase r, the whole pattern is matched.

The findstr utility can also be used to test for the occurrence of the sequence of characters color and colour, but the regular expression engine in the findstr utility has a limitation in that it lacks a metacharacter to signify an optional character. For many purposes, the * metacharacter, which matches zero, one, or more occurrences of the preceding character, will work successfully.

To look for lines that contain matches for colour and color using the findstr utility, enter the following at the command line:

findstr /N colo*r Colors.txt

The preceding command line assumes that the file Colors.txt is in the current directory.

Figure 3-15 shows the result from using the findstr utility on Colors.txt.

Figure 3-15

Notice that lines that contain the sequences of characters color and colour are successfully matched, whether as whole words or parts of longer words. However, notice, too, that the slightly strange “word” colouuuurful is also matched due to the * metacharacter’s allowing multiple occurrences of the lowercase letter u. In most practical situations, such bizarre “words” won’t be an issue for you, and the * quantifier will be an appropriate substitute for the ? quantifier when using the findstr utility. In some situations, where you want to match precisely zero or one specific characters, the findstr utility may not provide the functionality that you need, because it would also match a character sequence such as colonifier.

Having seen how we can use a single optional character in a regular expression pattern, let’s look at how you can use multiple optional characters in a single regular expression pattern.

58

Simple Regular Expressions

Matching Multiple Optional Characters

Many English words have multiple forms. Sometimes, it may be necessary to match all of the forms of a word. Matching all those forms can require using multiple optional characters in a regular expression pattern.

Consider the various forms of the word color (U.S. English) and colour (British English). They include the following:

color (U.S. English, singular noun)

colour (British English, singular noun)

colors (U.S. English, plural noun)

colours (British English, plural noun)

color’s (U.S. English, possessive singular)

colour’s (British English, possessive singular)

colors’ (U.S. English, possessive plural)

colours’ (British English, possessive plural)

The following regular expression pattern, which include three optional characters, can match all eight of these word forms:

colou?r’?s?’?

If you tried to express this in a semiformal way, you might have the following problem definition:

Match the U.S. English and British English forms of color (colour), including the singular noun, the plural noun, and the singular possessive and the plural possessive.

Let’s try it out, and then I will explain why it works and what limitations it potentially has.

Try It Out Matching Multiple Optional Characters

Use the sample file Colors2.txt to explore this example:

These colors are bright.

Some colors feel warm. Other colours feel cold.

A color’s temperature can be important in creating reaction to an image.

These colours’ temperatures are important in this discussion.

Red is a vivid colour.

59

Chapter 3

To test the regular expression, follow these steps:

1.Open OpenOffice.org Writer, and open the file Colors2.txt.

2.Use the keyboard shortcut Ctrl+F to open the Find and Replace dialog box.

3.Check the Regular Expressions check box and the Match Case check box.

4.In the Search for text box, enter the regular expression pattern colou?r’?s?’?, and click the Find All button. If all has gone well, you should see the matches shown in Figure 3-16.

Figure 3-16

As you can see, all the sample forms of the word of interest have been matched.

How It Works

In this description, I will focus initially on matching of the forms of the word colour/color.

How does the pattern colou?r’?s?’? match the word color? Assume that the regular expression engine is at the position immediately before the first letter of color. It first attempts to match lowercase c, because one lowercase c must be matched. That matches. Attempts are then made to match a subsequent

60

Simple Regular Expressions

lowercase o, l, and o. These all also match. Then an attempt is made to match an optional lowercase u. In other words, zero or one occurrences of the lowercase character u is needed. Because there are zero occurrences of lowercase u, there is a match. Next, an attempt is made to match lowercase r. The lowercase r in color matches. Then an attempt is made to match an optional apostrophe. Because there is no occurrence of an apostrophe, there is a match. Next, the regular expression engine attempts to match an optional lowercase s — in other words, to match zero or one occurrence of lowercase s. Because there is no occurrence of lowercase s, again, there is a match. Finally, an attempt is made to match an optional apostrophe.

Because there is no occurrence of an apostrophe, another match is found. Because a match exists for all the components of the regular expression pattern, there is a match for the whole regular expression pattern colour?r’?s?’?.

Now, how does the pattern colou?r’?s?’? match the word colour? Assume that the regular expression engine is at the position immediately before the first letter of colour. It first attempts to match lowercase c, because one lowercase c must be matched. That matches. Next, attempts are made to match a subsequent lowercase o, l, and another o. These also match. Then an attempt is made to match an optional lowercase u. In other words, zero or one occurrences of the lowercase character u are needed. Because there is one occurrence of lowercase u, there is a match. Next, an attempt is made to match lowercase r. The lowercase r in colour matches. Next, the engine attempts to match an optional apostrophe. Because there is no occurrence of an apostrophe, there is a match. Next, the regular expression engine attempts to match an optional lowercase s — in other words, to match zero or one occurrences of lowercase s. Because there is no occurrence of lowercase s, a match exists. Finally, an attempt is made to match an optional apostrophe. Because there is no occurrence of an apostrophe, there is a match. All the components of the regular expression pattern have a match; therefore, the entire regular expression pattern colour?r’?s?’? matches.

Work through the other six word forms shown earlier, and you’ll find that each of the word forms does, in fact, match the regular expression pattern.

The pattern colou?r’?s?’? matches all eight of the word forms that were listed earlier, but will the pattern match the following sequence of characters?

colour’s’

Can you see that it does match? Can you see why it matches the pattern? If each of the three optional characters in the regular expression is present, the preceding sequence of characters matches. That rather odd sequence of characters likely won’t exist in your sample document, so the possibility of false matches (reduced specificity) won’t be an issue for you.

How can you avoid the problem caused by such odd sequences of characters as colour’s’? You want to be able to express is something like this:

Match a lowercase c. If a match is present, attempt to match a lowercase o. If that match is present, attempt to match a lowercase l. If there is a match, attempt to match a lowercase o. If a match exists, attempt to match an optional lowercase u. If there is a match, attempt to match a lowercase r. If there is a match, attempt to match an optional apostrophe. And if a match exists here, attempt to match an optional lowercase s. If the earlier optional apostrophe was not present, attempt to match an optional apostrophe.

With the techniques that you have seen so far, you aren’t able to express ideas such as “match something only if it is not preceded by something else.” That sort of approach might help achieve higher specificity at the expense of increased complexity. Techniques where matching depends on such issues are presented in Chapter 9.

61