- •Introduction
- •Who This Book Is For
- •What This Book Covers
- •How This Book Is Structured
- •What You Need to Use This Book
- •Conventions
- •Source Code
- •Errata
- •p2p.wrox.com
- •What Are Regular Expressions?
- •What Can Regular Expressions Be Used For?
- •Finding Doubled Words
- •Checking Input from Web Forms
- •Changing Date Formats
- •Finding Incorrect Case
- •Adding Links to URLs
- •Regular Expressions You Already Use
- •Search and Replace in Word Processors
- •Directory Listings
- •Online Searching
- •Why Regular Expressions Seem Intimidating
- •Compact, Cryptic Syntax
- •Whitespace Can Significantly Alter the Meaning
- •No Standards Body
- •Differences between Implementations
- •Characters Change Meaning in Different Contexts
- •Regular Expressions Can Be Case Sensitive
- •Case-Sensitive and Case-Insensitive Matching
- •Case and Metacharacters
- •Continual Evolution in Techniques Supported
- •Multiple Solutions for a Single Problem
- •What You Want to Do with a Regular Expression
- •Replacing Text in Quantity
- •Regular Expression Tools
- •findstr
- •Microsoft Word
- •StarOffice Writer/OpenOffice.org Writer
- •Komodo Rx Package
- •PowerGrep
- •Microsoft Excel
- •JavaScript and JScript
- •VBScript
- •Visual Basic.NET
- •Java
- •Perl
- •MySQL
- •SQL Server 2000
- •W3C XML Schema
- •An Analytical Approach to Using Regular Expressions
- •Express and Document What You Want to Do in English
- •Consider the Regular Expression Options Available
- •Consider Sensitivity and Specificity
- •Create Appropriate Regular Expressions
- •Document All but Simple Regular Expressions
- •Document What You Expect the Regular Expression to Do
- •Document What You Want to Match
- •Test the Results of a Regular Expression
- •Matching Single Characters
- •Matching Sequences of Characters That Each Occur Once
- •Introducing Metacharacters
- •Matching Sequences of Different Characters
- •Matching Optional Characters
- •Matching Multiple Optional Characters
- •Other Cardinality Operators
- •The * Quantifier
- •The + Quantifier
- •The Curly-Brace Syntax
- •The {n} Syntax
- •The {n,m} Syntax
- •Exercises
- •Regular Expression Metacharacters
- •Thinking about Characters and Positions
- •The Period (.) Metacharacter
- •Matching Variably Structured Part Numbers
- •Matching a Literal Period
- •The \w Metacharacter
- •The \W Metacharacter
- •Digits and Nondigits
- •The \d Metacharacter
- •Canadian Postal Code Example
- •The \D Metacharacter
- •Alternatives to \d and \D
- •The \s Metacharacter
- •Handling Optional Whitespace
- •The \S Metacharacter
- •The \t Metacharacter
- •The \n Metacharacter
- •Escaped Characters
- •Finding the Backslash
- •Modifiers
- •Global Search
- •Case-Insensitive Search
- •Exercises
- •Introduction to Character Classes
- •Choice between Two Characters
- •Using Quantifiers with Character Classes
- •Using the \b Metacharacter in Character Classes
- •Selecting Literal Square Brackets
- •Using Ranges in Character Classes
- •Alphabetic Ranges
- •Use [A-z] With Care
- •Digit Ranges in Character Classes
- •Hexadecimal Numbers
- •IP Addresses
- •Reverse Ranges in Character Classes
- •A Potential Range Trap
- •Finding HTML Heading Elements
- •Metacharacter Meaning within Character Classes
- •The ^ metacharacter
- •How to Use the - Metacharacter
- •Negated Character Classes
- •Combining Positive and Negative Character Classes
- •POSIX Character Classes
- •The [:alnum:] Character Class
- •Exercises
- •String, Line, and Word Boundaries
- •The ^ Metacharacter
- •The ^ Metacharacter and Multiline Mode
- •The $ Metacharacter
- •The $ Metacharacter in Multiline Mode
- •Using the ^ and $ Metacharacters Together
- •Matching Blank Lines
- •Working with Dollar Amounts
- •Revisiting the IP Address Example
- •What Is a Word?
- •Identifying Word Boundaries
- •The \< Syntax
- •The \>Syntax
- •The \b Syntax
- •The \B Metacharacter
- •Less-Common Word-Boundary Metacharacters
- •Exercises
- •Grouping Using Parentheses
- •Parentheses and Quantifiers
- •Matching Literal Parentheses
- •U.S. Telephone Number Example
- •Alternation
- •Choosing among Multiple Options
- •Unexpected Alternation Behavior
- •Capturing Parentheses
- •Numbering of Captured Groups
- •Numbering When Using Nested Parentheses
- •Named Groups
- •Non-Capturing Parentheses
- •Back References
- •Exercises
- •Why You Need Lookahead and Lookbehind
- •The (? metacharacters
- •Lookahead
- •Positive Lookahead
- •Negative Lookahead
- •Positive Lookahead Examples
- •Positive Lookahead in the Same Document
- •Inserting an Apostrophe
- •Lookbehind
- •Positive Lookbehind
- •Negative Lookbehind
- •How to Match Positions
- •Adding Commas to Large Numbers
- •Exercises
- •What Are Sensitivity and Specificity?
- •Extreme Sensitivity, Awful Specificity
- •Email Addresses Example
- •Replacing Hyphens Example
- •The Sensitivity/Specificity Trade-Off
- •Sensitivity, Specificity, and Positional Characters
- •Sensitivity, Specificity, and Modes
- •Sensitivity, Specificity, and Lookahead and Lookbehind
- •How Much Should the Regular Expressions Do?
- •Abbreviations
- •Characters from Other Languages
- •Names
- •Sensitivity and How to Achieve It
- •Specificity and How to Maximize It
- •Exercises
- •Documenting Regular Expressions
- •Document the Problem Definition
- •Add Comments to Your Code
- •Making Use of Extended Mode
- •Know Your Data
- •Abbreviations
- •Proper Names
- •Incorrect Spelling
- •Creating Test Cases
- •Debugging Regular Expressions
- •Treacherous Whitespace
- •Backslashes Causing Problems
- •Considering Other Causes
- •The User Interface
- •Metacharacters Available
- •Quantifiers
- •The @ Quantifier
- •The {n,m} Syntax
- •Modes
- •Character Classes
- •Back References
- •Lookahead and Lookbehind
- •Lazy Matching versus Greedy Matching
- •Examples
- •Character Class Examples, Including Ranges
- •Whole Word Searches
- •Search-and-Replace Examples
- •Changing Name Structure Using Back References
- •Manipulating Dates
- •The Star Training Company Example
- •Regular Expressions in Visual Basic for Applications
- •Exercises
- •The User Interface
- •Metacharacters Available
- •Quantifiers
- •Modes
- •Character Classes
- •Alternation
- •Back References
- •Lookahead and Lookbehind
- •Search Example
- •Search-and-Replace Example
- •Online Chats
- •POSIX Character Classes
- •Matching Numeric Digits
- •Exercises
- •Introducing findstr
- •Finding Literal Text
- •Quantifiers
- •Character Classes
- •Command-Line Switch Examples
- •The /v Switch
- •The /a Switch
- •Single File Examples
- •Simple Character Class Example
- •Find Protocols Example
- •Multiple File Example
- •A Filelist Example
- •Exercises
- •The PowerGREP Interface
- •A Simple Find Example
- •The Replace Tab
- •The File Finder Tab
- •Syntax Coloring
- •Other Tabs
- •Numeric Digits and Alphabetic Characters
- •Quantifiers
- •Back References
- •Alternation
- •Line Position Metacharacters
- •Word-Boundary Metacharacters
- •Lookahead and Lookbehind
- •Longer Examples
- •Finding HTML Horizontal Rule Elements
- •Matching Time Example
- •Exercises
- •The Excel Find Interface
- •Escaping Wildcard Characters
- •Using Wildcards in Data Forms
- •Using Wildcards in Filters
- •Exercises
- •Using LIKE with Regular Expressions
- •The % Metacharacter
- •The _ Metacharacter
- •Character Classes
- •Negated Character Classes
- •Using Full-Text Search
- •Using The CONTAINS Predicate
- •Document Filters on Image Columns
- •Exercises
- •Using the _ and % Metacharacters
- •Testing Matching of Literals: _ and % Metacharacters
- •Using Positional Metacharacters
- •Using Character Classes
- •Quantifiers
- •Social Security Number Example
- •Exercises
- •The Interface to Metacharacters in Microsoft Access
- •Creating a Hard-Wired Query
- •Creating a Parameter Query
- •Using the ? Metacharacter
- •Using the * Metacharacter
- •Using the # Metacharacter
- •Using the # Character with Date/Time Data
- •Using Character Classes in Access
- •Exercises
- •The RegExp Object
- •Attributes of the RegExp Object
- •The Other Properties of the RegExp Object
- •The test() Method of the RegExp Object
- •The exec() Method of the RegExp Object
- •The String Object
- •Metacharacters in JavaScript and JScript
- •SSN Validation Example
- •Exercises
- •The RegExp Object and How to Use It
- •Quantifiers
- •Positional Metacharacters
- •Character Classes
- •Word Boundaries
- •Lookahead
- •Grouping and Nongrouping Parentheses
- •Exercises
- •The System.Text.RegularExpressions namespace
- •A Simple Visual Basic .NET Example
- •The Classes of System.Text.RegularExpressions
- •The Regex Object
- •Using the Match Object and Matches Collection
- •Using the Match.Success Property and Match.NextMatch Method
- •The GroupCollection and Group Classes
- •The CaptureCollection and Capture Class
- •The RegexOptions Enumeration
- •Case-Insensitive Matching: The IgnoreCase Option
- •Multiline Matching: The Effect on the ^ and $ Metacharacters
- •Right to Left Matching: The RightToLeft Option
- •Lookahead and Lookbehind
- •Exercises
- •An Introductory Example
- •The Classes of System.Text.RegularExpressions
- •The Regex Class
- •The Options Property of the Regex Class
- •Regex Class Methods
- •The CompileToAssembly() Method
- •The GetGroupNames() Method
- •The GetGroupNumbers() Method
- •GroupNumberFromName() and GroupNameFromNumber() Methods
- •The IsMatch() Method
- •The Match() Method
- •The Matches() Method
- •The Replace() Method
- •The Split() Method
- •Using the Static Methods of the Regex Class
- •The IsMatch() Method as a Static
- •The Match() Method as a Static
- •The Matches() Method as a Static
- •The Replace() Method as a Static
- •The Split() Method as a Static
- •The Match and Matches Classes
- •The Match Class
- •The GroupCollection and Group Classes
- •The RegexOptions Class
- •The IgnorePatternWhitespace Option
- •Metacharacters Supported in Visual C# .NET
- •Using Named Groups
- •Using Back References
- •Exercise
- •The ereg() Set of Functions
- •The ereg() Function
- •The ereg() Function with Three Arguments
- •The eregi() Function
- •The ereg_replace() Function
- •The eregi_replace() Function
- •The split() Function
- •The spliti() Function
- •The sql_regcase() Function
- •Perl Compatible Regular Expressions
- •Pattern Delimiters in PCRE
- •Escaping Pattern Delimiters
- •Matching Modifiers in PCRE
- •Using the preg_match() Function
- •Using the preg_match_all() Function
- •Using the preg_grep() Function
- •Using the preg_quote() Function
- •Using the preg_replace() Function
- •Using the preg_replace_callback() Function
- •Using the preg_split() Function
- •Supported Metacharacters with ereg()
- •Using POSIX Character Classes with PHP
- •Supported Metacharacters with PCRE
- •Positional Metacharacters
- •Character Classes in PHP
- •Documenting PHP Regular Expressions
- •Exercises
- •W3C XML Schema Basics
- •Tools for Using W3C XML Schema
- •Comparing XML Schema and DTDs
- •How Constraints Are Expressed in W3C XML Schema
- •W3C XML Schema Datatypes
- •Derivation by Restriction
- •Unicode and W3C XML Schema
- •Unicode Overview
- •Using Unicode Character Classes
- •Matching Decimal Numbers
- •Mixing Unicode Character Classes with Other Metacharacters
- •Unicode Character Blocks
- •Using Unicode Character Blocks
- •Metacharacters Supported in W3C XML Schema
- •Positional Metacharacters
- •Matching Numeric Digits
- •Alternation
- •Using the \w and \s Metacharacters
- •Escaping Metacharacters
- •Exercises
- •Introduction to the java.util.regex Package
- •Obtaining and Installing Java
- •The Pattern Class
- •Using the matches() Method Statically
- •Two Simple Java Examples
- •The Properties (Fields) of the Pattern Class
- •The CASE_INSENSITIVE Flag
- •Using the COMMENTS Flag
- •The DOTALL Flag
- •The MULTILINE Flag
- •The UNICODE_CASE Flag
- •The UNIX_LINES Flag
- •The Methods of the Pattern Class
- •The compile() Method
- •The flags() Method
- •The matcher() Method
- •The matches() Method
- •The pattern() Method
- •The split() Method
- •The Matcher Class
- •The appendReplacement() Method
- •The appendTail() Method
- •The end() Method
- •The find() Method
- •The group() Method
- •The groupCount() Method
- •The lookingAt() Method
- •The matches() Method
- •The pattern() Method
- •The replaceAll() Method
- •The replaceFirst() Method
- •The reset() Method
- •The start() Method
- •The PatternSyntaxException Class
- •Using the \d Metacharacter
- •Character Classes
- •The POSIX Character Classes in the java.util.regex Package
- •Unicode Character Classes and Character Blocks
- •Using Escaped Characters
- •Using Methods of the String Class
- •Using the matches() Method
- •Using the replaceFirst() Method
- •Using the replaceAll() Method
- •Using the split() Method
- •Exercises
- •Obtaining and Installing Perl
- •Creating a Simple Perl Program
- •Basics of Perl Regular Expression Usage
- •Using the m// Operator
- •Using Other Regular Expression Delimiters
- •Matching Using Variable Substitution
- •Using the s/// Operator
- •Using s/// with the Global Modifier
- •Using s/// with the Default Variable
- •Using the split Operator
- •Using Quantifiers in Perl
- •Using Positional Metacharacters
- •Captured Groups in Perl
- •Using Back References in Perl
- •Using Alternation
- •Using Character Classes in Perl
- •Using Lookahead
- •Using Lookbehind
- •Escaping Metacharacters
- •A Simple Perl Regex Tester
- •Exercises
- •Index
Visual Basic .NET and Regular Expressions
For each Match object, the position and value of the match are displayed using the Index property and
ToString() method:
Console.WriteLine(“At position {0}, the match ‘{1}’ was found”,
myMatch.Index, myMatch.ToString)
The Match object’s Groups property is assigned to the myGroupCollection variable:
myGroupCollection = myMatch.Groups
The inner For Each loop processes each Group object contained in the GroupCollection object:
For Each myGroup In myGroupCollection
The value and position of each group are displayed using the values of the Group object’s Value and Index properties:
Console.WriteLine(“Group containing ‘{0}’ found at position ‘{1}’.”, myGroup.Value, myGroup.Index)
Next Console.WriteLine()
Next
The first group output contains the group corresponding to the entire regular expression pattern. That group is present in all successful matching processes. Additional groups occur in the GroupCollection when paired parentheses occur in the regular expression pattern. Because the regular expression pattern ([A-Z])(\d+) has two pairs of parentheses, each GroupCollection contains three groups. You can see in Figure 21-8 that three groups are displayed for each match.
The CaptureCollection and Capture Class
The CaptureCollection object contains a collection of one or more Capture objects. Each Capture object represents the content of one capturing group of paired parentheses.
There is no public constructor for an instance of the Capture object. And the Capture object is immutable. It can be created only by a matching process, and each Capture object is part of the
CaptureCollection collection.
Try It Out |
The CaptureCollection Object and the Capture Class |
The code contained in Module1.vb of the CapturesDemo project is shown here:
Imports System.Text.RegularExpressions
Module Module1
Sub Main()
Dim myRegex = New Regex(“([A-Z])+(\d)+”)
Console.WriteLine(“Enter a string on the following line:”)
Dim inputString = Console.ReadLine()
Dim myMatchCollection = myRegex.Matches(inputString)
Console.WriteLine()
499
Chapter 21
Console.WriteLine(“There are {0} matches.”, myMatchCollection.Count) Console.WriteLine()
Dim myMatch As Match
Dim myGroupCollection As GroupCollection Dim myGroup As Group
For Each myMatch In myMatchCollection
Console.WriteLine(“At position {0}, the match ‘{1}’ was found”, myMatch.Index, myMatch.ToString)
Console.WriteLine(“This match has {0} groups.”, myMatch.Groups.Count) myGroupCollection = myMatch.Groups
For Each myGroup In myGroupCollection
Dim myCaptureCollection As CaptureCollection = myGroup.Captures Dim myCapture As Capture
Console.WriteLine(“Group containing ‘{0}’ found at position ‘{1}’.”, myGroup.Value, myGroup.Index)
For Each myCapture In myCaptureCollection
Console.WriteLine(“ |
Capture: ‘{0}’ at position ‘{1}’.”, |
myCapture.Value, myCapture.Index) |
|
Next |
|
Next |
|
Console.WriteLine() |
|
Next
Console.WriteLine()
Console.WriteLine(“Press Return to close this application.”)
Console.ReadLine()
End Sub
End Module
1.Create a new project in Visual Studio 2003 based on a console application template. Name the project CapturesDemo.
2.Edit the default module to match the preceding code. Save the code, and press F5 to run it.
3.In the command window, enter the test string ABC1 A123.
4.Press Return and inspect the results, as shown in Figure 21-9.
Figure 21-9
500
Visual Basic .NET and Regular Expressions
How It Works
The regular expression pattern for this example is importantly but subtly changed from the pattern in the preceding example:
Dim myRegex = New Regex(“([A-Z])+(\d)+”)
Notice that the character class [A-Z] is enclosed in paired parentheses and that the group is qualified by the + quantifier. That means that there is a group that captures a single character. If there is one uppercase alphabetic character in the test string, one group and one capture are created. If there are multiple alphabetic characters, multiple groups and multiple captures are created.
Similar considerations apply to the numeric part of the pattern, because (\d) creates a group that can occur one or more times, depending on the content of the test string. The pattern (\d)+ creates a group for each numeric digit captured, which is different from the pattern (\d+), which creates one group, whether there is one numeric digit or ten.
After accepting the user’s test string, ABC1 A123, the matches, groups, and captures are processed inside three nested For Each loops:
For Each myMatch In myMatchCollection
First, the value of each match is displayed:
Console.WriteLine(“At position {0}, the match ‘{1}’ was found”,
myMatch.Index, myMatch.ToString)
Then the number of groups for that match is displayed:
Console.WriteLine(“This match has {0} groups.”, myMatch.Groups.Count)
The myGroupCollection variable is assigned the Groups property of the myMatch variable:
myGroupCollection = myMatch.Groups
Each Group object in the GroupCollection object is processed next:
For Each myGroup In myGroupCollection
Each capture for the group is assigned to the myCaptureCollection variable:
Dim myCaptureCollection As CaptureCollection = myGroup.Captures
Dim myCapture As Capture
Each group is displayed. You may be surprised to see that only the final occurrence of a group that occurs more than once is displayed:
Console.WriteLine(“Group containing ‘{0}’ found at position
‘{1}’.”, myGroup.Value, myGroup.Index)
501
Chapter 21
Then each capture in the captures collection is displayed. Because each pair of parentheses captures only a single character, a character sequence such as ABC results in three captures being displayed:
For Each myCapture In myCaptureCollection
Console.WriteLine(“ |
Capture: ‘{0}’ at position ‘{1}’.”, |
myCapture.Value, myCapture.Index) |
|
The For Each loops each conclude with a Next statement:
Next
Next
Console.WriteLine()
Next
The RegexOptions Enumeration
The System.Text.RegularExpressions namespace includes a RegexOptions enumeration that controls the modes of operation of regular expression matching.
The following table summarizes the features of the RegexOptions enumeration.
Option |
Description |
|
|
None |
Specifies that no options are set. |
IgnoreCase |
Specifies that matching is case insensitive. |
Multiline |
Treats each line as a separate string for matching purposes. |
|
Therefore, the meaning of the ^ metacharacter is changed |
|
(matches the beginning of each line position), as is the $ |
|
metacharacter (matches the end of each line position). |
ExplicitCapture |
Changes the capturing behavior of parentheses. |
Compiled |
Specifies whether or not the regular expression is compiled |
|
to an assembly. |
SingleLine |
Changes the meaning of the period metacharacter so that it |
|
matches every character. Normally, it matches every charac- |
|
ter except \n. |
IgnorePatternWhitespace |
Interprets unescaped whitespace as not part of the pattern. |
|
Allows comments inline preceded by #. |
RightToLeft |
Specifies that pattern matching proceeds from right to left. |
ECMAScript |
Enables (limited) ECMAScript compatibility. |
CultureInvariant |
Specifies that cultural differences in language are ignored. |
|
|
Case-Insensitive Matching: The IgnoreCase Option
In Visual Basic .NET regular expressions, the default matching mode is case sensitive. To specify that matching be carried out in a case-insensitive way, the IgnoreCase option is used.
502
Visual Basic .NET and Regular Expressions
Try It Out |
Case-Insensitive Matching |
The code in Module1.vb in the IgnoreCaseDemo project is shown here:
Imports System.Text.RegularExpressions
Module Module1
Sub Main()
Dim myPattern As String = “[A-Z]+\d+” Console.WriteLine(“Enter a string on the following line:”) Dim inputString = Console.ReadLine()
Dim myMatchCollection = Regex.Matches(inputString, myPattern) Console.WriteLine(“This is case sensitive matching.”) Console.WriteLine(“There are {0} matches.”, myMatchCollection.Count) Console.WriteLine()
Dim myMatch As Match
For Each myMatch In myMatchCollection
Console.WriteLine(“At position {0}, the match ‘{1}’ was found”, myMatch.Index, myMatch.ToString)
Next Console.WriteLine()
myMatchCollection = Regex.Matches(inputString, myPattern, RegexOptions.IgnoreCase)
Console.WriteLine(“This is case insensitive matching.”) Console.WriteLine(“There are {0} matches.”, myMatchCollection.Count) Console.WriteLine()
For Each myMatch In myMatchCollection
Console.WriteLine(“At position {0}, the match ‘{1}’ was found”, myMatch.Index, myMatch.ToString)
Next Console.WriteLine()
Console.WriteLine(“Press Return to close this application.”) Console.ReadLine()
End Sub
End Module
1.Create a new project in Visual Studio 2003 using the console application template. Name the project CaseInsensitiveDemo.
2.Edit the code so that it reads the same as the preceding Module1.vb.
3.Save the code; then run it using the F5 key.
4.In the command window, enter the test text ABC123 abc123 DeF234.
5.Press the Return key, and inspect the results, as shown in Figure 21-10. Notice that when matching is case sensitive, there are two matches, and there are three matches when matching is case insensitive. Notice, too, that the case-sensitive match against DeF234 is F234, while the caseinsensitive match is DeF234.
503
Chapter 21
Figure 21-10
How It Works
The pattern assigned to the myPattern string (notice that it isn’t a Regex object) is [A-Z]+\d+. This, using the default case-sensitive matching in .NET, would match only when the user entered uppercase alphabetic characters:
Dim myPattern As String = “[A-Z]+\d+”
The MatchCollection object corresponding to the myMatchCollection variable is created using the shared Matches() method of the Regex class. No Regex object is instantiated. Notice that the Matches() method takes two arguments on this occasion (when case-sensitive matching is applied), the second argument being the string value containing the regular expression pattern:
Dim myMatchCollection = Regex.Matches(inputString, myPattern)
The character sequence ABC123 matches because only uppercase alphabetic characters are contained in the character sequence. The character sequence abc123, by the same measure, does not match. In the character sequence DeF234, the character sequence F234 matches because it contains one uppercase character followed by three numeric digits.
After the results of case-sensitive matching have been displayed, a new collection of Match objects is assigned to the myMatchCollection variable. Notice that on this occasion, the Matches() method takes three arguments. The third argument specifies one of the properties of the RegexOptions object, in this case, the IgnoreCase property:
myMatchCollection = Regex.Matches(inputString, myPattern,
RegexOptions.IgnoreCase)
When matching is case insensitive, the three character sequences, ABC123, abc123, and DeF234 all match the pattern [A-Z]+\d+.
504