Introduction to regular expressions

 Note: This is part 1 of a series of articles:

• Part 2 - Practical Parsing Using Groups in Regular Expressions

What will be covered

In this article I’ll demonstrate the use of the following:

·        Using the System.Text.RegularExpressions Namespace classes for text parsing

·        Using System.IO Namespace classes to read text files

Tools Used

·        Expresso – A tool for testing Regular Expressions

Introduction to regular expressions

Regular expressions (a.k.a “Regex”) are a very powerful text parsing language used widely in many parts of the technology industry.

Their main use is to find a particular pattern within a given string that matches whatever rules were expressed using this language.

In software applications, we will usually test whether the Regex Found a match for a pattern with the rules we defined, and based on the success of the search , perform certain actions such as validate an email address or finding HTTP link in a web page.

The Regex language can be used for both simple searches as well as for more complicated patterns such as finding all words in a given string that appear more then once within all the given text. Another use for Regex is to replace the string matching the pattern with another string (for example, making all lowercase words Uppercase, or replacing all replacing all “Five” to “5”).

A simple example would be to find a match for a pattern fitting an email address.

Let’s take a look at how an email address is built. Taking Royo@SomeWhere.com as an example:

We know there is a pattern here, since we know there will always be at least 5 parts:

-         A string

-         A “@” character

-         A string

-         A “.”

-         Another String

If you were to receive a string and determine whether it is a legitimate email address you would have several options:

·        Parse the string using cumbersome code for the existence of specific characters (which we mentioned above)

o       You need to check for hardcoded characters in your given string using String.IndexOf() and that sort of thing

o       You need to code lots of lines of code to perform complicated pattern searching (email pattern is very simple yet would take a pretty complicated “If” structure to apply correctly

·        Use the Regex Class found in the System.Text.RegularExpressions Namespace to check for an email pattern match

o        2 lines of code

A practical example to try

Installing and using Expresso

In order to practice the following example, you will need to download and install Expresso – A regular expression testing tool and much more. After you have installed it fire it up.

The main screen is divided into 3 main sections :

- The Regular expression input pane(upper part)

            This is where we try out different expressions and test for the parsing outcome in the output pane

- The Data Input Pane(Lower Left)

            The data to be parsed and tested. This is where we enter our sample data.

- The Output Pane (Lower right)

            The outcome of the Regex parsing engine.

Expresso starts up with a predefined sample expression already inside its expression pane.

You can tell by the "# Dates" inside  that this expression deals with parsing for dates inside text.

It already has some sample data filled in as well. To see what the parsing results look like, you can just click "Find Matches" button and see what happens.

The output pane is filled with all the strings that matched the particular pattern in the expression pane.

Notice that the results are also hierarchical, and can be expanded. This is another feature of Regex , which allows you to express groups to be captured and referred to by name, within an expression. But enough with this sample, since it is a bit to complicated for starters.

Let's do an easier sample, and I'll explain from the start.

Say we have some lines of text that we need to parse and retrieve email addresses from.

[Paste the following into the data pane in expresso]

This is a piece of text. It represents some email addresses.

For example royo@somewhere.com is an email address. Another one could be

Perhaps just junk@address.co.au or maybe it's @234.34.23.com.

For conclusion, here are some more junk email addresses. @. something@.

@somewhere. @.com something@.com @somwehre.com

This is the end of the text to be pasted into expresso.

Here’s a pattern to find a match for an email address using regular expressions :

\w*@\w*\.\w*((\.\w*)*)?

Like almost anything in languages, this can be expressed in more then one way, since the language itself contains many possibilities,

but let’s dissect this expression:

“\w “– An escape character meaning “Characters usually found in words”.

This means alpha numeric characters, but excludes spaces, punctuation marks and tabs.

Basically, anything that could be considered as being a non-word character is excluded.

“*” – The asterisk sign means “0 or more instances of” and operates on the expression to the left of it.

Thus, “\w*” means “match zero or more occurrences of any word-like character”

“@” – Match one instance of the character “@”

“\w*” - “match zero or more occurrences of any word-like character”

“\.” – Match one instance of the dot character (this requires an escape slash since “.” Is a reserved operator in the regex language)

“\w*” - “match zero or more occurrences of any word-like character”

“()“ – Groups what is inside the round brackets into a logical group

(\.\w*)* - Match zero or more instances of a dot followed by a zero or more instances of any word-like character

()? – optional group – Match this if it appears in the text

According to the last 3 explanations here’s an overall explanation to the last part of the expression:

((\.\w*)*)? – means there might be zero or more instance of a dot followed by a word (“.something.anotherthing.bla” would be matched)

To conclude, here’s the description of this expression in whole:

Match any word followed by an asterisk , followed by any word followed by a dot ,followed by any word. Followed by zero or more instances of a dot and a word after it.

Paste the pattern mentioned above into the expression pane of expresso. Once you've done that, click the "find matches" button.

You can see in the output pane, that the Regex engine recovered only part of the data that appears in the data pane. It returned only strings which match the expression that was given.

Can you see a problem yet?

Can you find a logical problem with this expression?

These strings were matched as well, although they are not legitimate email addresses.

@.

something@.

@somewhere.

@.com

something@.com

@somwehre.com

The problem is we tested for zero or more instances of a character, so this string was accepted as well. We needed to specify “1 or more instances of” instead.

This is easily expressed using the “+” operator instead of the "*" operator.

Try to fix the expression yourself and hit "Find Matches again"..

Here’s the fixed expression to catch only valid instances

\w+@\w+\.\w+((\.\w+)*)?

This will catch only the two legitimate email addresses.

As you can see, It's pretty easy to work with regular expressions, once you know the syntax.

A table of syntax for Regex can be found here.

Code Demo

Here's the source code listing for a simple form , which reads a text file name "data.txt" found in it's directory.

Data.txt contains the same text as mentioned above , which you have put in the "data" pane of expresso.

The form reads the file, then uses the Regex class to get a MatchCollection object, which is a collection of Match Objects.

Each Match Object contains the value of the string that was parsed.

using System;

using System.Drawing;

using System.Collections;

using System.ComponentModel;

using System.Windows.Forms;

using System.Data;

//we add the following import staements

using System.Text.RegularExpressions;

using System.IO;

using System.Text;

.

.

.

            //This is the method we will call on form load
    

            private void ParseData()

            {

                  listBox1.Items.Clear();

                  //Read the data.txt file to parse for emails

                  string data = GetDataFromFile();

                  //the pattern used to parse for emails

                  //notice the '@' at the start to avoid escape special

//characters treated as escape sequences

                  string expression = @"\w+@\w+\.\w+((\.\w+)*)?";

                  //get a collection of successfull matches

                  //from Regex using the specified pattern on the input data

                  MatchCollection matches = Regex.Matches(data,expression);

                  //Iterate through the matches, adding them to the list box

                  foreach(Match SingleMatch in matches)

                  {

                        listBox1.Items.Add(SingleMatch.Value);

                  }

            }

            private string GetDataFromFile()

            {

                  //Use the Path Class to generate FIle paths from a given folder and a file name

                  string filename = Path.Combine(Application.StartupPath,"data.txt");

                  //open a text file for reading

                  StreamReader reader = File.OpenText(filename);

                  //get all the text in the file

                  string ret = reader.ReadToEnd();

                  //dismiss the file handle

                  reader.Close();

                  //return the text in the file

                  return ret;

Conclusion

Using Regular expressions and the Regex object is very simple, and very powerful. Once you learn it, you'll use it all over the place – In search dialogs, Coding and sometimes even search engines. In future articles I will demonstrate more elaborate uses of regular expressions, and how they fit in an overall practical solution.

Appendix A

Here's a simple table displaying the syntax of a regular expression and any escape characters that can be used.

(You can find more on the syntax of Regex here.)

A regular expression is a pattern of text that consists of ordinary characters (for example, letters a through z) and special characters, known as metacharacters. The pattern describes one or more strings to match when searching a body of text. The regular expression serves as a template for matching a character pattern to the string being searched.

The following table contains the complete list of metacharacters and their behavior in the context of regular expressions: