In the last SQL post, I showed you SUBSTRING, and how you could use it to locate the area code in an un-formatted phone number.  The problem was the field we were working from was full of malformed numbers.  Today, we’ll start on cleaning up the numbers.

But first, I need to include my standard disclaimer:

I would like to point out that you have to be careful when and how often you use the techniques below.  A good rule to keep in mind is text manipulation is slow and painful to a server.  If you can leave the text manipulation to your middle-ware or front end, that would be better.  But we all have been stuck in a situation where we needed to alter a string before those two points, and so I bring you the following lesson.

And now back to our regularly scheduled post.

The Problem

phone

605-555-2862 (561)555-2700 904-555-5680 N/A 580-555-5371 2815558368 (254)555-8430 336-555-2797 3365557233 592-555-3181/4951 96615551222 Ext. 249 +44 7930 555271

Looking back at our column, we can see there are a lot of different ways the phone number can be formed.  Before we begin trying to pull out the area codes, we need to clean the data.  In this situation, I usually go for sanitizing the column of anything that is not part of a phone number.  Given the phone number itself is just numbers; we want to remove anything that is not a number from this field.

There are two methods to accomplish this, the first is using the REPLACE function on any character (or string of characters) that is not a number.  Using only the REPLACE function requires you to know all the variations of data you wish to remove.  Since you can’t ever know everything you don’t know (I keep on trying though!), in the next post I’ll show you a little regular expression technique.

The Solution

Let’s first look at the syntax for the REPLACE function.

REPLACE (searchString , findString , replaceWithString)

Using this function you can begin sanitizing the phone column.  Since we are going to be altering the source, I would advocate making a backup of the source to either a temp table, or make a full backup of the table before beginning.  Also,  when altering data, I am a big advocate of the following steps:

1. BEGIN TRANSACTION
2. SELECT the update you’re about to make
3. Make the UPDATE
4. SELECT the updated data and verify the results
5. If the results are correct:  COMMIT TRANSACTION
6. If the results are incorrect: ROLLBACK TRANSACTION

--BEGIN TRANSACTION
--use replace to remove spaces
SELECT TOP 100
   phone, REPLACE(phone, ‘ ‘, ‘’) AS [area code]
FROM sourceTable

UPDATE sourceTable SET
   Phone = REPLACE(phone, ‘ ‘, ‘’)
--ROLLBACK TRANSACTION
--COMMIT TRANSACTION

Remember, you will have to select just the “BEGIN TRANSACTION “, “COMMIT TRANSACTION”, or “ROLLBACK TRANSACTION” as needed.  I leave them commented back, so it doesn’t auto-rollback after each run.

That should remove spaces from the phone number; you can repeat those steps with variations to remove other bad data from phone.  Try the following to remove some more bad data from the phone field.

• REPLACE(phone, ‘+‘, ‘’)
• REPLACE(phone, ‘n/a‘, ‘’)
• REPLACE(phone, ‘-‘, ‘’)
• REPLACE(phone, ‘ext.‘, ‘’)

To proceed on this problem you’ll need a few more tools in your SQL tool belt.  Check out other articles I’ve published on string manipulation.  There you will discover how LTRIM, RTRIM, and more techniques for manipulating text strings can help solve this problem.

References

http://msdn.microsoft.com/en-us/library/ms186862.aspx

Syndication Sites

Experts Exchange