Hidden Gem TRY_CONVERT to troubleshoot data type conversion

Many of us have experienced a dreaded data type conversion error when doing tasks like importing files, converting to new database schemas, etc. Which record caused the problem? I have used a few tricks in the past for this, but TRY_CONVERT makes it easy. This was added in SQL 2012, but recently I have recommended it a couple times to some folks that did not know it existed.

Let’s just jump right into a demo to see how it works.

I created a database and pulled some data from good ol’ AdventureWorks to create a demo. Notice that I converted the ModifiedDate column to nvarchar. It is not uncommon that you may import raw data from a file or another RDBMS into a stage table with very generic data types just to get the raw data imported to work with.

USE master
GO

CREATE DATABASE TryConvert
GO

USE TryConvert
GO

-- pull some person data from Adventureworks
SELECT [BusinessEntityID]
      ,[PersonType]
      ,[Title]
      ,[FirstName]
      ,[MiddleName]
      ,[LastName]
	  ,CONVERT(nvarchar(20),[ModifiedDate],101) AS ModifiedDate
INTO TryConvert.dbo.Person
FROM Adventureworks.person.person;

--view the data
SELECT * FROM PERSON

Next, I updated a couple records to create data that I know will not convert to a datetime. I turned one record to a date with no year and another to test. Have you ever gotten a csv where there is a comma within a value? I sure as heck have… and it moves everything into the wrong column. Sometimes you may get a flat file from a 3^rd party or business partner, and they may add/remove a field.

--Lets update 1 date to have a missing year...or imagine you got the data
--as a csv yet there was a comma in a name which put alpha characters in the field.
UPDATE Person set ModifiedDate='03/18' WHERE BusinessEntityID=19253
UPDATE Person set ModifiedDate='NO DATE HERE' WHERE BusinessEntityID=19173

SELECT * FROM Person WHERE BusinessEntityID in (19253, 19173)

I then created a table which is where I would put the transformed data. You can see that I have the ModifiedDate as a datetime data type here.

-- create a new table for the date
CREATE TABLE [dbo].[Person_ConvertDateTime](
	[BusinessEntityID] int NOT NULL,
	[PersonType] nchar(2) NOT NULL,
	[Title] nvarchar(8) NULL,
	[FirstName] nvarchar(50) NOT NULL,
	[MiddleName] nvarchar(20) NULL,
	[LastName] nvarchar(50) NOT NULL,
	[ModifiedDate] datetime NOT NULL
)
ON [PRIMARY];

I tried to do a simple insert select into the table and got a “conversion failed” error.

--this will fail
INSERT INTO Person_ConvertDateTime
SELECT * FROM Person;

Here is where TRY_CONVERT can help you pinpoint which row has the bad data

--This will show you the bad records
SELECT *, TRY_CONVERT(datetime,ModifiedDate) FROM Person
WHERE TRY_CONVERT(datetime,ModifiedDate) is NULL

One gotcha with using TRY_CONVERT will not help you with string truncation errors. This is because a convert from a longer to a shorter varchar/nvarchar does not throw an error, so neither will TRY_CONVERT. The data will simply be truncated. I feel that this may be good thing as your business rules may allow for the data to be truncated, or you may want it to throw an error. See the below example:

--A GOTCHA is string truncation

-- create a new table limiting the length to 20
CREATE TABLE [dbo].[Person_ConvertString](
	[BusinessEntityID] int NOT NULL,
	[PersonType] nchar(2) NOT NULL,
	[Title] nvarchar(8) NULL,
	[FirstName] nvarchar(20) NOT NULL,
	[MiddleName] nvarchar(20) NULL,
	[LastName] nvarchar(20) NOT NULL,
	[ModifiedDate] nvarchar(20) NOT NULL
)
ON [PRIMARY];


--This will fail as some names are more than 20 characters
INSERT INTO Person_ConvertString
SELECT * FROM Person;

--Why does this not show us any bad rows?
SELECT LastName, TRY_CONVERT(nvarchar(5),LastName) FROM Person
WHERE TRY_CONVERT(nvarchar(5),LastName) is NULL

The last statement with the TRY_CONVERT will produce no results as all the rows will CONVERT. In this case I would suggest doing a CONVERT in your SELECT to naturally truncate the data or validate the data with LEN before inserting it into the destination table. You can see below the raw data, the TRY_CONVERT output and the CONVERT output:

--Since just doing a convert will truncate a string without error
--it will not capture string truncation errors

--You can see the conversion here
SELECT LastName, TRY_CONVERT(nvarchar(5),LastName) as TryConvLastName,
convert(nvarchar(5),LastName) as ConvertedLastName FROM Person

The data will simply be truncated. I feel that this may be good thing as your business rules may allow for the data to be truncated, or you may want it to throw an error.

Below is the full SQL script for you to try:

USE master
GO

CREATE DATABASE TryConvert
GO

USE TryConvert
GO

-- pull some person data from Adventureworks
SELECT [BusinessEntityID]
      ,[PersonType]
      ,[Title]
      ,[FirstName]
      ,[MiddleName]
      ,[LastName]
	  ,CONVERT(nvarchar(20),[ModifiedDate],101) AS ModifiedDate
INTO TryConvert.dbo.Person
FROM Adventureworks.person.person;

--view the data
SELECT * FROM PERSON

--Lets update 1 date to have a missing year...or imagine you got the data
--as a csv yet there was a comma in a name which put alpha characters in the field.
UPDATE Person set ModifiedDate='03/18' WHERE BusinessEntityID=19253
UPDATE Person set ModifiedDate='NO DATE HERE' WHERE BusinessEntityID=19173

SELECT * FROM Person WHERE BusinessEntityID in (19253, 19173)

-- create a new table for the date
CREATE TABLE [dbo].[Person_ConvertDateTime](
	[BusinessEntityID] int NOT NULL,
	[PersonType] nchar(2) NOT NULL,
	[Title] nvarchar(8) NULL,
	[FirstName] nvarchar(50) NOT NULL,
	[MiddleName] nvarchar(20) NULL,
	[LastName] nvarchar(50) NOT NULL,
	[ModifiedDate] datetime NOT NULL
)
ON [PRIMARY];


--this will fail
INSERT INTO Person_ConvertDateTime
SELECT * FROM Person;

--This will show you the bad records
SELECT *, TRY_CONVERT(datetime,ModifiedDate) FROM Person
WHERE TRY_CONVERT(datetime,ModifiedDate) is NULL



--A GOTCHA is string truncation

-- create a new table limiting the length to 20
CREATE TABLE [dbo].[Person_ConvertString](
	[BusinessEntityID] int NOT NULL,
	[PersonType] nchar(2) NOT NULL,
	[Title] nvarchar(8) NULL,
	[FirstName] nvarchar(20) NOT NULL,
	[MiddleName] nvarchar(20) NULL,
	[LastName] nvarchar(20) NOT NULL,
	[ModifiedDate] nvarchar(20) NOT NULL
)
ON [PRIMARY];


--This will fail as some names are more than 20 characters
INSERT INTO Person_ConvertString
SELECT * FROM Person;


--Why does this not show us any bad rows?
SELECT LastName, TRY_CONVERT(nvarchar(5),LastName) FROM Person
WHERE TRY_CONVERT(nvarchar(5),LastName) is NULL
--Since just doing a convert will truncate a string without error
--it will not capture string truncation errors

--You can see the conversion here
SELECT LastName, TRY_CONVERT(nvarchar(5),LastName) as TryConvLastName,
convert(nvarchar(5),LastName) as ConvertedLastName FROM Person