The concept of database containment basically means that all instance level dependencies are severed from the database.  If you have ever implemented log shipping or mirroring, you are probably aware of many of these.  Instance level logins, linked servers, SQL Agent jobs, SSIS packages, Tempdb collation, and even other databases often need to be manually copied and synchronized between instances when a database is being log shipped, mirrored, or part of an Availability Group.

The Database containment feature puts all of these items within the database itself.  This way when you copy a database from one instance to another, you can be sure you moved everything.  The end goal is to fully separate database management from application functions.

For SQL 2012, Partial Containment was implemented to resolve two of the most common dependencies, logins and collation.  This must be enabled at the instance level first, so that database owners cannot simply enable containment without the knowledge of the database administrator. It is a simple sp_configure command as follows:

sp_configure'contained database authentication', 1

Once contained database authentication is enabled, you can then set the Containment Type to partial in the database options tab in Management Studio.

It can also be done in an ALTER or CREATE DATABASE statement as follows:  

ALTERDATABASE<name> SETCONTAINMENT=PARTIAL    

Contained Logins:

Now that you have a partially contained database, you can create contained users that authenticate within the database. For SQL authentication, the authentication is at the database level instead of the instance level. The user is not mapped to a login and the password hash is stored within the user database,not master.  For users logging in to contained databases, authentication is first tried at the database level and then at the instance level if there is no contained user with that name. On the other hand, Windows users look relatively similar to before, but they have no matching login.  Authentication for Windows users tries at the instance level first, and then at the user level within the database.  You need to consider the order of authentication when you will have contained and non-contained usage of a user in multiple databases on an instance.  Here you can see what a contained user looks like in Management Studio:

There are some other considerations to take into account when using contained logins.  Setting AUTO CLOSE on will not allow the contained users to connect when there are no other connections tot he database as the database will be closed.  This can cause a denial of service type effect, so it is definitely recommended not to use AUTO CLOSE. Also, granting the ALTER ANY USER privilege at the database level allows users to be added. Since typically a login would need to be added first, it is not considered a huge security concern.  When the database is contained, then it is the equivalent of adding a new login, so in this case it is more of a security concern. Note that you can use the sp_migrate_user_to_contained stored procedure to migrate traditional database users to contained database users.

My next post we will explore contained databases and collation.  In the meantime, use the following script to enable containment, create databases, and create/migrate users. This will help you explore partial database containment on a test system.

--enable contained dbs

EXECsys.sp_configureN'contained database authentication',1

GO

RECONFIGURE     

GO    

--Create the 3 databases, all have different collation from the instance collation

--- Some of these will be used in the subsequent post.

CREATEDATABASE[MyDB]--not contained     

COLLATELatin1_General_CI_AS_KS_WS     

GO     

CREATEDATABASE[MyContainedDB] --partially contained     

CONTAINMENT=PARTIAL     

COLLATELatin1_General_CI_AS_KS_WS     

 GO     

 CREATEDATABASE[MyContainedDBToo]--partially contained to illustrate multiple collations     

 CONTAINMENT=PARTIAL     

 COLLATELatin1_General_CS_AS_KS     

 GO     

--Create a non-contained Login mapped to a user

USE[master]     

GO    

CREATELOGIN[TestSQLAccount]WITHPASSWORD=N'iL0V3SQL!',DEFAULT_DATABASE=[MyContainedDB]     

GO    

USE[MyContainedDB]     

GO    

CREATEUSER[TestSQLAccount]FORLOGIN[TestSQLAccount]     

GO    

--View  the TestSQLAccount User in Management studio under the MyContainedDB to see login affiliation

--Convert this to a contained user    

USE[MyContainedDB]     

GO    

EXECUTEsp_migrate_user_to_contained@username=N'TestSQLAccount',  @rename=N'keep_name',@disablelogin=N'disable_login';     

GO    

--Look in Management studio under general tab - you no longer see login affiliation and you see password info    

--If you want to log in as the contained user, you must specify the database name in the connection string.You can use the sample cmd below to log in as the contained user for testing purposes    

--sqlcmd -S <put your instance name here> -U TestSQLAccount -P iL0V3SQL! -d MyContainedDB    

-- Create a contained SQL user without login    

USE[MyContainedDB]     

GO    

CREATEUSERMyContainedUser     

WITHPASSWORD='iL0V3SQL!';     

GO    

Note: Here are some Limitations of partially contained databases:

Lisa Gardner - SQL Premier Field Engineer

In my last post, I went over partial database containment and contained users.  This post will focus on the other piece of functionality in partial containment… collation.  The collation of data determines how data is sorted and compared.  When all databases are using the same collation as the instance collation (selected during install), then there is really no need for containment of the collation.  The problem arises when you are hosting multiple application databases on your instance that require a separate collation.  The most typical example of this is when objects are created in tempdb.  They will have the instance default collation.  If the collations between the two are different then you may see an error similar to this one below:

Msg 468, Level 16, State 9, Line 4

Cannot resolve the collation conflict between "SQL_Latin1_General_CP1_CI_AS" and "Latin1_General_CI_AS_KS_WS" in the equal to operation.

In many cases, this can be resolved easily by using COLLATE to define the collation of the column to prevent this issue.  But what happens when you have a vendor application that will not allow you to change their code?  This is where having a contained collation will come into play. 

This problem was resolved by creating the catalog collation.  This collation is for system metadata and transient objects.  This means that any temporary metadata, variables, etc. will use the catalog default collation and not the database collation. The catalog collation is Latin1_General_100_CI_AS_WS_KS_SC,   and it is the same for all contained databases in the instance.  This does NOT mean that all contained databases on an instance must have the same collation,this is just the catalog collation that is predefined. The following chart will help illustrate the difference:

Many of us don’t deal with collation issues on a day to day basis, so I am including some scripts that will help you explore contained collation a little more.  I included the entire script in one piece at the bottom of the post for ease of copy/paste.

Run this first section to enable containment and create the databases (note that if you still have the 3 databases created from part 1 then you can skip over that part here):

--enable contained dbs

EXECsys.sp_configureN'contained database authentication',1

GO

RECONFIGURE

GO

--Create the 3 dbs, all have different collation from the instance collation

CREATEDATABASE[MyDB]--not contained

 COLLATELatin1_General_CI_AS_KS_WS

 GO

CREATEDATABASE[MyContainedDB] --partially contained

 CONTAINMENT=PARTIAL

 COLLATELatin1_General_CI_AS_KS_WS

 GO

 CREATEDATABASE[MyContainedDBToo]--partially contained to illustrate multiple collations

 CONTAINMENT=PARTIAL

 COLLATELatin1_General_CS_AS_KS

 GO

In the first database, there is no containment.

USEMyDB

GO

--Create Static Table

CREATETABLEMyTable

      (mycolumn1nvarchar,

      mycolumn2nvarchar);

GO

--Show column and server collation difference

SELECTname,collation_name

FROMsys.columns

WHEREnameLIKE'mycolumn%';

GO

selectSERVERPROPERTY('collation')

From the 2 above statements, you should see the following result set to show you that the columns in the table have a different collation than the instance:

Then create a temp table and join to it from MyTable, you will get a collation conflict.

CREATETABLE#MyTempTable   

(mycolumn1nvarchar,

      mycolumn2nvarchar);

GO

SELECTT1.mycolumn1,T2.mycolumn1

FROMMyTableT1

JOIN#MyTempTableT2

    ONT1.mycolumn1=T2.mycolumn1

Msg 468, Level 16, State 9, Line 10

Cannot resolve the collation conflict between "SQL_Latin1_General_CP1_CI_AS" and "Latin1_General_CI_AS_KS_WS" in the equal to operation.

Now drop the temp table and rerun the select with a collation conversion of the columns.  This works... until you find out that this is a vendor application and the mention the dreaded “unsupported” word.

--drop temp table

DROPTABLE#MyTempTable

--Now define the temp table with the db collation

--there will be no error when we select

CREATETABLE#MyTempTable   

(mycolumn1nvarcharCOLLATELatin1_General_CI_AS_KS_WS,

      mycolumn2nvarchar  COLLATELatin1_General_CI_AS_KS_WS);

SELECTT1.mycolumn1,T2.mycolumn1

FROMMyTableT1

JOIN#MyTempTableT2

    ONT1.mycolumn1=T2.mycolumn1

Database containment to the rescue! Here you will switch to a contained database.

--switch dbs

USEMyContainedDB

GO

--Create static table

CREATETABLEMyTable

      (mycolumn1nvarchar,

      mycolumn2nvarchar);

GO

--Show column and server collation difference

SELECTname,collation_name

FROMsys.columns

WHEREnameLIKE'mycolumn%';

GO

selectSERVERPROPERTY('collation')

Note here that your result set looks identical to the non-contained db.  That is OK.  Remember that contained databases use the catalog collation for temporary metadata.

Here is where we had an error before, but this time it will work!

--drop temp table

DROPTABLE#MyTempTable

--Create the temp table

--Temp objects use the CATALOG COLLATION for tempdb

--since this db is partially contained

CREATETABLE#MyTempTable   

(mycolumn1nvarchar,

      mycolumn2nvarchar);

GO

SELECTT1.mycolumn1,T2.mycolumn1

FROMMyTableT1

JOIN#MyTempTableT2

    ONT1.mycolumn1=T2.mycolumn1

I also included a separate database with a third collation just to show that you can have a number of databases with a number of collations.

--switch dbs

USEMyContainedDBToo

GO

--Create static table

CREATETABLEMyTable

      (mycolumn1nvarchar,

      mycolumn2nvarchar);

GO

--Show column and server collation difference

SELECTname,collation_name

FROMsys.columns

WHEREnameLIKE'mycolumn%';

GO

selectSERVERPROPERTY('collation')

--drop temp table

DROPTABLE#MyTempTable

--Create the temp table

--Temp objects use the CATALOG COLLATION for tempdb

--since this db is partially contained

CREATETABLE#MyTempTable   

(mycolumn1nvarchar,

      mycolumn2nvarchar);

GO

SELECTT1.mycolumn1,T2.mycolumn1

FROMMyTableT1

JOIN#MyTempTableT2

    ONT1.mycolumn1=T2.mycolumn1

Now you have an understanding of what contained databases can do.  Perhaps you have some candidates in your environment for database containment.

Lisa Gardner– SQL Premier Field Engineer

In this blog post I’ll (re)introduce you to a really neat tool to use alongside of SQL Server and use this tool to show you how SQL Server is handling IO under the covers.  We will view the IOs as they occur, and then tie the IO back to the pages that we pull into the buffer pool. 

First, I’ll create a table named NumbersTable that I’ll use for the subsequent demos, which I’ll create in a database named NumbersDB.  I’ll then load 500K worth of records into it.

CREATETABLENumbersTable

(

     NumberValueBIGINTNOTNULL

)

GO

INSERTINTONumbersTable (NumberValue)

SELECTTOP 500000NumberValue

FROM

(     SELECT NumberValue=row_number()over(orderbynewid()asc)

     FROMmaster..spt_valuesa

     CROSSAPPLYmaster..spt_valuesb

     WHEREa.type='P'ANDa.number<= 710 ANDa.number> 0 AND

     b.type='P'ANDb.number<= 710 ANDb.number> 0

)a

ORDERBYNumberValueASC

GO

Next, make sure is a clustered index on the table and that it isn’t fragmented after the rebuild.  I use MAXDOP to ensure that the operation is single threaded, which removes any opportunity for fragmentation.  Sometimes when a rebuild operation uses multiple worker threads the index can end up very slightly fragmented.  This removes that possibility.

CREATECLUSTEREDINDEXixNumbersTableONNumbersTable(NumberValue)

WITH(MAXDOP=1)

GO

Next, to setup for the following demo, run the following to clear out the buffer pool and flush anything necessary to the log file.  This will ensure that when I query the table later I’ll need to go out to disk to read the data pages.

DBCCDROPCLEANBUFFERS
CHECKPOINT
DBCCFREEPROCCACHE
GO

OK, now for the really cool stuff.  If you don’t have it downloaded already, navigate to http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx and download Process Monitor.  This is a great tool that allows you to view real-time process & thread activity – including SQL Server activity!  Once you download the file click on the Filter menu and enter the following filters.

Run the following query and watch the activity come through in Process Explorer.  Because we just cleared out all pages from the buffer pool, we will have to go out to the data file to pull data back in.  We can see these IOs as they occur through Process Monitor.  We care mainly about the Length portion of the Detail column for our purposes today.

 SELECTCOUNT(*)FROMNumbersTable

We can see that in most cases, the IO requests above are in 512KB blocks.  This is read-ahead at work.  The read-ahead mechanism allows the Database Engine to read up to 64 contiguous pages (512KB) from one file in a single IO. The read is performed as a single scatter-gather read (see this blog post for more information on IO basics - http://blogs.msdn.com/b/sqlblog/archive/2006/09/20/whitepaper-sql-server-i-o-basics.aspx)  to the appropriate number of (probably non-contiguous) buffers in the buffer cache. If any of the pages in the range are already present in the buffer cache, the corresponding page from the read will be discarded when the read completes. The range of pages may also be "trimmed" from either end if the corresponding pages are already present in the cache.  This makes the scanning of data very efficient in cases where the data on disk is not fragmented.  For more information on how fragmentation affects scan performance in SQL Server 2012, check out my post here:  http://blogs.msdn.com/b/timchapman/archive/2012/12/03/visualizing-index-fragmenation.aspx.

You’ll also notice that the smallest IO is 64K in size.  In this case we are reading extents because my buffer pool hadn’t ramped up quite yet. During the Ramp up phase which is before the buffer pool committed memory reaches the target memory, in order to expedite the ram up process every single page read is aligned into 8 page read requests.  If my buffer pool had been ramped up, then a single page ready would be an 8K IO.  We don’t always perform IOs on data files in 64K increments.  Log files are a bit different in that we will write very smaller units of work to the log file – as small as 512 bytes in size (up to a max of 60K). 

At this point, if you’ve loaded up the public symbols into Process Monitor (http://blogs.msdn.com/b/vijaysk/archive/2009/04/02/getting-better-stack-traces-in-process-monitor-process-explorer.aspx ), then you could view the functions calls by right clicking one of the ReadFile operations.  I’m not going to show this here because I’ve got the private symbols loaded up, but it is something to play with if you really want to geek out.

If I fire up Excel, I can post the values from the Process Monitor capture and do some quick math to see how many pages were actually read into SQL Server.  From my calculation, I can see that my scan query read in 1067 pages. 

Another way to view this same information is through Extended Events  (http://msdn.microsoft.com/en-us/library/bb630354(v=sql.105).aspx ) .  In 2012, the file_read_completed event includes the size field, which equates to (as you can guess) the size of the IO.  If I’ve done everything correctly, this should relate to the IOs that we captured in Process Monitor above.  Notice that I am using a histogram (previously called a Bucketizer in 2008) target, which will allow me to aggregate data based on fields (or actions) retrieved by the event.  This is a bit different than the event_counter target, which only counts occurrences of an event.

-- replace the Database ID by the appropriate DBID

CREATEEVENTSESSION[Reads]ONSERVER

ADDEVENTsqlserver.file_read_completed(

    WHERE ([database_id]=(8)))

ADDTARGETpackage0.histogram(SETfiltering_event_name=N'sqlserver.file_read_completed',

source=N'size',source_type=(0))

WITH

( MAX_MEMORY=4096 KB,EVENT_RETENTION_MODE=ALLOW_SINGLE_EVENT_LOSS,

  MAX_DISPATCH_LATENCY=30 SECONDS,

  MAX_EVENT_SIZE=0 KB,MEMORY_PARTITION_MODE=NONE,TRACK_CAUSALITY=OFF,

  STARTUP_STATE=ON

)

GO

And, here is what it looks like to configure the event through the fancy new Extended Events UI in SSMS 2012.  Notice that for an apples-to-apples comparison, I have to set the database_id value as a predicate.  If I didn’t, it would capture file_read_completed events for all databases. I set a similar predicate using a filter to the file path in Process Monitor.

Looking at the number of pages captured in my Extended Event, after I do some quick math I can see that my xEvent caught the same number of page reads as my Process Monitor capture did. Awesome!

So, let’s do some investigation to see what pages were read in.  First, I’ll somewhat redo my test and clear out my buffer pool.  I’ll then query sys.dm_os_buffer_descriptors to see what pages are immediately read back into the BP (this is done for misc. engine purposes). 

DBCCDROPCLEANBUFFERS
CHECKPOINT
DBCCFREEPROCCACHE

SELECTpage_type,count(*)asBPPageCount
FROMsys.dm_os_buffer_descriptorsd
WHEREdatabase_id=DB_ID()
GROUP BYpage_type

Cool, looks like we have around 9 pages.  Now, let’s scan the table again to see how many we read into the buffer pool.

SELECTCOUNT(*)FROMNumbersTable

If we compare the before and after page counts, we can see that the difference in pages is 1067 – which is the exact number we calculated from the IO reads from Process Monitor. Really cool!

Hope you think this is as cool as I do! 

Tim Chapman

Xperf/WPA is a powerful Windows tracing tool that can be used to assist in troubleshooting numerous issues on a Windows server.   The utility allows for in depth tracing without the high overhead typically associated with such tracing.   This is accomplished by making use of the Event Tracing for Windows architecture (ETW).     ETW is built into Windows and most other MS products.    This allows for the tracing of both the user mode and kernel mode activity on a server.   This provides an end to end look at the processes being traced. While most troubleshooting for sql server will be done with the normal toolset (profiler, xevents, perfmon, dmvs), there are a number of issues where xperf can fill in the gaps.  Notoriously difficult issues such as stalled or long IOs and high privileged time are prime candidates for xperf and its ability to hook into the kernel.

Kernel Tracing

While xperf has the ability to trace both kernel and application providers, I won't go into the app side in this post as SQL Server has its own ETW tool in Xevents.   In the past, trying to get insight into what is happening at the kernel level has been extremely difficult.   Generally, a debugger and possibly a crash dump were required to gain information of what was happening at the kernel level.  Xperf allows for quick and easy inspection of kernel events without the need for a debugger.  

Caution

While ETW is light weight, as with any tracing there is too much of a good thing.   In general, xperf tracing on the target server will only have a couple percent CPU overhead, however, the more kernel flags and providers added to the trace the impact will grow.    The other big consideration for running xperf is where the log file is and do you have the space.   Xperf traces tend to be large and can grow to multi-GB in times as little as 10 minutes.   You will want to ensure the trace files are written to a different drive than the sql databases to mitigate the impact of log generation on the performance of your system.   As with any tracing, if possible test your commands in lower environments before moving to production.

Where do I get Xperf?Xperf is included as a part of the Windows SDK under the Windows Performance Toolkit:

http://msdn.microsoft.com/en-us/windows/hardware/hh852363.aspx

The SDK has many components but if you are just interested in the Xperf piece, you can select just the Windows Performance Toolkit as a part of the download:

Basic Syntax

Before getting into any samples let's look at the basic syntax for getting a kernel trace with xperf.

To start a trace:

Xperf -on <kernel flags> -stackwalk <kernel flags>

To stop a trace:

Xperf -d <path to save the output to>

To view a trace:

You can still use the xperfview.exe, however, this is being deprecated in favor of using WPA.exe.   The WPA.exe is included in the Windows Performance Tools download.

This is a very small subset of the commands for xperf.   For more syntax info see: http://msdn.microsoft.com/en-us/library/windows/desktop/hh162920.aspx

Two useful commands to know are:

Xperf -providers KF

--will dump all the available kernel flags

And

Xperf -providers KG

--will dump all the available kernel groups

While you can specify a list of kernel flags to capture, you can also substitute a kernel group which is merely a collection of kernel flags.

For example:

Xperf -on PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+DPC+INTERRUPT+CSWITCH+PROFILE

is equivalent to:

Xperf -on Latency

Stackwalk

The -stackwalk flag is where xperf gets a lot of its power.   With stackwalk enabled for the necessary kernel flags, the xperf output will be able to show you stack information for the executing processes.   With this you can see what functions are being called and thus more directly where CPU activity or CPU Waits are occurring.    

One issue with capturing stackwalk is that you need to change the DisablePagingExecutive reg key before running xperf with stackwalk.   Xperf will check for this setting at startup and warn you if the change is not made.   The DisablePagingExecutive reg key controls whether kernel code is paged to disk.   If the code is paged we cannot get the stackwalk information.   Without the regkey you will be able to get the stack info for all code that is in memory but you may see random stacks not populate correctly.   Before running stackwalk it is recommended to change the

HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\DisablePagingExecutive to 1 (http://technet.microsoft.com/en-us/library/cc959492.aspx).   Note this does require a reboot to go into effect which may make it difficult to do in production environments.

Troubleshooting Samples

SQL high CPU

In general, for a high cpu scenario where sql server is driving cpu, traditional tools such as profiler, perfmon, and dmv's will be the preferred method of troubleshooting the issue.   However, this gives a good example to show the power of xperf and how it could supplement the traditional tools.

For this scenario, I will use the following xperf command:

Xperf -on Latency -stackwalk profile

I created a manufactured scenario to create high cpu on my system using ostress and some pretty nasty queries.   I collected xperf for just over one minute.  Note you will want to keep xperf traces short as the collections can get large quickly.   My one minute trace came out to 200MB.   This will vary depending on the number of processes running on a box and which kernel flags you set. 

After collecting the trace, I fired up WPA and opened the collected .etl file.

First you will want to make sure you have your symbols lined up.  This is important for making the Stackwalk information readable.   When xperf collects function and stack information it does not capture the actual function names.   Lining up symbols allows xperf to look at the symbols and present back to you the actual function names, as you will see below.   If symbols are not set, you will see question marks instead of function names.

For my symbols, I use a local cache of D:\symbols and point to the Microsoft public symbol server,

http://msdl.microsoft.com/download/symbols.

You can set an environment variable of _NT_SYMBOL_PATH equal to your path so that this is populated by default and you don't have to retype this each time.

Once the paths are set, you can then in the Trace drop down menu select "Load Symbols".   At this point symbols will either be found in your local cache or downloaded from the symbols site.   Depending on your network connectivity, this could take a few minutes.

On the left we can now open the Graph Explorer and see all the possible graphs we can look at.   There are many charts that breakdown the data in different ways which can be useful for different scenarios.   In this case since we have a known application, I will look at the "Utilization by Process".   You drag the chart from the Graph Explorer into the main window.

As we can see there is one process consuming the majority of the cpu.  By simply hovering over the high line on the cpu chart we will see which process that line represents. 

As expected we see SQL Server consuming the majority of the cpu.   To get more information about what SQL was doing we need to open the details table.  We open this by clicking on the "Display Graph and Chart" icon.

Once the detail table is open, we will need to add the stack column to get more information:

Now we can see the stack information for the processes.   With this we can follow the Weight column to see which process stack is using the CPU:

We will drill down and follow the weight to fully expand the stack that was using the most cpu.

As we drill down we see the weight start to split up at different function.   We are most interested in seeing the high level of where most the work was being done.  The Weight stays the same up to the point where we hit the FnProducerOpen function in sql server.   Without knowing the source, we can guess here that Parallel queries use the idea of producers and consumers.  Seeing as the stack is generating producers we can surmise the cpu is being eaten by parallel queries.  Looking a little further down we still see a large chunk of the CPU being used by the BuildSortTable function.  Again, this gives us information that we are dealing with Sort tables.   At this point we would definitely need to use this information along with profiler or sys.dm_exec_requests/sys.dm_exec_query_stats to see which queries may be running in parallel and possibly have ORDER BY clauses.  

As it happens, just by looking at some dvms with the following query:

select r.session_id, t.task_address, st.* from sys.dm_exec_requests r inner join sys.dm_exec_connections c on r.session_id = c.session_id

inner join sys.dm_os_tasks t on r.session_id = t.session_id

cross apply sys.dm_exec_sql_text(c.most_recent_sql_handle) st

order by r.session_id

I could see a large number of parallel queries all executing the following statement:

select distinct ts.dbid, tb.HashID, tbb.HashID from ReadTrace.tblUniqueStatements tb cross join ReadTrace.tblStatements ts cross join ReadTrace.tblBatches tbb

order by ts.DBID, tb.HashID, tbb.HashID

So I now have a pretty good place to start investigating as the above query was parallel and is doing an order by.

Again this was a bit of a manufactured scenario to demonstrate how we can use xperf.   In the future posts we will cover scenarios more suitable for xperf such as looking at privileged time and long IO's. 

Joe McTaggart– Premier Field Engineer

In this blog post we’ll take a look at how SQL Server interacts with the transaction log file using Extended Events and Process Monitor.  We will see how small transactions can have a negative impact on transaction log IO throughput and how larger transactions can actually be much faster than a lot of small transactions.

My goal with this post is to demonstrate the behavior of a Log Flush operation to the transaction log file for a given database and how the size of different transactions have an influence on the size of the IOs written to the log file.  While I will be showing cases where larger transactions do perform better than smaller transactions, it is not entirely appropriate for every environment.  Transaction scope should ultimately be dictated by business requirements, and not solely throughput.  Choices made in this area not only have an effect on performance, but also on concurrency and business needs, so choose wisely. 

First, let’s open up SSMS and create a table to use for our example.

USEAdventureWorks2012

GO

IFOBJECT_ID('dbo.Numbers')ISNOTNULL

DROPTABLEdbo.Numbers

GO

CREATETABLEdbo.Numbers

(

     NumINTNOTNULL,

     CONSTRAINTPK_NumbersPRIMARYKEYCLUSTERED(Num)

)

GO

Next, open up Process Monitor (which can be downloaded here:  http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx). We will only be looking for the sqlservr.exe Process and the Path to the transaction log file for AdventureWorks2012, in my case C:\SQL\AdventureWorks2012_log.ldf.  The idea here is to watch the size of the IO flush to the transaction log file.  The size will vary from the sector size on disk (usually 512 bytes) all the way up to 60k in size.  The smaller the log flush, the more IO we have to do to write log records out to the transaction log file (known as a log flush).  If the transaction is larger (closer to 60K) then the less work we do in terms of write frequency to the log because we can fit more log records into a single write.

Next I am going to dump 12,000 records into the Numbers table I previously created.  Since each iteration is a separate implicit transaction, we will have to flush the log records in the log buffer to the transaction log upon each transaction completion. There are 3 things that cause SQL Server to need to flush a log buffer to disk (to an adjoining log block in the transaction log). 

1. A transaction commits and that transaction has an active log record in a log buffer.  This will cause all log records in those buffer (again, up to 60K worth of transaction log record for each log buffer) to be flushed to the log file. 

2.  We hit the 60K size limit for the log buffer and must flush it to disk. 

3.  We have data pages that we must write to disk and there are log records related to those pages.  We must write those log records to the log file BEFORE we write the data pages to the data files (this supports our Write Ahead Logging (WAL) protocol).

SETSTATISTICSIOOFF

SETNOCOUNTON

DECLARE@NumberINT

SET@Number= 1

WHILE@Number<= 12000

BEGIN

     INSERTINTOdbo.Numbers(Num)

     SELECT@Number

     SET@Number=@Number+ 1

END

GO

We can see here that the above statement took around 6 seconds on my system.  Your mileage will vary here in aspect to duration.

And we can see that we did, in fact, insert 12,000 records into the table.

SELECTCOUNT(*)ASRecordCount

FROMdbo.Numbers

GO

More interestingly, looking at Process Monitor we can see the size of the writes to the transaction log file.  Because these were very small transactions, we see that we wrote many, many times to the transaction log file in 512 byte increments.  That is a lot of work to do, and we can see the result in the 6 second duration in the above INSERT statement loop.

Using extended events, I can use a histogram target to capture the size of the writes and how many times that write size occurred to the transaction log.  In this case, almost every one of the 12,000 inserts was in 512 byte increments.  I use the database_id of 5 in my example script below because my AdventureWorks2012 database_id is 5 on my system.

CREATEEVENTSESSION[LogRecords]ONSERVER

ADDEVENTsqlserver.log_flush_start(

    ACTION(sqlserver.database_name)

    WHERE ([database_id]=(5)))

ADDTARGETpackage0.histogram(SETfiltering_event_name=N'sqlserver.log_flush_start',

source=N'write_size',source_type=(0))

WITH (MAX_MEMORY=4096 KB,EVENT_RETENTION_MODE=ALLOW_SINGLE_EVENT_LOSS,

MAX_DISPATCH_LATENCY=30 SECONDS,MAX_EVENT_SIZE=0 KB,MEMORY_PARTITION_MODE=NONE,

TRACK_CAUSALITY=OFF,STARTUP_STATE=OFF)

GO

Now, what happens if we batch our INSERT statements up into one large transaction?  First, clear out the table using a TRUNCATE statement:

TRUNCATETABLEdbo.Numbers

GO

Next, run the same INSERT statement as before, this time just wrap the statement into an explicit transaction.

SETSTATISTICSIOOFF

SETNOCOUNTON

DECLARE@xINT

SET@x= 1

BEGINTRANSACTION

WHILE@x<= 12000

BEGIN

     INSERTINTOdbo.Numbers(Num)

     SELECT@x

     SET@x=@x+ 1

END

COMMITTRANSACTION

GO

We can see that the transaction is MUCH faster this time – less than a second in duration.

Looking at Process Monitor we see that the writes to the transaction log file are now mostly 60k in size rather than the 512 byte writes that we previous observed.  This means that there were many fewer writes to the log file.

The xEvent histogram backs up what we see with Process Monitor – all but a few of the transaction log writes are in 60k increments.  This happens to be much faster than writing in very small increments.  It is also worth noting here that the write nature to the transaction log is always sequential. A log flush always writes its transaction log records logically to the transaction log file in a place directly “after” the previous log flush.  The reason I stress using “after” here is because the transaction log is also circular in nature.  If when writing to the transaction log, the end of the file is reached, it is certainly possible to wrap back to the beginning of the file if the VLF boundaries at the beginning of the file are available for reuse.  For more information regarding the physical architecture of the transaction log, see this article:  http://technet.microsoft.com/en-us/magazine/2009.02.logging.aspx

Here are a couple of other articles that also dig into transaction log internals:

http://www.sqlskills.com/blogs/paul/benchmarking-1-tb-table-population-part-2-optimizing-log-block-io-size-and-how-log-io-works/

http://msdn.microsoft.com/en-us/library/cc917726.aspx

Tim Chapman

I encountered an interesting case with FileStream and Antivirus which I looked at with Bob Dorr, blogged about it on the PSS SQL Blog, thought I’d share that link here as well

http://blogs.msdn.com/b/psssql/archive/2013/03/29/the-case-of-anti-virus-filter-drive-interference-with-file-stream-restore.aspx

-Denzil Ribeiro

In this blog post I’ll take a look at what is actually stored in a non-clustered index and (re)introduce you to some tools that you can use for looking at the contents of a given data or index page.  Hopefully, by the end of the examples you’ll have a better idea as to what SQL Server is looking at when it is traversing an index and what is stored in the root and intermediate pages of an index.

First, I’ll create a database to use for our scenario. I’ll create a table named NumbersTable and load 40K records into it.

usemaster

go

SETSTATISTICSXMLOFF

SETSTATISTICSIOOFF

SETNOCOUNTON

go

IFDB_ID('HeapsDB')ISNOTNULL

BEGIN

     ALTERDATABASEHeapsDB

     SETSINGLE_USERWITHROLLBACKIMMEDIATE

     DROPDATABASEHeapsDB

END

GO

CREATEDATABASEHeapsDB

GO

ALTERDATABASEHeapsDB

SETRECOVERYSIMPLE

GO

USEHeapsDB

GO

CREATETABLENumbersTable

(

     NumberValueBIGINTNOTNULL,

     BiggerNumberBIGINTNOTNULL,

     CharacterColumnCHAR(50)

)

GO

INSERTINTONumbersTable

(

     NumberValue,BiggerNumber,CharacterColumn

)

SELECT

     NumberValue,

     NumberValue+ 5000000,

     LEFT(REPLICATE((CAST(NumberValueasVARCHAR(50))),50),50)

FROM

(

     SELECT

          NumberValue=row_number()over(orderbynewid()asc)

     FROMmaster..spt_valuesa

     CROSSAPPLYmaster..spt_valuesb

     WHEREa.type='P'ANDa.number<= 200 ANDa.number> 0 AND

     b.type='P'ANDb.number<= 200 ANDb.number> 0

)a

Next, cluster the table on the NumberValue column. Notice that I am adding the clustered index as a Primary Key constraint – which is unique.  Had a simply added a clustered index to the table and not specified that the clustered index be UNIQUE, then a “uniqueifier” would be added to the key definition to ensure uniqueness.  I’ll discuss this “uniqueifier” in a later tip.

I then add a non-clustered index on the BiggerNumber column.

ALTERTABLENumbersTable

ADDCONSTRAINTPK_NumbersTablePRIMARYKEYCLUSTERED (NumberValue)

GO

CREATENONCLUSTEREDINDEXidx_NumbersTable

ONNumbersTable(BiggerNumber)

GO

I can use sys.dm_db_index_physical_stats to view the depth of the B-tree non-clustered index.  Remember that a non-clustered index contains only a subset of the columns in the base table and is organized in a B-tree structure.  These NC indexes allow for very fast searching of data, potentially without needing to access the base table. (For more information regarding the overall structure of a NC Index, see this article:  http://msdn.microsoft.com/en-us/library/ms177484(v=sql.105).aspx)

--look at the depth of the trees on the NC index idx_NumbersTable

selectpage_count,index_level,record_count,index_depth

fromsys.dm_db_index_physical_stats(db_id(),object_id('NumbersTable'),2,null,'DETAILED');

I can see that there are 2 levels to this index -  a root and the leaf.  Had the index stored more rows (or wider rows) then it is possible that there be one or more intermediate levels.  However, for simplicity purposes I’m sticking to just two levels for this example. 

The root page is always a single page which points to an intermediate level of pages, or in this case simply a leaf level set of pages.  The query also indicates that the root page contains 109 records.  As it so happens, the leaf level of the index contains 109 pages.  Coincidence? I think not!  J  You see, the root page contains one record pointing to each page in the subordinate (in this case the leaf) level in the index.

If I use the sys.dm_db_database_page_allocations DMV, I can find the page ID for the root page in the index.  Note: This functionality existed in previous versions of SQL Server with the DBCC IND command. 

In this case, it is page 384.

--look at the linkages for the NC index

selectallocated_page_page_id,next_page_page_id,previous_page_page_id,page_level

fromsys.dm_db_database_page_allocations(db_id(),object_id('NumbersTable'),2,null,'DETAILED')

wherepage_type_descisnotnullandpage_type_desc='INDEX_PAGE'

andpage_level= 1

I can then use DBCC PAGE to look at the contents of the index page.  Note – since I am using the WITH TABLERESULTS option there is no need to turn on Trace Flag 3604.  If you want to view the results from DBCC PAGE without TABLERESULTS, you’ll need to enable that trace flag.  Looking at the second resultset from the DBCC PAGE call, I can see that 109 records are returned, along w/ the keys stored in the NC index in the root page.  I see that the BiggerNumber column is returned along with the NumberValue column, which is the clustered key on the table. We have to store the clustered key in the root page (and intermediate pages if they existed) in this case because we have to have a way to get to a unique key in the leaf level of the index. Since the NC index isn’t defined as unique, we store the clustered key as well (which is always unique).

Note:  We also do not store INCLUDED columns in the root or intermediate levels of a non-clustered index – only the leaf level.

DBCCPAGE(HeapsDB, 1, 384, 3)WITHTABLERESULTS

Running the same example again with a UNIQUE NC index, we can see the difference. 

CREATEUNIQUENONCLUSTEREDINDEXidx_NumbersTable

ONNumbersTable(BiggerNumber)

WITHDROP_EXISTING

Next, find the root page of the new index.

selectallocated_page_page_id,next_page_page_id,previous_page_page_id,page_level

fromsys.dm_db_database_page_allocations(db_id(),object_id('NumbersTable'),2,null,'DETAILED')

wherepage_type_descisnotnullandpage_type_desc='INDEX_PAGE'

andpage_level= 1

And pass the root page, 544 in this case, to the call to DBCC PAGE.  I see in this case that the BiggerNumber column is the only column value stored in the root page.  I also see that my row size is now 15 bytes, whereas before it was 23 bytes.  This 8 byte difference is due to the fact that the absent NumberValue column (the unique clustered key on the table) is a BIGINT, which is 8 bytes in size.

DBCCPAGE(HeapsDB, 1, 544, 3)WITHTABLERESULTS

So, what do these values in the BiggerNumber column actually point to in the root?  They point to child pages in the subordinate level in the index.  Using this approach, an index traversal can interrogate the values in the root page and determine which pages in the next level of the index need to be accessed. This works for singleton lookups as well as for read-ahead.  For example, let’s look at page 512 – which happens to be the first page in the leaf level of the index.  From the results above, I can see that ChildPageID has a BiggerNumber key value of NULL.  The next record in the root page points to the BiggerNumber key value of 5000369. So –the ChildPage 512 should contain values NULL all the way through 5000368.  If we look at DBCC PAGE for 512, we can see all the values it contains:

DBCCPAGE(HeapsDB, 1, 512, 3)WITHTABLERESULTS

In fact, this leaf level page does contain the BiggerNumber key values 5000001 – 5000368.  Also, since this is the leaf level there must be a pointer back to the base table (in this case the clustered index).  So, that is why the NumberValue column is present.

Hope this helps!

Tim Chapman

I was dealing with an issue where we had an AlwaysOn availability group with 2 replicas configured for Automatic failover. There was a 30 second glitch where folks could not connect to the Database on the primary and automatic failover did not kick in. Well, that was our initial impression at least. The purpose of this post is to expose you to the different logs available in troubleshooting AlwaysOn Availability group issues, not so much on this particular problem itself.

Symptoms on Primary:  Connections failed for a brief moment with the error below and then was all good.

Error: 983, Severity: 14, State: 1.

Unable to access database 'HADB' because its replica role is RESOLVING which does not allow connections. Try the operation again later.

So there were 3 questions to answer:

a.      What was the reason for the error?

b.      Why didn’t automatic failover kick in? Or did it?

c.      Was it supposed to fail over to the other node?

First of all we need to understand the FailureConditionLevel which controls when failover occurs both from an SQL FCI (failover cluster) or AlwaysOn Availability group Automatic failover perspective.  For detailed information regarding Failover Policies for Failover Cluster Instances, refer to this article:  http://msdn.microsoft.com/en-us/library/ff878664.aspx

In my case the FailoverConditionLevel is set to 5 (Default is 3).  This setting can be altered with the following TSQL script:

If I look at the article referenced above, I notice that the FailoverConditionLevel has the following attributes:

One thing to note here is that the Cluster action is only if any of the subsystems report an “error”, no action is taken on a warning.

So effectively what happens is the following:

•          Cluster service runs LooksAlive check

•          Sp_server_diagnostics results sent to Resource Monitor DLL

•          Resource Monitor DLL detects any error state and notifies the cluster service

•          Cluster Service takes the resource offline

•          Notifies SQL Server to issue an internal command to take the availability group offline.

• There is also the whole concept of a lease that is explained here:

http://blogs.msdn.com/b/psssql/archive/2012/09/07/how-it-works-sql-server-alwayson-lease-timeout.aspx

In order to understand this better I attempted and was able to reproduce the scenario.

I then looked at the Cluster Diagnostic extended event Log, the AlwaysOn extended event log, the cluster log, and the SQL Server error log to try to piece together what exactly happened.

Cluster Diagnostic Extended Event Log:

We see from this log that the System component did throw an error. Which equates to there were N number of dumps created and or a spinlock orhpaned after an Access violation or a detected Memory scribbler condition

The  Cluster Diagnostics Logs are located in the Log directory as shown below  and are different log files than the cluster log itself.

They are of the format : ServerName_InstanceName_SQLDIAG_*.xel

As we can see below, you see the component_health_result indicate that the system component of sp_server_diagnostics returned an error, when the resource monitor than interpretted as a Failure due to the FailureConditionLevel set, and propagated the resource “NOTHEALTH” to the cluster service which triggered the LooksAlive check to return “not alive” or false status.

AlwaysOn Extended Event log

The AlwaysOn Health Extended Event logs cover the Availability Group related diagnostics such as State changes for the Group or Replica or Databases, errors reported, lease expiration and any Availability Group Related DDL that is executed. The format of the logs is: AlwaysOn_health*.xel

If we look at the log snippet below, we see that the AG lease expired, and that triggered us to attempt a failover which in turn changes the state from PRIMARY_NORMAL to RESOLVING_NORMAL.

Cluster Log

Note: The times are in UTC so you have to convert them to match up with the other log files.

To generate the log: How to Generate a Cluster Log on Windows 2008 onwards

00006918.00015978::2013/04/03-18:54:37.251 INFO  [RES] SQL Server Availability Group: [hadrag] SQL Server component 'system' health state has been changed from 'warning' to 'error' at 2013-04-03 11:54:37.247

00006918.00014ef4::2013/04/03-18:54:37.970 ERR   [RES] SQL Server Availability Group: [hadrag] Failure detected, the state of system component is error

00006918.00014ef4::2013/04/03-18:54:37.970 ERR   [RES] SQL Server Availability Group< 2012AG>: [hadrag] Availability Group is not healthywith given HealthCheckTimeout and FailureConditionLevel

00006918.00014ef4::2013/04/03-18:54:37.970 ERR   [RES] SQL Server Availability Group< 2012AG>: [hadrag] Resource Alive result 0.

00006918.00014ef4::2013/04/03-18:54:37.970 ERR   [RES] SQL Server Availability Group< 2012AG>: [hadrag] Resource Alive result 0.

00006918.00014ef4::2013/04/03-18:54:37.970 WARN  [RHS] Resource 2012AG IsAlive has indicated failure.

00019d20.00000e5c::2013/04/03-18:54:37.970 INFO  [RCM] HandleMonitorReply: FAILURENOTIFICATION for '2012AG', gen(0) result 1.

00019d20.00000e5c::2013/04/03-18:54:37.970 INFO  [RCM] TransitionToState(2012AG) Online-->ProcessingFailure.

00019d20.00000e5c::2013/04/03-18:54:37.970 INFO  [RCM] rcm::RcmGroup::UpdateStateIfChanged: (2012AG, Online --> Failed)

00019d20.00000e5c::2013/04/03-18:54:37.970 INFO  [RCM] resource 2012AG: failure count: 1, restartAction: 2.

00019d20.00000e5c::2013/04/03-18:54:37.970 INFO  [RCM] Will restart resource in 500 milliseconds.

-        If you see the “restart action” highlighted above, a restart is attempted on the current node first before failing over to the other node and in this case the restart is successful so it doesn’t really fail over to the other node. If we take a look at the cluster Availability group Resource Properties, you can confirm that the Restart action does indicate that a restart will be attempted on the current node first

-       

00019d20.00019418::2013/04/03-18:55:06.079 INFO  [RCM] TransitionToState(2012AG) DelayRestartingResource-->OnlineCallIssued.

00019d20.00019418::2013/04/03-18:55:06.079 INFO  [RCM] HandleMonitorReply: ONLINERESOURCE for '2012AG', gen(1) result 997.

00019d20.00019418::2013/04/03-18:55:06.079 INFO  [RCM] TransitionToState(2012AG) OnlineCallIssued-->OnlinePending.

…

00006918.0001f1c0::2013/04/03-18:55:07.298 INFO  [RHS] Resource 2012AG has come online. RHS is about to report status change to RCM

SQL Server Errorlog

More details on Lease Expiration: http://blogs.msdn.com/b/psssql/archive/2012/09/07/how-it-works-sql-server-alwayson-lease-timeout.aspx 

2013-04-03 11:54:43.59 Server      Error: 19407, Severity: 16, State: 1.

2013-04-03 11:54:43.59 Server      The lease between availability group '2012AG' and the Windows Server Failover Cluster has expired. A connectivity issue occurred between the instance of SQL Server and the Windows Server Failover Cluster. To determine whether the availability group is failing over correctly, check the corresponding availability group resource in the Windows Server Failover Cluster.

2013-04-03 11:54:43.64 Server      AlwaysOn: The local replica of availability group '2012AG' is going offlinebecause either the lease expired or lease renewal failed. This is an informational message only. No user action is required.

2013-04-03 11:54:43.64 Server      The state of the local availability replica in availability group '2012AG' has changed from 'PRIMARY_NORMAL' to 'RESOLVING_NORMAL'. The replica state changed because of either a startup, a failover, a communication issue, or a cluster error. For more information, see the availability group dashboard, SQL Server error log, Windows Server Failover Cluster management console or Windows Server Failover Cluster log.

2013-04-03 11:54:43.84 spid31s     The availability group database "HADB" is changing roles from "PRIMARY" to "RESOLVING" because the mirroring session or availability group failed over due to role synchronization. This is an informational message only. No user action is required.

2013-04-03 11:54:43.84 spid27s     AlwaysOn Availability Groups connection with secondary database terminated for primary database 'HADB' on the availability replica with Replica ID: {89c5680c-371b-45b9-ae19-2042d8eec27b}. This is an informational message only. No user action is required.

n  The error below can occur if the Local Log records are hardened but quorum is lost with the cluster so the remote harden cannot be completed.

2013-04-03 11:54:45.16 spid58      Remote harden of transaction 'user_transaction' (ID 0x00000000001ee9e2 0000:000006eb) started at Apr  3 2013 11:54AM in database 'HADB' at LSN (37:28:204) failed.

2013-04-03 11:54:46.42 spid31s     Nonqualified transactions are being rolled back in database HADB for an AlwaysOn Availability Groups state change. Estimated rollback completion: 100%. This is an informational message only. No user action is required.

n  This phase is after the “restart action” as seen in the cluster log where we are attempting a restart on the same node before failing over to the other node.

2013-04-03 11:55:06.25 spid58      AlwaysOn: The local replica of availability group '2012AG' is preparing to transition to the primary role in response to a request from the Windows Server Failover Clustering (WSFC) cluster. This is an informational message only. No user action is required.

2013-04-03 11:55:07.27 spid58      The state of the local availability replica in availability group '2012AG' has changed from 'RESOLVING_NORMAL' to 'PRIMARY_PENDING'. The replica state changed because of either a startup, a failover, a communication issue, or a cluster error. For more information, see the availability group dashboard, SQL Server error log, Windows Server Failover Cluster management console or Windows Server Failover Cluster log.

2013-04-03 11:55:07.55 Server      The state of the local availability replica in availability group '2012AG' has changed from 'PRIMARY_PENDING' to 'PRIMARY_NORMAL'. The replica state changed because of either a startup, a failover, a communication issue, or a cluster error. For more information, see the availability group dashboard, SQL Server error log, Windows Server Failover Cluster management console or Windows Server Failover Cluster log.

So in answering the 3 prior questions I had with the logs

a.      The reason we got into this state was the system component reported an error ( was a bunch of exceptions), we won’t go into those here

b.      Failover was attempted, but initial attempt is to restart on the same node and it did end up coming online on that node.

c.      No, it should not have failed over to the other node

Hope the exposure to these logs is helpful in troubleshooting AlwaysON Availability group issues

-Denzil Ribeiro– Sr. SQL Premier Field Engineer

It is a widely known and discussed performance hit when you don’t have an optimal power plan set on your database servers.  This is one of those easy ways to get great gains by simply flicking a switch…almost literally.

But what happens if you have hundreds or thousands of servers under your responsibility?  The thought of that daunting task is enough to make the typical DBA shrug and push it off until another rainy day.  After all, no users are complaining about performance right now.

But alas, in one swift script you can easily reach across all of your servers to determine what the current active power plan is.  This gives you the time savings while not having to manually execute a monotonous task by going to each server.   My proposition is to user PowerShell and WMI in order to get this information.

Because I’m such a fan of reusable code, I will wrap all of my logic in a function that is good at doing just one thing:  Retrieving the active power plan of a list of servers.

FunctionGet-PowerPlan {

    param (

        [Parameter(Mandatory =$false, Position =0, ValueFromPipeline =$true, ValueFromPipelineByPropertyName =$true)]

        [String[]]$ServerNames=$env:COMPUTERNAME

    )

    ### function body – see the rest of the code below ###

}

The skeleton of the function appears above.  We declare only one single parameter, which is a string array that’ll store the list of server names to reach across.  It isn’t a mandatory parameter, as I wanted to have a default to allow a user to execute this function parameter-less to get the active power plan on the local machine.

Now onto the function’s body:

process {

    foreach ($ServerNamein$ServerNames) {

        try {

            Get-WmiObject-ComputerName$ServerName-ClassWin32_PowerPlan-Namespace"root\cimv2\power"|

                Where-Object {$_.IsActive -eq$true} |

                Select-Object @{Name ="ServerName"; Expression = {$ServerName}}, @{Name ="PowerPlan"; Expression = {$_.ElementName}}

        }

        catch {

            Write-Error$_.Exception

        }

    }

}

All the rest of this function does is loop through the list of server names supplied and by utilizing the Win32_PowerPlan WMI class and a few cmdlets to filter out and grab certain data.  The output of this function is simply the server name and the current active power plan on the server.  Here is a sample way to call this function:

$ServerNames="SERVER1","SERVER2","SERVER3"

$ServerNames| Get-PowerPlan

Most likely you will not have only three servers, so you will ideally want to grab this string array from a source listing of the server names.  If you store your server names in a text file (each server on its own line), you can use the Get-Content cmdlet to get the string array to pass to Get-PowerPlan:

ServerList.txt

SERVER1

SERVER2

SERVER3

Get-Content-Path"C:\Your\Directory\ServerList.txt"|Get-PowerPlan

Or, say you store your server names in a management database.  You can retrieve the list by using SMO, and pipe that string array to Get-PowerPlan.

Sample Server List Table

useTestDB;

go

createtabledbo.server_list

(

       namesysnamenotnull

);

go

insertintodbo.server_list

values

       ('SERVER1'),

       ('SERVER2'),

       ('SERVER3');

go

And the PowerShell code could resemble the following:

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") |Out-Null

$SqlServer=New-ObjectMicrosoft.SqlServer.Management.Smo.Server("YourSqlServer")

$SqlServer.Databases["TestDB"].ExecuteWithResults("select name from dbo.server_list;").Tables[0]|

    Select-Object-ExpandPropertyName|  Get-PowerPlan

In case you just want an output of all servers that aren’t using the “High performance” power plan, you can pipe the above output to a Where-Object cmdlet and specify a predicate for the PowerPlan property.

Get-Content-Path"C:\Your\Directory\ServerList.txt"|

Get-PowerPlan |  Where-Object {$_.PowerPlan -ne"High performance"}    

Likewise, you could pipe all of this to a file (such as a CSV) for later viewing and reporting. 

Get-Content-Path"C:\Your\Directory\ServerList.txt"|    

Get-PowerPlan|Export-Csv-Path"C:\Your\Directory\OutputPowerPlan.csv"-NoTypeInformation      

Note that on most newer servers, Power Management settings in the BIOS also need to be set correctly. If set incorrectly, despite the Windows power plan settings, the system could still favor electrical savings instead of performance.

Also from a Virtual environment, you would have to consider the host as well as the guest as in the case of VmWare Power Plan and Hyper-V Power Plan settings .

I have shown you today how to gather the active power Plans across a variable number of servers.  Enjoy!

Script: Download Powershell Script here

Thomas Stringer– SQL Server PFE

Twitter: @SQLife

We have introduced the System Health Session in SQL 2008 in order to capture some critical Events to make post mortem analysis much easier. However in SQL 2008, by default the system_health session was only collected to a ring buffer and not persisted to disk. SQL 2012 truly provides more of a black box recorder with the introduction of sp_server_diagnostics and the associated data it exposes.

There is a fair amount of documentation on the System Health Session. This post isn’t going to focus on System Health session itself, rather on visualizing that data. Subsequent posts could focus on the small blocks that make up this bigger picture.

More details on the System Health session can be found below:

http://msdn.microsoft.com/en-us/library/ff877955.aspx

http://blogs.msdn.com/b/psssql/archive/2012/03/08/sql-server-2012-true-black-box-recorder.aspx

http://www.sqlskills.com/blogs/joe/under-the-covers-with-sp_server_diagnostics-data-part-i/

Now this rich set of data is collected but visualizing it isn’t as easy for certain types of Extended Events collected. In particular the Extended Event GUI isn’t conducive to visualizing some of the payload columns that have XML data.

For example, if you look at the sp_server_diagnostics_component_result, the data is XML so getting trends across the whole session is difficult as is correlating it with the other XEvents.

In order to visualize this data, I have put together a set of reports along with some code to import the System Health Session into a database with a pre-defined schema and report off of it. We are NOT capturing any “new” data here, rather just visualizing what is already captured through the System Health Session. By default, we maintain 5 rollover files of 5 MB each for System Health session, I do recommend bumping that value up in order to cover a bit of a longer period of time. I would recommend bumping up that size to 5 files of 20MB each as indicated in the Readme which is part of the download.

Here is a snippet of what that Dashboard looks like with parameters to drill into a timeline as well as get am overall picture of

-        CPU utilization

-        Memory Resource utilization

-        Blocking that occurred > 30 seconds

-        Queries that were waiting > 30 seconds for some wait_types, > 15 for others

-        The status of all the sp_server_diagnostics subsystems and their underlying data

-        Other Ring buffer data such as security and connectivity ring buffers

As part of the download you will have 3 components

•         A Readme Word document that gives a gist of the import process and walks through the procedure and such.

•         A script that creates the necessary database and schema required

•         A set of reports that helps visualize the imported data ( Currently SQL 2008 R2 RDL files)  

• There are 2 basic “import” modes as documented in the Readme

•         Connect to an existing server, and it imports the System Health Session

•         Point the stored proc to system Health XEL files and import it into relational tables.

Each Report should have a short description on top as to what it entails, here are a few of them:

Waiting Queries

Connectivity Ring Buffer

Blocking > 30 seconds

System sp_Server_diagnostics component

Note: These reports only work with the System Health Session. Sp_server_diagnostics is also captured as part of the Cluster Diagnostics, however those XEL files do not have the other pieces of information and the XML structure if a bit different though these reports can be modified to include that scenario too.

System Health Session is “definitely” an extremely useful diagnostic collected out of the box. My hope is that these reports help enhance the experience and enable easy visualization.

Download SQL 2012 System Health reports

Amit Banerjee who is a SQL PFE in India also has some Dashboard reports for SQL 2008 System health Session  http://www.sswug.org/articles/viewarticle.aspx?id=62732

http://www.sswug.org/articlesection/default.aspx?TargetID=64148

Enjoy, and Feedback is always welcome J

-Denzil Ribeiro– Sr. Premier Field Engineer, SQL MCM

In a previous blog post (Troubleshooting SQL High CPU usage using Xperf), we covered the xperf basics, what types of scenarios are appropriate for xperf, and more specifically, how to look at CPU sampling within xperf.   In general, user CPU time would be investigated using the standard SQL Server tools such as Profiler/Extended Events, DMV's, and Perfmon.   A more real world use for SQL DBA's to use Xperf is in the event of slow IO and/or stalled IO's.  

Currently to troubleshoot IO issues within SQL Server, we would generally make use of Perfmon to determine if IO response times and throughput are within expected ranges.    We may also use DMV's such as sys.dm_io_pending_io_requests or sys.dm_io_virtual_file_stats to monitor for stalled IO and couple that with sys.dm_os_wait_stats to determine if IO performance is effecting SQL Server performance.   This can give us a solid idea that IO performance is an issue but it does not give us very much information as to where the problem actually lies.    All the DMV's in SQL Server are tracking the IO's from the point a file operation (READ, WRITE, etc) occurs.    The temptation is to relate these delays to a physical disk or SAN performance issue.   However if we look at the full IO stack for an application, we can see there are many levels an IO must go through to get to the actual physical disk and back.   Below is a visualization of the Windows Storage architecture from: http://technet.microsoft.com/en-us/library/ee619734(v=WS.10).aspx

From <http://technet.microsoft.com/en-us/library/ee619734(v=WS.10).aspx>

SQL Server is monitoring the IO's at the application level, and perfmon gets us a bit further down the stack to where it plugs in after the Partition Manager, but still leaves some ambiguity as to where IO delays are coming from exactly.  Many times IO delays are introduced by driver issues which are not related to the backend physical disk performance.   This leads to finger pointing between DBA's and Storage Admins as the DBA's see slow perf and blame the disks and the storage admins say perf on the disks are great.   With perfmon and standard sql tools, we cannot easily uncover driver issues. 

This is where Xperf can greatly help troubleshoot slow or stalled IO.  With the Drivers and FLT_IO_* kernel flags, we can see how much time is spent in the different levels of the IO stack down to the StorPort driver.   This functionality was added in Windows 2008 R2 and the ntdebugging blog has some great details here.  Let's look at an example SQL Server IO Stall from the SQL Server Errorlog:

2013-02-26 07:49:31.500 spid8s       SQL Server has encountered 1 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [E:\slowio\compressed.ndf] in database [slowio] (13).  The OS file handle is 0x0000000000000B9C.  The offset of the latest long I/O is: 0x00000005c00000

I have captured an xperf trace at the same time this error was raised.  I used the following command to capture detailed IO information:

XPERF -on PROC_THREAD+LOADER+

FLT_IO_INIT+FLT_IO+FLT_FASTIO+FLT_IO_FAILURE+

FILENAME+FILE_IO+FILE_IO_INIT+

DISK_IO+HARD_FAULTS+DPC+

INTERRUPT+CSWITCH+PROFILE+DRIVERS+DISPATCHER

 -stackwalk MiniFilterPreOpInit+MiniFilterPostOpInit+

CSWITCH+PROFILE+ThreadCreate+ReadyThread+

DiskReadInit+DiskWriteInit+DiskFlushInit+

FileCreate+FileCleanup+FileClose+FileRead+FileWrite

-BufferSize 1024 -MaxBuffers 1024 -MaxFile 1024 -FileMode Circular

Note, I was having issues with my initial traces having a large number of dropped events.  You will drop events if the buffers used by Xperf fill and Xperf cannot write them to target file fast enough.   To help prevent dropped events 1) set the -f parameter to a drive that is not used by the application you are troubleshooting and 2) increase the buffers and buffer size.   In this case, I added the -BufferSize and       -MaxBuffers to my command line to give me a 1GB buffer in memory.   As mentioned in the previous blog, xperf logs do grow fairly quickly and a command like above will potentially grow rapidly.  Because of this I have set the –MaxFile size to 1024MB and –FileMode to Circular.  This will keep my output to 1GB but it does mean that the trace will start to overwrite itself once it has collected 1GB of data.   With this it is important to stop the trace as soon as the problem occurs to prevent losing the data for the event you wanted to investigate.

Now that I have the trace, as with before it is important to load symbols since I collected the stackwalk option. 

Opening the trace with Windows Performance Analyzer, I'm starting with the File I/O Count by Type graph.  This graph shows the timing and count of file level operations.   In this situation, this would be at the top of the IO Stack with sql server issuing READFILE/WRITEFILE commands against the data files.

After opening the FileIO graph, since we know the process and the filename, I have setup my grouping on Process, File Name, and Event Type.   We can then drill down to the problem file we saw in the event log.

If we then expand out the Read Event Type, we will see each individual Read operation and details about that operation.   The details show us Duration, Start and End Time, Size, Offset, etc.   The Offset is of particular interest as we can use the Offset column in Xperf to find the Offset found in the sql server errorlog: 0x00000005c00000.    Note that Xperf stores offsets as decimal so we have to convert it to decimal: 0x00000005c00000 = 96468992

I have the IO highlighted here and now we can see this particular read took 20 seconds to complete, that it was a 512KB read operation, and we have the start and end time.   Note the start and end times are the number of seconds from the start of the trace.  Right away from Xperf we can see this was not an isolated event.   In my graph we can see many other IO's on this file taking longer than 15 seconds.   Also if I were to look at other database files I would see the same thing so this is not isolated to a single file.

While this tells us what the IO's look like from the file level, we need to know where the delays are coming from.   For that we can look at two other graphs:  Mini-Filter Delays and Device I/O.    Let's start by looking at the mini-filter delays chart:

In the above chart,  we can see which mini-filter drivers (Anti-virus, encryption) and how much time was spent in each filter drive.   AV software or encryption could introduce delays and this graph gives us insight into any issues here.   In my example, we see there were 22 seconds of delay due to the MpFilter.sys.   A quick Bing search and we can find that the MpFilter.sys is related to Forefront endpoint protection (anti-virus).   We can dig further to see if the AV software is impacting sql server and if so what file:

In this case, we do see SQL Server is being impacted and that a specific cleanup operation was delayed for almost 3 seconds on the slowIOwrites.mdf.    For my repro, I purposely did not set any exceptions to the AV filter thus why AV is scanning my .mdf files and I was running a scan of the drive at the time of the trace.   The simple solution here is to go in and review the AV configuration and ensure proper exceptions are set as documented in http://support.microsoft.com/kb/309422.  The broader point here being that with xperf, we can now see how much time is being spent in filter drivers and we can perform fact based research to determine if things like anti-virus software are at the root of our issues.

Now let's look at the device IO graph for more info about what is happening throughout the IO stack:

Above, we can see all the different device drivers that an IO must go through to satisfy an IO request.   Note that this is a stack structure so delays in a lower driver will show in a higher level driver.   For instance here we see the fltmgr.sys and ntfs.sys consuming a lot of time.   The ntfs.sys is below fltmgr.sys so the fltmgr.sys also accounts for the time spent in ntfs.sys.   What we do see is that there is a ~40 second difference between the two and if we go back and look at the Mini-Filter chart we'll see where much of that time came from.  

Looking back at the IO stack chart above, we note that the storeport.sys is about as low on the stack as we can see before an IO is handed off to the disk.   In general, if we were having issues specific to the disks, we should see high duration in the storport.sys.    Interestingly, in my example, there is very little time spent in storport.sys so we know that our issue is not likely with the drive itself.    The vast majority of delay is in the ntfs.sys.   In this case, I know I can let the storage team off the hook.    The more likely culprit for my repro was that this is a virtual machine using dynamic disks on the host NTFS volume which was attached by USB.   Also, I was creating contention on the USB drive with other disk operations on the host drive.   The host drive and the guest drive had bit-locker enabled.   Basically I threw every bad thing I could at this drive and thus the root cause could be a number of things, but in the end it is a configuration issue with the VM and guest drive and not an issue with the physical disk itself. 

As you can see, xperf gives us much greater visibility into the process of satisfying an IO request.   We can now use a much more fact based approach to troubleshooting delayed IO and provide a more targeted approach to these issues.

Joe McTaggart -- Premier Field Engineer

It is a common check to see if all of the nodes in a cluster are equally patched.  The reason behind this is because in order to ensure stable operations and a consistent experience, it is best practice to have all nodes of a failover cluster instance (FCI) at the same build, including the SQL Server version, Windows Server version, and the OS hot fixes installed.

In order to get this information quickly and efficiently, my recommendation would be to use PowerShell and WMI (and a dash of SMO) in order to accomplish this task.  There are a few benefits to this route:

1.      No need to failover anything

2.      It is scripted, and therefore has the ability to be reused for future requirements

3.      It takes minimal time to complete this action

Diving right into the script, it includes two different functions (one to compare the SQL Server versions, and the other to compare the OS versions and hotfixes), and then a little looping and logic directly in the script.

The script starts with a param block specifying a required single parameter:

## $SqlServerName : the clustered instance name as input ##

param (

    [Parameter(Mandatory =$true, Position =0)]

    [String]$SqlServerName

)

This mandatory parameter is simply the SQL Server instance name that you are looking to test for the cluster.

The first function is created in order to compare the SQL Server versions across two nodes:

FunctionCompare-SqlVersion {

    param ([Parameter(Mandatory =$true, Position =0)]

        [String]$Server1,

        [Parameter(Mandatory =$true, Position =1)]

        [String]$Server2,

        [Parameter(Mandatory =$true, Position =2)]

        [String]$SqlServerName,

        [Parameter(Mandatory =$true, Position =3)]

        [Int32]$SqlVersion,

        [Parameter(Mandatory =$true, Position =4)]

        [String]$SqlServiceName

    )

    # get the version of the first server

    $Server1Version=Get-WmiObject-ComputerName$Server1-ClassSqlServiceAdvancedProperty-Namespace"root\Microsoft\SqlServer\ComputerManagement$SqlVersion"|

        Where-Object {$_.SqlServiceType -eq1-and$_.PropertyName -eq"FILEVERSION"-and$_.ServiceName -eq$SqlServiceName} |

        Select-Object-ExpandPropertyPropertyStrValue

    # get the version of the second server

    $Server2Version=Get-WmiObject-ComputerName$Server2-ClassSqlServiceAdvancedProperty-Namespace"root\Microsoft\SqlServer\ComputerManagement$SqlVersion"|

        Where-Object {$_.SqlServiceType -eq1-and$_.PropertyName -eq"FILEVERSION"-and$_.ServiceName -eq$SqlServiceName} |

        Select-Object-ExpandPropertyPropertyStrValue

    # if either are null, then one server isn't participating in the given FCI

    # so exist and notify the script that SQL Server for this FCI doesn't live here

    if (!$Server1Version-or!$Server2Version) {

        return"NonexistentSql"

    }

    # otherwise compare the two versions

    Compare-Object$Server1Version$Server2Version|

        Select-Object @{Name ="InstanceName"; Expression = {$SqlServerName}},

            @{Name ="OwningNode"; Expression = {

                if ($_.SideIndicator -eq"<=") {

                    $Server1

                }

                elseif ($_.SideIndicator -eq"=>") {

                    $Server2

                }

            }},

            @{Name ="Issue"; Expression = {"SqlVersionMismatch"}},

            @{Name ="Discrepancy"; Expression = {$_.InputObject}}

}

As per the comments in the script, all this function does is compare the SQL Server version on two servers and returns that result if they differ.  If either of the servers doesn’t house a node of the FCI, then there is no need to compare these two servers and return with a status of “NonexistentSql” for conditional logic later on in the script.

The second function is to compare the Operating System side, both with OS version and hotfixes installed:

FunctionCompare-OsVersion {

    param ([Parameter(Mandatory =$true, Position =0)]

        [String]$Server1,

        [Parameter(Mandatory =$true, Position =1)]

        [String]$Server2,

        [Parameter(Mandatory =$true, Position =2)]

        [String]$SqlServerName

    )

    # compare the OS versions

    Compare-Object (Get-WmiObject-ComputerName$Server1-ClassWin32_OperatingSystem|Select-Object-ExpandPropertyVersion) (Get-WmiObject-ComputerName$Server2-ClassWin32_OperatingSystem|Select-Object-ExpandPropertyVersion) |

        Select-Object @{Name ="InstanceName"; Expression = {$SqlServerName}},

            @{Name ="OwningNode"; Expression = {

                if ($_.SideIndicator -eq"<=") {

                    $Server1

                }

                elseif ($_.SideIndicator -eq"=>") {

                    $Server2

                }

            }},

            @{Name ="Issue"; Expression = {"OsVersionMismatch"}},

            @{Name ="Discrepancy"; Expression = {$_.InputObject}}

    # get the first server's hotfixes installed

    $Server1HotFix=Get-HotFix-ComputerName$Server1|

        Select-Object-ExpandPropertyHotFixID

    # get the second server's hotfixes installed

    $Server2HotFix=Get-HotFix-ComputerName$Server2|

        Select-Object-ExpandPropertyHotFixID

    # if the first server has no hotfixes, set to empty string

    if (!$Server1HotFix) {

        $Server1HotFix=""

    }

    # if the second server has no hotfixes, set to empty string

    if (!$Server2HotFix) {

        $Server2HotFix=""

    }

    # compare the hotfixes installed on each server

    Compare-Object$Server1HotFix$Server2HotFix|

            Select-Object @{Name ="InstanceName"; Expression = {$SqlServerName}},

                @{Name ="OwningNode"; Expression = {

                    if ($_.SideIndicator -eq"<=") {

                        $Server2

                    }

                    elseif ($_.SideIndicator -eq"=>") {

                        $Server1

                    }

                }},

                @{Name ="Issue"; Expression = {"OsHotfixMissing"}},

                @{Name ="Discrepancy"; Expression = {$_.InputObject}} |

            Where-Object {$_.Discrepancy -ne""}

}

As the comments explain, this function first compares the OS version, and then subsequently compares the hotfixes installed on the provided two servers.

Now comes to the looping and the logic to get our final result set of disparate versioning on all of the nodes in the cluster.  First we need to utilize SMO in order to connect to the instance.  We also need to get the service name for the given FCI: If is a default instance it’ll be “MSSQLSERVER”, and if it is a named instance then it’ll be in the format of “MSSQL$InstanceName”.

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") |

    Out-Null

# connect to the SQL Server instance to be tested

$SqlServer=New-ObjectMicrosoft.SqlServer.Management.Smo.Server($SqlServerName)

# in order to get the service name, we will do some logic to get

#  the service name (default: MSSQLSERVER, named: MSSQL$InstanceName)

if ($SqlServer.InstanceName -eq"") {

    $SqlServiceName="MSSQLSERVER"

}

else {

    $SqlServiceName='MSSQL$'+$SqlServer.InstanceName

}

Now we need to get an actual listing of all the nodes that the given cluster contains.  We will use this node list in order to loop through each of the servers.  With that list of node names, we will then utilize our above two worker functions to compare the SQL Server version, and the OS version and hotfixes installed.  And you will see below that there are nested FOR loops.  I didn’t want to be constrained to a two-node cluster, and the nested FOR loops allow me to compare each server to every other server, regardless of how many nodes the cluster has.

# ensure first that the instance is clustered

if ($SqlServer.IsClustered) {

    # retrieve the cluster node names directly from SQL Server

    $SqlNodesStats=$SqlServer.Databases["master"].ExecuteWithResults("select NodeName from sys.dm_os_cluster_nodes;").Tables[0]|

        Select-Object-ExpandPropertyNodeName 

    # save the results in the an array

    # loop through each server, for each server

    # this will compare all servers to each other

    $IssuesResults=for ($i=0; $i-lt ($SqlNodesStats.Count -1); $i++) {

        for ($j=0; $j-lt$SqlNodesStats.Count; $j++) {

            if ($i-ne$j) {

                # compare the SQL Server version

                $SqlVersionReturn=Compare-SqlVersion-Server1$SqlNodesStats[$i]-Server2$SqlNodesStats[$j]-SqlServerName$SqlServerName-SqlVersion$SqlServer.VersionMajor -SqlServiceName$SqlServiceName

                # check to make sure that the server houses the FCI node

                if ($SqlVersionReturn-ne"NonexistentSql") {

                    # if the server has a SQL Server FCI node retrieve the SQL and OS versions, and hotfixes

                    $SqlVersionReturn

                    Compare-OsVersion-Server1$SqlNodesStats[$i]-Server2$SqlNodesStats[$j]-SqlServerName$SqlServerName

                }

            }

        }

    }

    # sort the results and remove duplicates

    $IssuesResults|

        Sort-Object-PropertyInstanceName,Issue,OwningNode,Discrepancy-Unique

    if (!$IssuesResults) {

        Write-Host"No identified version/hotfix mismatches"

    }

}

else {

    Write-Host"$($SqlServer.Name) is not a clustered instance"

}

The last statement is used to sort the results and remove duplicates (since we’re comparing each server to every other server, there is a high chance for duplicate version/hotfix mismatch). On my test cluster (with intentionally mismatched versions), my output looks like the following.

Here today I’ve illustrated one way using PowerShell to verify if all of the cluster nodes are utilizing the same version.  This method gives you added flexibility and the opportunity for code reusability. Enjoy!

Download : Compare SQL and OS Failover Cluster Node Versions 

Thomas Stringer - SQL Server Premier Field Engineer

Twitter: @SQLife

Choosing the database recovery model is one of the more important choices DBAs have.  If the business does not need to recover a database to a point in time (PIT) then the SIMPLE recovery model is the choice for you.  However, if PIT recovery is necessary, you must use the FULL recovery model.  The other recovery model, BULK LOGGED exists so that you have the ability to make certain operations faster while still being able to potentially recover to a PIT.  However, going the way of BULKED LOGGED is not without its potential dangers.  In this blog post I want to take a look at why you shouldn’t use the BULK LOGGED operation as a full-time database recovery model and expect to be able to recover to a point in time should a failure occur.

First, I’ll create a database to use for the demos.

USE[master]

GO

IFDB_ID('MinimalLoggingRecovery')ISNOTNULL

BEGIN

      ALTERDATABASEMinimalLoggingRecovery

      SETSINGLE_USERWITHROLLBACKIMMEDIATE

      DROPDATABASEMinimalLoggingRecovery

END

GO

CREATEDATABASEMinimalLoggingRecoveryON  PRIMARY

(NAME=N'MinimalLoggingRecovery_Data',FILENAME=N'C:\SQL\MinimalLoggingRecovery_Data.mdf',

SIZE= 1024MB,MAXSIZE=UNLIMITED,FILEGROWTH= 50MB)

LOGON

(NAME=N'MinimalLoggingRecovery_Log',FILENAME=N'C:\SQL\MinimalLoggingRecovery_log.LDF',

SIZE= 100MB,MAXSIZE= 1024MB,FILEGROWTH= 25MB)

GO

Next, make sure the database is set to the BULK LOGGED recovery model.

ALTERDATABASEMinimalLoggingRecovery

SETRECOVERYBULK_LOGGED

GO

USEMinimalLoggingRecovery

GO

Now I am going to create two tables to use in the example.  The first table, ImportantClusteredTable, will mimic an OLTP table used by a mission critical application.  Imagine that records are inserted into this table on a regular basis and the table supports the day to day needs of the business.  The other table, NonImportantHeapTable, will be a user table that a staff DBA creates to load some data into for maintenance operations.  This table supports no business function.

CREATETABLEImportantClusteredTable

(

 Fld1INTNOTNULL,Fld2INTNOTNULL,Fld3INTNOTNULL,Fld4INTIDENTITY(1,1)PRIMARYKEYCLUSTERED

)

GO

CREATETABLENonImportantHeapTable

(

    Fld1INTNOTNULL,Fld2INTNOTNULL,Fld3INTNOTNULL

)

GO

Create a full database backup so that the database does not behave as though it is in the SIMPLE recovery model.

BACKUPDATABASEMinimalLoggingRecovery

TODISK='C:\SQL\MinimalLoggingRecoveryFull.bak'WITHINIT

GO

My first operation will be to insert 5000 records into the ImportantClusteredTable table.  This will be a fully logged operation (assuming you do not have Trace Flag 610 enabled).

DECLARE@ValueINT

SET@Value= 1

INSERTINTOImportantClusteredTable(Fld1,Fld2,Fld3)

SELECTTOP 5000 @Value,SalesOrderDetailID+ 100,SalesOrderDetailID+ 99

FROMAdventureWorks2012.Sales.SalesOrderDetail

GO

To prove that the operation was fully logged, use the sys.fn_dblog function to return the log records generated associated with that specific table name.

SELECTLogRecordCount1=COUNT(*)

FROMsys.fn_dblog(null,null)

WHEREAllocUnitNameLIKE'%ImportantClusteredTable%'

GO

The query above returned 5172 log records on my system.

Next, perform a checkpoint followed by a log backup.  This will ensure that the log records are flushed to disk and that the fn_dblog function no longer returns information associated w/ the log records flushed to disk.

CHECKPOINT

GO

BACKUPLOGMinimalLoggingRecovery

TODISK='C:\SQL\MinimalLoggingRecoveryLog1.bak'WITHINIT

GO

Perform the same INSERT operation again and view the number of log records generated.  This number will likely be slightly different from the number above.

DECLARE@ValueINT

SET@Value= 2

INSERTINTOImportantClusteredTable(Fld1,Fld2,Fld3)

SELECTTOP 5000 @Value,SalesOrderDetailID+ 100,SalesOrderDetailID+ 99

FROMAdventureWorks2012.Sales.SalesOrderDetail

GO

SELECTLogRecordCount2=COUNT(*)

FROMsys.fn_dblog(null,null)

WHEREAllocUnitNameLIKE'%ImportantClusteredTable%'

GO

In my case I see 5153 log records.

Perform another checkpoint and log backup.

CHECKPOINT

GO

BACKUPLOGMinimalLoggingRecovery

TODISK='C:\SQL\MinimalLoggingRecoveryLog2.bak'WITHINIT

GO

Now, update all of the records from the original INSERT statement so that Fld1 is 9999.  View the log records generated from this UPDATE statement.  Again, this is a fully logged operation.  (See link below for operations that are minimally logged)

UPDATEImportantClusteredTable

SETFld3= 9999

WHEREFld1= 1

GO

SELECTLogRecordCountAfterUpdate=COUNT(*)

FROMsys.fn_dblog(null,null)

WHEREAllocUnitNameLIKE'%ImportantClusteredTable%'

GO

In this case I had 5000 log records – which is the same number as the number of records I updated. 

Get the current timestamp on the system and wait 5 seconds.  After waiting 5 seconds, perform a BULK LOGGED operation.  In this case the NonImportantHeapTable is INSERTed into using the TABLOCK hint.  This will cause the INSERT operation to be minimally logged.  This is to mimic a scenario where a user or application performs a minimally logged operation while in the BULK LOGGED recovery mode. 

SELECTGETDATE()

WAITFORDELAY'0:0:5'

DECLARE@ValueINT

SET@Value= 3

INSERTINTONonImportantHeapTableWITH(TABLOCK)(Fld1,Fld2,Fld3)

selectTOP 5000 @Value,SalesOrderDetailID+ 100,SalesOrderDetailID+ 99

FROMAdventureWorks2012.Sales.SalesOrderDetail

GO

SELECTLogRecordCountAfterBL=COUNT(*)

FROMsys.fn_dblog(null,null)

WHEREAllocUnitNameLIKE'%NonImportantHeapTable%'

Looking at the log records generated for the NonImportantHeapTable, I see 64 log records.  Since this was a minimally logged operation, only allocation page changes were captured in the transaction log.

Capture a point in time after the BULK LOGGED operation occurs.

SELECTGETDATE()  --make sure we know what time to restore to

GO

Here are the values I captured from the SELECT GETDATE() statements above.

2013-04-12 14:07:18.913  --before

2013-04-12 14:07:24.190  --after

Now, assume an unfortunate event occurs and puts the state of the business in jeopardy – maybe a user accidentally deletes all sales made in the past 45 minutes (assume it’s a LOT of sales).  For the given scenario, you determine that best course of action is to restore the database to a point in time BEFORE the error occurs.  No sweat – you’ve been taking log backups every hour, right?

So, the first thing you’ll want to do is take a tail log backup to ensure that you preserve all log records currently in the transaction log but not yet reflected in a log backup.  You decide to use WITH NORECOVERY to ensure no new modifications can occur that modify the state of the database.

usemaster

go

BACKUPLOGMinimalLoggingRecovery