Archive
Temporal data support in SQL Server 2016 – Part 1
This coming release of new SQL Server 2016 will also support for Temporal Data as a new feature called Temporal Tables or System-Versioned Tables.
A Temporal Data is tightly related to a Time period and used to handle the time-varying data. It provides the ability to look at data trends, types of data changes, and the overall data evolution within your database.

A Temporal Table is a table for which a PERIOD definition exists and which contains system columns with a datatype of datetime2 into which the period of validity is recorded by the system. The Temporal table has an associated History Table into which the system records all prior versions of each record with their period of validity.
With a Temporal Table, the value of each record at any point in time can be determined, rather than just the current value of each record.
The Temporal Data is closely related to Slowly Changing Dimensions (SCD) Type-6 method, but the whole mechanism is internally and automatically managed by the SQL Server DB Engine.
–> With Temporal Data support you can:
1. Time Travel back & forth and understand business trends over time.
2. Track and Audit all data changes.
3. Maintain a Slowly Changing Dimension (SCD) for decision support applications
4. Recover from accidental data changes by repairing record-level corruptions.
–> To create a Temporal Table following clauses are used with CREATE TABLE statement:
1. Two additional Start & End date Audit columns of datetime2 datatype for capturing the validity of records. You can use any meaningful column name here, we will use ValidFrom and ValidTo column names in our example below.
2. Both the column names have to be specified in and as PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo) clause with the column list.
3. Specify WITH (SYSTEM_VERSIONING = ON) option at the end of the CREATE TABLE statement with optional (HISTORY_TABLE = {History_Table_Name}) option.
The above CREATE TABLE statement will create 2 tables:
1. one the parent Temporal or System-Versioned Table
2. and second the History Table
–> Now, as I mentioned in Step #3 above, you have an option to specify the History Table name or not. So let’s check both the options here:
–> Option #1: Create Temporal Table [dbo].[Department] with automatically named History table:
USE [TestManDB]
GO
CREATE TABLE dbo.Department
(
DepartmentID int NOT NULL IDENTITY(1,1) PRIMARY KEY CLUSTERED,
DepartmentName varchar(50) NOT NULL,
ManagerID int NULL,
ValidFrom datetime2 GENERATED ALWAYS AS ROW START NOT NULL,
ValidTo datetime2 GENERATED ALWAYS AS ROW END NOT NULL,
PERIOD FOR SYSTEM_TIME (
ValidFrom,
ValidTo
)
)
WITH ( SYSTEM_VERSIONING = ON ); -- No History table name given here
GO
– Check the History Table nested under the Temporal Table dbo.Department . The auto-naming convention goes like this MSSQL_TemporalHistoryFor_{parent_temporal_table_object_id}:
– Let’s Check the metadata of both the tables:
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name -- Department
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.Department', 'U')
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name -- MSSQL_TemporalHistoryFor_1397580017
FROM SYS.TABLES
WHERE object_id = (
SELECT history_table_id
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.Department', 'U')
)
GO
– Output:
– To DROP both the Tables, first you need to switch OFF the System Versioning on the parent Temporal Table:
ALTER TABLE [dbo].[Department] SET ( SYSTEM_VERSIONING = OFF ) GO DROP TABLE [dbo].[Department] GO DROP TABLE [dbo].[MSSQL_TemporalHistoryFor_1525580473] GO
–> Option #2: Create Temporal Table [dbo].[Department] with a named History table [dbo].[DepartmentHistory]:
CREATE TABLE dbo.Department
(
DepartmentID int NOT NULL IDENTITY(1,1) PRIMARY KEY CLUSTERED,
DepartmentName varchar(50) NOT NULL,
ManagerID int NULL,
ValidFrom datetime2 GENERATED ALWAYS AS ROW START NOT NULL,
ValidTo datetime2 GENERATED ALWAYS AS ROW END NOT NULL,
PERIOD FOR SYSTEM_TIME (
ValidFrom,
ValidTo
)
)
WITH ( SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.DepartmentHistory) );
GO
– Check the History Table dbo.DepartmentHistory nested under the parent Temporal Table dbo.Department:
– Let’s Check the metadata of both the tables:
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.Department', 'U')
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.DepartmentHistory', 'U')
GO
– Output:
– Final Cleanup, As mentioned above to DROP both the Tables, first you need to switch OFF the System Versioning on the parent Temporal Table by ALTER statement:
ALTER TABLE [dbo].[Department] SET ( SYSTEM_VERSIONING = OFF ) GO DROP TABLE [dbo].[Department] GO DROP TABLE [dbo].[DepartmentHistory] GO
– Otherwise you will get following error message:
Msg 13552, Level 16, State 1, Line 82
Drop table operation failed on table ‘TestManDB.dbo.Department’ because it is not supported operation on system-versioned temporal tables.
–> Limitation of Temporal Tables:
1. Temporal querying over Linked Server is not supported.
2. History table cannot have constraints (PK, FK, Table or Column constraints).
3. INSERT and UPDATE statements cannot reference the SYSTEM_TIME period columns.
4. TRUNCATE TABLE is not supported while SYSTEM_VERSIONING is ON
5. Direct modification of the data in a history table is not permitted.
6. INSTEAD OF triggers are not permitted on either the tables.
7. Usage of Replication technologies is limited.
–> You can also check this demo in this video:
In my [next post] we will see a DEMO of Temporal data in SQL Server and how you can Time-Travel with your data and get details about point in time history without any extra effort.
In-memory enhancements and improvements in SQL Server 2016
In-Memory tables were introduced in SQL Server 2014 and were also known as Hekaton tables. I’ve written previously about In-memory tables for SQL Server 2014 and you can check in my [previous posts] to know more about these type of tables with some Hands-on examples and demos.
–> In-memory tables as new concept in SQL Server 2014 had lot of limitations compared to normal tables. But with the new release of SQL Server 2016 some limitations are addressed and other features have been added for In-Memory tables. These improvements will enable scaling to larger databases and higher throughput in order to support bigger workloads. And compared to previous version of SQL Server it will be easier to migrate your applications to and leverage the benefits of In-Memory OLTP with SQL Server 2016.
–> I have collated all the major improvements here in the table below:
* Collation Support
1. Non-BIN2 collations in index key columns
2. Non-Latin code pages for (var)char columns
3. Non-BIN2 collations for comparison and sorting in native modules
–> You can check more about In-Memory tables for SQL Server 2016 in MSDN BoL [here].
Check the above details explained in the video below:
ColumnStore Indexes evolution from SQL Server 2012, 2014 to 2016
ColumnStore Indexes were first introduced in SQL Server 2012, and this created a new way to store and retrieve the Index or Table data in an efficient manner.
ColumnStore uses xVelocity technology that was based on Vertipaq engine, this uses a new Columnar storage technique to store data that is highly Compressed and is capable of In-memory Caching and highly parallel data scanning with Aggregation algorithms.
ColumnStore or Columnar data format does not store data in traditional RowStore fashion, instead the data is grouped and stored as one column at a time in Column Segments.
The Traditional RowStore stores data for each row and then joins all the rows and store them in Data Pages, and is still the same storage mechanism for Heap and Clustered Indexes.
Thus, a ColumnStore increases the performance by reading less data that is highly compressed, and in batches from disk to memory by further reducing the I/O.
–> To know more about ColumnStore Indexes check [MSDN BoL].
–> Let’s see the journey of ColumnStore index from SQL Server 2012 to 2016:
✔ We will start with SQL Server 2012 offering for ColumnStore Indexes:
1. A Table (Heap or BTree) can have only one NonClustered ColumnStore Index.
2. A Table with NonClustered ColumnStore Index becomes readonly and cannot be updated.
3. The NonClustered ColumnStore Index uses Segment Compression for high compression of Columnar data.
✔ With SQL Sever 2014 some new features were added, like:
1. You can create one Clustered ColumnStore Index on a Table, and no further Indexes can be created.
2. A Table with Clustered ColumnStore Index can be updated with INSERT/UPDATE/DELETE operations.
3. Both Clustered & NonClustered ColumnStore Index has new Archival Compression options i.e. COLUMNSTORE_ARCHIVE to further compress the data.
✔ Now with SQL Server 2016 new features have been added and some limitations are removed:
1. A Table will still have one NonClustered ColumnStore Index, but this will be updatable and thus the Table also.
2. Now you can create a Filtered NonClustered ColumnStore Index by specifying a WHERE clause to it.
3. A Clustered ColumnStore Index table can have one or more NonClustered RowStore Indexes.
4. Clustered ColumnStore Index can now be Unique indirectly as you can create a Primary Key Constraint on a Heap table (and also Foreign Key constraints).
5. Columnar Support for In-Memory (Hekaton) tables, as you can create one one ColumnStore Index on top of them.
–> Here is the full feature summary of ColumnStore Indexe evolution from SQL Server 2012 to 2016:
–> Check more about this on [MSDN BoL].
Check the above details explained in the video below:
Now truncate “Table Partitions” with TRUNCATE TABLE statement – SQL Server 2016
TRUNCATE TABLE has got a new definition now!!!
Now with SQL Server 2016 as per MSDN BoL: “TRUNCATE TABLE removes all rows from a Table or Specified Partitions of a Table, without logging the individual row deletions”.
This is kind of a filtration to TRUNCATE TABLE statement at a Partition level, just like a WHERE clause option with DELETE TABLE statement at a Row level.
With previous versions ( SQL Server 2014 and back) you could only Truncate a whole Table, but not its Partition.
–> You can now use a WITH PARTITIONS() option with TRUNCATE TABLE statement to truncate Table Partition(s) like:
TRUNCATE TABLE dbo.PartitionedTable WITH (PARTITIONS (2, 4, 6 TO 8))
MSDN reference for [TRUNCATE TABLE]
Check my [previous post] on differences between TRUNCATE & DELETE statements.
See Execution Plans running with Live Query Statistics – SQL Server 2016
In SQL Server 2016 Live Query Statistics is going to be the most used feature among Developers & DBAs to check the live Execution Plan of an active Query running in parallel.
The Live Query/Execution Plan will provide Real-time insights into the Query Execution process as the Control flows from one Operator to the another. It will display the overall Query Progress and Operator-level Run-time Execution Stats such as:
1. Number of Rows processed
2. Completion Estimate
3. Operator progress – Elapsed time & Percentage done for each Operator
4. Elapsed time & Percentage done for overall Query progress
–> The feature can be enable from the Toolbar, just besides the “Actual Execution Plan” icon:
–> The Live Execution plan running can be seen in below animated image:
The dotted moving lines above shows the Operators currently in execution. As soon as the Operator finishes execution these dotted lines will change to solid lines.
Thus, by using this feature the user will not have to wait for a Query to complete its execution and then see the Execution plan stats. As soon as the user will run the Query, the Live Execution Plan will also start showing all the Operators and their progress. This will help users in checking the long running queries where actually they are taking time, and help debugging Query performance issues.
You can also check the full demo here in this video:














