Archive
Implementing “Row Level Security” (RLS) with “Filter Predicate” in SQL Server 2016 – Part 1
For an Organization its Data is the most important thing, and the Org’s IT department takes lot of measures to make sure data does not fall in wrong hands. The DBA’s and Application programmers setup different layers of security on top of data so that the user is only able to see the Filtered data/rows from a table that he has access to. DB programmers normally create abstracted Views or Stored Procedures with complex logic on top of the Tables by using other master-mapping tables that contains the user-data access key. Sometimes these security logics are not upto the standards, and may have various versions on a database and thus it becomes difficult to track, update and make them fail-proof. So, with SQL Server 2016 the new Row Level Security feature is going to handle this Out of the Box and as a Standard.
RLS or Row Level Security is a feature that enables fine grained control over access to rows in a table, allowing you to easily control which users can access which data with complete transparency to the application.
With this feature rows are filtered based on the Execution Context of the Query, rather than the current user access rights. A secure logic can be created to determine which user can see which rows and restrict any kind of data (rows) by designing a flexible and robust Security policy for a table.
–> To setup Row Level Security (RLS) on a particular table it needs only these simple steps:
1. Create Users and Grant Read/SELECT access for a Particular table.
2. Create a new Inline Table-Valued Function that will contain the Filter Predicate for that table. This Function Predicate can be a sophisticated business logic with multiple JOINs or a simple WHERE ManagerCode = ‘userHR’.
3. Create a new Security Policy for this table and add the above Function (Filter) Predicate to it.
Please note: that these Functions & Security Policies should be unique for a table. So to create RLS for an another table, you will need to create separate Function & Security Policy.
–> Step 1.a. Let’s create some test accounts: I will create three users for:
1. The CEO, over-all admin of the company data.
2. HR department head
3. Finance department head
CREATE USER userCEO WITHOUT LOGIN; GO CREATE USER userHR WITHOUT LOGIN; GO CREATE USER userFin WITHOUT LOGIN; GO
–> Create a sample table [dbo].[Employee]: with a self-referencing Manager ID column.
CREATE TABLE dbo.Employees ( [EmpCode] NVARCHAR(50), -- Employee ID [EmpName] NVARCHAR(250), -- Employee/Manager Full Name [Salary] MONEY, -- Fictious Salary [MgrCode] NVARCHAR(50) -- Manager ID ); GO
-> Now insert some test records:
-- Top Boss CEO
INSERT INTO dbo.Employees VALUES ('userCEO' , 'CEO Top Boss' , 800, NULL)
-- Next 2 levels under CEO
INSERT INTO dbo.Employees VALUES ('userHR' , 'HR User' , 700, 'userCEO');
INSERT INTO dbo.Employees VALUES ('userFin' , 'Finance User' , 600, 'userCEO');
-- Employees under Kevin
INSERT INTO dbo.Employees VALUES ('manojp' , 'Manoj Pandey' , 100, 'userHR');
INSERT INTO dbo.Employees VALUES ('saurabhs', 'Saurabh Sharma', 400, 'userHR');
INSERT INTO dbo.Employees VALUES ('deepakb' , 'Deepak Biswal' , 500, 'userHR');
-- Employees under Amy
INSERT INTO dbo.Employees VALUES ('keshavk' , 'Keshav K' , 200, 'userFin');
INSERT INTO dbo.Employees VALUES ('viveks' , 'Vivek S' , 300, 'userFin');
GO
–> Let’s check the records before applying “Row Level Security”:
SELECT * FROM dbo.Employees; -- 8 rows GO
As a normal SEELCT and without RLS, it just ignores my Execution Context and execute the Query and return all the 8 rows.
–> The Traditional way to setup the Row Level Security till now was as follows (a simple example):
-- Stored Procedure with User-Name passed as parameter: CREATE PROCEDURE dbo.uspGetEmployeeDetails (@userAccess NVARCHAR(50)) AS BEGIN SELECT * FROM dbo.Employees WHERE [MgrCode] = @userAccess OR @userAccess = 'userCEO'; -- CEO, the admin should see all rows END GO -- Execute the SP with different parameter values: EXEC dbo.uspGetEmployeeDetails @userAccess = 'userHR' -- only 3 rows GO EXEC dbo.uspGetEmployeeDetails @userAccess = 'userFin' -- only 2 rows GO EXEC dbo.uspGetEmployeeDetails @userAccess = 'userCEO' -- all 8 rows GO
The above method is prone to issues, like SQL Injection and any other user can apply other user’s User-Name and get the information that he is not allowed to see. With this type of method you have to apply another security layer at the application level so whenever a user executes the SP it gets executed with the same user’s User-Name.
–> The new Row Level Security feature let you:
– apply this security at the database level
– and there is no need to apply the WHERE clause filter for the User-Name.
This makes the security system more reliable and robust by reducing the surface area of your security system.
–> Step 1.b. Grant Read/SELECT access on the dbo.Employee table to all 3 users:
GRANT SELECT ON dbo.Employees TO userCEO; GO GRANT SELECT ON dbo.Employees TO userHR; GO GRANT SELECT ON dbo.Employees TO userFin; GO
–> Step 2. Let’s create an Inline Table-Valued Function to write our Filter logic:
CREATE FUNCTION dbo.fn_SecurityPredicateEmployee(@mgrCode AS sysname)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN SELECT 1 AS fn_SecurityPredicateEmployee_result
-- Predicate logic
WHERE @mgrCode = USER_NAME()
OR USER_NAME() = 'userCEO'; -- CEO, the admin should see all rows
GO
This function returns value 1 when:
– a row in the MgrCode (i.e. the Manager Code) column is the same as the user executing the query (@@mgrCode = USER_NAME())
– or if the user executing the query is the Top Boss user (USER_NAME() = ‘userCEO’)
–> Step 3. Create a security policy adding the function as a filter predicate:
CREATE SECURITY POLICY ManagerFilter ADD FILTER PREDICATE dbo.fn_SecurityPredicateEmployee(MgrCode) -- Filter Column from dbo.Employee table ON dbo.Employees WITH (STATE = ON); -- The state must be set to ON to enable the policy. GO
The above Security Policy takes the Filter Predicate Logic from the associated Function and applies it to the Query as a WHERE clause.
–> Now let’s again check the records after applying “Row Level Security”:
SELECT * FROM dbo.Employees; -- 0 rows, GO
The simple “SELECT *” statement will fetch me zero rows after applying RLS, as my user ID is not configured to have access to any of those rows & Table.
–> And if you check in the Execution Plan of above SELECT statement without WHERE clause, it will show you the Filter Predicate that is added by the Security Policy defined in Step #3 for applying RLS on this table, which looks like this:
[TestManDB].[dbo].[Employee].[MgrCode]=user_name()
OR user_name()=N’userCEO’
–> Let’s check the 3 users we created and provided them customized access to the dbo.Employee table and rows in it:
-- Execute as our immediate boss userHR (3 rows): EXECUTE AS USER = 'userHR'; SELECT * FROM dbo.Employees; -- 3 rows REVERT; GO -- Execute as our immediate boss userFin: EXECUTE AS USER = 'userFin'; SELECT * FROM dbo.Employees; -- 2 rows REVERT; GO -- Execute as our Top boss userCEO (8): EXECUTE AS USER = 'userCEO'; SELECT * FROM dbo.Employees; -- 8 rows REVERT; GO
–> The results of the above 3 SELECTs looks like this:
So, as you can see the three users we created resulted in different results:
– The HR & Finance users got just 3 & 2 rows, respectively.
– But the admin CEO user got all the 8 rows.
–> Thus, by using RLS Filter Predicate feature in SQL Server 2016 you can create your own customized Security by creating an Inline Table-Valued Function that is linked to the Security Policy for your Table.
–> Check Block Predicates with RLS in my [next post, Part 2].
–> Final Cleanup
DROP SECURITY POLICY [dbo].[ManagerFilter] GO DROP FUNCTION [dbo].[fn_SecurityPredicateEmployee] GO DROP TABLE [dbo].[Employee] GO DROP PROCEDURE dbo.uspGetEmployeeDetails GO
Check the same demo on YouTube:
new CTP 2.1 update for SQL Server 2016 available for download
Today Microsoft announced an update of the recently released Community Technology Preview (CTP) 2 version of SQL Server 2016, the CTP 2.1.
[Register and Download the CTP 2.1 Evaluation version (180 days) here]
Direct download link:
– SQLServer2016CTP2.1-x64-ENU.box
– SQLServer2016CTP2.1-x64-ENU.exe
As per the Microsoft [SQL Server Blog], now people do not have to wait for several months for the release of next CTP versions. Now customers can experience the Rapid Preview models and bits for their dev & test purpose.
–> This rapid release includes some improvements and fixes to the new features added in SQL Server 2016 release, and as follows:
1. Stretch Database – Demo video
2. Query Store
3. Temporal Data, added support for computed columns and Period columns with HIDDEN flag – Demo video.
4. Columnstore Index, improved seek/scan performance – Demo video.
Check the [SQL Server blog] for all these updates in detail.
Time Travel with Temporal Tables in SQL Server 2016 – Part 2
In my [previous post] I discussed about Temporal Data, how it will be supported in SQL Server and how to CREATE Temporal or System-Versioned Tables.
–> Here in this post we will see how we can enable System-Versioning in an existing table containing data. I will also tweak the data to show you the demo on how you can point to a time back in history and get details relative to that time. This will be like Time Travelling to past and seeing record details as if its current data, without much change to the current table’s data-structure.
1. Let’s create a sample table [dbo].[Employee] and populated it by JOINing some tables on [AdventureWorks] Database:
USE [TestManDB] GO ;WITH CTE AS ( SELECT E.BusinessEntityID, P.FirstName, P.LastName, D.Name AS DepartmentName, ROW_NUMBER() OVER(PARTITION BY E.BusinessEntityID ORDER BY D.ModifiedDate DESC) as RN FROM [AdventureWorks2014].[HumanResources].[Employee] E JOIN [AdventureWorks2014].[Person].[Person] P ON P.BusinessEntityID = E.BusinessEntityID JOIN [AdventureWorks2014].[HumanResources].[EmployeeDepartmentHistory] DH ON DH.BusinessEntityID = E.BusinessEntityID JOIN [AdventureWorks2014].[HumanResources].[Department] D ON D.DepartmentID = DH.DepartmentID ) SELECT BusinessEntityID, FirstName, LastName, DepartmentName INTO dbo.Employee FROM CTE WHERE RN = 1 GO
2. The System Versioned Temporal table [dbo].[Employee] must have Primary Key defined: otherwise you will get the same error message (Msg 13553).
ALTER TABLE dbo.Employee ADD CONSTRAINT PK_BusinessEntityID PRIMARY KEY (BusinessEntityID) GO
3. Now to make [dbo].[Employee] table System Versioned we will add:
– Two Audit columns of datetime2 datatype to store Start & End datetime.
– Use PERIOD FOR SYSTEM_TIME clause to associate these two columns as System Time.
ALTER TABLE dbo.Employee ADD
StartDate datetime2 GENERATED ALWAYS AS ROW START NOT NULL
DEFAULT CAST('1900-01-01 00:00:00.0000000' AS DATETIME2),
EndDate datetime2 GENERATED ALWAYS AS ROW END NOT NULL
DEFAULT CAST('9999-12-31 23:59:59.9999999' AS DATETIME2),
PERIOD FOR SYSTEM_TIME (
StartDate,
EndDate
)
GO
4. After all pre-requisites let’s enable the System-Versioning on [dbo].[Employee] table:
ALTER TABLE dbo.Employee SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.EmployeeHistory)) GO
–> So, as soon you enable the System-Versioning the SQL Engine creates an another History table with similar schema and nests it under the main Temporal table, let’s check both the tables columns and contents:
SELECT TOP 10 * FROM dbo.Employee SELECT TOP 10 * FROM dbo.EmployeeHistory GO
You can see above the History table is empty as there are no updates on the Parent table. I’ve rounded the 5th row because I will update this row in next step for the demo.
5. Let’s make some updates on the parent Temporal Table (5th row): SQL Engine will automatically populate the History table.
UPDATE dbo.Employee SET FirstName = 'Gabriel' WHERE BusinessEntityID = 5 GO UPDATE dbo.Employee SET DepartmentName = 'Research and Development' WHERE BusinessEntityID = 5 GO UPDATE dbo.Employee SET DepartmentName = 'Executive' WHERE BusinessEntityID = 5 GO -- Let's check the records again: SELECT * FROM dbo.Employee WHERE BusinessEntityID = 5 SELECT * FROM dbo.EmployeeHistory WHERE BusinessEntityID = 5 GO
You will see that after doing 3 UPDATEs on the parent Temporal Table the History table [dbo].[EmployeeHistory] is populated with 3 rows that contains the older versions on data in [dbo].[Employee] table across all columns.
6. Ok, now I’ll do some tweaks on the System Time column values of [dbo].[Employee] table.
– First of all I will switch OFF the System-Versioning on dbo.Employee table.
– Now I will update the date of System Time columns, set it to back in history (5-10 days back for an effective demo).
– Enable back the System-Versioning
ALTER TABLE [dbo].[Employee] SET ( SYSTEM_VERSIONING = OFF ) GO update dbo.EmployeeHistory set EndDate = '2015-06-01 18:47:07.5566710' where BusinessEntityID = 5 AND EndDate = '2015-06-09 18:47:07.5566710' update dbo.EmployeeHistory set StartDate = '2015-06-01 18:47:07.5566710', EndDate = '2015-06-05 18:47:28.0153416' where BusinessEntityID = 5 AND StartDate = '2015-06-09 18:47:07.5566710' update dbo.EmployeeHistory set StartDate = '2015-06-05 18:47:28.0153416' where BusinessEntityID = 5 AND StartDate = '2015-06-09 18:47:28.0153416' GO ALTER TABLE [dbo].[Employee] SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.EmployeeHistory)) GO -- Check the data after datetime changes: SELECT * FROM dbo.Employee WHERE BusinessEntityID = 5 SELECT * FROM dbo.EmployeeHistory WHERE BusinessEntityID = 5 GO
–> Ok, now here comes the most interesting part of the topic i.e. “Time Travel”:
SQL Server engine provides you FOR SYSTEM_TIME AS OF {datetime_value} option with your SELECT query to get details pointing to any time in the history, let’s check this here:
SELECT * FROM dbo.Employee FOR SYSTEM_TIME AS OF '2015-01-01' WHERE BusinessEntityID = 5 SELECT * FROM dbo.Employee FOR SYSTEM_TIME AS OF '2015-06-03' WHERE BusinessEntityID = 5 SELECT * FROM dbo.Employee FOR SYSTEM_TIME AS OF '2015-06-07' WHERE BusinessEntityID = 5 SELECT * FROM dbo.Employee FOR SYSTEM_TIME AS OF '2015-06-10' WHERE BusinessEntityID = 5 GO
The above four Queries will show you results from the History Table by pulling out the records for the particular date you mentioned by checking the date ranges in the Audit columns:
–> Let’s check the Execution Plan of the 4th SELECT statement:
So, you can see that you are just querying the dbo.Employee Temporal table, but SQL Engine is internally also querying the dbo.EmployeeHistory table and concatenating (UNION) the rows from both the operators.
–> Final Cleanup: before Dropping the Temporal & History tables, you need to switch OFF the System Versioning feature in the parent table.
ALTER TABLE [dbo].[Employee] SET ( SYSTEM_VERSIONING = OFF ) GO DROP TABLE [dbo].[Employee] GO DROP TABLE [dbo].[EmployeeHistory] GO
–> You can also check this demo in this video:
Temporal data support in SQL Server 2016 – Part 1
This coming release of new SQL Server 2016 will also support for Temporal Data as a new feature called Temporal Tables or System-Versioned Tables.
A Temporal Data is tightly related to a Time period and used to handle the time-varying data. It provides the ability to look at data trends, types of data changes, and the overall data evolution within your database.

A Temporal Table is a table for which a PERIOD definition exists and which contains system columns with a datatype of datetime2 into which the period of validity is recorded by the system. The Temporal table has an associated History Table into which the system records all prior versions of each record with their period of validity.
With a Temporal Table, the value of each record at any point in time can be determined, rather than just the current value of each record.
The Temporal Data is closely related to Slowly Changing Dimensions (SCD) Type-6 method, but the whole mechanism is internally and automatically managed by the SQL Server DB Engine.
–> With Temporal Data support you can:
1. Time Travel back & forth and understand business trends over time.
2. Track and Audit all data changes.
3. Maintain a Slowly Changing Dimension (SCD) for decision support applications
4. Recover from accidental data changes by repairing record-level corruptions.
–> To create a Temporal Table following clauses are used with CREATE TABLE statement:
1. Two additional Start & End date Audit columns of datetime2 datatype for capturing the validity of records. You can use any meaningful column name here, we will use ValidFrom and ValidTo column names in our example below.
2. Both the column names have to be specified in and as PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo) clause with the column list.
3. Specify WITH (SYSTEM_VERSIONING = ON) option at the end of the CREATE TABLE statement with optional (HISTORY_TABLE = {History_Table_Name}) option.
The above CREATE TABLE statement will create 2 tables:
1. one the parent Temporal or System-Versioned Table
2. and second the History Table
–> Now, as I mentioned in Step #3 above, you have an option to specify the History Table name or not. So let’s check both the options here:
–> Option #1: Create Temporal Table [dbo].[Department] with automatically named History table:
USE [TestManDB]
GO
CREATE TABLE dbo.Department
(
DepartmentID int NOT NULL IDENTITY(1,1) PRIMARY KEY CLUSTERED,
DepartmentName varchar(50) NOT NULL,
ManagerID int NULL,
ValidFrom datetime2 GENERATED ALWAYS AS ROW START NOT NULL,
ValidTo datetime2 GENERATED ALWAYS AS ROW END NOT NULL,
PERIOD FOR SYSTEM_TIME (
ValidFrom,
ValidTo
)
)
WITH ( SYSTEM_VERSIONING = ON ); -- No History table name given here
GO
– Check the History Table nested under the Temporal Table dbo.Department . The auto-naming convention goes like this MSSQL_TemporalHistoryFor_{parent_temporal_table_object_id}:
– Let’s Check the metadata of both the tables:
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name -- Department
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.Department', 'U')
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name -- MSSQL_TemporalHistoryFor_1397580017
FROM SYS.TABLES
WHERE object_id = (
SELECT history_table_id
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.Department', 'U')
)
GO
– Output:
– To DROP both the Tables, first you need to switch OFF the System Versioning on the parent Temporal Table:
ALTER TABLE [dbo].[Department] SET ( SYSTEM_VERSIONING = OFF ) GO DROP TABLE [dbo].[Department] GO DROP TABLE [dbo].[MSSQL_TemporalHistoryFor_1525580473] GO
–> Option #2: Create Temporal Table [dbo].[Department] with a named History table [dbo].[DepartmentHistory]:
CREATE TABLE dbo.Department
(
DepartmentID int NOT NULL IDENTITY(1,1) PRIMARY KEY CLUSTERED,
DepartmentName varchar(50) NOT NULL,
ManagerID int NULL,
ValidFrom datetime2 GENERATED ALWAYS AS ROW START NOT NULL,
ValidTo datetime2 GENERATED ALWAYS AS ROW END NOT NULL,
PERIOD FOR SYSTEM_TIME (
ValidFrom,
ValidTo
)
)
WITH ( SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.DepartmentHistory) );
GO
– Check the History Table dbo.DepartmentHistory nested under the parent Temporal Table dbo.Department:
– Let’s Check the metadata of both the tables:
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.Department', 'U')
SELECT object_id, temporal_type, temporal_type_desc, history_table_id, name
FROM SYS.TABLES
WHERE object_id = OBJECT_ID('dbo.DepartmentHistory', 'U')
GO
– Output:
– Final Cleanup, As mentioned above to DROP both the Tables, first you need to switch OFF the System Versioning on the parent Temporal Table by ALTER statement:
ALTER TABLE [dbo].[Department] SET ( SYSTEM_VERSIONING = OFF ) GO DROP TABLE [dbo].[Department] GO DROP TABLE [dbo].[DepartmentHistory] GO
– Otherwise you will get following error message:
Msg 13552, Level 16, State 1, Line 82
Drop table operation failed on table ‘TestManDB.dbo.Department’ because it is not supported operation on system-versioned temporal tables.
–> Limitation of Temporal Tables:
1. Temporal querying over Linked Server is not supported.
2. History table cannot have constraints (PK, FK, Table or Column constraints).
3. INSERT and UPDATE statements cannot reference the SYSTEM_TIME period columns.
4. TRUNCATE TABLE is not supported while SYSTEM_VERSIONING is ON
5. Direct modification of the data in a history table is not permitted.
6. INSTEAD OF triggers are not permitted on either the tables.
7. Usage of Replication technologies is limited.
–> You can also check this demo in this video:
In my [next post] we will see a DEMO of Temporal data in SQL Server and how you can Time-Travel with your data and get details about point in time history without any extra effort.
In-memory enhancements and improvements in SQL Server 2016
In-Memory tables were introduced in SQL Server 2014 and were also known as Hekaton tables. I’ve written previously about In-memory tables for SQL Server 2014 and you can check in my [previous posts] to know more about these type of tables with some Hands-on examples and demos.
–> In-memory tables as new concept in SQL Server 2014 had lot of limitations compared to normal tables. But with the new release of SQL Server 2016 some limitations are addressed and other features have been added for In-Memory tables. These improvements will enable scaling to larger databases and higher throughput in order to support bigger workloads. And compared to previous version of SQL Server it will be easier to migrate your applications to and leverage the benefits of In-Memory OLTP with SQL Server 2016.
–> I have collated all the major improvements here in the table below:
* Collation Support
1. Non-BIN2 collations in index key columns
2. Non-Latin code pages for (var)char columns
3. Non-BIN2 collations for comparison and sorting in native modules
–> You can check more about In-Memory tables for SQL Server 2016 in MSDN BoL [here].
Check the above details explained in the video below:

















