SQL Basics – What are System databases | master, model, msdb, tempdb, resource
While working on SQL Server Management Studio you might have seen some default databases in System Database folder. And you would have observed the same set of databases on other machines or database servers you might have worked on or seen, as shown in the image below (red-circled):
There are total 5 System Databases, out of these you can see 4 under the System Databases folder (above image, ignore the blue circled). The 5th database is Resource which does not appear on SSMS.
–> Let’s check about these databases individually here:
1. master: database records all the system-level information for an instance of SQL Server, which includes instance-wide metadata such as logon accounts, endpoints, linked servers, and system configuration settings.
It records the existence of all other databases and the location of those database files and records the initialization information for SQL Server.
SQL Server cannot start if the master database is unavailable. If in case you would like to move the master database or have moved it and ran into issues, check this post Move Master database.
Starting SQL Server 2005 and ahead, system objects are no longer stored in the master database, instead, they are stored in the Resource database, mentioned below (5th type).
2. model: database is used as the template for all databases created on an instance of SQL Server. The entire contents of the model database, including database options, are copied to the new database. Thus, if you modify the model database, all databases created afterward will inherit those changes.
Because tempdb is created every time SQL Server is started, the model database must always exist on a SQL Server system.
3. msdb: database is used by SQL Server Agent for scheduling alerts and jobs and by other features such as SSMS, Service Broker and Database Mail.
SQL Server automatically maintains a complete online backup-and-restore history within tables in msdb, which includes the name of the party that performed the backup, the time of the backup, and the devices or files where the backup is stored.
4. tempdb: database is a workspace for holding temporary objects or intermediate result sets.
tempdb is a global resource that is available to all users connected to the instance of SQL Server and is used to hold the following:
– Temporary user objects that are explicitly created, such as: global or local temporary tables, temporary stored procedures, table variables, or cursors.
– Internal objects that are created by the SQL Server Database Engine, for example, work tables to store intermediate results for spools or sorting.
– Row versions that are generated by data modification transactions in a database that uses read-committed using row versioning isolation or snapshot isolation transactions.
– Row versions that are generated by data modification transactions for features, such as: online index operations, Multiple Active Result Sets (MARS), and AFTER triggers.
tempdb is re-created every time SQL Server is started so that the system always starts with a clean copy of the database.
Temporary tables and stored procedures are dropped automatically on disconnect, and no connections are active when the system is shut down. Thus, there is never anything in tempdb to be saved from one session of SQL Server to another.
5. Resource: database does not appear on SSMS, and is a read-only database that contains all the system objects that are included with SQL Server.
SQL Server system objects, such as sys.objects, are physically persisted in the Resource database, but they logically appear in the sys schema of every database. The Resource database does not contain user data or user metadata.
The ID of the Resource database is always 32767.
The physical file names of the Resource database are mssqlsystemresource.mdf and mssqlsystemresource.ldf, and are located in :\Program Files\Microsoft SQL Server\MSSQL.\MSSQL\Binn\ folder.
Check the Resource Database version and the last updated date:
SELECT SERVERPROPERTY('ResourceVersion') AS ResourceVersion;
GO
SELECT SERVERPROPERTY('ResourceLastUpdateDateTime') AS ResourceLastUpdateDateTime;
GO
Check my other post on Resource Database.
–> You can also query all these system databases, except the Resource database.
And you can see above first 4 database IDs, 1-4 are reserved for master, tempdb, model and msdb databases.
–> There are two more databases created while you install SQL Server with Reporting Services (SSRS). You can see these databases appearing in SSMS in the image above (circled blue):
1. reportServer: stores following:
– Items managed by a report server (reports and linked reports, shared data sources, report models, folders, resources) and all of the properties and security settings that are associated with those items.
– Subscription and schedule definitions.
– Report snapshots (which include query results) and report history.
– System properties and system-level security settings.
– Report execution log data.
– Symmetric keys and encrypted connection and credentials for report data sources.
2. reportServerTempdb: Each report server database uses a related temporary database to store session and execution data, cached reports, and work tables that are generated by the report server. Background server processes will periodically remove older and unused items from the tables in the temporary database.
SQL DBA – Configure multiple TempDB Database Files while installing SQL Server 2016 (new feature)
SQL Server 2016 allows you to scale up your database with Enhanced Database Caching, using support for multiple TempDB files per instance for multi-core environments. This reduces metadata- and allocation contention for TempDB workloads, improving performance and scalability.
Now you can configure the number of TempDB Database Files during the installation of a new Instance. While installation process in the Database Engine Configuration page you will see an extra option to set number of TempDB files.
Or, you can specify the number of files by using the new command line parameter: /SQLTEMPDBFILECOUNT
setup.exe /Q /ACTION="INSTALL" /IACCEPTSQLSERVERLICENSETERMS /FEATURES="SqlEngine" /INSTANCENAME="SQL15" .. /SQLTEMPDBDIR="D:\tempdb" /SQLTEMPDBFILECOUNT="4"
While installing via UI the label besides the Input Control below mentions: “The default value is 8 or the number of cores, whichever is lower. This value can be increased up to the number of cores”.
Update: Now with CTP 2.4 release the installer provides a separate tab for tempdb files configuration.
So, in my [C:\Program Files\Microsoft SQL Server\MSSQL13.SQLSERVER2016\MSSQL\DATA\] folder I could see 8 data files, with 1 log file:
Implementing “Row Level Security” (RLS) with “Filter Predicate” in SQL Server 2016 – Part 1
For an Organization its Data is the most important thing, and the Org’s IT department takes lot of measures to make sure data does not fall in wrong hands. The DBA’s and Application programmers setup different layers of security on top of data so that the user is only able to see the Filtered data/rows from a table that he has access to. DB programmers normally create abstracted Views or Stored Procedures with complex logic on top of the Tables by using other master-mapping tables that contains the user-data access key. Sometimes these security logics are not upto the standards, and may have various versions on a database and thus it becomes difficult to track, update and make them fail-proof. So, with SQL Server 2016 the new Row Level Security feature is going to handle this Out of the Box and as a Standard.
RLS or Row Level Security is a feature that enables fine grained control over access to rows in a table, allowing you to easily control which users can access which data with complete transparency to the application.
With this feature rows are filtered based on the Execution Context of the Query, rather than the current user access rights. A secure logic can be created to determine which user can see which rows and restrict any kind of data (rows) by designing a flexible and robust Security policy for a table.
–> To setup Row Level Security (RLS) on a particular table it needs only these simple steps:
1. Create Users and Grant Read/SELECT access for a Particular table.
2. Create a new Inline Table-Valued Function that will contain the Filter Predicate for that table. This Function Predicate can be a sophisticated business logic with multiple JOINs or a simple WHERE ManagerCode = ‘userHR’.
3. Create a new Security Policy for this table and add the above Function (Filter) Predicate to it.
Please note: that these Functions & Security Policies should be unique for a table. So to create RLS for an another table, you will need to create separate Function & Security Policy.
–> Step 1.a. Let’s create some test accounts: I will create three users for:
1. The CEO, over-all admin of the company data.
2. HR department head
3. Finance department head
CREATE USER userCEO WITHOUT LOGIN; GO CREATE USER userHR WITHOUT LOGIN; GO CREATE USER userFin WITHOUT LOGIN; GO
–> Create a sample table [dbo].[Employee]: with a self-referencing Manager ID column.
CREATE TABLE dbo.Employees ( [EmpCode] NVARCHAR(50), -- Employee ID [EmpName] NVARCHAR(250), -- Employee/Manager Full Name [Salary] MONEY, -- Fictious Salary [MgrCode] NVARCHAR(50) -- Manager ID ); GO
-> Now insert some test records:
-- Top Boss CEO
INSERT INTO dbo.Employees VALUES ('userCEO' , 'CEO Top Boss' , 800, NULL)
-- Next 2 levels under CEO
INSERT INTO dbo.Employees VALUES ('userHR' , 'HR User' , 700, 'userCEO');
INSERT INTO dbo.Employees VALUES ('userFin' , 'Finance User' , 600, 'userCEO');
-- Employees under Kevin
INSERT INTO dbo.Employees VALUES ('manojp' , 'Manoj Pandey' , 100, 'userHR');
INSERT INTO dbo.Employees VALUES ('saurabhs', 'Saurabh Sharma', 400, 'userHR');
INSERT INTO dbo.Employees VALUES ('deepakb' , 'Deepak Biswal' , 500, 'userHR');
-- Employees under Amy
INSERT INTO dbo.Employees VALUES ('keshavk' , 'Keshav K' , 200, 'userFin');
INSERT INTO dbo.Employees VALUES ('viveks' , 'Vivek S' , 300, 'userFin');
GO
–> Let’s check the records before applying “Row Level Security”:
SELECT * FROM dbo.Employees; -- 8 rows GO
As a normal SEELCT and without RLS, it just ignores my Execution Context and execute the Query and return all the 8 rows.
–> The Traditional way to setup the Row Level Security till now was as follows (a simple example):
-- Stored Procedure with User-Name passed as parameter: CREATE PROCEDURE dbo.uspGetEmployeeDetails (@userAccess NVARCHAR(50)) AS BEGIN SELECT * FROM dbo.Employees WHERE [MgrCode] = @userAccess OR @userAccess = 'userCEO'; -- CEO, the admin should see all rows END GO -- Execute the SP with different parameter values: EXEC dbo.uspGetEmployeeDetails @userAccess = 'userHR' -- only 3 rows GO EXEC dbo.uspGetEmployeeDetails @userAccess = 'userFin' -- only 2 rows GO EXEC dbo.uspGetEmployeeDetails @userAccess = 'userCEO' -- all 8 rows GO
The above method is prone to issues, like SQL Injection and any other user can apply other user’s User-Name and get the information that he is not allowed to see. With this type of method you have to apply another security layer at the application level so whenever a user executes the SP it gets executed with the same user’s User-Name.
–> The new Row Level Security feature let you:
– apply this security at the database level
– and there is no need to apply the WHERE clause filter for the User-Name.
This makes the security system more reliable and robust by reducing the surface area of your security system.
–> Step 1.b. Grant Read/SELECT access on the dbo.Employee table to all 3 users:
GRANT SELECT ON dbo.Employees TO userCEO; GO GRANT SELECT ON dbo.Employees TO userHR; GO GRANT SELECT ON dbo.Employees TO userFin; GO
–> Step 2. Let’s create an Inline Table-Valued Function to write our Filter logic:
CREATE FUNCTION dbo.fn_SecurityPredicateEmployee(@mgrCode AS sysname)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN SELECT 1 AS fn_SecurityPredicateEmployee_result
-- Predicate logic
WHERE @mgrCode = USER_NAME()
OR USER_NAME() = 'userCEO'; -- CEO, the admin should see all rows
GO
This function returns value 1 when:
– a row in the MgrCode (i.e. the Manager Code) column is the same as the user executing the query (@@mgrCode = USER_NAME())
– or if the user executing the query is the Top Boss user (USER_NAME() = ‘userCEO’)
–> Step 3. Create a security policy adding the function as a filter predicate:
CREATE SECURITY POLICY ManagerFilter ADD FILTER PREDICATE dbo.fn_SecurityPredicateEmployee(MgrCode) -- Filter Column from dbo.Employee table ON dbo.Employees WITH (STATE = ON); -- The state must be set to ON to enable the policy. GO
The above Security Policy takes the Filter Predicate Logic from the associated Function and applies it to the Query as a WHERE clause.
–> Now let’s again check the records after applying “Row Level Security”:
SELECT * FROM dbo.Employees; -- 0 rows, GO
The simple “SELECT *” statement will fetch me zero rows after applying RLS, as my user ID is not configured to have access to any of those rows & Table.
–> And if you check in the Execution Plan of above SELECT statement without WHERE clause, it will show you the Filter Predicate that is added by the Security Policy defined in Step #3 for applying RLS on this table, which looks like this:
[TestManDB].[dbo].[Employee].[MgrCode]=user_name()
OR user_name()=N’userCEO’
–> Let’s check the 3 users we created and provided them customized access to the dbo.Employee table and rows in it:
-- Execute as our immediate boss userHR (3 rows): EXECUTE AS USER = 'userHR'; SELECT * FROM dbo.Employees; -- 3 rows REVERT; GO -- Execute as our immediate boss userFin: EXECUTE AS USER = 'userFin'; SELECT * FROM dbo.Employees; -- 2 rows REVERT; GO -- Execute as our Top boss userCEO (8): EXECUTE AS USER = 'userCEO'; SELECT * FROM dbo.Employees; -- 8 rows REVERT; GO
–> The results of the above 3 SELECTs looks like this:
So, as you can see the three users we created resulted in different results:
– The HR & Finance users got just 3 & 2 rows, respectively.
– But the admin CEO user got all the 8 rows.
–> Thus, by using RLS Filter Predicate feature in SQL Server 2016 you can create your own customized Security by creating an Inline Table-Valued Function that is linked to the Security Policy for your Table.
–> Check Block Predicates with RLS in my [next post, Part 2].
–> Final Cleanup
DROP SECURITY POLICY [dbo].[ManagerFilter] GO DROP FUNCTION [dbo].[fn_SecurityPredicateEmployee] GO DROP TABLE [dbo].[Employee] GO DROP PROCEDURE dbo.uspGetEmployeeDetails GO
Check the same demo on YouTube:
Pass an array of multiple values with a single Parameter in a Stored Procedure – SQL Server
This post comes from an old discussion on MSDN T-SQL Forum[link], where someone asked about the same topic.
Check my reply there and here with this post I’m going to collate all the different methods I’ve blog previously that can be used for the same purpose.
Stored Procedures accept only a fixed and pre-defined number of parameters thus there is a limitation you cannot provide dynamic number of params, like you can do with other languages, like C, C#, Java, etc. In these languages there is concept of Arrays which you can pass in a method/function, but in SQL there are no array variables and it does not have any datatype that support arrays. Thus if you have to provide multiple values to any parameter you cannot do it directly, though there are some workarounds.
–> Here are some of the methods or workarounds by which we can pass multiple values as a single Parameter in a Stored Procedure or a Function:
Method #1 – Passing a CSV: list of strings as a parameter to a (N)VARCHAR datatype parameter, then splitting/parsing it inside the SP or UDF, check here.
Method #2 – Passing an XML: string as an XML datatype parameter. We will need to parse the XML inside the SP, check here.
Method #3 – Using a temp table: inside an SP which is created outside just before its execution. Here there is no need to pass any parameter with the SP, check here.
Method #4 – Using TVPs: With SQL Server 2008 and above you can create TVPs or Table Valued Parameters and declare them by using user-defined table types. These TVPs can then be used to send multiple rows of data to SPs or UDFs, without creating a temp table or multiple parameters, check here.
Method #5 – Passing a JSON string: as a NVARCHAR datatype parameter. We will need to parse the JSON inside the SP, check here.
Thanks a lot, please provide your valuable comments and suggestions on this topic.
>> Check & Subscribe my [YouTube videos] on SQL Server.
Bug with ISNUMERIC() – MSDN TSQL forum
–> Question:
Has anyone seen this SQL bug before?
If you use the IsNumeric function with 12345 and replace number 3 with an “e” and no other letter, SQL still thinks it’s numeric.
If ISNUMERIC('12e45') = 1
print 'Is Numeric'
else
Print 'No'
–> My Answer:
Yes, “12e45” is still numeric, notice the small ‘e’ which is a symbol for exponent for representing a big number.
But yes, NUMERIC() function does results incorrect results for some values, like:
SELECT
ISNUMERIC('123') as '123'
,ISNUMERIC('.') as '.' --Period
,ISNUMERIC(',') as ',' --Comma
SELECT
ISNUMERIC('123') as '123'
,ISNUMERIC('-') as '-'
,ISNUMERIC('+') as '+'
,ISNUMERIC('$') as '$'
,ISNUMERIC('\') as '\'
… gives you 1 for all these non-numeric values.
After release of SQL Server 2012 and ahead you must be using TRY_PARSE() instead of ISNUMERIC().
Check my blog post on how to use this – Use new TRY_PARSE() instead of ISNUMERIC() | SQL Server 2012
–> Answer by CELKO:
The use of E or e for floating point numbers started in the 1950’s with FORTRAN and later in Algol. You wrote “twelve times ten to the forty-fifth power” in the standard notation. The only difference is that some languages have to start with a digit and some can start with the E.
Please look up how floating numbers are displayed. This has been true for over 60 years! The E or e is for “exponent” in the notation that goes back to FORTRAN I.
–> Answer by Erland Sommarskog:
In addition to other posts, on SQL2012 or later, you can use try_convert to check if the number is convertible to a specific data type:
SELECT
CASE WHEN try_convert(int, @str) IS NOT NULL
THEN 'Converts'
ELSE 'Not convertible'
END
–> Answer by Dan Guzman:
ISNUMERIC isn’t particularly useful for the common use case of checking for a string containing only the digits 0 through 9. Consider a CASE expression instead to get the desired behavior:
CASE WHEN @value LIKE '%[^0-9]%' OR @value = '' THEN 0 ELSE 1 END
Ref link.













