Archive
SQL Basics – UDF | User Defined Functions – Scalar, Table Valued (TVF), MultiStatement (MTVF)
UDF or User Defined Functions are a set or batch of code where one can apply any SQL logic and return a single scalar value or a record set.
According to MS BOL UDFs are the subroutines made up of one or more Transact-SQL statements that can be used to encapsulate code for reuse. These reusable subroutines can be used as:
– In TSQL SELECT statements at column level.
– To create parametrized view or improve the functionality of in indexed view.
– To define a column and CHECK constraints while creating a table.
– To replace a stored procedures and views.
– Join complex logic with a table where a stored procedure fails.
– Faster execution like Stored procedures, reduce compliation cost by caching the execution query plans.
Apart from the benefits UDF’s has certain limitations:
– Can not modify any database objects, limited to update table variables only.
– Can not contain the new OUTPUT clause.
– Can only call extended stored procedures, no other procedures.
– Can not define TRY-CATCH block.
– Some built-in functions are not allowed here, like:GETDATE(), because GETDATE is non-deterministic as its value changes every time it is called. On the other hand DATEADD() is allowed as it is deterministic, because it will return same result when called with same argument values.
A UDF can take 0 or upto 1024 parameters and returns either a scalar value or a table record set depending on its type.
SQL Server supports mainly 3 types of UDFs:
1. Scalar function
2. Inline table-valued function
3. Multistatement table-valued function
1. Scalar function: Returns a single value of any datatype except text, ntext, image, cursor & timestamp.
-- Example: --// Create Scalar UDF [dbo].[ufn_GetContactOrders] SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO CREATE FUNCTION [dbo].[ufn_GetContactOrders](@ContactID int) RETURNS varchar(500) AS BEGIN DECLARE @Orders varchar(500) SELECT @Orders = COALESCE(@Orders + ', ', '') + CAST(SalesOrderID as varchar(10)) FROM Sales.SalesOrderHeader WHERE ContactID = @ContactID RETURN (@Orders) END --// Usage: -- Used at COLUMN level with SELECT SELECT ContactID, dbo.ufn_GetContactOrders(ContactID) FROM Person.Contact WHERE ContactID between 100 and 105 -- Output below -- Used while defining a computed column while creating a table. CREATE TABLE tempCustOrders (CustID int, Orders as (dbo.ufn_GetContactOrders(CustID))) INSERT INTO tempCustOrders (CustID) SELECT ContactID FROM Person.Contact WHERE ContactID between 100 and 105 SELECT * FROM tempCustOrders -- Output below DROP TABLE tempCustOrders
Output of both the selects above:
ContactID OrdersCSV
100 51702, 57021, 63139, 69398
101 47431, 48369, 49528, 50744, 53589, 59017, 65279, 71899
102 43874, 44519, 46989, 48013, 49130, 50274, 51807, 57113, 63162, 69495
103 43691, 44315, 45072, 45811, 46663, 47715, 48787, 49887, 51144, 55310, 61247, 67318
104 43866, 44511, 45295, 46052, 46973, 47998, 49112, 50215, 51723, 57109, 63158, 69420
105 NULL
Note: If this was a temp(#) table then the function also needs to be created in tempdb, cause the temp table belongs to tempdb. The tables in function should also have the database name prefixed, i.e. [AdventureWorks].[Sales].[SalesOrderHeader]
2. Inline table-valued function: Returns a table i.e. a record-set. The function body contains just a single TSQL statement, which results to a record-set and is returned from here.
-- Example: --// Create Inline table-valued UDF [dbo].[ufn_itv_GetContactSales] SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO CREATE FUNCTION [dbo].[ufn_itv_GetContactSales](@ContactID int) RETURNS TABLE AS RETURN ( SELECT h.[ContactID], h.[SalesOrderID], p.[ProductID], p.[Name], h.[OrderDate], h.[DueDate], h.[ShipDate], h.[TotalDue], h.[Status], h.[SalesPersonID] FROM Sales.SalesOrderHeader AS h JOIN Sales.SalesOrderDetail AS d ON d.SalesOrderID = h.SalesOrderID JOIN Production.Product AS p ON p.ProductID = d.ProductID WHERE ContactID = @ContactID ) --// Usage: SELECT * FROM ufn_itv_GetContactSales(100)
3. Multistatement table-valued function: Also returns a table (record-set) but can contain multiple TSQL statements or scripts and is defined in BEGIN END block. The final set of rows are then returned from here.
-- Example: --// Create Multistatement table-valued UDF [dbo].[ufn_mtv_GetContactSales] SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO CREATE FUNCTION [dbo].[ufn_mtv_GetContactSales](@ContactID int) RETURNS @retSalesInfo TABLE ( [ContactID] INT NOT NULL, [SalesOrderID] INT NULL, [ProductID] INT NULL, [Name] NVARCHAR(50) NULL, [OrderDate] DATETIME NULL, [DueDate] DATETIME NULL, [ShipDate] DATETIME NULL, [TotalDue] MONEY NULL, [Status] TINYINT NULL, [SalesPersonID] INT NULL) AS BEGIN IF @ContactID IS NOT NULL BEGIN INSERT @retSalesInfo SELECT h.[ContactID], h.[SalesOrderID], p.[ProductID], p.[Name], h.[OrderDate], h.[DueDate], h.[ShipDate], h.[TotalDue], h.[Status], h.[SalesPersonID] FROM Sales.SalesOrderHeader AS h JOIN Sales.SalesOrderDetail AS d ON d.SalesOrderID = h.SalesOrderID JOIN Production.Product AS p ON p.ProductID = d.ProductID WHERE ContactID = @ContactID END -- Return the recordsets RETURN END --// Usage: SELECT * FROM ufn_mtv_GetContactSales(100)
— Output:
More MSDN & MS BOL links:
http://msdn.microsoft.com/en-us/library/aa175085(SQL.80).aspx
http://msdn.microsoft.com/en-us/library/aa214363(SQL.80).aspx
http://msdn.microsoft.com/en-us/library/ms186755.aspx
http://msdn.microsoft.com/en-us/library/ms191007.aspx
http://msdn.microsoft.com/en-us/magazine/cc164062.aspx
SET ANSI_NULLS, QUOTED_IDENTIFIER, ANSI_PADDING – SQL Server
You have to create a Stored-Procedure and you need a template. You expand the Object Explorer go to Database -> Programmability -> “Stored Procedure” -> (right click) -> select “New Stored Procedure”.
A new editor opens up with lot of commented & flawless code. There you see some predefined statements.
Sometimes while working with Stored Procedures and other SQL Scripts a those few statements makes you confusing and clueless. Most of the time people remove them or just ignore them, but one should know why are they placed there. The 2 mains SET statements that are placed by default are described below with examples:
–// SET ANSI_NULLS (http://msdn.microsoft.com/en-us/library/ms188048.aspx)
Specifies ISO compliant behavior of the Equals (=) and Not Equal To (<>) comparison operators when they are used with null values.
Syntax: SET ANSI_NULLS { ON | OFF }
create table #tempTable (sn int, ename varchar(50)) insert into #tempTable select 1, 'Manoj' UNION ALL select 2, 'Pankaj' UNION ALL select 3, NULL UNION ALL select 4, 'Lokesh' UNION ALL select 5, 'Gopal' SET ANSI_NULLS ON select * from #tempTable where ename is NULL -- (1 row(s) affected)
sn ename
3 NULL
select * from #tempTable where ename = NULL -- (0 row(s) affected) select * from #tempTable where ename <> NULL -- (0 row(s) affected) SET ANSI_NULLS OFF select * from #tempTable where ename is NULL -- (1 row(s) affected) select * from #tempTable where ename = NULL -- (1 row(s) affected)
sn ename
3 NULL
select * from #tempTable where ename is not NULL -- (4 row(s) affected) select * from #tempTable where ename <> NULL -- (4 row(s) affected)
sn ename
1 Manoj
2 Pankaj
4 Lokesh
5 Gopal
drop table #tempTable
–// SET QUOTED_IDENTIFIER (http://msdn.microsoft.com/en-us/library/ms174393.aspx)
Causes SQL Server to follow the ISO rules regarding quotation mark delimiting identifiers and literal strings. Identifiers delimited by double quotation marks can be either Transact-SQL reserved keywords or can contain characters not generally allowed by the Transact-SQL syntax rules for identifiers.
When SET QUOTED_IDENTIFIER is ON, identifiers can be delimited by double quotation marks, and literals must be delimited by single quotation marks.
When SET QUOTED_IDENTIFIER is OFF, identifiers cannot be quoted and must follow all Transact-SQL rules for identifiers.
Syntax: SET QUOTED_IDENTIFIER { ON | OFF }
-- Example 1 SET QUOTED_IDENTIFIER ON create table "#tempTable" (sn int, ename varchar(50)) -- Command(s) completed successfully. SET QUOTED_IDENTIFIER OFF create table "#tempTable" (sn int, ename varchar(50)) -- Incorrect syntax near '#tempTable'. drop table #tempTable -- Example 2 SET QUOTED_IDENTIFIER ON select "'My Name is Manoj'" as Col1, "Let's play" as Col2 -- Invalid column name ''My Name is Manoj''. Invalid column name 'Let's play'. SET QUOTED_IDENTIFIER OFF select "'My Name is Manoj'" as Col1, "Let's play" as Col2 -- (1 row(s) affected)
Col1 Col2
'My Name is Manoj' Let's play
-- Example 3 SET QUOTED_IDENTIFIER ON select '''My Name is Manoj''' as Col1, 'Let''s play' as Col2 -- (1 row(s) affected)
Col1 Col2
'My Name is Manoj' Let's play
SET QUOTED_IDENTIFIER OFF select '''My Name is Manoj''' as Col1, 'Let''s play' as Col2 -- (1 row(s) affected)
Col1 Col2
'My Name is Manoj' Let's play
–// SET ANSI_PADDING (http://msdn.microsoft.com/en-us/library/ms187403.aspx)
Controls the way the column stores values shorter than the defined size of the column, and the way the column stores values that have trailing blanks in char, varchar, binary, and varbinary data.
Syntax: SET ANSI_PADDING { ON | OFF }
-- Example 1 SET ANSI_PADDING ON create table #tempTable1 (charcol char(20), varcharcol varchar(20)) insert into #tempTable1 select 'Manoj', 'Manoj' UNION ALL select 'Pandey', 'Pandey' select '['+charcol+']' as charcol, '['+varcharcol+']' as varcharcol from #tempTable1
charcol varcharcol
[Manoj ] [Manoj]
[Pandey ] [Pandey]
-- Example 2 SET ANSI_PADDING OFF create table #tempTable2 (charcol char(20), varcharcol varchar(20)) insert into #tempTable2 select 'Manoj', 'Manoj' UNION ALL select 'Pandey', 'Pandey' select '['+charcol+']' as charcol, '['+varcharcol+']' as varcharcol from #tempTable2
charcol varcharcol
[Manoj] [Manoj]
[Pandey] [Pandey]
drop table #tempTable1 drop table #tempTable2
Other SET statements MSDN BOL: http://msdn.microsoft.com/en-us/library/ms190356.aspx
TSQL Interview Questions – Part 1
These are the following common question that I faced in various TSQL interviews.
… I’ll be updating this post with more questions.
1. Difference between SQL Server 2000 & SQL Server 2005 features, or new features of 2005 vs 2000.
– Ranking functions (ROW_NUMBER, RANK, DENSE_RANK, NTILE), [YouTube]
– Exception handling (TRY-CATCH block)
– CTE (Common Table Expressions)
– PIVOT, UNPOVIT
– CUBE, ROLUP & GROUPING SET
– SYNOMYMS
2. What tools do you use for performance tuning?
Query Analyzer, Profiler, Index Wizard, Performance Monitor
Link: http://www.sqlteam.com/article/sql-server-2000-performance-tuning-tools
3. What is SQL Profiler? What it does? What template do you use?
More Info: http://www.extremeexperts.com/SQL/Articles/TraceTemplate.aspx
http://msdn.microsoft.com/en-us/library/ms190176.aspx
4. How can you execute an SQL query from command prompt?
OSQL & SQLCMD
More Info: http://msdn.microsoft.com/en-us/library/ms162773.aspx
http://blog.sqlauthority.com/2009/01/05/sql-server-sqlcmd-vs-osql-basic-comparison/
http://www.databasejournal.com/features/mssql/article.php/3654176/SQL-Server-2005-Command-Line-Tool-SQLCMD–Part-I.htm
5. Difference between DELETE & TRUNCATE statement? Which statement can be Rollbacked?
– With DELETE we can provide conditional WHERE clause to remove/delete specific rows, which is not possible with TRUNCATE.
– TRUNCATE is faster than DELETE as Delete keeps log of each row it deletes in transaction logs, but truncate keeps log of only de-allocated pages in transaction logs.
– Both statements can be rolled backed if provided in a transaction (BEGIN TRANS). If not then none of them can be rollbacked.
– DELETE is DML just like INSERT, UPDATE, but TRANCATE is DDL, just like CREATE, ALTER, DROP
More differences on Delete & Truncate: http://wp.me/p12rgl-7 | YouTube.
6. What are extended stored procedures? Can you create your own extended stored-proc?
More Info: http://msdn.microsoft.com/en-us/library/ms175200.aspx
http://msdn.microsoft.com/en-us/library/ms164627.aspx
7. How can you execute a DOS command from SQL or through SQL query by using xp_cmdshell?
exec xp_cmdshell 'dir c:\*.exe'
More Info: http://msdn.microsoft.com/en-us/library/aa260689%28SQL.80%29.aspx
http://msdn.microsoft.com/en-us/library/ms175046.aspx
8. How will you insert result set of the above proc in a table?
insert into exec xp_cmdshell 'dir c:\*.exe'
9. What are Cursors and their types? What type do you use most and which one is fast?
FORWARD-ONLY, FAST-FORWARD or READ-ONLY cursors.
Fastest to slowest: Dynamic, Static, and Keyset.
More Info: http://msdn.microsoft.com/en-us/library/ms180169.aspx
Check how to use Cursors optimally: https://sqlwithmanoj.com/2011/02/07/avoid-cursors-or-use-them-optimally/
10. Why you should not use a cursor? What are its alternatives?
Alternatives: while loops with temp tables, derived tables, correlated sub-queries, CASE stmt
More Info: http://www.sql-server-performance.com/tips/cursors_p1.aspx
http://sqlserverpedia.com/wiki/Cursor_Performance_Issues
http://searchsqlserver.techtarget.com/feature/Part-3-Cursor-disadvantages
11. Difference between LEFT JOIN with WHERE clause & LEFT JOIN with no WHERE clause.
OUTER LEFT/RIGHT JOIN with WHERE clause can act like an INNER JOIN if not used wisely or logically.
JOINs and its types: https://sqlwithmanoj.com/2009/03/12/sql-server-joins-and-types/
12. How will you migrate an SSIS package from Development to Production environment?
Do not include db connections and file paths in your workflow, instead create configuration files. This will help in deploying the pkg created in DEV server to Testing and finally to the PROD environment.
More Info: http://msdn.microsoft.com/en-us/library/cc966389.aspx
http://www.wpconfig.com/2010/03/26/ssis-package-configurations/
13. Multiple ways to execute a dynamic query.
EXEC sp_executesql, EXECUTE()
More Info: https://sqlwithmanoj.com/2011/03/22/execute-or-exec-vs-sp_executesql/
14. Difference between COALESCE() & ISNULL()
More Info: https://sqlwithmanoj.com/2010/12/23/isnull-vs-coalesce/
15. Difference between Temporary Table and Table Variable.
Check here: http://wp.me/p12rgl-r | YouTube
16. What should be the ideal combination with IN & UNION (ALL) in terms of performance?
a. SELECT *FROM
WHERE
IN (SELECT… UNION SELECT…)
OR
b. SELECT * FROM
WHERE
IN (SELECT… UNION ALL SELECT…)
17. What is an IDENTITY column and its usage in INSERT statements?
IDENTITY column can be used with a tables column to make it auto incremental, or a surrogate key.
Check my all blog posts related to IDENTITY: https://sqlwithmanoj.com/tag/identity/
18. Can you create a Primary key without clustered index?
Yes, you can create a Primary Key with a Non Clustered Index. But by default creation of PK automatically creates a clustered index upon the table, check here: https://sqlwithmanoj.com/2015/05/24/sql-myth-primary-key-pk-always-creates-clustered-index/
19. There are two tables one Master and another Feed table, both with 2 columns: ID & Price. Feed table gets truncated and re-populated every day.
Master Table Feed Table ID, Price ID, Price 1 100 1 200 3 200 2 250 5 300 4 500 6 400 6 750 7 500 7 800
Create a job with an optimal script that will update the Master table by the Feed table.
Use MERGE statement to INSERT any new record form the Feed table, update any existing record in Master table and DELETE existing records in Master table that is not present in Feed table.
20. What are CUBE & ROLLUP sets?
CUBE & ROLLUP are the grouping sets used with GROUP BY clause and are very helpful in creating reports.
More Info: https://sqlwithmanoj.com/tag/cube/
21. What new indexes are introduced in SQL Server 2005 in comparison to 2000?
– Spatial
– XML
More Info: http://msdn.microsoft.com/en-us/library/ms175049.aspx
22. What are XML indexes, what is their use?
More Info: http://msdn.microsoft.com/en-us/library/ms345121%28SQL.90%29.aspx
23. How many types of functions (UDF) are there in SQL Server? What are inline functions?
– Scalar functions
– Inline Table-valued functions
– Multi-statement Table-valued functions
More Info: https://sqlwithmanoj.com/2010/12/11/udf-user-defined-functions/
http://msdn.microsoft.com/en-us/library/ms189593.aspx
24. How will you handle exceptions in SQL Server programming?
By using TRY-CATCH constructs, putting our SQL statements/scripts inside the TRY block and error handling in the CATCH block, https://sqlwithmanoj.com/2010/06/16/try-catch-exception-handling/.
25. How would you send an e-mail form SQL Server?
Configure Database mail here: https://sqlwithmanoj.com/2010/09/29/database-mail-setup-sql-server-2005/.
… more questions on next post [Part-2].
SQL DBA – Querying SQL Server Metadata, INFORMATION_SCHEMA, sysObjects, System Catalog, etc
While writing complex code & business logic in my scrits & stored-procdedures most of the time I wonder…
– What table does a particular column belongs to?
– What all Stored Procedures effect a particular table?
– How can I see what particular constraint does my tables have?
– What all Foreign Keys defined in table’s columns are linked to?
These and many more questions can be answered by querying the SQL Server System Catalog and metadata that SQL Server manages very beautifully.
The SQL Server system catalogs is a set of views that show metadata that describes the objects in an instance of SQL Server. Metadata is data that describes the attributes of objects in a system. SQL Server-based applications can access the information in the system catalogs by using some of the following:
-> Information Schema, views to quickly retrieve metadata
-> Catalog Views, recommened by MS.
–// Information Schema Views: They present the catalog information in a format that is independent of any catalog table implementation and therefore are not affected by changes in the underlying catalog tables.
-- INFORMATION_SCHEMA.TABLES select * from INFORMATION_SCHEMA.TABLES
TABLE_CATALOG TABLE_SCHEMA TABLE_NAME TABLE_TYPE
AdventureWorks Production ProductProductPhoto BASE TABLE
AdventureWorks Sales StoreContact BASE TABLE
AdventureWorks Person Address BASE TABLE
AdventureWorks Production ProductReview BASE TABLE
AdventureWorks Production TransactionHistory BASE TABLE
AdventureWorks Person AddressType BASE TABLE
select distinct TABLE_SCHEMA from INFORMATION_SCHEMA.TABLES
TABLE_SCHEMA
dbo
HumanResources
Person
Production
Purchasing
Sales
select distinct TABLE_TYPE from INFORMATION_SCHEMA.TABLES
TABLE_TYPE
BASE TABLE
VIEW
-- INFORMATION_SCHEMA.COLUMNS select * from INFORMATION_SCHEMA.COLUMNS select TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, DATA_TYPE, CHARACTER_MAXIMUM_LENGTH, CHARACTER_OCTET_LENGTH, NUMERIC_PRECISION, NUMERIC_PRECISION_RADIX, NUMERIC_SCALE from INFORMATION_SCHEMA.COLUMNS select * from INFORMATION_SCHEMA.COLUMNS where COLUMN_NAME = 'ContactID' -- INFORMATION_SCHEMA.VIEWS select * from INFORMATION_SCHEMA.VIEWS select * from INFORMATION_SCHEMA.VIEW_TABLE_USAGE select * from INFORMATION_SCHEMA.VIEW_COLUMN_USAGE -- INFORMATION_SCHEMA.ROUTINES select * from INFORMATION_SCHEMA.ROUTINES select distinct ROUTINE_SCHEMA from INFORMATION_SCHEMA.ROUTINES
ROUTINE_SCHEMA
dbo
HumanResources
select distinct ROUTINE_TYPE from INFORMATION_SCHEMA.ROUTINES
ROUTINE_TYPE
FUNCTION
PROCEDURE
select ROUTINE_SCHEMA, ROUTINE_NAME, ROUTINE_TYPE, DATA_TYPE, ROUTINE_BODY, ROUTINE_DEFINITION, IS_DETERMINISTIC, SQL_DATA_ACCESS, CREATED, LAST_ALTERED from INFORMATION_SCHEMA.ROUTINES select * from INFORMATION_SCHEMA.ROUTINES where ROUTINE_DEFINITION like '%ContactID%' select * from INFORMATION_SCHEMA.ROUTINE_COLUMNS
–// Catalog Views: provide access to metadata that is stored in every database on the server.
-- The following SQL statement will pull all Stored-Procedures & Functions in the Database. select c.id, object_name(c.id) as obj_name_c, o.name as obj_name_o, o.xtype, c.text, o.crdate, o.refdate from sys.syscomments c join sys.sysobjects o on o.id = c.id where xtype in ('P', 'FN') -- This following SQL statement will pull all Objects information with in the Database. select c.id, object_name(c.id) as obj_name_c, -- o.name case xtype when 'C' then 'Check Constraint' when 'D' then 'Default Constraint' when 'F' then 'Foreign Key Constraint' when 'L' then 'Log' when 'P' then 'Stored Procedure' when 'PK' then 'Primary Key Constraint' when 'RF' then 'Replication Filter Procedure' when 'S' then 'System Table' when 'TR' then 'Trigger' when 'U' then 'User Table' when 'UQ' then 'Unique Constraint' when 'V' then 'View' when 'X' then 'Extended Procedure' when 'FN' then 'User Defined Function' else 'N/A' end as obj_type, c.text, o.crdate, o.refdate from sys.syscomments c join sys.sysobjects o on o.id = c.id -- Pulls Foreign Key and its links select fk.name as fk_name, fk.object_id, object_name(fk.parent_object_id) as table_name, col_name(fc.parent_object_id, fc.parent_column_id) as constraint_col_name, object_name(fk.referenced_object_id) as referenced_table, COL_NAME(fc.referenced_object_id, fc.referenced_column_id) AS referenced_col_name, fk.delete_referential_action_desc from sys.foreign_keys fk join sys.foreign_key_columns fc on fk.object_id = fc.constraint_object_id
Note: All examples are executed on SQL Server 2005 AdventureWorks DB.
More info from MSDN BOL: http://msdn.microsoft.com/en-us/library/ms189082(v=SQL.90).aspx
MSDN FAQs: http://msdn.microsoft.com/en-us/library/ms345522.aspx
MS SQL Server (TSQL) Interview Questions
Here is a collection of MS SQL Server (TSQL) Server Interview Questions. Some I faced in interviews and some common questions:
1. TSQL Interview Questions – Part 1
2. TSQL Interview Questions – Part 2
3. TSQL Interview Questions – Part 3
4. TSQL Interview Questions – Part 4
5. TSQL Interview Questions – Part 5
Comments, suggestions & more question appreciated.