Archive

Posts Tagged ‘RUNNING TOTALS’

Enhanced OVER() Clause in SQL Server 2012 – Part2

January 2, 2013 5 comments

First of all I would like to wish my readers a very Happy and Prosperous New Year – 2013!!!

This is my 1st post in this new year and I will try to come up with more informative and interesting topics related to SQL Server.

Coming back to the topic, sometime back in Part-1 [link] I discussed about how to use both PARTITION BY & ORDER BY clause within OVER() clause when using it with AGGREGATE functions, like SUM().

Here in this post we will see other features that are added to the OVER() clause.

ROWS/RANGE are new the 2 new clauses that you can use inside the OVER() clause to limit the records within the Partition by specifying start and end within the partition. They use a new set of keywords to define the boundaries within the partition, i.e. PREECEDING, FOLLOWING mixed with UNBOUNDED or CURRENT, or number of rows. The PRECEDING and FOLLOWING rows are defined based on the ordering in the ORDER BY clause of the query.

#1. Using ROWS/RANGE UNBOUNDED PRECEDING with OVER() clause:
Let’s check with 1st example where I want to calculate Cumulative Totals or Running Totals at each row. This total is calculated by the SUM of current and all previous rows.
We will use UNBOUND PRECEDING option here, which means that the window starts from the 1st row till the current row.

USE [AdventureWorks2012]
GO

-- To Calculate Cumulative SUM or Running Totals:
;WITH CTE AS (
	SELECT BusinessEntityID AS SalesPersonID, CAST([Rate] AS DECIMAL(10,0))AS Salary, [ModifiedDate] AS SalDate
	FROM [HumanResources].[EmployeePayHistory]
	WHERE BusinessEntityID <= 10
	)
SELECT SalesPersonID, SalDate, Salary
   ,SUM(Salary) OVER (ORDER BY SalesPersonID ROWS UNBOUNDED PRECEDING) AS CumulativeSumByRows
   ,SUM(Salary) OVER (ORDER BY SalesPersonID RANGE UNBOUNDED PRECEDING) AS CumulativeSumByRange
FROM CTE
ORDER BY SalesPersonID, SalDate

OUTPUT:
SQL2012_OVER_Enhanced_Clause_2_1

In the Output above you can see the was the SUM is calculated, as we go down all previous and current rows are getting summed at all levels.
Here you can also see a difference between the way ROW & RANGE are calculated, for SalesPersonID=4:
– ROWS shows different and continuous calculation at different levels.
– But RANGE shows same calculation of all 3 records at different levels.


#2. Using ROWS/RANGE CURRENT ROW & UNBOUNDED FOLLOWING with OVER() clause:
Now, let’s say you’ve to do the same calculation, but in reverse order. So, here we will use BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING options to define our Window. This means the window starts from current row and ends at the last row.

-- To Calculate Cumulative SUM or Running Totals, but in REVERSE order:
;WITH CTE AS (
	SELECT BusinessEntityID AS SalesPersonID, CAST([Rate] AS DECIMAL(10,0))AS Salary, [ModifiedDate] AS SalDate
	FROM [HumanResources].[EmployeePayHistory]
	WHERE BusinessEntityID <= 10
	)
SELECT SalesPersonID, SalDate, Salary
   ,SUM(Salary) OVER (ORDER BY SalesPersonID ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) AS ReverseCumulativeSumByRows
   ,SUM(Salary) OVER (ORDER BY SalesPersonID RANGE BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) AS ReverseCumulativeSumByRange
FROM CTE
ORDER BY SalesPersonID, SalDate

OUTPUT:
SQL2012_OVER_Enhanced_Clause_2_2

In the Output above we can see that the Running totals are now in reverse order, compared to the 1st example.
Also, the ROWS & RANGE difference can be seen for the SalesPersonID=4 similar to the 1st example.


#3. Using ROWS/RANGE PRECEDING & FOLLOWING with OVER() clause, without UNBOUNDED:
Now, if you want to calculate the SUM only of those values that are just adjacent to a current row. Here we will use combination of row numbers with CURRENT/PRECEDING & FOLLOWING keywords.
Let’s us check this by following example:

-- To Calculate Moving SUM, by taking 3 moving rows:
;WITH CTE AS (
	SELECT BusinessEntityID AS SalesPersonID, CAST([Rate] AS DECIMAL(10,0))AS Salary, [ModifiedDate] AS SalDate
	FROM [HumanResources].[EmployeePayHistory]
	WHERE BusinessEntityID <= 10
	)
SELECT SalesPersonID, SalDate, Salary
   ,SUM(Salary) OVER (ORDER BY SalesPersonID ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) AS MovingSumByRowsPrevCurrentNext
   ,SUM(Salary) OVER (ORDER BY SalesPersonID ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING) AS MovingSumByRowsCurrentNext2
FROM CTE
ORDER BY SalesPersonID, SalDate

OUTPUT:
SQL2012_OVER_Enhanced_Clause_2_3

In the output above for:
– Column MovingSumByRowsPrevCurrentNext: it calculates the SUM of just one Previous, Current & one Next row.
– Column MovingSumByRowsCurrentNext2: it calculates the SUM of Current and next two rows.


So, by above 3 examples we saw how to use new ROWS & RANGE clauses with a mix of UNBOUNDED PRECEDING, UNBOUNDED FOLLOWING, CURRENT ROW, etc. Similarly there could be many different cases and scenarios where we can use there clauses and options.

I will try to post more on these features as I see anything new and exciting around this.

For more info on OVER() Clause check MS BOL: http://msdn.microsoft.com/en-us/library/ms189461.aspx

SQL Server 2012 (a.k.a Denali) – New feature | Enhanced OVER() Clause – Part1

April 13, 2012 1 comment

OVER() clause in SQL Server was introduced in version 2005 and it was the same in 2008 and even 2008-R2.

Ranking functions like ROW_NUMBER(), RANK(), DENSE_RANK() & NTILE() can use the OVER() clause to Parition and Order a record-set before actually applying their logic. Thus a large record-set can be divided into several partitions and calculations can be applied to each set in a set-based approach rather than going with Loops or Cursors.

With the new version of SQL Server i.e. 2012 the OVER() clause has been extended completely to other aggregates, like SUM(), AVG(), etc. Prior to this the OVER() clause can be partially used for these Aggregate Functions.

Here we will see solving a classic problem of calculating Running Totals with this new feature.
I’ve already discussed about solving this problem with JOINS in version 2005,2008, 2008-R2 in my previous post at following [link].

Let’s see how can we use the SUM() Aggregate Function with OVER() clause to solve this:

USE [AdventureWorks2012]
GO

-- With 2012, calculating Running totals by using Aggregare function SUM() with extended Window Function OVER():
;with RunTot as (
	select 
		row_number() over(order by s.OrderDate) as row,
		s.SalesPersonID, p.FirstName, p.LastName, s.TotalDue, s.OrderDate
	from Sales.SalesOrderHeader s
	join Person.Person p
	on s.SalesPersonID = p.BusinessEntityID
	)
SELECT 
	SalesPersonID, FirstName, LastName, OrderDate, TotalDue, 
	SUM(TotalDue) OVER(partition by SalesPersonID ORDER BY row)
FROM RunTot
order by SalesPersonID, row
GO


-- Prior to 2012 calculating Running totals by using SELF-JOIN:
;with RunTot as (
	select 
		row_number() over(partition by s.SalesPersonID order by s.OrderDate) as row,
		s.SalesPersonID, p.FirstName, p.LastName, s.TotalDue, s.OrderDate
	from Sales.SalesOrderHeader s
	join Person.Person p
	on s.SalesPersonID = p.BusinessEntityID)
select a.row, a.SalesPersonID, a.FirstName, a.LastName, a.OrderDate, a.TotalDue, sum(b.TotalDue) as RunTotal
from RunTot a
join RunTot b on a.SalesPersonID = b.SalesPersonID and a.row >= b.row
group by a.row, a.SalesPersonID, a.FirstName, a.LastName, a.TotalDue, a.OrderDate
order by a.SalesPersonID, a.row

On comparing performance of both the queries within the batch:
– The first query with SUM() window function costs just 38%.
– While the second query without the SUM() window function by using JOINS costs 68%.

I still like to call SQL Server’s latest release by its code name “Denali”, not “SQL Server 2012″.

Stay tuned to other new features of Denali in my forthcomming posts.

For more information about the OVER() clause check MS BOL link here: http://msdn.microsoft.com/en-us/library/ms189461.aspx

Calculating Running Totals

July 4, 2011 3 comments

A running total is the summation of a sequence of numbers which is updated each time a new number is added to the sequence, simply by adding the value of the new number to the running total.

Let’s see how to get these Running totals by creating a simple SQL queries below:

USE [AdventureWorks]
GO

-- Method 1: (Query Cost 9%)
;with RunTot as (
select row_number() over(partition by s.SalesPersonID order by s.OrderDate) as row,
s.SalesPersonID, p.FirstName, p.LastName, s.TotalDue, s.OrderDate
from Sales.SalesOrderHeader s
join Person.Contact p
on s.SalesPersonID = p.ContactID)
select  a.row, a.SalesPersonID, a.FirstName, a.LastName, a.OrderDate, a.TotalDue, sum(b.TotalDue) as RunTotal
from RunTot a
join RunTot b on a.SalesPersonID = b.SalesPersonID and a.row >= b.row
group by a.row, a.SalesPersonID, a.FirstName, a.LastName, a.TotalDue, a.OrderDate
order by a.SalesPersonID, a.row

-- Method 2: (Query Cost 91%)
;with RunTot as (
select row_number() over(partition by s.SalesPersonID order by s.OrderDate) as row,
s.SalesPersonID, p.FirstName, p.LastName, s.TotalDue, s.OrderDate
from Sales.SalesOrderHeader s
join Person.Contact p
on s.SalesPersonID = p.ContactID)
select  row, SalesPersonID, FirstName, LastName, OrderDate, TotalDue,
(select SUM(TotalDue) from RunTot b where b.SalesPersonID=a.SalesPersonID and b.row<=a.row) as RunTotal
from RunTot a

-- Output Listed below:

Categories: SQL Tips Tags:

TSQL Interview Questions – Part 4

July 1, 2011 4 comments

Next part of TSQL Interview Questions, contd. from my previous post.

76. What are Running totals and how would you calculate them? Create a single SQL statement starting with SELECT or ;WITH.
http://sqlwithmanoj.wordpress.com/2011/07/04/calculating-running-totals/

77. What are the various SSIS logging mechanisms?
– Text file
– SQL Server Profiler
– SQL Server
– Windows Event Log
– XML File
MS BOL link: http://msdn.microsoft.com/en-us/library/ms140246.aspx

78. On which table the SSIS SQL Server logs are stored?
On 2005 its sysdtslog90 & on 2008 its sysssislog.
More on SSIS logging on: http://sqlwithmanoj.wordpress.com/2011/06/15/logging-in-ssis-using-sql-server-log-provider/

79. Reverse a String value without using REVERSE() function and WHILE loop. Should be a single SQL statement/query.
http://sqlwithmanoj.wordpress.com/2011/08/18/reverse-a-string-without-using-tsqls-reverse-function/

80. What is the use of BCP utility in SQL Server and how will you use it?
http://sqlwithmanoj.wordpress.com/2011/09/09/bcp-in-out-queryout-the-bcp-util/

81. Is there any difference between Excel Source in SSIS 2005 & 2008?
http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/ea770da2-6f0c-41c4-8093-52b5df73f460/#4267c1f1-74a0-4e2c-b5a4-e2909bf4e5f4

82. Difference between Bookmark lookup & RID lookup?

83. How nested transactions behave?
Here is a scenario:
I have a transaction T and inside this there are 2 transactions T1 AND T2.
If TRANSACTION T2 fails then what happens to transaction T and T1

84. How will you check if a stored procedure with 500 line of code is running very slow? What steps will you take to optimize it?

85. New features in SQL Server 2005 compared to SQL Server 2000 you’ve worked with.
PIVOT, UNPIVOT, Ranking functions (row_number, rank, dense_rank, ntile), CTEs, Grouping sets (ROLUP, CUBE), Intersect, Except, OUTPUT clause, Merge statement, Try-Catch, BIDS (SSIS, SSRS, SSAS), CLR, SMO.
New datatypes: XML, VARCHAR(max), NVARCHAR(max), VARBINARY(max) deprecating the TEXT, NTEXT AND IMAGE datatypes.
XML  indexes.
Database (SMTP) mail, SSMS, DMVs, Express Edition, Service Broker, Data Encryption, MARS

86. How will you copy unique records from duplicates in source to destination in SSIS?

87. What transformation will you use to concatenate First name and Last name in SSIS?

88. What do you mean by selectivity of a column/table?
The selectivity is what goes with the cardinality concept. The “cardinality” refers to the number of “distinct” values, as in the set theory so, take a column “SEX”.  The possible values are “male” and “female” (ignoring for the moment other possible values like “unknown” or even “other”) … so, your cardinality for that column would be 2, no matter how many rows you have in that table.

The selectivity is the “number of rows” / “cardinality”, so if you have 10K customers, and search for all “female”, you have to consider that the search would return 10K/2 = 5K rows, so a very “bad” selectivity.

The column for the primary key on the other side is “unique”, and hence the cardinality is equal to the number of rows, by definition.  So, the selectivity for searching a value in that column will be 1, by definition, which is the best selectivity possible.

89. Difference between EXISTS & IN, which one gives good performance?

90. What output will “SELECT 1/2″ statement give?
0, it will give zero.

91. What is Database Partitioning?
This involves 4 steps:
1. Create Database with different file groups
2. Create Partition Function
3. Create Partition Scheme
4. Create Partitioned Table or Index

92. What is the use of NOLOCK option?

93. How many types of recovery models are available for a database?
1. Simple
2. Bulk logged
3. Full

94. How many types of temporary tables are there in SQL Server?
– Local Temp tables (#)
– Global temp tables (##)
– Table variables (@)

95. In how many ways you can get a table’s row count?

--// 1. Using COUNT(*)
Select count(*) from Person.Contact

--// 2. using COUNT(1)
select count(1) from Person.Contact

--// 3. Using SUM() aggregate function
select sum(1) from Person.Contact

--// 4. Using sysindexes view
select object_name(id), rows from sys.sysindexes where object_name(id) = 'Contact' and indid<2

--// 5. Using sp_spaceused system SP
exec sp_spaceused 'Person.Contact' 

--// 6. Using DBCC CHECKTABLE function
DBCC CHECKTABLE('Person.Contact') 

--// Note: Before running 5 & 6 you may need to run this script:
DBCC UPDATEUSAGE ('AdventureWorks','Person.Contact') WITH COUNT_ROWS

Reference

96. In how many ways you can select distinct records from a table?

97. In how many ways you can select top 1 row?

98. What are the new features introduced in SQL Server 2008 R2?

99. What are the new features added SQL Server 2012 (Denali)?

100. What new feature has been added to TRY-CATCH construct?
… more questions on next post Part-5.

Follow

Get every new post delivered to your Inbox.

Join 405 other followers