Wednesday, December 8, 2010

SQL Query runs fast in MSSMS but slow as stored procedure

 

It was the weirdest thing.  I had a fairly straight-forward query that when run in my Server Management Studio editor would only take a second or 2 but after building a stored procedure with the same exact query and executing it (either from a .NET page/object or even from a T-SQL call, the stored procedure would take FOREVER to execute.

After a few hours of Google, hair-pulling and searching for a white towel to throw in, I found a solution that works 99% of the time.

 

The cause is an issue called Parameter Scanning and occurs "sometimes" in procedures that pass a parameter in.

The fix is to use internalized parameters - basically creating a new parameter inside of the stored procedure and assigning it a value of the one being passed in.

I don't know the answer as to WHY this happens and why this fix corrects it but all I know is that it does work

 

Examples:

A typical stored procedure with a single passed-parameter might be something like:

ALTER PROC [dbo].[uspREPORTING_Sales]
(@parSALESPERSONID int)
AS

SET NOCOUNT ON

SELECT SUM(GROSS_PROFIT) FROM tblSALESCONTRACTS WHERE SALESPERSON_ID = @parSALESPERSONID



This same procedure rewritten to internalize the parameter would look something like




ALTER PROC [dbo].[uspREPORTING_Sales]
(@parSALESPERSONID int)
AS

SET NOCOUNT ON

DECLARE @intSALESPERSONID int
SET @intSALESPERSONID = @parSALESPERSONID

SELECT SUM(GROSS_PROFIT) FROM tblSALESCONTRACTS WHERE SALESPERSON_ID = @intSALESPERSONID



You're basically referencing the internally-declared parameter instead of the one passed to the procedure. 



Hope this helps those who are having this problem

Wednesday, April 28, 2010

AutoCompleteExtender using Application-Cached Data

A recent project required me to build a search function that looked for existing customers by name.  Our company has thousands of customers so simply putting the list into a dropdownlist or bulleted list was not feasible.  I decided to build a simple textbox input search function which was wired up with the AJAX AutoCompleteExtender.  The user would enter a name of a customer and the list would filter down until either a match was made or it wasn't. 

Working with the extender is quite simple.  You add a textbox control to your form, associate the extender with it and point the extender to a webservice.  Each character entered into the textbox triggers a webmethod of the webservice which queries data and returns results as a string array [].

On the backend the webservice would execute a simple query against a table on our SQL Server based on the contents of the text entered in the textbox.  As more and more people use this function however, the traffic and performance of all of this activity becomes a concern.   Since the data being queried against rarely changed (names of customers), I decided to utilize an application cache which the webmethod would use as the basis of finding matches via the extender.

The original setup of my webmethod was quite simple.   It would execute a query against the database that returned X amount of rows (to limit the number shown in the autocomplete results).  These rows (containing 1 column) would be inserted into a string array and returned to the extender for display.

Original code:

[WebMethod]
public string[] GetCustomers(string prefixText, int ResultsReturned)
{
string strSQL = "SELECT TOP " + ResultsReturned.ToString() + " CUSTOMER_NAME + ' - ' + CUSTOMER_STATUS FROM tblCUSTOMERS ";
strSQL += "WHERE REPLACE(REPLACE(UPPER(CUSTOMER_NAME),'.',''),',','') LIKE '%" + prefixText.ToUpper().Replace(".", "").Replace(",", "") + "%' ";
strSQL += "ORDER BY CUSTOMER_NAME";
SqlConnection conn = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["myConn"].ToString());
conn.Open();
SqlDataAdapter sda = new SqlDataAdapter(strSQL, conn);
DataSet ds = new DataSet();
sda.Fill(ds);
conn.Close();
string[] items = new string[ds.Tables[0].Rows.Count];
int i = 0;
foreach (DataRow dr in ds.Tables[0].Rows)
{
items.SetValue(dr[0].ToString(), i);
i++;
}
ds.Dispose();
sda.Dispose();
conn.Dispose();

return items;
}

To utilize caching I had to determine the best scope to use.  If the data being cached was different or would change depending on the user accessing that cache, I would use a Session-level cache.   If the data was the same across the board, regardless of the user, an Application cache is the best bet.   Since this was the case, I went the application route. 


Implementing a cached data source takes a little extra coding since I now have to filter my data from a persistant (sortof) dataset as opposed to requerying the database and building a new dataset each time.  The first step is to add a constructor that can build a basic, unfiltered dataset (using a DataTable) and add it into the Application.Cache space as such:


private DataTable GetCustomersDT()
{
DataTable dt = (DataTable)HttpContext.Current.Cache["CustomersDT"];
if (dt == null)
{
string strSQL = "SELECT CUSTOMER_NAME, CUSTOMER_STATUS FROM tblCUSTOMERS";

SqlConnection conn = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["myConn"].ToString());
conn.Open();
SqlDataAdapter sda = new SqlDataAdapter(strSQL, conn);
DataSet ds = new DataSet();
sda.Fill(ds);
dt = ds.Tables[0];
HttpContext.Current.Cache.Insert("CustomersDT", dt, null, Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(5));
sda.Dispose();
ds.Dispose();
conn.Close();
}
else
{
dt = (DataTable)HttpContext.Current.Cache["CustomersDT"];
}
return dt;
}

Now that we have a Cached DataTable containing the data we'll use to filter for our extender, lets look at the WebMethod that utilizes it:


[WebMethod]
public string[] GetCustomersFromCache(string prefixText, int ResultsReturned)
{
DataTable dt = GetCustomersDT();
DataView dv = new DataView(dt);
dv.RowFilter = "CUSTOMER_NAME LIKE '%" + prefixText.ToUpper().Replace(".", "").Replace(",", "") + "%'";
dv.Sort = "CUSTOMER_NAME";
int rowcount = dv.Count;
int counter = 5;
if (rowcount < 5)
{
counter = rowcount;
}
string[] items = new string[counter];

int x = 0;
foreach (DataRowView drv in dv)
{
if (x < counter)
{
items.SetValue(drv[0].ToString(), x);
x++;
}
else
{
break;
}

}
return items;
}

And there you have it.   You'll see that the first line of the WebMethod calls GetCustomersDT() which returns the DataTable used for filtering our results.  This DataTable will either exist in the Application's Cache or be created and then added to the Cache.    This should save many, many round-trips to the database.

Thursday, March 25, 2010

Avoiding "Timeout Expired" errors from long running SELECT statements

a SQLDataSource object has a default command timeout of 90 seconds. Often you may be tasked with returning large amounts of data or doing some other tedious processing on the database side that takes while. If you exceed this you'll get the pain-in-the-ass runtime exception - Sys.WebForms.PageRequestManagerServerErrorException: Timeout expired.

 error

 

To work-around this when all other methods have failed, I increase the command timeout value of the SqlDataSource object via it's SELECTING event as such:

protected void sdsYourSQLDataSource_Selecting(object sender, SqlDataSourceSelectingEventArgs e)
{
e.Command.CommandTimeout = 360;
}


The value is set in SECONDS. Hope this helps