Stored Procedure for Paging in SQL

Implementing paging correctly is critically important. If you’re shop still uses, or insists on stored procedures, that’s fine. It’s just as easy to write a stored procedure for paging in SQL.

This example was developed on Microsoft SQL server 2016, although I believe it will work fine back to SQL 2012, not sure about SQL 2008.

As with any solution, the idea is to keep the data at rest and only pull out what you need to return. This example uses the AdventureWorks database.

This example has all of the typical parameters that I use on many paging implementations. You can trim out the ones not needed for your implementation.

USE [AdventureWorks]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

-- Created by Jack Yasgar, www.jackyasgar.net
CREATE PROCEDURE [dbo].[uspGetOrderDetailsYTGI]
	@PageNumber			INT,
	@PageSize			INT,
	@SearchTerm			VARCHAR(100) = NULL,
	@SortDirection		VARCHAR(4) = 'ASC',
	@SortColumn			VARCHAR(100) = 'lastname',
	@Filter				VARCHAR(50) = '',
	@ActiveOnly			BIT
AS
BEGIN
    SET NOCOUNT ON;

 	IF ((@PageNumber = 0) OR (@PageNumber IS NULL))
	BEGIN
		SET @PageNumber = 1;
	END

	IF ((@PageSize = 0) OR (@PageSize IS NULL))
	BEGIN
		SET @PageSize = 25;
	END

	DECLARE @FirstRec INT, @LastRec INT;
	DECLARE @OnlineOrder BIT;
	SET @OnlineOrder = (SELECT CONVERT(BIT, @Filter));

	SELECT @FirstRec = (@PageNumber - 1) * @PageSize
	SELECT @LastRec = (@PageNumber * @PageSize + 1);
	

	SELECT 
		sod.OrderQty,
		sod.ProductID,
		p.[Name] AS ProductName,
		p.ProductNumber,
		soh.ShipDate,
		sod.UnitPrice,
		soh.OnlineOrderFlag
	FROM [Sales].[SalesOrderHeader] soh WITH (NOLOCK)
		LEFT JOIN [Sales].[SalesOrderDetail] sod WITH (NOLOCK) ON soh.SalesOrderID = sod.SalesOrderID
		LEFT JOIN [Production].[Product] p WITH (NOLOCK) ON p.ProductID = sod.ProductID
	WHERE 
		(ISNULL(@SearchTerm, '') = '' OR @SearchTerm = '' OR p.[Name] LIKE '%' + @SearchTerm + '%')
		AND
		(ISNULL(@Filter, 0) = 0 OR @Filter = 0 OR soh.OnlineOrderFlag = @OnlineOrder)
		AND
		(ISNULL(@ActiveOnly, 0) = 0 OR @ActiveOnly = 0 OR soh.ShipDate > '2013-01-01')
	ORDER BY 
		CASE WHEN @SortDirection = 'ASC' THEN
			CASE @SortColumn 
				WHEN 'ProductName'		THEN p.[Name]
				WHEN 'ProductNumber'	THEN p.ProductNumber
			END
		END,
		CASE WHEN @SortDirection = 'DESC' THEN
			CASE @SortColumn 
				WHEN 'ProductName'		THEN p.[Name]
				WHEN 'ProductNumber'	THEN p.ProductNumber
			END
		END DESC
	OFFSET @PageSize * (@PageNumber - 1) ROWS
	FETCH NEXT @PageSize ROWS ONLY;

		-- Return the full count for pager
	SELECT COUNT(*) AS TotalCount FROM (
			SELECT DISTINCT sod.SalesOrderDetailID
			FROM [Sales].[SalesOrderHeader] soh WITH (NOLOCK)
				LEFT JOIN [Sales].[SalesOrderDetail] sod WITH (NOLOCK) ON soh.SalesOrderID = sod.SalesOrderID
				LEFT JOIN [Production].[Product] p WITH (NOLOCK) ON p.ProductID = sod.ProductID
			WHERE 
				(ISNULL(@SearchTerm, '') = '' OR @SearchTerm = '' OR p.[Name] LIKE '%' + @SearchTerm + '%')
				AND
				(ISNULL(@Filter, 0) = 0 OR @Filter = 0 OR soh.OnlineOrderFlag = @OnlineOrder)
				AND
				(ISNULL(@ActiveOnly, 0) = 0 OR @ActiveOnly = 0 OR soh.ShipDate > '2013-01-01')
		) AS Counting

END

This query uses the SQL OFFSET command to start where we need to based upon the PageNumber and PageSize values passed in as arguments. It then uses the FETCH command to retrieve a specific row. You can take a look at the documentation on FETCH here:

https://docs.microsoft.com/en-us/sql/t-sql/language-elements/fetch-transact-sql?view=sql-server-ver16

Once you receive back the DataSet in your Repository, you’ll need to stuff the data into a collection of DTO’s in your ServiceLayer for transfer through your web service: Stuffing DataSets into your objects.

DataLayer Level Paging

This entry is part 2 of 7 in the series Ultimate MVC Paging

This post was revised and updated on 10/24/2020.

If we’re going to implement paging in our UI, the first thing we’ll need is code that will return a paged set of data from the database. The ultimate goal is to only retrieve the exact quantity of filtered, sorted records from the database.

I’m going to be using the AdventureWorks database for some of these examples. The code in the GitHub project DOES NOT use AdventureWorks, it uses the YTGI Lookups database tables. You can get the AdventureWorks database from Microsoft’s site here: AdventureWorks Download

The complete solution that I show in this Series can cloned here: Paging Demo Download. This solution was created in Visual Studio 2019 Professional using MVC in .NET Core 3.1.

Here are three posts showing retrieving data in a paged format. Once you look at these, you can return to this post to continue to the next step.

Paging with a Stored Procedure

Paging with Entity Framework – Simple

Paging with Entity Framework – Advanced

Ultimate Guide to MVC Paging!

This entry is part 1 of 7 in the series Ultimate MVC Paging
MVC Paging Partial Views

After searching the web for MVC paging techniques, I realized that 90% or more of the examples I found on the web would not solve my problem. Some of them are so bad that they could get me fired if I followed their example. (copy -> paste). They so oversimplify the need, that they render their solutions useless. The intentions are good, but the implementation is lacking.

Have you:

  • Wanted to add paging to your MVC site, but think it’s too complicated?
  • Implemented it on one page and hope you never have to do it again?
  • After implementing paging thought, “There’s got to be a better way?”

The most popular problem with most examples that I found is that they show pulling back the entire collection from the database and paging the results in the UI (in the controller). This is fine if you’re absolutely positive, sure beyond belief, willing to bet your job, that the results will never exceed a hundred records or so…

If you want to become an expert on how to implement fast, extremely flexible paging in all your MVC applications, then figure that this is going to take an hour or two. If you’re looking to “turn on paging”, then feel free to click on the “Back” button, I won’t be offended.

Not implementing paging, or implementing it incorrectly can cause serious problems with the performance of your web site. Below are some symptoms that I’ve seen that leads me to believe there could be a problem with the paging implementation on a web site:

  1. Complaints that the web site slows down, but nobody can see high utilization on the web server or database server (and no expensive queries)
  2. Complaints that the web site sometimes stops responding, occasionally for minutes at a time
  3. Complaints that the web site sometimes suddenly throws users back to the login screen while they’re working
  4. Complaints that the slow downs are intermittent, and can’t be reproduced in the test environment

I know you may be thinking, what does this have to do with paging? Well, it’s tangentially related, as it has to do with writing bad code without thinking about what the consequences are. The reason these issues don’t tax the servers is that doing a SELECT * FROM Clients doesn’t really tax the CPU that much. Now, if you were watching the NIC card traffic, that would be a different story.

The biggest issue that I see with code examples on the web:

  1. They show bringing back every row in the table and then page in the controller (No, no, please don’t do this!)
  2. They show using a LINQ query in the controller to Skip and Take
  3. They assume that I’m pulling back my data directly from the database in my Web Application

// Example of code that is ridiculous outside of a POC app
return View(MyRepository.GetClients().Skip((page - 1 ?? 0) * (rows ?? 3)).Take(rows ?? 3));

This LINQ query will work well if:

  1. The MyRepository is an EntityFramework repository
  2. AND you’re implementing your Repository in your web site
  3. AND it doesn’t dispose of the Entity context inside the method
  4. AND the GetClients() method doesn’t call ToList() inside of it
  5. AND GetClients() is not executing a stored procedure

If any of the above is not true, then this method is no better than the first method that pulls the entire table back.

Another issue with this example is that it assumes that the Repository is part of your web project AND you’re using the objects that are in your Entity Model. So no DTO’s, no business objects, and absolutely no Web Service in the middle.

The reason I’m showing this, is that I’ve never seen these issues discussed in the posts. A developer could easily implement this on a GetClients() that again, returns the entire table contents and then pages on the UI side.

Just in case you forgot, the web is stateless, so if IIS or most other Web Servers feel that they’ve had about enough with the application memory usage, they just reset themselves. Shouldn’t cause a problem right? Except if you’re depending on session to keep people logged in, then they get kicked back to the login screen. (BTW, this only happens if you have your own custom code looking for a session object to verify the user is logged in. If you use the proper MVC standard Authorization, they shouldn’t be sent to the login screen.) But not to minimize the impact, because all users get kicked out, not just the one user.

Almost every web, and even windows project I’ve ever worked on followed the Repository pattern in that there were separations of concerns, i.e.:

Database → DataLayer → ServiceLayer → WebService → ProxyClass → ServiceLayer → UI

I don’t want to digress into a discussion of patterns. But the ServiceLayer behind the WebService will convert the Entity Framework object or DataSet to a DTO collection. Then the ServiceLayer behind the UI will likely convert it to a ViewModel. We don’t want to do all that work on 5000 records so that we can ultimately display 25 to the user…


This search on Google returned 466,000 records. Do you think that they were all returned and paged in the controller? I think not…

The reason I’m discussing this is to get your imagination started. If you didn’t want to transfer tons of data over the wire from your WebService, then where is the best place to implement Paging? You guessed it, in the DataLayer (also referred to as Data Access Layer (DAL)). As close to the data as you can possibly get.

Alright, let’s get to the nuts and bolts. I’m going to show you a full implementation, so you can trim it down if your particular need is less complicated.

Anatomy of a Paging Request

The following are the properties that I’ve been able to accomplish almost any paging task I’ve ever needed on the client side:

Property Type Notes
SortedResults List<T> The list of results, without it, we don’t need paging!
TotalItems int The total number of items that would be returned, if we returned them all at once
CurrentPage int The current page the user is on
PageSize int The size of the page, usually an option for the user to select
TotalPages int The total number of pages (TotalItems/PageSize rounded up)
StartPage int The start page, for when you have more pages than will fit on one widget
EndPage int The end page, for when you have more pages than will fit on one widget
SearchFilter string A filter to use in addition to any search criteria, usually selected from a dropdown, i.e. year, division, salesman, etc.
SearchTerm string A search term, usually typed in a search box by the user
SortExpression string The column name to sort the results by
RecordCount int The quantity of records in this particular result set
SortOrder string The order that you want the result set, i.e.: “ASC”, “DESC”
ActiveOnly bool Another filter flag, if you have an IsActive flag in your table, or any other criteria to knock out inactive rows
RefId int Typically used when there is a Parent/Child relationship. i.e. Your SortedList is a list of Project tasks, then you’ll need to know the parent project PK Id

Paging with Entity Framework (Advanced)

This example is only called “Advanced” because it addresses multiple tables for paging. This will often be the case, but you can view a simple example that addresses one table here: Paging with Entity Framework (Simple)

The example below accepts all the common parameters and handles assembling a Linq query. It DOES NOT execute the query until the entire statement is built. There is a necessary evil call to get the count, once the WHERE clause criteria is completed. You’ll notice that this call is done before adding the OrderBy so as not to burden the Count query with sorting.

The OrderBy statement is not standard in Linq. It uses an extension method that you can take a look at here: OrderBy Extension for Linq Queries

You’ll also notice that this method returns a PagedResult object. This is just a custom DTO that brings back a strongly typed list and a total count. You can see the code for this object here. PagedResult for Paging

        /// <summary>
        /// Complicated Paged results from multiple tables in EF Query
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        /// <param name="PageNumber"></param>
        /// <param name="PageSize"></param>
        /// <param name="SearchTerm"></param>
        /// <param name="SearchFilter"></param>
        /// <param name="SortExpression"></param>
        /// <param name="SortOrder"></param>
        /// <param name="ActiveOnly"></param>
        /// <returns></returns>
        public PagedResult<Models.SalesProduct> GetSalesDetailsByEF(int PageNumber,
                                                                    int PageSize,
                                                                    string SearchTerm,
                                                                    string SearchFilter,
                                                                    string SortExpression,
                                                                    string SortOrder,
                                                                    bool ActiveOnly)
        {
            try
            {
                int _skipRows = (PageNumber - 1) * PageSize; // if this is not the first call, need move forward
                int _totalCount = 0; // placeholder for the total amount of records

                using (EFModels.AWEntities _efEntities = new EFModels.AWEntities())
                {
                    // Using var because this is returning an anonymous type
                    var _entityrows = (from sheader in _efEntities.SalesOrderHeaders
                                       join sdetail in _efEntities.SalesOrderDetails on sheader.SalesOrderID equals sdetail.SalesOrderID
                                       join product in _efEntities.Products on sdetail.ProductID equals product.ProductID
                                       select new Models.SalesProduct()
                                       {
                                           OrderQuantity = sdetail.OrderQty,
                                           ProductId = sdetail.ProductID,
                                           ProductName = product.Name,
                                           ProductNumber = product.ProductNumber,
                                           ShipDate = sheader.ShipDate ?? DateTime.MinValue,
                                           UnitPrice = sdetail.UnitPrice,
                                           OnlineOrderFlag = sheader.OnlineOrderFlag
                                       });

                    if (ActiveOnly)
                    {
                        // Showing how to use ActiveOnly without a boolean flag as an example
                        _entityrows = _entityrows.Where(er => er.ShipDate.Year > 2013);
                    }

                    if (!string.IsNullOrWhiteSpace(SearchFilter))
                    {
                        // This can be customized for each implementation
                        bool _onlineOrder;
                        bool.TryParse(SearchFilter, out _onlineOrder);
                        _entityrows = _entityrows.Where(f => f.OnlineOrderFlag == _onlineOrder);
                    }

                    if (!string.IsNullOrWhiteSpace(SearchTerm))
                    {
                        // This can be customized for each implementation
                        _entityrows = _entityrows.Where(f => f.ProductName.Contains(SearchTerm.Trim()));
                    }

                    // Getting count will execute a SELECT COUNT(*)
                    // Like to do this before adding sort criteria
                    _totalCount = _entityrows.Count();

                    if (!string.IsNullOrWhiteSpace(SortExpression))
                    {
                        bool IsSortDESC = false;
                        if (SortOrder.ToLower() == "asc") { IsSortDESC = true; }

                        _entityrows = _entityrows.OrderBy<Models.SalesProduct>(SortExpression, IsSortDESC);
                    }

                    _entityrows = _entityrows.Skip(_skipRows).Take(PageSize);

                    return new PagedResult<Models.SalesProduct>(_entityrows.ToList(), _totalCount);


                }
            }
            catch (Exception)
            {
                throw;
            }
        }

If you’re viewing this as part of the Paging series, continue with that series here: Business Objects for Paging

Business Object used for Search and Paging

When I’m implementing a paged query, I wanted a simple object that I could return a strongly typed list of values along with a total count from my query. The business object used for search and paging also has the return of a List<T> of objects along with a TotalCount property from my Repository. You’ll see this as the return object in many of my sample paging posts.

This is basically a DTO, it doesn’t have any specific functionality.

    /// <summary>
    /// Yasgar Technology Group, Inc.
    /// http://www.ytgi.com
    /// </summary>
    /// <typeparam name="TEntity"></typeparam>
    [DataContract(Name = "PagedResult", Namespace = "", IsReference = true)]
    public class PagedResult<TEntity> : IPagedResult<TEntity>
    {

        #region Constructors

        /// <summary>
        /// Primary constructor
        /// </summary>
        /// <param name="items"></param>
        /// <param name="totalCount"></param>
        public PagedResult(IEnumerable<TEntity> items, int totalCount)
        {
            m_items = items;
            m_totalCount = totalCount;
        }

        #endregion // Constructors

        #region Fields

        private IEnumerable<TEntity> m_items;
        private int m_totalCount = 0;

        #endregion // Fields

        #region Properties

        /// <summary>
        /// IEnumerable list of items returned
        /// </summary>
        [DataMember]
        public IEnumerable<TEntity> Items
        {
            get { return m_items; }
            protected set { m_items = value; }
        }

        /// <summary>
        /// Total count of records from query
        /// </summary>
        [DataMember]
        public int TotalCount
        {
            get { return m_totalCount; }
            protected set { m_totalCount = value; }
        }

        #endregion // Properties

    }

This data transfer object (DTO) is used to house the results from a paging query that has the minimum amount of information needed to return from the database. It does not need to have search criteria etc, as it is assumed that the calling method already has all that data. This will help you have a standard return object that not only has the collection of objects, but the total count that would have been returned had there not been any filters.

Here is an example of this object in use:

Paging with Entity Framework (Simple)

Whenever you have a need to show a paged list of your items, it is a must that this is done in your repository in Entity Framework so that you’re not passing around more data than is necessary. There are too many examples of paging on the web which show paging at the UI level, especially terrible ones are showing doing a LINQ to SQL query with ToList() after it. Horrible!

DON’T EVER DO THIS!

var _result = dbContext.Customers.ToList().Skip(page).Take(pageSize);

The following example is only called “Simple” because it addresses one table for paging. This will often be the case, but you can view a more complicated query here: Paging with Entity Framework (Advanced)

The example below accepts all the common parameters and handles assembling a Linq query. It DOES NOT execute the query until the entire statement is built. There is a necessary evil call to get the count, once the WHERE clause criteria is completed. You’ll notice that this call is done before adding the OrderBy so as not to burden the Count query with sorting.

The OrderBy statement is not standard in Linq. It uses an extension method that you can take a look at here: OrderBy Extension for Linq Queries

You’ll also notice that this method returns a PagedResult object. This is just a custom DTO that brings back a strongly typed list and a total count. You can see the code for this object here. PagedResult for Paging

This LINQ code was pulled directly from the LookupsRep class in the YTG MVC Lookups and Paging Demo Download.

/// <summary>
/// Simple Paged results from multiple tables in EF Query
/// Yasgar Technology Group, Inc. - www.ytgi.com
/// </summary>
/// <param name="Page"></param>
/// <param name="PageSize"></param>
/// <param name="SearchTerm"></param>
/// <param name="SearchFilter"></param>
/// <param name="SortColumn"></param>
/// <param name="SortOrder"></param>
/// <param name="ActiveOnly"></param>
/// <returns></returns>
public async Task<PagedResult<LuCategories>> GetLuCategoriesByEFAsync(int Page,
            int PageSize,
            string SearchTerm,
            string SearchFilter,
            string SortColumn,
            string SortOrder,
            bool ActiveOnly)
{
    try
    {
        int _skipRows = (Page - 1) * PageSize; // if this is not the first call, need move forward
        int _totalCount = 0; // placeholder for the total amount of records

        // Using var because this is returning an anonymous type
        var _entityrows = (from item in LuContext.LuCategories.Include(a => a.LuItems)
                           select item);

        if (ActiveOnly)
        {
            // Showing how to use ActiveOnly without a boolean flag as an example
            _entityrows = _entityrows.Where(er => er.IsActive == true);
        }

        if (!string.IsNullOrWhiteSpace(SearchFilter))
        {
            // This can be customized for each implementation
            _entityrows = _entityrows.Where(f => f.ShortName == SearchFilter);
        }

        if (!string.IsNullOrWhiteSpace(SearchTerm))
        {
            // This can be customized for each implementation
            _entityrows = _entityrows.Where(f => f.Name.Contains(SearchTerm.Trim()));
        }

        // Getting count will execute a SELECT COUNT(*)
        // Like to do this before adding sort criteria
        _totalCount = _entityrows.Count();

        if (!string.IsNullOrWhiteSpace(SortColumn))
        {
            bool IsSortDESC = false;
            if (SortOrder.ToLower() == "desc") { IsSortDESC = true; }

            _entityrows = _entityrows.OrderBy(SortColumn, IsSortDESC);
        }

        _entityrows = _entityrows.Skip(_skipRows).Take(PageSize);

        return new PagedResult<Models.LuCategories>(await _entityrows.ToListAsync(), _totalCount);

    }
    catch (Exception)
    {
        throw;
    }
}


If you’re viewing this as part of the Paging series, continue with that series here: Business Objects for Paging

Linq OrderBy Extension using a string

When writing more complicated Linq queries against an ORM, it is often desirable to be able to randomly pass in a field name as a string and sort order (direction) on the fly. This comes in especially handy for the paging project that you’ll also find on this Blog. I developed this Linq OrderBy Extension using a string and boolean value to determine if should sort ascending or descending.

Basically, with out of the box Linq, you can’t easily change the sort column dynamically. There is a Dynamic Linq Library (System.Linq.Dynamic) that you can install using a NuGet package, but that seemed overkill to me for this need. After looking that over, if you feel it’s a better solution for you, you won’t need to use this extension.

This extension method allows you to pass in the collection as IQueryable, so you can pass in Linq queries that haven’t been executed yet. Then it accepts your OrderByProperty and whether you want the query set to sort the results Descending as a boolean value.

You implement it like this:

using (EFModels.AWEntities _efEntities = new EFModels.AWEntities())
{
    var _entityrows = (from sheader in _efEntities.SalesOrderHeaders
                       select sheader);

    if (!string.IsNullOrWhiteSpace(SortExpression))
    {
        bool IsSortDESC = false;
        if (SortOrder.ToLower() == "asc") { IsSortDESC = true; }
  
        _entityrows = _entityrows.OrderBy(SortExpression, IsSortDESC);
    }

	return _entityrows.ToList();
	
}

Put the following Extension in your extensions class and make sure to put a using to that location at the top of the class where you’re going to use it.

    /// <summary>
    /// Implementation of OrderBy Extension Method for IQueryable Collections
    /// </summary>
    /// <typeparam name="TEntity">Generic Type Object</typeparam>
    /// <param name="p_Source">The collection to order</param>
    /// <param name="p_OrderByProperty">The property to order by</param>
    /// <param name="p_Descending">True to sort Descending</param>
    /// <returns></returns>
    public static IQueryable<TEntity> OrderBy<TEntity>(this IQueryable<TEntity> p_Source, string p_OrderByProperty, bool p_Descending)
    {
        try
        {
            string command = p_Descending ? "OrderByDescending" : "OrderBy";
            var type = typeof(TEntity);
            var property = type.GetProperty(p_OrderByProperty);
            var parameter = Expression.Parameter(type, "p");
            var propertyAccess = Expression.MakeMemberAccess(parameter, property);
            var orderByExpression = Expression.Lambda(propertyAccess, parameter);
            var resultExpression = Expression.Call(typeof(Queryable), command, new Type[] { type, property.PropertyType },
                                          p_Source.Expression, Expression.Quote(orderByExpression));
            return p_Source.Provider.CreateQuery<TEntity>(resultExpression);
        }
        catch (ArgumentNullException)
        {
            throw new Exception("The OrderByProperty value of: '" + p_OrderByProperty + "', was empty or is not a proper column name!");
        }
        catch (Exception)
        {
            throw;
        }
    }

If you like this, take a look at some other extension methods that I use all the time:

Knock out 1/1/0001 Dates in MVC

Back when I coded in Visual Studio 2003, we used to joke that if you wrote a small app that had 100 lines of code, 98 of them would be checking for nulls. It actually wasn’t that far off.

Because of that, I often force my DTO’s to default to DateTime.MinValue instead of nulls and have my code look for MinValue rather than checking for nulls all the time. This has one side effect that I didn’t like, when you use your model in an MVC view, it will display in the text box like “01/01/0001 00:00:00” etc. This is obviously not desirable.

The secondary issue is that when you use Date/Time pickers in the view, they don’t like being bound to DateTime properties, they work better on a string based text box. So what I’ve done is resolve both issues with one solution.

First part is in the ViewModel. I often just inherit my DTO and add or override properties with Attributes/Decorations that I need for the View. I’ll use my ProjectViewModel as an example:

    public class ProjectViewModel : Project
    {
        /// <summary>
        /// Gets or sets the Project Start Date Display
        /// Use to get rid of 01/01/0001 displays
        /// </summary>
        [Display(Name = "Start Date: ")]
        public string StartDateDisplay
        {
            get
            {
                return StartDate.ToShortDateDisplay();
            }
            set
            {
                StartDate = value.ToDateFromString();
            }
        }
    }

In this case, my Project DTO has a StartDate property that is a DateTime. It could be null, or it could be DateTime.MinValue, or it could be a real date. In this implementation, I don’t care about the time, but you could build this out to include time very easily.

When I use this in a view, I use the StartDateDisplay property, rather than the StartDate property.

@Html.TextBoxFor(model => model.StartDateDisplay, new { @class = "form-control", @id = "txtStartDate", @title = "Project Start Date", @type = "text", autocomplete = "off" })

Now this will play nice with the Date/Time picker because it is a string property.

In order to get the dates formated, you’ll notice the ToShortDateDisplay() and the ToDateFromString() extension methods. Those are defined as:

        /// <summary>
        /// Convert a date to a short date string, empty if min or max value
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        /// <param name="value"></param>
        /// <returns></returns>
        public static string ToShortDateDisplay(this DateTime value)
        {
            if (value == null) { return string.Empty; }

            if ((value == DateTime.MaxValue) || (value == DateTime.MinValue))
            {
                return string.Empty;
            }
            else
            {
                return value.ToShortDateString();
            }
        }

        /// <summary>
        /// Convert a string in MM/DD/CCYY format to a valid date
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        /// <param name="value"></param>
        /// <returns></returns>
        public static DateTime ToDateFromString(this string value)
        {
            if (!string.IsNullOrWhiteSpace(value))
            {
                if ((value == "99999999") || (value == "99/99/9999"))
                {
                    return DateTime.MaxValue;
                }
                else
                {
                    if (!string.IsNullOrWhiteSpace(value))
                    {
                        DateTime _value;
                        DateTime.TryParse(value, out _value);
                        return _value;
                    }
                    else
                    {
                        return DateTime.MinValue;
                    }
                }
            }

            return DateTime.MinValue;

        }

Solved two issues with one solution. No checking for nulls, no checking for DateTime.MinValue.

Google reCAPTCHA in HTML and JavaScript

This post is using two HTML pages that have only JavaScript enabled. Feel free to replace some of the code with JQuery if you are loading those scripts. Otherwise, you can implement Google reCAPTCHA in HTML and JavaScript.

Google reCAPTCHA
Google reCAPTCHA

Google has a client side implementation for their reCAPTCHA on your web pages. Their documentation is great at explaining what it is, but it lacks in specific examples for how to implement in different environments. This causes confusion with some developers when they paste the two lines of code in their web page, but they are still able to submit the page, even when they don’t fill in the CAPTCHA.

I’ve noticed during testing that it may be possible to get through the CAPTCHA the first time. On subsequent requests, probably based on IP address, it creates a popup that you have to select photos from. That should stop most bot engines. Just mentioning so you don’t think there’s a problem if you still occasionally get a form submit that looks like it could be a bot.

The getting started section of the Google Developer’s Guide is fine for getting started, but I’ll still cover it here, as I strongly dislike blog posts that only show 80% of the solution.

First thing you need is the actual URL that your going to deploy the application on. So if you haven’t registered one yet, you should do that now. I don’t know how Google handles it when two people try to register the same domain with reCAPTCHA, but I would assume that it would be questioned at some level. Maybe I’ll do an investigation in the future when I’m bored. I just feel that I don’t want to setup a domain under my Google account and then find out later someone else registered the domain and I’ve made a potential problem for them.

Okay, so let’s get started:

  1. Register your domain, as previously mentioned.
  2. Sign up for your reCAPTCHA at Google.
    1. Save your site and private/secret key somewhere in your source control
  3. Place the script call on your page, preferably in the header, but it doesn’t have to be if you use frames etc.
<script src="https://www.google.com/recaptcha/api.js" async defer> </script>

4. WITHIN THE FORM TAGS of your page, place the widget. This is the “Implicit” method of displaying the reCAPTCHA widget.

<div class="g-recaptcha"></div>

5. Create a javascript function that you can call to validate the CAPTCHA response from Google.

function VerifyCAPTCHA() {
	var response = grecaptcha.getResponse();

    alert(response);

    if(response.length == 0)
    {
        // reCaptcha not verified
        var textbox = document.getElementById('errortext');
        textbox.value = 'reCAPTCHA failed you BOT!';
        return false;
    }
    else
    {
		window.location.assign('http://www.yourdomain.com/MyResultsPage.html');
    }
};
6. Create the button that will validate your CAPTCHA on submit.
        <div>
            <input type="button" name="submitButton" value="Show Results!" onclick="javascript: VerifyCAPTCHA();" />
        </div>

A few things to note:
1) Notice that the button is of type “button”, not “submit”. This is because we’re going to depend on our VerifyCAPTCHA() function to redirect us on success.
2) You don’t need your Private/Secret key for this type of implementation.

Now when you run your page with the CAPTCHA on it, it will have to pass this response test before redirecting. On the page you’re redirecting to, you should have some code to verify that the referrer is your CAPTCHA page. This is because any BOT can look for window.location statements and follow them on their own, so you’ll want to stop deep linking in it’s tracks.

The version 1.0 of reCAPTCHA used to allow you to debug using localhost without issue. The new version doesn’t. I can only assume that this was done for security reasons.

I hope that this post helps you to get up and running quickly with Google’s reCAPTCHA without the two or three hours of frustration that I had.

Google reCAPTCHA in .NET MVC

This post is using Microsoft .NET in C# with Visual Studio Community 2015 edition.

Google reCAPTCHA
Google reCAPTCHA

Google has a client side implementation for their reCAPTCHA on your web pages. Their documentation is great at explaining what it is, but it lacks in specific examples for how to implement in different environments. This causes confusion with some developers when they paste the two lines of code in their web view, but they are still able to submit the page, even when they don’t fill in the CAPTCHA.

There are a few Nuget packages that have widget wrappers, but that’s not really necessary.

Another issue is that there is an older Version 1.0 and the newer 2.0 (“I’m not a robot”) implementation. I think most folks would prefer the newer one.

Also, I’ve noticed during testing that it may be easily possible to get through the CAPTCHA the first time. On subsequent requests, probably based on IP address, it creates a popup that you have to select photos from. That should stop most bot engines. Just mentioning so you don’t think there’s a problem if you still occasionally get a form submit that looks like it could be a bot.

The getting started section of the Google Developer’s Guide is fine for getting started, but I’ll still cover it here, as I strongly dislike blog posts that only show 80% of the solution.

First thing you need is the actual URL that your going to deploy the application on. So if you haven’t registered one yet, you should do that now. I don’t know how Google handles it when two people try to register the same domain with reCAPTCHA, but I would assume that it would be questioned at some level. Maybe I’ll do an investigation in the future when I’m bored. I just feel that I don’t want to setup a domain under my Google account and then find out later someone else registered the domain and I’ve made a potential problem for them.

Okay, so let’s get started:

  1. Register your domain, as previously mentioned.
  2. Sign up for your reCAPTCHA at Google.
    1. Save your site and private/secret key somewhere in your source control
  3. Right click on your solution and select “Manage NuGet Packages for Solution…” (Note, many MVC solutions may already have this installed)
    1. Click on “Browse”
    2. Search for “Newtonsoft.Json”
    3. Highlight it in the results
    4. Check the box next to your Web Application and click “Install”
  4. Place the script call on your view, preferably in the header, but it doesn’t have to be if you use a _Layout view. I have it right after the @using and @model statements.
<script src="https://www.google.com/recaptcha/api.js" async defer></script>

5. WITHIN THE FORM TAGS of your view, place the widget. This is the “Implicit” method of displaying the reCAPTCHA widget.

@using (Html.BeginForm("Register", "Account", FormMethod.Post))
{
    <div class="g-recaptcha"></div>
}

6. Create a class to hold the response from Google

using System;
using System.Collections.Generic;
using Newtonsoft.Json;

namespace TimeTracker.web
{
	public class CaptchaResponse
	{

		[JsonProperty("success")]
		public bool Success { get; set; }

		[JsonProperty("error-codes")]
		public List<string> ErrorCodes { get; set; }

		[JsonProperty("challenge_ts")]
		public DateTime TimeStamp { get; set; }

		[JsonProperty("hostname")]
		public string HostName { get; set; }

	}
}

6. Create a private method in your controller to verify the reCAPTCHA

///
/// Check if the reCAPTCHA challenge was successful
///
private bool VerifyCaptcha()
{
	var response = Request["g-Recaptcha-Response"];
	string secret = "Your_Private/Secret_Key_From_Google";

	var client = new WebClient();
	var reply =
	client.DownloadString(

	string.Format("https://www.google.com/recaptcha/api/siteverify?secret={0}&response={1}",secret, response));

	CaptchaResponse captchaResponse = JsonConvert.DeserializeObject(reply);

	// Optionaly, look for messages if response is false, caution, response collection could still be null
	if (!captchaResponse.Success)
	{
		return false;
	}

	return true;

}

7. Handle the checking in your [HttpPost] controller method for the view.

[HttpPost]
public ActionResult Create(Models.TaxWorksheetModel model)
{
	if (!ModelState.IsValid)
	{
		return View(model)
	}

	if (!VerifyCaptcha())
	{
		ModelState.AddModelError("", "There was an error validating the Captcha, please try again.");
		return View(model);
	}
	else
	{
		return RedirectToAction("Thanks");
	}
}

I’ll do my best to put up a post on how to debug this code in the near future. I hope that this post helps you to get up and running quickly with Google’s reCAPTCHA without the frustration that I had.