Business Objects for Paging

This entry is part 3 of 7 in the series Ultimate MVC Paging

The .NET framework contains lots of ways to pass around data generically now. Tuples, Dynamic Type, Anonymous types, ExpandoObject, etc. But my favorite is a standard style DTO using Generic collections that I can use for business objects for paging.

For paging in an N-Tier, Repository Pattern environment, I use two different Data Transfer Objects (DTO). The first, is an object used to transfer the basic data from the data layer that contains the collection of return objects or DataRows. The second is a more detailed and flexible object that contains all the properties to satisfy 90% of my paging needs.

DataLayer PageResults

Since my service layer already knows the intimate details of my paging request, when I receive back the response from the data layer, I really only need two things:

  1. The collection I asked for
  2. The total number of items that fit my filters

I’ve always felt that if the API was made public, that less is more.

Yes, I could use the same object that I use in the service layer that has more properties, but I’ve always felt that if the API was made public, that less is more. The data layer return object is one that I call PagedResults, it’s structured like:

using System;
using System.Collections.Generic;

namespace YTG.Models
{

    /// <summary>
    /// Yasgar Technology Group, Inc.
    /// http://www.ytgi.com
    /// </summary>
    /// <typeparam name="TEntity"></typeparam>
    public class PagedResult<TEntity> : IPagedResult<TEntity>
    {

        /// <summary>
        /// Primary constructor
        /// </summary>
        /// <param name="items"></param>
        /// <param name="totalCount"></param>
        public PagedResult(List<TEntity> items, int totalCount)
        {
            Items = items;
            TotalCount = totalCount;
        }

        private Lazy<List<TEntity>> m_items = new Lazy<List<TEntity>>(() => new List<TEntity>(), true);

        /// <summary>
        /// IEnumerable list of items returned
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public List<TEntity> Items
        {
            get { return m_items.Value; }
            set
            {
                m_items = new Lazy<List<TEntity>>(() => value, true);
            }
    }

        /// <summary>
        /// Total count of records from query
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int TotalCount { get; set; } = 0;

    }
}

You can read a little more about this object here:

Your data layer will pull back a dataset or IQueryable resultset. You can stuff that into the PagedResult with the following code:

return new PagedResult<Models.LuCategory>(_entityrows.ToList(), _totalCount);

I usually do this in a service layer before my web service front end, so something like:

  1. Pull back a DataSet or Entity IQueryable collection
  2. Return that to the service layer
  3. Loop through and map to a DTO and stuff into the Paging DTO

A ha! I know what you’re thinking, I would just stuff the DataSet or Entity collection in there and send it back, save some time. Right? I don’t recommend that for one main reason; you’re tying your UI to the structure of your database or ORM. I don’t want to digress in this post as to why this is not desirable, but I promise to discuss it in another post soon.

Hate writing redundant mapping code? Read this post: Stuffing DataSets into your objects.

I return this object from my system REST API or WCF web service. In my service layer in the web site, I combine the results into my UI based DTO that has the rest of the properties needed for paging, called SearchModel. It is IMPERATIVE that this DTO be based on the Interface in the project. The SearchModel object looks like this:

using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Text.Json.Serialization;

namespace YTG.MVC.Lookups.Models
{

    /// <summary>
    /// View Model for Searching and holding results
    /// </summary>
    public class SearchModel<TEntity> : ISearchModel
    {

        /// <summary>
        /// Default constructor
        /// </summary>
        public SearchModel() { }


        public SearchModel(int totalItems, int? page, int pageSize = 10)
        {
            TotalItems = totalItems;
            CurrentPage = page ?? 1;
            PageSize = pageSize;
        }

        private int m_TotalItems = 0;
        private int m_CurrentPage = 1;
        private int m_PageSize = 0;
        private Lazy<List<TEntity>> m_SortedResults = new Lazy<List<TEntity>>(() => new List<TEntity>(), true);

        /// <summary>
        /// Gets or set the total number of items to be paged
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int TotalItems
        {
            get
            { return m_TotalItems; }
            set
            {
                m_TotalItems = value;
                if (m_PageSize > 0)
                { TotalPages = (m_TotalItems + (m_PageSize - 1)) / m_PageSize; }
            }
        }

        /// <summary>
        /// Gets or sets the current page being displayed
        /// Automatically populates the values of other properties
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int CurrentPage
        {
            get
            { return m_CurrentPage; }
            set
            {
                m_CurrentPage = value < 1 ? 1 : value;
                StartPage = m_CurrentPage - 5;
                EndPage = m_CurrentPage + 4;

                if (StartPage <= 0)
                {
                    EndPage -= (StartPage - 1);
                    StartPage = 1;
                }

                if (EndPage > TotalPages)
                {
                    EndPage = TotalPages;
                    if (EndPage > 10)
                    {
                        StartPage = EndPage - 9;
                    }
                }
            }
        }

        /// <summary>
        /// Gets or sets the page size currently set
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int PageSize
        {
            get
            { return m_PageSize; }
            set
            {
                m_PageSize = value;
                if (m_TotalItems > 0)
                { TotalPages = (m_TotalItems + (m_PageSize - 1)) / m_PageSize; }
                // To get EndPage set correctly if this property is set last
                CurrentPage = m_CurrentPage;
            }
        }

        /// <summary>
        /// Gets or set the total number of pages based on the TotalItems and PageSize
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int TotalPages { get; set; } = 0;

        /// <summary>
        /// Gets or sets the start page for the pager display
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int StartPage { get; set; } = 0;

        /// <summary>
        /// Gets or sets the end page for the pager display
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int EndPage { get; set; } = 0;

        /// <summary>
        /// Gets or sets the search filter for the paged items
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public string SearchFilter { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the search term for the paged items
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public string SearchTerm { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the sort expression for the paged items
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public string SortColumn { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the previous sort expression for the paged items when posting back to controller
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public string PrevSortColumn { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the count of records in the current page
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public int RecordCount { get; set; } = 0;

        /// <summary>
        /// Gets or sets the sort order, default is asc
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public bool SortDescending { get; set; } = false;

        /// <summary>
        /// Gets or sets whether the paged items only contain active records
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        [Display(Name = "Active Only")]
        public bool ActiveOnly { get; set; } = true;

        /// <summary>
        /// Gets or sets the Reference Id, to be used for a parent object
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public long RefId { get; set; } = 0;

        /// <summary>
        /// Gets or sets the Reference Unique Id, to be used for a parent object
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public Guid RefUniqueId { get; set; } = Guid.Empty;

        /// <summary>
        /// Gets or sets the List of pre-sorted results
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public List<TEntity> SortedResults
        {
            get { return m_SortedResults.Value; }
            set
            {
                m_SortedResults = new Lazy<List<TEntity>>(() => value, true);
            }
        }


        /// <summary>
        /// Gets or sets the controller name for this Model
        /// Needed for Pagination partial views
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public string controllerName { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the action name for this model
        /// Needed for Pagination partial views
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        public string actionName { get; set; } = string.Empty;

    }
}

I use the simpler PagedResults to return the data collection and the total count from my repository. Then push this data, combined with whatever filter options were sent in back to the View.

You can take a look at how these objects are used in my sample app on GitHub:

https://github.com/YTGI/YTG-MVC-Lookups

Stuffing DataSets into your objects

One thing that most developers loath is writing mapping classes. One common need is stuffing datasets into your objects from a return of your SQL Client code. Yes, I know that there are utilities that are out there like AutoMapper, but here’s something to consider.

The power of many of these utilities are their ability to map properties that have the same name, but different case without any code. This is definitely a time saver. I would highly recommend their use in the service layer of your UI if you’re mapping one object, or a small collection of objects like a paged response of 25 or 50 objects. But don’t use these tools if you are processing batch size collections. Why you ask?

Think about it, how would you find out if there was a property in an object with a particular name regardless of case? Reflection of course, but Reflection has a serious processing cost. So using it for a small collection of objects being displayed to a user is one thing, but when you’re processing thousands, or millions of objects, there is nothing that beats declarative code for speed.

I want to make it clear, this method has lots of advantages, as well as disadvantages, there’s never a panacea…

Advantages

  1. One standard place that maps ALL your return values to your DTO
  2. No maintenance of numerous areas mapping data. Think of the scenario where a PK or FK is changed from int to long…
  3. The entire team ALWAYS knows where code is mapped, they don’t have to hunt through multiple service layers and mapping classes
  4. Your object can consume DataSets from multiple stored procedures seamlessly and still populate, even if the field names are different.

Disadvantages

  1. This is only recommended if you’re getting back DataSets, as otherwise, you’d be putting a dependency for an EF entity in your DTO. (Don’t do this, as then you have to reference your Repository in your UI project, which is, how do doctors say it, contra-recommended)
  2. If you’re filling your object from multiple stored procedures, it will hide the fact that a particular column your expecting is absent, which is a code smell.

With the aforementioned caveats, let’s show the concept. The reason I only recommend this for DataSets is that it’s part of the .NET framework. I vehemently recommend that DTO’s don’t have any references to objects that don’t exist either 1) In the .NET framework, or 2) In your common utilities libraries.

Here’s an example of the constructors you would have in such an object:

        /// <summary>
        /// Default constructor
        /// </summary>
        public SalesProduct() { }

        /// <summary>
        /// Constructor with a DataSet to populate
        /// </summary>
        /// <param name="p_DataRow"></param>
        public SalesProduct(DataRow p_DataRow)
        {
            Populate(p_DataRow);
        }

Remember that when you code a constructor, like the second one, you stop the ability to instantiate your object without a parameter. So, in most cases, you want to declare an empty “Default” constructor as well. Remember, if you don’t have an empty constructor, you won’t be able to deserialize the object from a web service call…

Next step is to have the Populate(DataSet) method in your object like this example:

        /// <summary>
        /// Populate this object from a DataRow
        /// www.ytgi.com
        /// </summary>
        /// <param name="p_DataRow"></param>
        private void Populate(DataRow p_DataRow)
        {

            if (p_DataRow.Table.Columns.Contains("ProductID"))
            { this.ProductId = p_DataRow.IsNull("ProductID") ? 0 : p_DataRow.Field<int>("ProductID"); }

            if (p_DataRow.Table.Columns.Contains("ProductName"))
            { this.ProductName = p_DataRow.IsNull("ProductName") ? string.Empty : p_DataRow.Field<string>("ProductName"); }
            if (p_DataRow.Table.Columns.Contains("Description"))
            { this.ProductName = p_DataRow.IsNull("Description") ? string.Empty : p_DataRow.Field<string>("Description"); }


            if (p_DataRow.Table.Columns.Contains("UnitPrice"))
            { this.UnitPrice = p_DataRow.IsNull("UnitPrice") ? 0m : p_DataRow.Field<decimal>("UnitPrice"); }

            if (p_DataRow.Table.Columns.Contains("OnlineOrderFlag"))
            { this.OnlineOrderFlag = p_DataRow.IsNull("OnlineOrderFlag") ? false : p_DataRow.Field<bool>("OnlineOrderFlag"); }

            if (p_DataRow.Table.Columns.Contains("ShipDate"))
            { this.ShipDate = p_DataRow.IsNull("ShipDate") ? DateTime.MinValue : p_DataRow.Field<DateTime>("ShipDate"); }

        }

Notice one thing in the above code that’s my favorite feature of using this method, the ProductName property is being populated by either the “ProductName” column or the “Description” column, depending on the stored procedure you’re calling.

I typically use this implementation when I return a DataSet from a stored procedure and load the objects like this:

List<Models.SalesProduct> _sales = new List<Models.SalesProduct>();

// Would normally do this in the ServiceLayer
if (_returnData != null)
{
   foreach (DataRow _row in _returnData.Tables[0].Rows)
   {
       _sales.Add(new Models.SalesProduct(_row));
   }
}

I hope this helps folks to think of different ways to consolidate code that doesn’t violate the patterns they like to use.

If you’re going through the Paging series, visit the next post to continue: Business Objects for Paging.

Ultimate Guide to MVC Paging!

This entry is part 1 of 7 in the series Ultimate MVC Paging
MVC Paging Partial Views

After searching the web for MVC paging techniques, I realized that 90% or more of the examples I found on the web would not solve my problem. Some of them are so bad that they could get me fired if I followed their example. (copy -> paste). They so oversimplify the need, that they render their solutions useless. The intentions are good, but the implementation is lacking.

Have you:

  • Wanted to add paging to your MVC site, but think it’s too complicated?
  • Implemented it on one page and hope you never have to do it again?
  • After implementing paging thought, “There’s got to be a better way?”

The most popular problem with most examples that I found is that they show pulling back the entire collection from the database and paging the results in the UI (in the controller). This is fine if you’re absolutely positive, sure beyond belief, willing to bet your job, that the results will never exceed a hundred records or so…

If you want to become an expert on how to implement fast, extremely flexible paging in all your MVC applications, then figure that this is going to take an hour or two. If you’re looking to “turn on paging”, then feel free to click on the “Back” button, I won’t be offended.

Not implementing paging, or implementing it incorrectly can cause serious problems with the performance of your web site. Below are some symptoms that I’ve seen that leads me to believe there could be a problem with the paging implementation on a web site:

  1. Complaints that the web site slows down, but nobody can see high utilization on the web server or database server (and no expensive queries)
  2. Complaints that the web site sometimes stops responding, occasionally for minutes at a time
  3. Complaints that the web site sometimes suddenly throws users back to the login screen while they’re working
  4. Complaints that the slow downs are intermittent, and can’t be reproduced in the test environment

I know you may be thinking, what does this have to do with paging? Well, it’s tangentially related, as it has to do with writing bad code without thinking about what the consequences are. The reason these issues don’t tax the servers is that doing a SELECT * FROM Clients doesn’t really tax the CPU that much. Now, if you were watching the NIC card traffic, that would be a different story.

The biggest issue that I see with code examples on the web:

  1. They show bringing back every row in the table and then page in the controller (No, no, please don’t do this!)
  2. They show using a LINQ query in the controller to Skip and Take
  3. They assume that I’m pulling back my data directly from the database in my Web Application

// Example of code that is ridiculous outside of a POC app
return View(MyRepository.GetClients().Skip((page - 1 ?? 0) * (rows ?? 3)).Take(rows ?? 3));

This LINQ query will work well if:

  1. The MyRepository is an EntityFramework repository
  2. AND you’re implementing your Repository in your web site
  3. AND it doesn’t dispose of the Entity context inside the method
  4. AND the GetClients() method doesn’t call ToList() inside of it
  5. AND GetClients() is not executing a stored procedure

If any of the above is not true, then this method is no better than the first method that pulls the entire table back.

Another issue with this example is that it assumes that the Repository is part of your web project AND you’re using the objects that are in your Entity Model. So no DTO’s, no business objects, and absolutely no Web Service in the middle.

The reason I’m showing this, is that I’ve never seen these issues discussed in the posts. A developer could easily implement this on a GetClients() that again, returns the entire table contents and then pages on the UI side.

Just in case you forgot, the web is stateless, so if IIS or most other Web Servers feel that they’ve had about enough with the application memory usage, they just reset themselves. Shouldn’t cause a problem right? Except if you’re depending on session to keep people logged in, then they get kicked back to the login screen. (BTW, this only happens if you have your own custom code looking for a session object to verify the user is logged in. If you use the proper MVC standard Authorization, they shouldn’t be sent to the login screen.) But not to minimize the impact, because all users get kicked out, not just the one user.

Almost every web, and even windows project I’ve ever worked on followed the Repository pattern in that there were separations of concerns, i.e.:

Database → DataLayer → ServiceLayer → WebService → ProxyClass → ServiceLayer → UI

I don’t want to digress into a discussion of patterns. But the ServiceLayer behind the WebService will convert the Entity Framework object or DataSet to a DTO collection. Then the ServiceLayer behind the UI will likely convert it to a ViewModel. We don’t want to do all that work on 5000 records so that we can ultimately display 25 to the user…


This search on Google returned 466,000 records. Do you think that they were all returned and paged in the controller? I think not…

The reason I’m discussing this is to get your imagination started. If you didn’t want to transfer tons of data over the wire from your WebService, then where is the best place to implement Paging? You guessed it, in the DataLayer (also referred to as Data Access Layer (DAL)). As close to the data as you can possibly get.

Alright, let’s get to the nuts and bolts. I’m going to show you a full implementation, so you can trim it down if your particular need is less complicated.

Anatomy of a Paging Request

The following are the properties that I’ve been able to accomplish almost any paging task I’ve ever needed on the client side:

Property Type Notes
SortedResults List<T> The list of results, without it, we don’t need paging!
TotalItems int The total number of items that would be returned, if we returned them all at once
CurrentPage int The current page the user is on
PageSize int The size of the page, usually an option for the user to select
TotalPages int The total number of pages (TotalItems/PageSize rounded up)
StartPage int The start page, for when you have more pages than will fit on one widget
EndPage int The end page, for when you have more pages than will fit on one widget
SearchFilter string A filter to use in addition to any search criteria, usually selected from a dropdown, i.e. year, division, salesman, etc.
SearchTerm string A search term, usually typed in a search box by the user
SortExpression string The column name to sort the results by
RecordCount int The quantity of records in this particular result set
SortOrder string The order that you want the result set, i.e.: “ASC”, “DESC”
ActiveOnly bool Another filter flag, if you have an IsActive flag in your table, or any other criteria to knock out inactive rows
RefId int Typically used when there is a Parent/Child relationship. i.e. Your SortedList is a list of Project tasks, then you’ll need to know the parent project PK Id

Knock out 1/1/0001 Dates in MVC

Back when I coded in Visual Studio 2003, we used to joke that if you wrote a small app that had 100 lines of code, 98 of them would be checking for nulls. It actually wasn’t that far off.

Because of that, I often force my DTO’s to default to DateTime.MinValue instead of nulls and have my code look for MinValue rather than checking for nulls all the time. This has one side effect that I didn’t like, when you use your model in an MVC view, it will display in the text box like “01/01/0001 00:00:00” etc. This is obviously not desirable.

The secondary issue is that when you use Date/Time pickers in the view, they don’t like being bound to DateTime properties, they work better on a string based text box. So what I’ve done is resolve both issues with one solution.

First part is in the ViewModel. I often just inherit my DTO and add or override properties with Attributes/Decorations that I need for the View. I’ll use my ProjectViewModel as an example:

    public class ProjectViewModel : Project
    {
        /// <summary>
        /// Gets or sets the Project Start Date Display
        /// Use to get rid of 01/01/0001 displays
        /// </summary>
        [Display(Name = "Start Date: ")]
        public string StartDateDisplay
        {
            get
            {
                return StartDate.ToShortDateDisplay();
            }
            set
            {
                StartDate = value.ToDateFromString();
            }
        }
    }

In this case, my Project DTO has a StartDate property that is a DateTime. It could be null, or it could be DateTime.MinValue, or it could be a real date. In this implementation, I don’t care about the time, but you could build this out to include time very easily.

When I use this in a view, I use the StartDateDisplay property, rather than the StartDate property.

@Html.TextBoxFor(model => model.StartDateDisplay, new { @class = "form-control", @id = "txtStartDate", @title = "Project Start Date", @type = "text", autocomplete = "off" })

Now this will play nice with the Date/Time picker because it is a string property.

In order to get the dates formated, you’ll notice the ToShortDateDisplay() and the ToDateFromString() extension methods. Those are defined as:

        /// <summary>
        /// Convert a date to a short date string, empty if min or max value
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        /// <param name="value"></param>
        /// <returns></returns>
        public static string ToShortDateDisplay(this DateTime value)
        {
            if (value == null) { return string.Empty; }

            if ((value == DateTime.MaxValue) || (value == DateTime.MinValue))
            {
                return string.Empty;
            }
            else
            {
                return value.ToShortDateString();
            }
        }

        /// <summary>
        /// Convert a string in MM/DD/CCYY format to a valid date
        /// Yasgar Technology Group, Inc. - www.ytgi.com
        /// </summary>
        /// <param name="value"></param>
        /// <returns></returns>
        public static DateTime ToDateFromString(this string value)
        {
            if (!string.IsNullOrWhiteSpace(value))
            {
                if ((value == "99999999") || (value == "99/99/9999"))
                {
                    return DateTime.MaxValue;
                }
                else
                {
                    if (!string.IsNullOrWhiteSpace(value))
                    {
                        DateTime _value;
                        DateTime.TryParse(value, out _value);
                        return _value;
                    }
                    else
                    {
                        return DateTime.MinValue;
                    }
                }
            }

            return DateTime.MinValue;

        }

Solved two issues with one solution. No checking for nulls, no checking for DateTime.MinValue.

Google reCAPTCHA in .NET MVC

This post is using Microsoft .NET in C# with Visual Studio Community 2015 edition.

Google reCAPTCHA
Google reCAPTCHA

Google has a client side implementation for their reCAPTCHA on your web pages. Their documentation is great at explaining what it is, but it lacks in specific examples for how to implement in different environments. This causes confusion with some developers when they paste the two lines of code in their web view, but they are still able to submit the page, even when they don’t fill in the CAPTCHA.

There are a few Nuget packages that have widget wrappers, but that’s not really necessary.

Another issue is that there is an older Version 1.0 and the newer 2.0 (“I’m not a robot”) implementation. I think most folks would prefer the newer one.

Also, I’ve noticed during testing that it may be easily possible to get through the CAPTCHA the first time. On subsequent requests, probably based on IP address, it creates a popup that you have to select photos from. That should stop most bot engines. Just mentioning so you don’t think there’s a problem if you still occasionally get a form submit that looks like it could be a bot.

The getting started section of the Google Developer’s Guide is fine for getting started, but I’ll still cover it here, as I strongly dislike blog posts that only show 80% of the solution.

First thing you need is the actual URL that your going to deploy the application on. So if you haven’t registered one yet, you should do that now. I don’t know how Google handles it when two people try to register the same domain with reCAPTCHA, but I would assume that it would be questioned at some level. Maybe I’ll do an investigation in the future when I’m bored. I just feel that I don’t want to setup a domain under my Google account and then find out later someone else registered the domain and I’ve made a potential problem for them.

Okay, so let’s get started:

  1. Register your domain, as previously mentioned.
  2. Sign up for your reCAPTCHA at Google.
    1. Save your site and private/secret key somewhere in your source control
  3. Right click on your solution and select “Manage NuGet Packages for Solution…” (Note, many MVC solutions may already have this installed)
    1. Click on “Browse”
    2. Search for “Newtonsoft.Json”
    3. Highlight it in the results
    4. Check the box next to your Web Application and click “Install”
  4. Place the script call on your view, preferably in the header, but it doesn’t have to be if you use a _Layout view. I have it right after the @using and @model statements.
<script src="https://www.google.com/recaptcha/api.js" async defer></script>

5. WITHIN THE FORM TAGS of your view, place the widget. This is the “Implicit” method of displaying the reCAPTCHA widget.

@using (Html.BeginForm("Register", "Account", FormMethod.Post))
{
    <div class="g-recaptcha"></div>
}

6. Create a class to hold the response from Google

using System;
using System.Collections.Generic;
using Newtonsoft.Json;

namespace TimeTracker.web
{
	public class CaptchaResponse
	{

		[JsonProperty("success")]
		public bool Success { get; set; }

		[JsonProperty("error-codes")]
		public List<string> ErrorCodes { get; set; }

		[JsonProperty("challenge_ts")]
		public DateTime TimeStamp { get; set; }

		[JsonProperty("hostname")]
		public string HostName { get; set; }

	}
}

6. Create a private method in your controller to verify the reCAPTCHA

///
/// Check if the reCAPTCHA challenge was successful
///
private bool VerifyCaptcha()
{
	var response = Request["g-Recaptcha-Response"];
	string secret = "Your_Private/Secret_Key_From_Google";

	var client = new WebClient();
	var reply =
	client.DownloadString(

	string.Format("https://www.google.com/recaptcha/api/siteverify?secret={0}&response={1}",secret, response));

	CaptchaResponse captchaResponse = JsonConvert.DeserializeObject(reply);

	// Optionaly, look for messages if response is false, caution, response collection could still be null
	if (!captchaResponse.Success)
	{
		return false;
	}

	return true;

}

7. Handle the checking in your [HttpPost] controller method for the view.

[HttpPost]
public ActionResult Create(Models.TaxWorksheetModel model)
{
	if (!ModelState.IsValid)
	{
		return View(model)
	}

	if (!VerifyCaptcha())
	{
		ModelState.AddModelError("", "There was an error validating the Captcha, please try again.");
		return View(model);
	}
	else
	{
		return RedirectToAction("Thanks");
	}
}

I’ll do my best to put up a post on how to debug this code in the near future. I hope that this post helps you to get up and running quickly with Google’s reCAPTCHA without the frustration that I had.

Google reCAPTCHA in .NET ASPX

This post is using Microsoft .NET in C# with Visual Studio Community 2015 edition.

Google reCAPTCHA
Google reCAPTCHA

Google has a client side implementation for their reCAPTCHA on your web pages. Their documentation is great at explaining what it is, but it lacks in specific examples for how to implement in different environments. This causes confusion with some developers when they paste the two lines of code in their web page, but they are still able to submit the page, even when they don’t fill in the CAPTCHA.

There are a few Nuget packages that have widget wrappers, but that’s not really necessary.

Another issue is that there is an older Version 1.0 and the newer 2.0 (“I’m not a robot”) implementation. I think most folks would prefer the newer one.

Also, I’ve noticed during testing that it may be easily possible to get through the CAPTCHA the first time. On subsequent requests, probably based on IP address, it creates a popup that you have to select photos from. That should stop most bot engines. Just mentioning so you don’t think there’s a problem if you still occasionally get a form submit that looks like it could be a bot.

The getting started section of the Google Developer’s Guide is fine for getting started, but I’ll still cover it here, as I strongly dislike blog posts that only show 80% of the solution.

First thing you need is the actual URL that your going to deploy the application on. So if you haven’t registered one yet, you should do that now. I don’t know how Google handles it when two people try to register the same domain with reCAPTCHA, but I would assume that it would be questioned at some level. Maybe I’ll do an investigation in the future when I’m bored. I just feel that I don’t want to setup a domain under my Google account and then find out later someone else registered the domain and I’ve made a potential problem for them.

Okay, so let’s get started:

  1. Register your domain, as previously mentioned.
  2. Sign up for your reCAPTCHA at Google.
    1. Save your site and private/secret key somewhere in your source control
  3. Right click on your solution and select “Manage NuGet Packages for Solution…”
    1. Click on “Browse”
    2. Search for “Newtonsoft.Json”
    3. Highlight it in the results
    4. Check the box next to your Web Application and click “Install”
  4. Place the script call on your page, preferably in the header, but it doesn’t have to be if you use master pages etc. I have it right after the asp:Content start key.
<script src="https://www.google.com/recaptcha/api.js" async defer></script>

NOTE: My IntelliSense in VS recognizes the two keywords “async” and “defer”, but still flags them with an HTML5 warning. You can ignore this.

5. WITHIN THE FORM TAGS of your page, place the widget. This is the “Implicit” method of displaying the reCAPTCHA widget.

<form runat="server" class="form-horizontal">
	<div class="g-recaptcha" data-sitekey="Site_Key_Provided_By_Google"><div>
<form>

6. Create a class to hold the response from Google

using System;
using System.Collections.Generic;
using Newtonsoft.Json;

namespace TimeTracker.web
{
	public class CaptchaResponse
	{

		[JsonProperty("success")]
		public bool Success { get; set; }

		[JsonProperty("error-codes")]
		public List<string> ErrorCodes { get; set; }

		[JsonProperty("challenge_ts")]
		public DateTime TimeStamp { get; set; }

		[JsonProperty("hostname")]
		public string HostName { get; set; }

	}
}
  1. Create a private method in your code behind to verify the reCAPTCHA
/// <summary>
/// Check if the reCAPTCHA challenge was successful
/// </summary>
/// <returns></returns>
private bool VerifyCaptcha()
{
	var response = Request.Form["g-Recaptcha-Response"];
	string secret = "Your_Private/Secret_Key_From_Google";

	var client = new WebClient();
	var reply = client.DownloadString(
				string.Format("https://www.google.com/recaptcha/api/siteverify?secret={0}&response={1}",
								secret, response));

	CaptchaResponse captchaResponse = JsonConvert.DeserializeObject<CaptchaResponse>(reply);

	// Optionaly, look for messages if response is false, caution, response collection could still be null
	if (!captchaResponse.Success)
	{
		return false;
	}

	return true;

}
  1. Handle the checking in your button submit event
protected void btnSubmit_Click(object sender, EventArgs e)
{
	if (VerifyCaptcha())
	{
		Response.Redirect("Your_SubmitSuccessPage.aspx");
	}
	else
	{
		// Send shock to user's chair
	}
}

The version 1.0 of reCAPTCHA used to allow you to debug using localhost without issue. The new version doesn’t. I can only assume that this was done for security reasons.

Update: If you add localhost as an approved domain in the reCAPTCHA setup, you should be able to debug locally.

I hope that this post helps you to get up and running quickly with Google’s reCAPTCHA without the two or three hours of frustration that I had.

List Shuffle / Random Order

Had a project where we needed to be able to process items in a list in random order. The implementation of the random order was a little verbose. This extension method on IList<T> works pretty well for me.

static readonly Random _random = new();

/// <summary>
/// This is a Shuffle extension to shuffle the order of Lists randomly
/// </summary>
/// <typeparam name="T"><typeparam>
/// <param name="list"></param>
public static void Shuffle<T>(this IList<T> list)
{
	int intListCount = list.Count;
	while (intListCount > 1)
	{
		intListCount--;
		int intRandom = _random.Next(intListCount + 1);
		T value = list[intRandom];
		list[intRandom] = list[intListCount];
		list[intListCount] = value;
	}
}

The above sample uses the System.Random implementation to retrieve a random number by passing in the count of the list + 1. The reason I’m adding 1 here is because the Next method is zero (0) based, so it will return a number from 0 to your list count – 1. There is another overload that you can pass in the starting value and the ending value.

int intRandom = _random.Next(1, intListCount);

So this extension method will shuffle your list and give you back a list that is in random order each time.

If you’d like to see more extension methods, take a look at my post Jack’s Top 10 String Extension Methods