Stuffing DataSets into your objects

One thing that most developers loath is writing mapping classes. One common need is stuffing datasets into your objects from a return of your SQL Client code. Yes, I know that there are utilities that are out there like AutoMapper, but here’s something to consider.

The power of many of these utilities are their ability to map properties that have the same name, but different case without any code. This is definitely a time saver. I would highly recommend their use in the service layer of your UI if you’re mapping one object, or a small collection of objects like a paged response of 25 or 50 objects. But don’t use these tools if you are processing batch size collections. Why you ask?

Think about it, how would you find out if there was a property in an object with a particular name regardless of case? Reflection of course, but Reflection has a serious processing cost. So using it for a small collection of objects being displayed to a user is one thing, but when you’re processing thousands, or millions of objects, there is nothing that beats declarative code for speed.

I want to make it clear, this method has lots of advantages, as well as disadvantages, there’s never a panacea…

Advantages

  1. One standard place that maps ALL your return values to your DTO
  2. No maintenance of numerous areas mapping data. Think of the scenario where a PK or FK is changed from int to long…
  3. The entire team ALWAYS knows where code is mapped, they don’t have to hunt through multiple service layers and mapping classes
  4. Your object can consume DataSets from multiple stored procedures seamlessly and still populate, even if the field names are different.

Disadvantages

  1. This is only recommended if you’re getting back DataSets, as otherwise, you’d be putting a dependency for an EF entity in your DTO. (Don’t do this, as then you have to reference your Repository in your UI project, which is, how do doctors say it, contra-recommended)
  2. If you’re filling your object from multiple stored procedures, it will hide the fact that a particular column your expecting is absent, which is a code smell.

With the aforementioned caveats, let’s show the concept. The reason I only recommend this for DataSets is that it’s part of the .NET framework. I vehemently recommend that DTO’s don’t have any references to objects that don’t exist either 1) In the .NET framework, or 2) In your common utilities libraries.

Here’s an example of the constructors you would have in such an object:

        /// <summary>
        /// Default constructor
        /// </summary>
        public SalesProduct() { }

        /// <summary>
        /// Constructor with a DataSet to populate
        /// </summary>
        /// <param name="p_DataRow"></param>
        public SalesProduct(DataRow p_DataRow)
        {
            Populate(p_DataRow);
        }

Remember that when you code a constructor, like the second one, you stop the ability to instantiate your object without a parameter. So, in most cases, you want to declare an empty “Default” constructor as well. Remember, if you don’t have an empty constructor, you won’t be able to deserialize the object from a web service call…

Next step is to have the Populate(DataSet) method in your object like this example:

        /// <summary>
        /// Populate this object from a DataRow
        /// www.ytgi.com
        /// </summary>
        /// <param name="p_DataRow"></param>
        private void Populate(DataRow p_DataRow)
        {

            if (p_DataRow.Table.Columns.Contains("ProductID"))
            { this.ProductId = p_DataRow.IsNull("ProductID") ? 0 : p_DataRow.Field<int>("ProductID"); }

            if (p_DataRow.Table.Columns.Contains("ProductName"))
            { this.ProductName = p_DataRow.IsNull("ProductName") ? string.Empty : p_DataRow.Field<string>("ProductName"); }
            if (p_DataRow.Table.Columns.Contains("Description"))
            { this.ProductName = p_DataRow.IsNull("Description") ? string.Empty : p_DataRow.Field<string>("Description"); }


            if (p_DataRow.Table.Columns.Contains("UnitPrice"))
            { this.UnitPrice = p_DataRow.IsNull("UnitPrice") ? 0m : p_DataRow.Field<decimal>("UnitPrice"); }

            if (p_DataRow.Table.Columns.Contains("OnlineOrderFlag"))
            { this.OnlineOrderFlag = p_DataRow.IsNull("OnlineOrderFlag") ? false : p_DataRow.Field<bool>("OnlineOrderFlag"); }

            if (p_DataRow.Table.Columns.Contains("ShipDate"))
            { this.ShipDate = p_DataRow.IsNull("ShipDate") ? DateTime.MinValue : p_DataRow.Field<DateTime>("ShipDate"); }

        }

Notice one thing in the above code that’s my favorite feature of using this method, the ProductName property is being populated by either the “ProductName” column or the “Description” column, depending on the stored procedure you’re calling.

I typically use this implementation when I return a DataSet from a stored procedure and load the objects like this:

List<Models.SalesProduct> _sales = new List<Models.SalesProduct>();

// Would normally do this in the ServiceLayer
if (_returnData != null)
{
   foreach (DataRow _row in _returnData.Tables[0].Rows)
   {
       _sales.Add(new Models.SalesProduct(_row));
   }
}

I hope this helps folks to think of different ways to consolidate code that doesn’t violate the patterns they like to use.

If you’re going through the Paging series, visit the next post to continue: Business Objects for Paging.

Stored Procedure for Paging in SQL

Implementing paging correctly is critically important. If you’re shop still uses, or insists on stored procedures, that’s fine. It’s just as easy to write a stored procedure for paging in SQL.

This example was developed on Microsoft SQL server 2016, although I believe it will work fine back to SQL 2012, not sure about SQL 2008.

As with any solution, the idea is to keep the data at rest and only pull out what you need to return. This example uses the AdventureWorks database.

This example has all of the typical parameters that I use on many paging implementations. You can trim out the ones not needed for your implementation.

USE [AdventureWorks]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

-- Created by Jack Yasgar, www.jackyasgar.net
CREATE PROCEDURE [dbo].[uspGetOrderDetailsYTGI]
	@PageNumber			INT,
	@PageSize			INT,
	@SearchTerm			VARCHAR(100) = NULL,
	@SortDirection		VARCHAR(4) = 'ASC',
	@SortColumn			VARCHAR(100) = 'lastname',
	@Filter				VARCHAR(50) = '',
	@ActiveOnly			BIT
AS
BEGIN
    SET NOCOUNT ON;

 	IF ((@PageNumber = 0) OR (@PageNumber IS NULL))
	BEGIN
		SET @PageNumber = 1;
	END

	IF ((@PageSize = 0) OR (@PageSize IS NULL))
	BEGIN
		SET @PageSize = 25;
	END

	DECLARE @FirstRec INT, @LastRec INT;
	DECLARE @OnlineOrder BIT;
	SET @OnlineOrder = (SELECT CONVERT(BIT, @Filter));

	SELECT @FirstRec = (@PageNumber - 1) * @PageSize
	SELECT @LastRec = (@PageNumber * @PageSize + 1);
	

	SELECT 
		sod.OrderQty,
		sod.ProductID,
		p.[Name] AS ProductName,
		p.ProductNumber,
		soh.ShipDate,
		sod.UnitPrice,
		soh.OnlineOrderFlag
	FROM [Sales].[SalesOrderHeader] soh WITH (NOLOCK)
		LEFT JOIN [Sales].[SalesOrderDetail] sod WITH (NOLOCK) ON soh.SalesOrderID = sod.SalesOrderID
		LEFT JOIN [Production].[Product] p WITH (NOLOCK) ON p.ProductID = sod.ProductID
	WHERE 
		(ISNULL(@SearchTerm, '') = '' OR @SearchTerm = '' OR p.[Name] LIKE '%' + @SearchTerm + '%')
		AND
		(ISNULL(@Filter, 0) = 0 OR @Filter = 0 OR soh.OnlineOrderFlag = @OnlineOrder)
		AND
		(ISNULL(@ActiveOnly, 0) = 0 OR @ActiveOnly = 0 OR soh.ShipDate > '2013-01-01')
	ORDER BY 
		CASE WHEN @SortDirection = 'ASC' THEN
			CASE @SortColumn 
				WHEN 'ProductName'		THEN p.[Name]
				WHEN 'ProductNumber'	THEN p.ProductNumber
			END
		END,
		CASE WHEN @SortDirection = 'DESC' THEN
			CASE @SortColumn 
				WHEN 'ProductName'		THEN p.[Name]
				WHEN 'ProductNumber'	THEN p.ProductNumber
			END
		END DESC
	OFFSET @PageSize * (@PageNumber - 1) ROWS
	FETCH NEXT @PageSize ROWS ONLY;

		-- Return the full count for pager
	SELECT COUNT(*) AS TotalCount FROM (
			SELECT DISTINCT sod.SalesOrderDetailID
			FROM [Sales].[SalesOrderHeader] soh WITH (NOLOCK)
				LEFT JOIN [Sales].[SalesOrderDetail] sod WITH (NOLOCK) ON soh.SalesOrderID = sod.SalesOrderID
				LEFT JOIN [Production].[Product] p WITH (NOLOCK) ON p.ProductID = sod.ProductID
			WHERE 
				(ISNULL(@SearchTerm, '') = '' OR @SearchTerm = '' OR p.[Name] LIKE '%' + @SearchTerm + '%')
				AND
				(ISNULL(@Filter, 0) = 0 OR @Filter = 0 OR soh.OnlineOrderFlag = @OnlineOrder)
				AND
				(ISNULL(@ActiveOnly, 0) = 0 OR @ActiveOnly = 0 OR soh.ShipDate > '2013-01-01')
		) AS Counting

END

This query uses the SQL OFFSET command to start where we need to based upon the PageNumber and PageSize values passed in as arguments. It then uses the FETCH command to retrieve a specific row. You can take a look at the documentation on FETCH here:

https://docs.microsoft.com/en-us/sql/t-sql/language-elements/fetch-transact-sql?view=sql-server-ver16

Once you receive back the DataSet in your Repository, you’ll need to stuff the data into a collection of DTO’s in your ServiceLayer for transfer through your web service: Stuffing DataSets into your objects.

Paging with Entity Framework (Simple)

Whenever you have a need to show a paged list of your items, it is a must that this is done in your repository in Entity Framework so that you’re not passing around more data than is necessary. There are too many examples of paging on the web which show paging at the UI level, especially terrible ones are showing doing a LINQ to SQL query with ToList() after it. Horrible!

DON’T EVER DO THIS!

var _result = dbContext.Customers.ToList().Skip(page).Take(pageSize);

The following example is only called “Simple” because it addresses one table for paging. This will often be the case, but you can view a more complicated query here: Paging with Entity Framework (Advanced)

The example below accepts all the common parameters and handles assembling a Linq query. It DOES NOT execute the query until the entire statement is built. There is a necessary evil call to get the count, once the WHERE clause criteria is completed. You’ll notice that this call is done before adding the OrderBy so as not to burden the Count query with sorting.

The OrderBy statement is not standard in Linq. It uses an extension method that you can take a look at here: OrderBy Extension for Linq Queries

You’ll also notice that this method returns a PagedResult object. This is just a custom DTO that brings back a strongly typed list and a total count. You can see the code for this object here. PagedResult for Paging

This LINQ code was pulled directly from the LookupsRep class in the YTG MVC Lookups and Paging Demo Download.

/// <summary>
/// Simple Paged results from multiple tables in EF Query
/// Yasgar Technology Group, Inc. - www.ytgi.com
/// </summary>
/// <param name="Page"></param>
/// <param name="PageSize"></param>
/// <param name="SearchTerm"></param>
/// <param name="SearchFilter"></param>
/// <param name="SortColumn"></param>
/// <param name="SortOrder"></param>
/// <param name="ActiveOnly"></param>
/// <returns></returns>
public async Task<PagedResult<LuCategories>> GetLuCategoriesByEFAsync(int Page,
            int PageSize,
            string SearchTerm,
            string SearchFilter,
            string SortColumn,
            string SortOrder,
            bool ActiveOnly)
{
    try
    {
        int _skipRows = (Page - 1) * PageSize; // if this is not the first call, need move forward
        int _totalCount = 0; // placeholder for the total amount of records

        // Using var because this is returning an anonymous type
        var _entityrows = (from item in LuContext.LuCategories.Include(a => a.LuItems)
                           select item);

        if (ActiveOnly)
        {
            // Showing how to use ActiveOnly without a boolean flag as an example
            _entityrows = _entityrows.Where(er => er.IsActive == true);
        }

        if (!string.IsNullOrWhiteSpace(SearchFilter))
        {
            // This can be customized for each implementation
            _entityrows = _entityrows.Where(f => f.ShortName == SearchFilter);
        }

        if (!string.IsNullOrWhiteSpace(SearchTerm))
        {
            // This can be customized for each implementation
            _entityrows = _entityrows.Where(f => f.Name.Contains(SearchTerm.Trim()));
        }

        // Getting count will execute a SELECT COUNT(*)
        // Like to do this before adding sort criteria
        _totalCount = _entityrows.Count();

        if (!string.IsNullOrWhiteSpace(SortColumn))
        {
            bool IsSortDESC = false;
            if (SortOrder.ToLower() == "desc") { IsSortDESC = true; }

            _entityrows = _entityrows.OrderBy(SortColumn, IsSortDESC);
        }

        _entityrows = _entityrows.Skip(_skipRows).Take(PageSize);

        return new PagedResult<Models.LuCategories>(await _entityrows.ToListAsync(), _totalCount);

    }
    catch (Exception)
    {
        throw;
    }
}


If you’re viewing this as part of the Paging series, continue with that series here: Business Objects for Paging