Add headers to all downstream requests with DelegatingHandler example

When calling APIs from your website, you may wish to send headers and make other HTTP request modifications. In my case, I’d always want to ensure I send a specific correlation ID with each downstream request to any microservice.

When I create an HttpClient for the HttpFactory in the Dependency Injection Service Container, I don’t yet have access to the HttpContext of the incoming request. This is a bigger problem in APIs, where another application may be passing an existing correlation ID that I may want to grab and forward to any downstream requests.

On a website, I usually use a user identifier such as the user’s IdP unique ID. If I need or want to keep it more private, I just use the session ID.

Rather than add the correlation ID to each request in the repository classes, I want to have all transactions automatically check for an incoming correlation ID and, if so, use it as the correlation ID for downstream requests to any other APIs. If one doesn’t exist, then I’ll either generate one or use a different value.

To do this, I’ll create a special helper class to retrieve the value and override the SendAsync method to add it to any downstream request during the life of the current request.

    /// <summary>
    /// Handler for HTTP requests to propagate specific headers from the incoming request to outgoing requests
    /// </summary>
    public class RequestHeaderPropagationHandler(IHttpContextAccessor CurrentContext) : DelegatingHandler
    {

        /// <summary>
        /// Handle the SendAsync Event
        /// </summary>
        /// <param name="request"></param>
        /// <param name="cancellationToken"></param>
        /// <returns></returns>
        protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            Microsoft.AspNetCore.Http.HttpContext? httpContext = CurrentContext.HttpContext;

            if (httpContext != null)
            {
                string _corId = httpContext.GetCorrelationID();
                request.Headers.TryAddWithoutValidation("Accept", "application/json");
                request.Headers.TryAddWithoutValidation("X-CORRELATION-ID", _corId);
            }

            return base.SendAsync(request, cancellationToken);

        }
    }

You’ll notice I have an Extension method called GetCorrelationID() that I use to see if the incoming request has one I want to use.

        /// <summary>
        /// Get the CorrelationID for this HttpContext or replies with 
        /// </summary>
        /// <param name="value"></param>
        /// <param name="newCorrID"></param>
        /// <returns></returns>
        public static string GetCorrelationID(this HttpContext value, string newCorrID = "")
        {
            string _corId = string.Empty;
            StringValues header = value.Request.Headers["X-CORRELATION-ID"];

            if (header.Count > 0)
            {
                _corId = header.First() + string.Empty;
            }

            // Use the new correlation ID if provided
            if (!string.IsNullOrWhiteSpace(newCorrID))
            {
                _corId = newCorrID;
            }

            // No X-CORRELATION-ID sent in headers
            if (string.IsNullOrWhiteSpace(_corId))
            {
                _corId = Guid.NewGuid().ToString();
            }

            return _corId;

        }

In the above extension method, if there is no existing correlation ID in the incoming request, it will replace it with the one I’ve passed in; if not, it will generate a new GUID.

Startup.cs

In the SendAsync override, IHttpContextAccessor is not normally available in the dependency injection service collection. In the Startup.cs, make sure to enable that functionality:

// REQUIRED: Give downstream services access to the current HTTP context
services.AddHttpContextAccessor();

This will allow access the the HttpContext in any method within the application.

Next in the Startup.cs, you’ll need to register the RequestHeaderPropagationHandler. I make sure that this is registered as a transient, so it doesn’t store values from requests from other callers/users.

// REQUIRED: This is used in to propogate headers to downstream requests
services.AddTransient<RequestHeaderPropagationHandler>();

The last step is to add an http message handler to the Http Client in the Statup.cs.

services.AddHttpClient("lookups", c =>
{
    c.BaseAddress = new Uri(Configuration["AppSettings:LookupsBaseURL"]!);
})
.AddHttpMessageHandler<RequestHeaderPropagationHandler>();

Summary

So now you have the code that attempts to add an X-CORRELATION-ID header and value to any outgoing HTTP request made by this application.

C# MVC Detect which button was clicked

Detecting which button was clicked to cause a post back is very easy, once you know how to do it!

I often have screens that have multiple submit buttons on a single form that require a post back. I needed to pass the button value to the controller in MVC. For instance, I may have a data entry screen where a user can click a button that says “Save” or “Save as New”. I need to be able to detect which button they clicked when the page posts back.

Fortunately, there is an easy way to tell, or determine, which button the user selected when the page posts back on a submit action.

First, you need to have a button setup like so:

<form id="rvWidgetForm" method="post" enctype="application/x-www-form-urlencoded" asp-controller="Search" asp-action="SearchBegin">

    <button id="btnTopTen" type="submit" name="submitButton" class="btn text-center" value="TopTen">
        TOP 10 SEARCH
    </button>

    <button id="btnTraditional" type="submit" name="submitButton" class="btn  text-center" value="Trad">
        TRADITIONAL SEARCH
    </button>

</form>

The buttons must be of type=submit. Having type=button won’t post back. You have a choice here, to use the value= or not use it. If you don’t declare a value attribute, then what you’ll receive in the controller is the text of the button. While this is okay, you or another developer may change the text in the future and not realize they are going to break your code. I recommend using the value= like I’ve used above. It’s less likely to change in the future.

I recommend using the value attribute as it’s less likely than the button text to change in the future.

The next most important part is the name attribute. Every button that will post back should have the same name. This will be the name of your parameter in your controller as well and they must match. The value you declare on the button will be the argument passed to your controller method.

public async Task<ActionResult> SearchBegin([FromForm] SearchPageModel _searchModelIn, [FromForm] string submitButton)
    {
        // If you "name" your buttons as "submitButton"
        // then you can grab the value of the button
        // here to make decisions on which button was clicked
        switch (submitButton)
        {
            case "TopTen":
                return TopTen(_searchModelIn);
            case "Trad":
                return Traditional(_searchModelIn);
            default:
                break;
        }

        return View("~/");  // Go home

 }

The parameter name in your method must match the name attribute on your buttons exactly. The type passed in will be a string, although I imagine if your value attribute on all your buttons was numeric, that you could declare it as an int.

Once you’re in your method, you can use logic in a switch statement to detect the value passed in and make a decision how to proceed.

Get one list of Task return objects

Using the Task Async/Await pattern for grabbing data can be a real performance enhancement. When you thread off the calls, it’s pretty normal to want Task return objects to be in one single usable collection. The example I can give is a method that needs to gather up several different categories of lookup items. These calls all return a collection of the same type.

When you await the tasks, you generally have a few options:

Await each item individually

            List<Task<List<LuItem>>> _allLus = new();
            List<LuItem> _return = new();

            _allLus.Add(LookupsSvc.GetLuItemsByCatShortNameAsync("URLTYPES"));
            _allLus.Add(LookupsSvc.GetLuItemsByCatShortNameAsync("RVFUELTYPES"));
            _allLus.Add(LookupsSvc.GetLuItemsByCatShortNameAsync("GENERATORFUELTYPES"));

            Task<List<LuItem>>.WaitAll(_allLus.ToArray());

            List<LuItem> _task1 = await _allLus[0];
            List<LuItem> _task2 = await _allLus[1];
            List<LuItem> _task3 = await _allLus[2];

            _return.AddRange(_task1);
            _return.AddRange(_task2);
            _return.AddRange(_task3);

            return _return;

Not sure how you feel, but this is horrible. I’m sure I’ve done something like this in the past, but I’d prefer not to think about it.

Use WhenAll to retrieve them in an Array

The Task.WhenAll, when declared with a type, will return an array of the return type. So in this case, it would return an Array of List<LuItem>. We can then do a simple LINQ query to push them all into one collection.

            List<Task<List<LuItem>>> _allLus = new();
            List<LuItem> _return = new();

            _allLus.Add(LookupsSvc.GetLuItemsByCatShortNameAsync("URLTYPES"));
            _allLus.Add(LookupsSvc.GetLuItemsByCatShortNameAsync("RVFUELTYPES"));
            _allLus.Add(LookupsSvc.GetLuItemsByCatShortNameAsync("GENERATORFUELTYPES"));

            List<LuItem>[] _await = await Task<List<LuItem>>.WhenAll(_allLus);
            _await.ToList().ForEach(lus => _return.AddRange(lus));

            return _return;

In this example, we await the Task with WhenAll, which has a return type, as opposed to WaitAll which does not. As stated earlier, this example will return a collection as Task<List<LuItem>[]>. So we’re most of the way there. We use the ToList().ForEach LINQ query to transform the Array of Lists into a single list called _return.\

Sum of a list of values in a collection

Summing a collection that is within a collection without using nested foreach loops can be easily done with LINQ

It’s hard to think of a good name for this post. But if you have a collection and each item has a collection of values that you need to get a sum on, you can do that easily with LINQ.

Say you have a List<CartItem> in a shopping cart. Each item has a list of DecimalCost, possibly the user has ordered different sizes or colors and they each have an associated cost.

decimal _sum;
_return.CartItems.ForEach(c => _sum = c.DecimalCost.Sum());

Above we’re basically setting up an inline ForEach loop and then summing on the DecimalCost field which is actually a List<decimal>.

LINQ Safely Remove Items from a Collection

I’m sure we’ve all experienced the great idea of looping through a collection and trying to remove an item from a collection that doesn’t need to be there. You’ll get the infamous “Collection was modified; enumeration operation may not execute”. You can create a new collection and add the ones you want to that one, but that’s extra overhead.

Collection was modified; enumeration operation may not execute

This is a method that you can use that is outside of a foreach loop:

entity.TheBody.Elements.RemoveAll(a => string.IsNullOrWhiteSpace(a.Key));

In this example, I have an email named “entity” with a “TheBody” property that has a collection of Elements. The Elements have two properties, “Key” and “Value”, basically like a Dictionary entry. Creating a new list of elements means a new List<EmailElement> and then a .Clear and .AddRange, which kills more CPU cycles and milliseconds.

However, executing the above line will remove all the items from the collection that meet the criteria in the lambda.

Update Property Values in Collection using LINQ

There are many times that I wanted to be able to quickly update the property values in a collection without needing to create a foreach loop. Sometimes it’s because I needed to do it within a larger query, other times, just because it’s a relatively simple update and like being able to do it in one line of code.

Take for instance this example. I have a list of objects and I want to add a counter value to each. I’m doing this because they collection is sorted, but later processing is threaded, so they come out of that method unsorted again. I wanted a way to quickly get them sorted again so I didn’t have to pass around the sortColumn and sortOrder properties.

List<Contacts> _contacts = GetContactsPagedAsync(1, 25, "LastName", "desc");

int _counter = 1;
_contacts.Select(c => { c.SortOrdinal = _counter++; return c; });

The above code gets the collection and then updates the SortOrdinal property with a counter value.

You can get a little more complex pretty easily, take this:

List<Contacts> _contacts = GetContactsPagedAsync(1, 25);

_contacts.Select(c => { c.FullName = c.FirstName + " " + c.LastName ; return c; });

You can easily call a method from within your code as well, but just keep in mind that this runs synchronously. If the method is simple, we could rewrite the above like:

List<Contacts> _contacts = await GetContactsPagedAsync(1, 25);

_contacts.Select(c => { c.FullName = GetFullName(c); return c; });

Happy LINQing!

Get Records in one table with a Foreign Key to a related one to many table

There are times when you need to get records in one table with a foreign key to a related one to many table. This is a difficult need to describe, so I’ll give you the exact business scenario.

I have designed and used a Process Tracking system for many years. It currently has two basic components in the database:

  1. A FileProcess table that tracks a file (name, date, paths, app that processed it, etc.)
  2. A StatusLog table that I punch in records as this file goes through the process of being imported, validated, etc.

Often, I have multiple applications that process a batch of records from a file. I designed a stored procedure that would allow me to check for any file, by a particular application, that was in a particular status, but not past that status.

So here’s the scenario, we have a process that I have assigned the following status log values:

10 – File Parsed
20 – File Imported
30 – Data Validated
40 – Data Archived

Ok, so one application parses the file and imports it, let’s say it’s an SQL SSIS package just for fun. So it punches two status records in while it’s working, a 10 and a 20.

So now I have another validation application that checks every few minutes for something to do. I want it to be able to find any file that is in a status of 20, but NOT higher than that. So then I know it’s ready to be validated.

In order to do this, I have the following LINQ to SQL query that seems to do the job for me. I hope looking at this code will help you with whatever similar type of issue you’re trying to solve:

public async Task<List<FileProcess>> GetFileProcessesForAStatusByAppIdAsync(int AppId, int StatusId)
        {
            try
            {
                var _entityrows = (from st in _appLogContext.StatusLogs
                                   join fp in _appLogContext.FileProcess.Include(a => a.App) on st.FileProcessId equals fp.Id
                                   where st.AppId == AppId
                                    && st.StatusId == StatusId
                                    && st.StatusId == (_appLogContext.StatusLogs.Where(f => f.FileProcessId == fp.Id).OrderByDescending(p => p.StatusId).FirstOrDefault().StatusId)
                                   select fp).AsNoTracking();


                return await _entityrows.ToListAsync();

            }
            catch (Exception)
            {
                throw;
            }
        }

For those of you that are database jockeys, here’s the SQL code that this replaces:

     @AppId AS INT = NULL,
     @StatusId AS INT = NULL

    SELECT 
        [Id],
        [AppId],
        [FileName],
        [DateProcessed],
        [Inbound]
    FROM
        [FileProcess]
    WHERE
        Id IN (
    SELECT
        s.FileProcessId
    FROM
        (SELECT DISTINCT MAX(StatusId) 
            OVER(PARTITION BY FileProcessId) 
            AS ProperRow, FileProcessId, AppId
            FROM StatusLogs) AS s
    WHERE 
        s.ProperRow = @StatusId 
        AND AppId = @AppId
        )

The instance of entity type cannot be tracked because another instance with the same key value for {‘Id’} is already being tracked

When using Entity Framework (EF) Core, by default, EF Core will track any records that it pulls from the database so that it can tell if it has changes when you go to save it again. If you attempt to add the same record again etc, it will complain with a “The instance of entity type cannot be tracked because another instance with the same key value for {‘Id’} is already being tracked” error.

If you do N-Tier development, then having EF track your objects in the Repository or DataLayer of your API is of no use. It will start to cause problems when you go to save the object through a different endpoint that has created a copy of the repository model and a SaveChanges() is attempted.

In order to work around this, you can declare the Dependency Injected (DI) instance of your DB context to not use Query Tracking by using this type of code in your Startup.cs.

services.AddDbContext<AppLogContext>(o => 
o.UseSqlServer(_AppLoggingConnString)
.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking));

You can also accomplish this on each and every query, especially if your not using .NET Core and/or Dependency Injection as:

var _entityrows = (from al in _ale.AppLogs
                   select al).AsNoTracking();

You also have the option to set this behavior on the context at some other point in your code:

 _ale.AppLogs.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;

Thanks for Reading!

System.InvalidOperationException: Cannot consume scoped service

Using Dependency Injection can have challenges, along with rewards.

System.InvalidOperationException: Cannot consume scoped service

Copyright 2020 Microsoft 🙂

This error occurred when I modified my AppLogging REST Service to have an internal service that logged errors directly to the database. Can’t have the AppLogging Service call itself if there’s an error right?

After the modification, I recieved the following error:

System.InvalidOperationException: Cannot consume scoped service 'Enterprise.Logging.Repository.Context.AppLogContext' from singleton 'WF.Library.Shared.Logging.IAppLocalLoggingSvc`1[Enterprise.Logging.App.Rest.Controllers.AppMastersController]'. 

After some head tapping, I realized that I had modified the internal service class to now accept the DBContext, so that I could log errors directly to the database.

public AppLoggingSvc(AppLogContext appLogContext, IOptionsMonitor<WFAppSettings> appSettings)

I had the Dependency Injection (DI) setup like:

// Add DI reference to AppLoggingSvc that is a generic type
services.AddSingleton(typeof(IAppLocalLoggingSvc<>), typeof(Services.AppLoggingSvc<>));

I found the problem was that when you use AddDBContext to add the Database Context to your Dependency Injection collection, it is added as “Scoped”. So I was adding my IAppLocalLoggingSvc as a Singleton, but it was accepting a DI component in the constructor that was Scoped. These two scenarios are incompatible.

I found that using AddTransient resolved the issue:

// Add DI reference to AppLoggingSvc that is a generic type
services.AddTransient(typeof(IAppLocalLoggingSvc&lt;>), typeof(Services.AppLoggingSvc&lt;>));

Thanks for reading! Happy Coding.

Full Error Listing:

System.InvalidOperationException: Cannot consume scoped service 'Enterprise.Logging.Repository.Context.AppLogContext' from singleton 'WF.Library.Shared.Logging.IAppLocalLoggingSvc`1[Enterprise.Logging.App.Rest.Controllers.AppMastersController]'.
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteValidator.VisitScopeCache(ServiceCallSite scopedCallSite, CallSiteValidatorState state)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteValidator.VisitConstructor(ConstructorCallSite constructorCallSite, CallSiteValidatorState state)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteValidator.VisitRootCache(ServiceCallSite singletonCallSite, CallSiteValidatorState state)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteValidator.ValidateCallSite(ServiceCallSite callSite)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.Microsoft.Extensions.DependencyInjection.ServiceLookup.IServiceProviderEngineCallback.OnCreate(ServiceCallSite callSite)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetService(IServiceProvider sp, Type type, Type requiredBy, Boolean isDefaultParameterRequired)
   at lambda_method(Closure , IServiceProvider , Object[] )
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerActivatorProvider.&lt;>c__DisplayClass4_0.&lt;CreateActivator>b__0(ControllerContext controllerContext)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerFactoryProvider.&lt;>c__DisplayClass5_0.&lt;CreateControllerFactory>g__CreateController|0(ControllerContext controllerContext)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State&amp; next, Scope&amp; scope, Object&amp; state, Boolean&amp; isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.&lt;InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.&lt;InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.&lt;Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

MVC AJAX JSON Null DTO in Controller

A few months ago, I was enabling paging on a .NET Core 3.1 MVC application and had my search model passed into a controller method via AJAX. Well, it didn’t work. I received a NULL DTO object no matter what I tried. Trying to figure out what to do about an MVC Ajax JSON Null DTO in a controller method had me chasing my tail.

Fast forward to a few days ago, and guess what, another web app, same use case, same issue. Problem was, I couldn’t remember how I resolved it. Well, after another two hours of tinkering around with different objects, removing default settings in my DTO, and more endless googling, I finally found the issue… again.

Main issue I had is that System.Text.Json is not really usable. I found out that unless all your properties are strings, you have to setup custom comparers for each type. That about sums it up. Unless you’re passing in a very simple object that only has string properties, you can pretty much forget about using this library out of the box.

Take a look at the things that are not supported in System.Text.Json: https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-migrate-from-newtonsoft-how-to

For those of you in a hurry, here is a summary of what I did. Details of the implementation will follow:

  1. Make sure you have “FromBody” in your controller method. I already had this, but it’s what most blog posts focus on.
[HttpPost]
public async Task<IActionResult> CatDisplay([FromBody] SearchModel<LuCategory> searchModelIn)

2. Change the default JSON serializer in your Startup.cs

using Microsoft.AspNetCore.Mvc;
public void ConfigureServices(IServiceCollection services)
{
    services.AddControllersWithViews().AddNewtonsoftJson();

If you get the little squigglies under this method name, then add the Nuget package: Microsoft.AspNetCore.Mvc.NewtonsoftJson

Just so you can see how I’m calling this, here is the Javascript/Jquery/JSON that I’m sending in:

    function GetPaging(ToPage) {

        var _url = "/@Model.controllerName/@Model.actionName";

        // Set the global values for sorting post back
        var searchModel = {};
        searchModel.SortColumn = '@Model.SortColumn';
        searchModel.PrevSortColumn = ''; // Leave blank so sorting doesn't kick;
        searchModel.CurrentPage = ToPage;
        searchModel.PageSize = @Model.PageSize;
        searchModel.SearchTerm = '@Model.SearchTerm';
        searchModel.SearchFilter = '@Model.SearchFilter';
        searchModel.SortDescending = '@Model.SortDescending';
        searchModel.ActiveOnly = '@Model.ActiveOnly';
        searchModel.RefId = @Model.RefId;
        searchModel.RefUniqueId = '@Model.RefUniqueId';

        $.ajax({
            type: "POST",
            url: _url,
            async: true,
            contentType: "application/json",
            data: JSON.stringify(searchModel),
            dataType: "html",
            success: function (result, status, xhr) {
                $("#gridPartial").html(result)
            },
            error: function (xhr, status, error) {
                alert("Result: " + status + " " + error + " " + xhr.status + " " + xhr.statusText)
            }
        });

    }

3. The last problem I ran into was boolean values. In the above example, the boolean value was coming from the Model, so there is no issue. However, if you are trying to get a boolean value from javascript or jquery, big problems. In order to be sure that what is being passed as a value can be deserialized into an object, you should have code like:

searchModel.SortDescending = $('descending').val().toLowerCase() != 'false') ? true : false;

This issue has hit me several times, so I’ve used this code to resolve it.

Thanks for reading and happy coding!