Month: November 2021

Storing .NET NuGet Packages in GitHub

I can’t believe that I didn’t write this post two years ago when I figured this out. I apologize to everyone that had to figure this out on their own.

When I first tried to create NuGet packages from my apps and upload them, I wasn’t using Jenkins. But no matter, the concepts were the hard part, not the application that was doing the creation and uploading (push).

Scenario

Your doing software development and have one or more components that you want to re-use in many projects. Whether it’s a collection of extensions or models for an API. The goal is to have a versioned download that other developers can grab which is versioned and easily added to their application. I won’t bore you here with why this is a good idea, even when many corporate developers argue about what a pain it is to use NuGet packages for this.

The main concern is that you don’t want this proprietary code to be publicly available on NuGet.org. The three choices that I’ve worked with are File Share, Azure Artifacts and GitHub Packages. I’m not going to discuss Azure Artifact here, because they actually are a little easier, because if your using Azure Devops, then the authentication is coordinated, where as in GitHub, it’s not so easy.

I spent many hours trying get this to work. After a few hours, I actually got a NuGet package to build and then it would upload to GitHub. Then the next day I would try it and it would fail again. I opened a ticket with GitHub and had a few email exchanges. The last email basically said “If you figure it out, let us know how you did it.” Well, I have to admit that I never did, as much as I don’t like the saying that “It’s not my job”, I’m not being paid to educate GitHub support staff.

GitHub Support: “If you figure it out, let us know how you did it.”

Anyway, enough with the small talk, let’s get down to it.

Concepts

I need to state that this solution applies to all versions of .NET Core and .NET 5.x versions that I’ve been using for a few years. If you’re trying to do this with older .NET Framework versions, then some of this may not apply.

There are three major things you’ll have to be aware of when creating a NuGet package.

Your Project File

The default project file in .NET Core or .NET 5.x is not sufficient to create a NuGet package, unless you hard code all the data in the {ApplicationName}.nuspec file covered below. I recommend embellishing the csproj file.

There are several things that a NuGet package requires, id (PackageId) , title, etc. being a few. You need to make sure that all this data is in your csproj file. I have a sample below:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>library</OutputType>
    <TargetFramework>net5.0</TargetFramework>
    <PackageId>YTG-Framework-V2</PackageId>
    <Version>2.1.1</Version>
    <Authors>Jack Yasgar</Authors>
    <Company>Yasgar Technology Group, Inc.</Company>
    <PackageDescription>Shared resources for all Yasgar Technology Group applications</PackageDescription>
    <RepositoryUrl>https://github.com/YTGI/YTG-Framework-V2</RepositoryUrl>
    <Description>Shared framework methods and objects used throughout the enterprise.</Description>
    <Copyright>@2021 - Yasgar Technology Group, Inc</Copyright>
    <AssemblyVersion>2.1.1.0</AssemblyVersion>
    <FileVersion>2.1.1.0</FileVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Logging" Version="3.1.3" />
    <PackageReference Include="Microsoft.Extensions.Logging.Configuration" Version="3.1.3" />
  </ItemGroup>
</Project>

Almost every line in this file is important. The “MOST” important one is the “Version” key highlighted. When you first get your upload (nuget push) to work, it will work fine, but it will get an exception the second time of you don’t increment the version. The NuGet upload process ignores the “AssemblyVersion” and “FileVersion”.

{ApplicationName}.nuspec

This is the file that the nuget packager actually looks at to create a NuGet package. The file has a variable base syntax that will pull data from the csproj file, hence my recommendation that you use the csproj file as the source of truth. You have the option of hard coding values in here if you wish. Why use the variables you ask? Because, if you use the csproj file as the source of truth, then your {ApplicationName}.nuspec can have the same content in every project you have. I think that makes this process simpler if you plan to have several NuGet packages.

<?xml version="1.0" encoding="utf-8"?>
<package >
  <metadata>
    <id>$packageid$</id>
    <version>$version$</version>
    <title>$title$</title>
    <authors>$author$</authors>
    <owners>$author$</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <license type="expression">MIT</license>
    <projectUrl>$repositoryurl$</projectUrl>
    <description>$packagedescription$</description>
    <releaseNotes>Release</releaseNotes>
    <copyright>$copyright$</copyright>
    <tags>Utilities Extensions</tags>
  </metadata>
</package>

As you can see above in my live example, the only thing you might want to adjust is the <tags> entry. All the rest will pull from the project file as variables.

Now for the next tricky part. Here’s where your existing knowledge may hurt you. If you said to yourself that you don’t need a NuGet.config file in your project, your right, if you just want to fetch packages. But if your creating packages on Azure Pipelines or GitHub Actions, you’ll need it. If you’re creating your package on Jenkins, then you can just have a reusable NuGet.config in the C:\Users\{serviceid}\AppData\Roaming\NuGet folder. {serviceid} is the service account that Jenkins is running as, often LocalSystem, which means the {serviceid} = “Default”.

Okay, so now our project is ready for for a NuGet package build. So how do we get it pushed up. Well, that’s different depending on your environment. I’ll show you a few.

Command Line

First thing is to make sure your NuGet package gets created. You can do that in the CLI (Command Prompt):

dotnet pack {yourprojectfolder}\{yourprojectname}.csproj --configuration Release --output nupkgs

This should build your project and put the NuGet package in a folder called “nupkgs”. If this doesn’t work, you need to review all the messages and review the above configuration steps. Remember, this process is not concerned that you’re in a hurry.

If you wind up with a .nupkg file in the folder, then pat yourself on the back, you’re most of the way there. Here’s where I ended up opening a ticket with GitHub, because I had a nupkg, but I couldn’t get it to upload consistently. It turned out that I, and GitHub support, didn’t understand the different credentials.

I and GitHub support didn’t understand the different credentials.

In your NuGet.confg, there are two different sections.

packageSourceCredentials

	<packageSourceCredentials>
		<github>
			<add key="Username" value="JYasgarYTGI" />
			<add key="ClearTextPassword" value="ghp_asdsfsfjAbunchofBScharacters" />
		</github>
	</packageSourceCredentials>

and apikeys

	<apikeys>
		<add key="https://nuget.pkg.github.com/YTGI/index.json" value="awholebunchofbullshit==" />
	</apikeys>

The “packageSourceCredentials” are exactly what they say, it is the credentials to READ the nuget packages. So that’s where I got sidetracked, it has nothing to do with uploading the package after it’s created.

In order to actually PUSH (upload) a file, you need to have apikeys credentials. It DOES NOT use your regular GitHub access in the “packageSourceCredentials ” to upload packages. That means you have to have another section in your nuget.config file that give you access to push files. This is the ef’d up thing, it is often the exact same credentials, but just in a different place.

RECOMMENDATION: You should create a different account that has is very generic to create API access. This will mitigate the issue of a developer leaving the team that has all the tokens under their account for push access.

GitHub Token

In order to get the proper credentials to use for push (uploading) the nuget package, you should log in as your generic (DevOps) account in GitHub if you have one. If not, use your current account. MAKE SURE YOU GO INTO THE “Settings” FROM YOUR ACCOUNT ICON IN THE UPPER RIGHT CORNER and not the “Settings” on the account or project menu.

Once you’re in there, click “Personal access tokens” and click “Generate new token”.

Give your new token a name, select the expiration days, I suggest 90, but “No expiration” is your decision.

Select the following options:

  • workflow
  • write:packages
  • read:packages
  • delete:packages
  • admin:org
  • admin:write:org
  • admin:read:org
  • user:user:email

Make sure you copy and SAVE the token, because you will never be able to see it again!

You can encode the password that needs access to push the NuGet package with the following command. Please note that if you don’t put the -ConfigFile entry, then it will update the version of the NutGet.config file in the C:\users folder and NOT the version in your project config. This causes the issue were it works when you do it, but all downstream deployments fail, i.e.: Jenkins, Azure etc.

The command to add this to your project nuget.confg file is:

nuget setApiKey ghp_{theapikeyprovidedbyGitHub} -source https://nuget.pkg.github.com/YTGI/index.json -ConfigFile nuget.config

I’m going to repeat that you must notice the -Configfile param. If you don’t add it, the command will update the nuget.config in our current user \roaming\nuget folder, which is not what you want if this project is being deployed from a different device, i.e. Jenkins or Azure.

GitHub NuGet Push

The PUSH process is the following:

nuget push %WORKSPACE%\nupkgs*.nupkg -ConfigFile "{yourprojectfolder}\nuget.config" -src https://nuget.pkg.github.com/{yourgithuborgname}/index.json -SkipDuplicate

Notice the -SkipDuplicate argument in the above CLI command. It will cause the push (upload) command to ignore the fact that you’re trying to upload a duplicate version instead of raising an error that will fail a build. Just keep in mind that if you forget to change your version in the csproj file, your change will not show up. It is against convention to make changes to an existing version, as that could cause real problems for consuming applications. If you made a mistake, or left something out and you want to get rid of the version you just built, you’ll need to go into GitHub packages, Versions and delete the version you just built to upload it again with the same version number.

If this command works, then you have setup your nuget.config file correctly and you’re good to go.

NOTE: It will sometimes take up to a minute or so for the package to show up in the GitHub Packages list, so be patient and don’t assume it didn’t work until you give it some time.

Update Property Values in Collection using LINQ

There are many times that I wanted to be able to quickly update the property values in a collection without needing to create a foreach loop. Sometimes it’s because I needed to do it within a larger query, other times, just because it’s a relatively simple update and like being able to do it in one line of code.

Take for instance this example. I have a list of objects and I want to add a counter value to each. I’m doing this because they collection is sorted, but later processing is threaded, so they come out of that method unsorted again. I wanted a way to quickly get them sorted again so I didn’t have to pass around the sortColumn and sortOrder properties.

List<Contacts> _contacts = GetContactsPagedAsync(1, 25, "LastName", "desc");

int _counter = 1;
_contacts.Select(c => { c.SortOrdinal = _counter++; return c; });

The above code gets the collection and then updates the SortOrdinal property with a counter value.

You can get a little more complex pretty easily, take this:

List<Contacts> _contacts = GetContactsPagedAsync(1, 25);

_contacts.Select(c => { c.FullName = c.FirstName + " " + c.LastName ; return c; });

You can easily call a method from within your code as well, but just keep in mind that this runs synchronously. If the method is simple, we could rewrite the above like:

List<Contacts> _contacts = await GetContactsPagedAsync(1, 25);

_contacts.Select(c => { c.FullName = GetFullName(c); return c; });

Happy LINQing!