Vaughan Reid's blog

How to throttle http calls on the client using a DelegatingHandler

A common issue that you encounter when you need to consume an external api is when there is a rate limit for users of the service. The api will only allow x requests in a time period otherwise it will return a HTTP 429 Too Many Requests Error.

read more

Calculating combinations of an array

A friend of mine has started to dabble in C# and he was surprised that there doesn’t seem to be an equivalent to the Python Combinations library. To be honest its something that I didn’t really know I was missing and my first thought was that its a simple enough problem to solve. But when I started trying to solve it, it turned out that I was very wrong.

read more

Can NDepend replace the need for code reviews?

No most probably not but with the right setup it can make it a quicker process. People often see code reviews only as a way to enforce coding standards in an application. While this is important, for me its more about communicating changes to each other so that information isn’t siloed. That is why its perfectly reasonable for a junior developer to code review a seniors changes.

read more

C# 9.0: Readability improvements

With the recent announcement of .NET 5.0 I’d like to spend some time going over some of the nice features you will get out the box if you upgrade.

read more

Using Refit in ASP.NET Core to remove boilerplate code when consuming REST endpoints

I recently started using Refit in a project and its a really nice way to consume rest endpoints. Once you set it up, all you have to worry about is making a call against a interface and Refit will do all the work for you.

read more

Code challenges: Finding number of sub arrays with a sum of zero

It can be fun to challenge yourself with algorithm type problems and to try find an optimal solution. Unfortunately the only way to get better at them is to practice. After enough practice you might start to spot common patterns to solve a new problem just based off experience.

read more

Using MediatR in ASP.NET Core

We recently started using MediatR in a project and its a great way to make your services simpler as requirements grow. I want to show a simple example to give an idea on why you would use it.

read more

What is AyncLocal in C#?

An interesting class that is useful to know about but you most probably won’t directly use is AsyncLocal. It stores data local to the async flow. This means that if a method further down the chain changes the underlying value, the change will only apply to that and its children. A potential use case might be if you want to have access to a trace id on an internal request without having to manage some sort of cache.

read more

Why you should start using property based testing

I want to convince you to try Property-based testing in your projects. Its a really powerful approach when combined with classic unit testing.

read more

Classifying tweet sentiment with ML.NET

Following from my previous post on getting paged data from twitter, I thought it would be interesting to use ML.NET to do a little analysis. A common problem solved with machine learning is doing sentiment analysis on text to evaluate how positive or negative something is. To solve this problem with ML.NET really isn’t that much code and so its a nice entry point into their machine learning offering.

read more

Consuming paged Twitter data with async enumerable

A really nice feature in C# 8 is async enumerables. A popular use case is if you want to consume paged data and potentially aggregate each page of data without having to get everything first. Previously the consuming code would have to handle the paging but now your method can just enumerate over the IAsyncEnumerable and process the data after each call. Its actually simpler than I’m making it sound! I created a small console app that will hopefully show you what I mean. If you want to run it locally you can get the source here.

read more

Creating an AWS Lambda function written in C# .NET Core

With more and more companies moving to the cloud, the idea of serverless execution of your code has more and more appeal. AWS has Lambda and Azure has Functions. The idea is the same: You give them a method that you want to execute and the provider works out the scaling. As you get more requests, it will create more and more instances. The obvious benefit is that you don’t need to worry about where or how your application is running. Its a cheap and easy way to execute code in the cloud.

read more

ASP.NET Core integration tests with WebApplicationFactory

We all agree that we want to have tests that cover as much of our application as possible. Right? The general approach with ASP.NET applications has been to extract meaningful business logic out of your controllers. Pretty much everything that needs to be tested would be injected with an interface so that its easy to mock and test your business logic.

read more

Performance with ref readonly returns

When looking at performance of an application, there is a trade off between when you should be using value types and when you should be using reference types. People often talk about minimizing allocations to the heap for reference types but allocations themselves are actually very quick. The reason why its an issue is that those allocations need to be garbage collected eventually. This is time that the application is spending cleaning up memory instead of running your code. When looking at allocations, value types are great because they generally are just on the stack. Once the method is finished they are gone. Of course with everything there are trade offs, the stack is more limited than the heap and so you have size constraints on them and the fact that the whole object needs to be copied when passing it around.(side note: value types aren’t always on the stack. They can be on the heap in some circumstances! eg: when boxed, fields of a class or elements of an array.)

read more

Understanding default interface members in .NET Core

A new feature in C# 8.0 and .NET Core 3.0 is that of default interface members. The motivation for it is where authors of a public interface would like to add functionality to an existing interface but don’t want to force all the consuming code to implement it. You are able to add a default implementation to your new method so that consuming code isn’t forced to.

read more

Using the new Index and range operators

A great C# feature that was recently released is the new Index and Range operators. Two new system objects were created to make accessing elements or range of elements in arrays shorter and more convenient.

read more

A case for pattern matching

Over the years developers using more functional languages like F# have boasted about how much easier their code is to read and how imperative C# can be. NO LONGER! Well maybe not much longer.

read more

Why is async void return bad?

When writing async code, your method is allowed to return either void, Task, Task, or [ValueTask](https://www.vaughanreid.com/2020/06/when-to-consider-using-valuetask-over-task-with-async-code)). Since early on with async code, the advice has been to not use void return methods. But in real life there are times when you don't really have a choice. Since we can't always get away from using them, its good to understand whats the difference to async code returning a Task and what we can potentially do to help make it less *bad*.

read more

Running a .NET Core HostedService as a Windows service

If you need to run your long running .NET Core worker application on a windows server, you can easily install it as a windows service with minimal changes.

read more

When to consider using ValueTask over Task with async code

As of .NET Core 2.0, you can use ValueTask and ValueTask as return types for your async methods. Its important to understand when and if you should use these instead of a Task and what the trade-offs are if you decide to use it.

read more

Counting sheep with BenchmarkDotNet

When trying to improve the performance of your code, the first thing you need to do is measure. If you don’t measure then you won’t know if your changes are actually improving performance. In most cases, there is a trade-off between readability and performance. Often better performing code is more complex and harder to maintain. Its for this reason that you have to be very careful when making changes without actually understanding what the real impact will be.

read more

Using a custom action constraint to version an endpoint in ASP.NET Core

There are times where you want to add a new action with the same route but you don’t want to break backwards compatibility. One option that could help with this is to use a custom action constraint.

read more

Why concurrency with async beats parallelism for scalability

Two terms that are often used interchangeably are concurrency and parallelism. It doesn’t help that in English, doing something concurrently means that you are doing more than one thing at a time. In software its a little more complicated. A good explanation on the difference that I have read before is as follows:

read more

Using route constraints in ASP.NET Core

A small but useful feature available in ASP.NET Core is to define route constraints to disambiguate similar routes. **Its important to note that the documentation makes it clear that you shouldn’t be using this for input validation. **

read more

What I learnt attending the Microsoft build online event

A major positive of the whole world working from home is that events that you would normally have to travel to are now online for everyone. Thats a huge saving in time and money for someone in South Africa. This year I was lucky to be able to attend the Microsoft Build event hosted in Seattle. Because of time zone issues I only attending the event on the 20th May but there was still quite a lot of content that was available in my timezone.

read more

Measuring .Net Core: 5. Creating and analyzing a dump file when an ASP.NET Core application crashes

This is the fifth in a series of posts covering measuring a .Net Core application. If you want to try follow the code then you can have a look at this repo: Blog-Diagnostics.

read more

Measuring .Net Core: 4. Using dotnet-gcdump to investigate memory leaks

This is the fourth in a series of posts covering measuring a .Net Core application. If you want to try follow the code then you can have a look at this repo: Blog-Diagnostics.

read more

ASP.NET Core doesn't use a SynchronizationContext

Yesterday I was trying to show how to diagnose a deadlock situation when using blocking code over async. This was a pretty common mistake in ASP.NET and a nice scenario to add to my measurement series. I created a controller like this in my ASP.NET Core sample application and strangely it worked without error.

read more

Measuring .Net Core: 3. dotnet-counters with a custom provider

This is the third in a series of posts covering measuring a .Net Core application. If you want to try follow the code then you can have a look at this repo: Blog-Diagnostics.

read more

Measuring .Net Core: 2. Using dotnet-counters

This is the second in a series of posts covering measuring a .Net Core application. If you want to try follow the code then you can have a look at this repo: Blog-Diagnostics.

read more

Measuring .Net Core: 1. Setting up local diagnostics tools

This is the first in a series of posts covering measuring a .Net Core application. If you want to try follow the code then you can have a look at this repo: Blog-Diagnostics.

read more

Using a custom authorization filter in ASP.NET Core

In a previous post I showed how you can use custom middleware to disable endpoints based on configuration and attributes. It is a good way to show how middleware works but there is an easier way. You can achieve the same result by creating a custom authorization filter.

read more

Using in-line middleware in ASP.NET Core

I previously showed how you can create your own middleware and use the app.UseMiddleware method to insert it into the pipeline.

read more

Authenticating to IdentityServer4 in ASP.NET Core

Authenticating to an IdentityServer4 service with an ASP.NET Core application is actually quite easy once you see it. It does require a basic understanding of the OpenID framework though. I’m first going to explain some basics and then I’ll show the code at the end.

read more

Creating a CSV text to model custom model binder in ASP.NET Core

ASP.NET Core has quite a large selection of model binders built in that are able to convert http requests to simple and complex models in your actions. If there isn't one that matches your needs, you can always create a custom one. To show how, I'm going to create a custom model binder that converts csv formatted text to an IEnumerable of models using the great CsvHelper library.

read more

Using the swagger-codegen-cli Docker image to generate c# classes from a swagger api

Not everyone knows that swagger gives you more than just a nice UI to expose your APIs. It actually exposes a JSON endpoint that defines your API. Tools like swagger codegen can use that endpoint to create a complete consuming project in an array of programming languages.

read more

Using middleware to conditionally allow endpoints

There are times when you have controllers or actions that are very useful for debugging issues but you don't really want to expose them in a production environment. One way to achieve this is to take advantage of the separation of the Endpoint Routing Middleware and the Endpoint Middleware components. Another way would be to use a custom authorisation filter but in this case I’ll use middleware.

read more

Why you should use the IHttpClientFactory in ASP.NET Core

One surprising issue when using the HttpClient, is the issue of port exhaustion. This often happens when an application creates a lot of HttpClient instances for short requests. When there is sufficient load the machine may run out of tcp ports and then it won't be able to make another request until a port opens up. The reason for this is the fact that disposing a HttpClient will still keep a tcp connection open in TIME_WAIT status. This then relies on the system configured timeout which is often a couple of minutes. Its the worst kind of bug because it often happens in production when you are under heavy load.

read more

Using automapper in ASP.NET Core

To be honest I have mixed feelings on using something like automapper by default but there are definitely cases when it heavily reduces the amount of code you write and therefore your bugs.

read more

ASP.NET Core: Injecting dependencies directly into controller actions

One feature in ASP.NET Core that I was surprised by is the ability to inject a service into a specific action of a controller. Normally you inject into a constructor of a controller and keep the dependencies as local variables. So imagine a controller like this:

read more

ASP.NET Core: Using fluent validations in your api controllers

If you need to validate the inputs to your api actions I really do recommend trying the FluentValidation.AspNetCore package. Its very easy to integrate and you can do quite powerful validations. As an example I created a Person controller.

read more

Semantic logging with ASP.NET Core

One subtle adjustment that some of the older .net developers most probably need to make is that idea of semantic logging in ASP.NET core. This is subtle because a lot of us started used string interpolation a few years ago. The examples in the documentation look very similar to how my string interpolation strings look. See if you can spot the issue with below:

read more

ASP.NET Core IOptions Validation

I've spoken previously about IOptions in ASP.NET core. On their own, they are a very useful way to inject configuration into your services. What I didn't mention is that they allow you to build in validation of your configuration. One reason that this might be needed is that the IOptions services are normally very forgiving on missing configuration. If you don't have a value in your appsettings.json then it will just populate with a default value. This could cause serious hidden errors at runtime for your application.

read more

ASP.NET Core dependency injection: Registering open generics

A really interesting trick you can do with the ASP.NET core dependency injection is that you can register an open generic type. The prime example is the ILogger interface that is built in. Each class can inject a ILogger of its type without having to register it specifically. It can be quite a powerful technique in the right circumstance.

read more

ASP.Net Core Hosted services

In the containerized application and Kubernetes world, quite often you need to create services that aren't traditional rest endpoints. Your application could for example rather listen to a message queue to process requests. It would be nice to still have all the nice infrastructure of .NET Core applications and things like health checks but have it work closer to a windows service.

read more

ASP.NET Core configuration in Docker containers

In the old days of .NET applications we had a simple xml configuration file that would ship with each environment. This was tried and tested and was well understood. Then came Docker. The promise was that I could build a container on my machine and then ship that exact container to production. This is great except for my trusty configuration file. You don't really want to update the file system of the container for each environment it goes through. There must be a better way!

read more

Docker with certificates

Sometimes when connecting to internal systems you need trusted certificates to be able to authenticate. Generally the base docker image you use when running ASP.NET core applications would not have your internal company certificates. You need a way to deploy these in your container so that your endpoints work.

read more

ASP.NET Core with IOptions

One of my favorite improvements when moving to ASP.NET Core applications is that configuration feels like a first class citizen in the infrastructure. Previously I often had to do some manual work in the DI registration to create configuration objects that could be injected into my classes. No longer!

read more

Why I started contributing to open source

About a month ago I made my first commit to the grpc-dotnet project. It was a small change but it was more about the intent for me. Since then I’ve submitted a few other pull requests to that project and also the aspnetcore project. Some have been merged and others are waiting.

read more

ConcurrentDictionary internals

The nice thing about .net being opensource is that at any time you can look at the source of interesting classes that you rely on. In this post I look at the ConcurrentDictionary class to get a better idea of how Microsoft writes threadsafe classes. As a recap: generally a normal dictionary is implemented internally as an array of buckets. When you add an element, the hash of the key will get you to the right bucket and then each item in the bucket will be compared for equality to see if it exists. It is for this reason that you need to be careful of the hashing algorithm that you implement because ideally for quicker lookups you want more buckets with less items than a small amount of buckets with many elements. A worst case example would be a dictionary resolving to a single bucket for all its items which will then have a o(n) lookup instead of a o(1) lookup. ConcurrentDictionary is quite a large and complex class and so I will try cover the core parts of it. Central to how it works is in this internal class within it.

read more

Present value in code

To try show some of the financial concepts in practice I've create a library on github to track it - FinanceLibrary. I'm not going to include the tests below but I added tests to each class as I created it. Have a look at the project for a more complete view. The first calculation I want to show is Present Value. To do this I need some initial setup code. I'm going to start by declaring that all instruments have cashflows.

read more

The problem with the BlockingCollection

The BlockingCollection is a really powerful thread-safe collection that can be a really powerful tool to use where appropriate. Generally you could consider it where you have a common instance of a class accessed from multiple threads. It gives the class control on how many items are added or taken in a thread-safe way. Calling the Take when the collection is empty or an Add when the collection is full blocks the thread until something is added/removed. So an example is to imagine your class gets data from a wcf client. To improve performance you might want to have multiple shared client instances. A BlockingCollection could really help you there because you could bound it to say max 4 instances of the client contract. Each consuming thread will try take an instance from the collection and add it back once done. If all 4 instances are checked out then the next call will block and wait until there is one available. This is incredibly powerful and it would keep the code maintaining this quite small.

read more

What is present value?

Present value is an important calculation across a bank as its use to calculate a traders positions total mark-to-market. ie: How much are all their trades worth?

read more

Rx as part of your standard toolkit

For years I heard about RX in .NET but never really found a compelling reason to use it. About a year ago we started using it in my team and let me just say that things have changed. Now I don’t think theres a compelling reason NOT to use it as part of your standard toolset. Its a great tool with a really interesting way to look at things but I'd argue that you should strongly consider using it whenever you are making multi-threaded applications. The reason is that it has a really nice solution to one of the biggest issues with multi-threaded code. Testability. Without a test proving a piece of work can you really say its working as expected? What happens if your elegant code has a bug? How sure are you that your fix to the bug isn't breaking something else? I'm not saying that Rx can/should be used for all multi-threaded code but since its built with testing in mind its a very useful tool in the belt. Let me show you an example.

read more

The interviewing process

Over the years I’ve been to a lot of technical interviews on both sides of the table and if there’s one thing I’ve learnt and thats the fact that its really hard to get right. How do you tell if someone will be good at writing good quality code in an hour or 2 of talking to them? Do you even know after a week of them working? Are the people asking about TDD(insert trendy buzz word at the time) because they actually do it or because they want to do it. Do they actually understand what it means?

read more

Setting up my blog

As a heavy consumer of blogs I’ve eventually decided to start blogging myself. I’ve owned this domain since 2012 and so its been a plan for a while but I could never bring myself to getting around to setting it all up. I think one of my main obstacles was the fact that I knew websites took work to develop and maintain. I’ve created a fair amount of websites and even the simplest ones take time when you have to think about browser support, accessibility and extensibility. I knew most people used CMS’s to create it but I’m a developer. I know my way around css, javascript and html. Why would I waste my time building it on someone elses framework?? So this is where I was stuck for 3 years.

read more

Hello world!

So I’ve finally created my blog after owning a domain since 2012. Most of my posts will be on development problems that I’m interested in but I’d also like to write about financial concepts that I’ve learnt along the way.

read more