Performance issues can appear when you least expect it. This can have negative consequences for your customers. As your user base grows, your application may lag because it is unable to meet the demand. Fortunately, there are tools and techniques available to address these issues in a timely manner.
We created this article in partnership with Site24x7. Thank you for supporting the partners who make SitePoint possible.
In this take, I’ll explore performance bottlenecks in a .NET 6 application. The focus will be on a performance issue I’ve personally seen in production. The intent is for you to be able to reproduce the issue in your local development environment and address the issue.
Feel free to download the sample code from GitHub or follow along. The solution has two APIs, unimaginatively named First.Api and Second.Api. The first API calls the second API to get weather data. This is a common use case, because APIs can call other APIs, so data sources remain decoupled and can be scaled individually.
First, make sure you have the .NET 6 SDK installed on your machine. Then open a terminal or console window:
> dotnet new webapi –name First.Api –use-minimal-apis –no-https –no-openapi > dotnet new webapi –name Second.Api –use-minimal-apis –no-https – -no-openapi
The above can go to a solution folder like performance-bottleneck-net6. This creates two web projects with minimal APIs, no HTTPS, and no swagger or open API. The tool uses the folder structure, so take a look at the example code if you need help setting up these two new projects.
The solution file can go in the solution folder. This allows you to open the entire solution using an IDE like Rider or Visual Studio:
dotnet new sln –name Performance.Bottleneck.Net6 dotnet sln add First.Api\First.Api.csproj dotnet sln add Second.Api\Second.Api.csproj
Next, make sure to set the port numbers for each web project. In the example code, I set them to 5060 for the first API and 5176 for the second. The specific number doesn’t matter, but I’ll use them to reference the APIs throughout the sample code. So make sure you change your port numbers or keep what the scaffold generates and stay consistent.
The infringing application
Open the Program.cs file in the second API and place the code that responds with the weather data:
var builder = WebApplication.CreateBuilder(args); var app = builder.Build(); var summaries = new[]
{ “Frozen”, “Strengthening”, “Cold”, “Cool”, “Mild”, “Warm”, “Bald”, “Hot”, “Suffocating”, “Scorching” }; app.MapGet(“/weatherForecast”, async () => { await Task.Delay(10); return Enumerable .Range(0, 1000) .Select(index => new WeatherForecast ( DateTime.Now.AddDays(index), Random.Shared.Next(-20, 55), abstracts[Random.Shared.Next(summaries.Length)]
) ) .ToArray()[..5]; }); app.Run(); public record WeatherForecast(DateTime Date, int TemperatureC, string? Summary) { public int TemperatureF => 32 + (int)(TemperatureC / 0.5556); }
.NET 6’s minimal API feature helps keep code small and concise. This will loop through a thousand records and make a task delay to simulate asynchronous data processing. In a real project, this code might call a distributed cache or a database, which is an IO-bound operation.
Now, go to the Program.cs file of the first API and write the code that uses this weather data. You can simply copy and paste this and replace what the scaffolding generates:
var builder = WebApplication.CreateBuilder(args); builder.Services.AddSingleton(_ => new HttpClient(new SocketsHttpHandler { PooledConnectionLifetime = TimeSpan.FromMinutes(5) }) { BaseAddress = new Uri(” }); var app = builder.Build(); app.MapGet(” async ( HttpClient client ) => { var result = new List
The HttpClient is injected as a singleton, because this makes the client scalable. In .NET, a new client creates sockets to the underlying operating system, so a good technique is to reuse these connections by reusing the class. Here, the HTTP client sets a connection pool lifetime. This allows the customer to hang onto the plugs for as long as needed.
A base address simply tells the client where to go, so make sure it points to the correct port number set in the second API.
When a request comes in, the code goes through a hundred times and then calls the second API. This is to simulate, for example, a number of records needed to make calls to other APIs. Iterations are hardcoded, but in a real project it can be a list of users, which can grow without limit as the business grows.
Now, turn your attention to the loop, because this has implications in performance theory. In an algorithmic analysis, a single loop has linear Big-O, or O(n) complexity. But, the second API also loops, which increases the algorithm to quadratic or O(n^2) complexity. Also, the loop goes through an IO limit to boot, which affects performance.
This has a multiplier effect, because for every iteration of the first API, the second API loops a thousand times. There are 100 * 1000 iterations. Remember that these lists are not linked, which means that performance will degrade exponentially as the datasets grow.
When angry customers are spamming your call center asking for a better user experience, use these tools to try to figure out what’s going on.
CURL and NBomber
The first tool will help identify which API to focus on. When optimizing code, it is possible to optimize everything endlessly, so avoid premature optimizations. The goal is to get performance to be “good enough”, and this tends to be subjective and driven by business demands.
First, call each API individually via CURL, for example to get a feel for latency:
> curl -i -o /dev/null -s -w %{time_total} > curl -i -o /dev/null -s -w %{time_total}
Port number 5060 belongs to the first API and 5176 belongs to the second. Please validate that these are the correct ports for your machine.
The second API responds in fractions of a second, which is good enough and probably not the culprit. But the first API takes almost two seconds to respond. This is unacceptable, because web servers can exhaust requests that take so long. Also, a two-second latency is too slow from the client’s perspective, because it’s a disruptive delay.
Then a tool like NBomber will help benchmark the problematic API.
Return to the console and, inside the root folder, create a test project:
dotnet new console -n NBomber.Tests cd NBomber.Tests dotnet add package NBomber dotnet add package NBomber.Http cd .. dotnet sln add NBomber.Tests\NBomber.Tests.csproj
In the Program.cs file, type the benchmarks:
using NBomber.Contracts; using NBomber.CSharp; using NBomber.Plugins.Http.CSharp; var step = Step.Create( “fetch_first_api”, clientFactory: HttpClientFactory.Create(), execute: async context => { var request = Http .CreateRequest(“GET”, “/”) .WithHeader(“Accept”, “application ” /json”); var response = await Http.Send(request, context); return response.StatusCode == 200 ? Response.Ok( statusCode: response.StatusCode, sizeBytes: response.SizeBytes): Response.Fail( ); } ); var scenario = ScenarioBuilder .CreateScenario(“first_http”, step) .WithWarmUpDuration(TimeSpan.FromSeconds(5)) .WithLoadSimulations( Simulation.InjectPerSec(rate: 1, during: TimeSpan.FromSeconds(5)), Simulation (Inject.PerSec(5)), 2, while: TimeSpan.FromSeconds(10)), Simulation.InjectPerSec(rate: 3, while: TimeSpan.FromSeconds(15)) ); NBomberRunner .RegisterScenarios(scenario) .Run() ;
The NBomber only spams the API at the rate of one request per second. Then, at intervals, twice per second for the next ten seconds. Finally, three times per second for the next 15 seconds. This prevents the local development machine from being overloaded with too many requests. NBomber also uses network sockets, so be careful when both the target API and the reference tool are running on the same machine.
The test step tracks the response code and sets it to the return value. This tracks API errors. In .NET, when the Kestrel server receives too many requests, it rejects those with an error response.
Now, inspect the results and check latencies, concurrent requests, and performance.
The P95’s latencies show 1.5 seconds, which is what most customers will experience. Performance is still poor, because the tool was calibrated to only reach three requests per second. On a local development machine, it is difficult to figure out the concurrency, because the same resources running the reference tool are also needed to service the requests.
dotTrace analysis
Then choose a tool that can do algorithmic analysis like dotTrace. This will help further isolate where the performance issue might be.
To analyze, run dotTrace and take a snapshot after NBomber sends the API as hard as possible. The goal is to simulate a heavy load to identify where the slowness is coming from. The already established benchmarks are good enough, so make sure you use dotTrace together with NBomber.
According to this analysis, about 85% of the time is spent on the GetFromJsonAsync call. Looking at the tool reveals that this is coming from the HTTP client. This correlates with performance theory, because this shows that the asynchronous loop with O(n^2) complexity could be the problem.
A benchmark tool that runs locally will help identify bottlenecks. The next step is to use a…