Patterns & Practices for efficiently handling C# async/await cancel processing and timeouts

One important use of async/await is proper cancellation processing. Create a CancellationTokenSource, pass a CancellationToken, and handle OperationCanceledException. All this, however, faces some difficulties to implement efficiently.

Based on a pattern I faced when implementing a high-speed network client called AlterNats, I will show how to write a process for situations where “external cancellation, timeout, and dispose of the main source” are combined, appropriate exception handling, and finally an efficient zero-allocation CancellationTokenSource handling method that also includes ObjectPools.

CreateLinkedTokenSource Pattern

The simplest pattern for cancellation processing is to provide a CancellationToken at the end of the argument and propagate it to the inner methods. If propagated properly, the innermost process will properly handle the CancellationToken and throw an OperationCanceledException when it detects a cancellation.

In addition to optional cancellations, it is often desirable to include a timeout process, especially in communication systems, and the basic idea behind timeouts in async/await is that a timeout is also a cancellation process.

CancellationTokenSource has a method called CancelAfter that fires a cancel after a certain period of time, so using this method to pass a CancellationToken will result in a timeout.

UniTask provides a method called CancelAfterSlim, which is a Unity-friendly implementation because Cancel uses the thread pool while CancelAfterSlim runs on the PlayerLoop. However, the way to stop the internal timer is different in CancelAfterSlim. However, there are some differences in implementation, such as the fact that the internal timer stop method requires the return value of CancelAfterSlim to be disposed of.

Since the timeout period is usually fixed, having the user use CancelAfter each time is a difficult design to use. Therefore, CancelAfter should be executed inside the SendAsync method. To combine such an internal timeout CancellationToken and an external CancellationToken into a single CancellationToken, use the CancellationTokenSource.CreateLinkedTokenSource.

A CancellationTokenSource generated by CreateLinkedTokenSource will itself be cancelled when any of the linked CancellationTokenSource is cancelled. A Cancel can also be fired from itself.

This is complete, but there is a problem with exception handling as it is.

OperationCanceledException has a property called CancellationToken, which allows the caller to determine the cause of the cancellation. As an example, we may catch an OperationCanceledException and then add a further judgment to branch the code as shown below.

CancellationToken that concatenated with CreateLinkedTokenSource, which is meaningless information and cannot be used to determine the cause of the exception.

Another problem is that timeouts are treated as an OperationCanceledException, which is a special exception that is often omitted from logging as a known exception (e.g. (For example, it is common for a web server to be canceled due to a client disconnecting (e.g., closing the browser during a request), but logging every time an error occurs would be an error fest). ) A timeout is an obvious anomaly, and should be distinguished from such cancellations, and should be an exception, not an OperationCanceledException.

.NET also raised an Issue as HttpClient throws TaskCanceledException on timeout #21965, which caused a heated debate. The reason is that the implementation was using CreateLinkedTokenSource as shown in the example above, and did not do any handling.

.NET’s compatibility policy to change the exception type of a class once it has been released to the world (in fact, changing it would have a significant impact) so they put a TimeoutException in the InnerException so that judgments could be made via that exception.

This is only a compromise, so make sure that any new code you create is handled correctly.

Finally, let’s assume that the Client itself can dispose, and let’s make the code react to that.

The only difference is that CreateLinkedTokenSource increases the number of tokens to be linked and the number of branches for exception handling.

Make to the Zero Allocation

In most cases, the above pattern is perfectly fine, but you may be concerned about creating a new CancellationTokenSource with CreateLinkedTokenSource each time. Either way, if the async method is executed asynchronously, the asynchronous state machine itself will be allocated, so it is not something to be concerned about. However, if you have an asynchronous implementation that avoids allocation, using IValueTaskSource or PoolingAsyncValueTaskMethodBuilder, it becomes a matter of concern. Also, although it is not a big deal for the frequency of HTTP/1 REST calls, you may want to pay attention to this issue if, for example, the server is exposed to a large amount of parallel execution and the client communicates in every frame in real-time communication.

So, let’s proceed with zero-allocation. Note that the SendAsync method itself will remain async Task for simplicity of explanation here.

First, let’s look at the case with no external cancellation, just a timeout. Since timeouts do not fire in the case of normal systems, that is, they do not fire in most cases, we should use CancellationTokenSource for non-firing cases.

There are various implementations of ObjectPool, but for simplicity of explanation, I used Microsoft.Extensions.ObjectPool (you must reference Microsoft.Extensions.ObjectPool from NuGet). If a timeout is triggered, it is not reusable and should not be returned to the pool. Note that CancellationTokenSource.TryReset is a method from .NET 6. Prior to that, there is a hack to call CancelAfter(Timeout.InfiniteTimeSpan) to put in a change that stretches the timer time to infinity (internally the Timer is changed).

If an external cancellation is to be entered, do not create a LinkedToken, but cancel the CancellationTokenSource for the timer with CancellationToken.UnsafeRegister.

CancellationToken.UnsafeRegister is a method from .NET 6 that is more efficient because it does not Capture the ExecutionContext. NET 6, you can use Register or the hack of doing an ExecutionContext.SuppressFlow/RestoreFlow before and after the call (see UniTask’s RegisterWitho (UniTask’s RegisterWithoutCaptureExecutionContext uses this implementation).

If a callback is made to a CancellationToken, there is a possibility that a race condition may occur. In this case, if a CancellationTokenSource for Timeout is returned to the pool and then a Cancel occurs, it would be a disaster. To prevent this, always call CancellationTokenRegistration.Dispose before TryReset. The beauty of CancellationTokenRegistration.Dispose is that if the callback is running It will block and wait until the execution is finished to ensure that it is done. This prevents multithreading timing problems that can slip through.

Although CancellationTokenRegistration also has a DisposeAsync function, it is more efficient to call Dispose than to call the DisposeAsync function because most of the time, it is like lock. There is also an Unregister method in CancellationTokenRegistration, which is useful for fire-and-forget type unregistration. This method is useful for fire-and-forget type deregistration. It is a matter of usage.

UnsafeRegister” method is used to register a callback to a CancellationToken, which generates a slot for callback registration the first time. The second and subsequent times, however, the slot is reused. This is another advantage over creating a new (Linked)CancellationTokenSource.

Let’s continue with an implementation that adds a CancellationToken that is hooked into the lifetime of the Client itself. However, we simply add a Register.

Of course, exception handling is also required! but this is the same as in the first example made with LinkedToken.

So, this is the final form.

As to whether it is easy or not, I think it is quite difficult if you are told to think about it each time you do it. So it is important to learn the pattern.

StackExchange.Redis also does not accept the asynchronous method, CancellationToken, and it is a very difficult problem to add support for CancellationToken while pursuing performance. NET 6 generation, however, the number of methods has increased considerably, and it can be done if you want to do it.

Starting with AlterNats, I will continue to push the libraries I offer without making any compromises.