MicroBatchFramework — Cloud Native Batch Framework for C#
In my previous post, I made an announcement about the release of the MagicOnion library. In this post, I will discuss the release of the new open source C# CLI/Batch library.
A Framework for CLI Tools
There are many analysis tools for command-line arguments in C#. There are a lot, but when you use them, you are not really looking to analyze command-line arguments, what you actually want is parameter binding. That’s why I decided to make it able to call up methods like a web framework.
It’s made so you can call it up with SampleApp.exe -name “foo” -repeat 5.
It will take care of everything suitably with no annoying standard argument analysis, as long as you define the method.
With applications created with .NET Core, you can make an executable file for Windows/Linux/Mac without a need for installing a runtime. Also, you can use container bases like CircleCI with as much efficacy as other languages. For example, you can let CI create an executable file with input like that shown below.
With regards to distribution, things like NPM Global Tools, at .NET Core Global Tools , are also available.
Also, you can make the Daemon tool because MicroBatchFramework manages the lifecycle of the batch and can check by CancellationToken.
A Framework for Batch
In order to be able to manage efficiently with a single console application, I have prepared a system that can be separated by just defining classes and methods.
You can change the method you are executing with the first argument.
SampleApp.exe Foo.Echo -msg “aaaaa”
SampleApp.exe Foo.Sum -x 100 -y 200
SampleApp.exe Bar.Hello2
Now, I recommend that you make any applications made like this into containers. This is easy even in C# if you are using .NET Core.
For example, if you bring it up with CircleCI in AWS ECR, it will probably come our in the following config.
It’s simple to write, with nothing special needed to get C# into a standard ecosystem.
With a container done up like this, you can now execute it simply as a container, but you can define both execution and scheduling (which you can set with the event triggers in CloudWatch) with relative ease with, for example, AWS Batch . You can then check the standard output in the log on CloudWatch.
In the case that you want to define a complicated work flow, you should be able to use Luigi or Apache Airflow.
If you use the managed kuberenetes services, you can use kuberenetes CronJob for batch scheduling.
Web interfaces with Swagger
Sometimes it can be a pain to type in a command every time you want to check something during development, etc., so I added a function to host it on the web. All it does is use RunBatchEngineWebHosting instead of RunBatchEngineAsync on startup.
This is hosted through Swagger, which makes it easier to confirm execution, etc.
.NET Generic Host
MicroBatchFramework is new in that is constructed console applications on .NET Generic Host.
.NET Generic Host has standard support for logging, config loading, DI. It fully supports the standard methods of config mapping, logging, etc.
It would be great if anyone feels how easy it is to use CLI tools in C#! It really works on all platforms; writing CLI in C# is no problem, for real. I would be even more happy if I manage to get across the fact that C# will work, and is working in normal ecosystems, like CicleCI or Docker.
https://github.com/Cysharp/MicroBatchFramework
It is easy to try by NuGet. Please try it.
