How We Used Akka.Net to process thousands of installments instantly

Akka.Net is a toolkit to build highly distributed, event driven applications using .NET.

Originally written for Java/Scala, this community managed project for .NET is not just a top-notch/high performance port, but a great source code reference to those who want to learn the art of writing high-quality code.

HyperTrends was tasked with a challenge to process thousands of installments on a daily basis. The number could go as high as hundreds of thousands.

In the past, we used CRON jobs that would wake every minute to check if there were installments to process, if there were, the CRON job would run them one by one (or at best create sub-jobs). We used Quartz.Net (another great platform) quite a bit in the past. While it worked, it came with its issues. We had to write a lot of state management code to accomplish similar outcomes.

With Akka, the whole process of performing such tasks is greatly simplified. We built two “Actors” to take care of this process. One called the “Installment Monitor Actor” was responsible for identifying the users that need installments processed, and the other “Installment Billing Actor” was responsible to perform the actual transaction processing. The billing actor’s job was to perform the transaction and report the outcome to the monitor. The monitor would then handle exception scenarios, or terminate the actor as necessary.

Akka has the ability to schedule an activity at periodic intervals. Every Actor System has a built in “Scheduler” that can do this for you.

In this case, we wrote logic to wake up the monitor every 15mins and send it a message to identify the Users that needed installments processed.

var installmentMonitor = TsActorSystem.ActorOf(TsActorSystem.DI().Props<InstallmentBillingMonitor>(),
                    TsConstants.ActorNames.InstallmentBillingMonitor);

                TsActorSystem.Scheduler.ScheduleTellRepeatedly(new TimeSpan(0, 1, 0, 0),
                    new TimeSpan(0, 15, 0), installmentMonitor, new ProcessInstallmentsMessage(), null);

var installmentMonitor = TsActorSystem.ActorOf(TsActorSystem.DI().Props<InstallmentBillingMonitor>(),

TsConstants.ActorNames.InstallmentBillingMonitor);

TsActorSystem.Scheduler.ScheduleTellRepeatedly(new TimeSpan(0, 1, 0, 0),

new TimeSpan(0, 15, 0), installmentMonitor, new ProcessInstallmentsMessage(), null);

Once the Users needing installments are identified, we simply create Child “Installment Billing Actors” (one per user) to process the transactions. These actors can process each individual installment and process the results and gracefully exit.

The logic of building installment actors is as follow:

private void ProcessInstallments(ProcessInstallmentsMessage message)
        {

            var result = _ordersService.GetInstallmentsDue(DateTime.UtcNow);
            if (result.Succeeded)
            {
                _logger.Information("{ActorName} Found {InstallmentCount} installments to process.", Self.Path.Name,
                    result.Value.Count);
                foreach (var installmentItem in result.Value)
                {
                    if (_installmentIds.Contains(installmentItem.Key)) continue;

                    _installmentIds.Add(installmentItem.Key);
                    var installmentActor = GetInstallmentActor(installmentItem.Key);
                    installmentActor.Tell(new RunInstallmentMessage(installmentItem.Key, installmentItem.Value));
                }
            }
            else
            {
                _logger.Information("No Installments found to process.");
            }
        }

private void ProcessInstallments(ProcessInstallmentsMessage message)

{

var result = _ordersService.GetInstallmentsDue(DateTime.UtcNow);

if (result.Succeeded)

{

_logger.Information("{ActorName} Found {InstallmentCount} installments to process.", Self.Path.Name,

result.Value.Count);

foreach (var installmentItem in result.Value)

{

if (_installmentIds.Contains(installmentItem.Key)) continue;

_installmentIds.Add(installmentItem.Key);

var installmentActor = GetInstallmentActor(installmentItem.Key);

installmentActor.Tell(new RunInstallmentMessage(installmentItem.Key, installmentItem.Value));

}

else

{

_logger.Information("No Installments found to process.");

}

Each child Actor is referenced in a Hash Set so that it can be tracked and processed at a later stage. To create the Child Actor, we simply used logic like so:

private static IActorRef GetInstallmentActor(string id)
        {
            var childName = "installment-" + id;
            var actor = Context.Child(childName);
            if (!actor.Equals(Nobody.Instance)) return actor;

            var props = Context.DI().Props<InstallmentBillingActor>();
            actor = Context.ActorOf(props, childName);

            return actor;
        }

private static IActorRef GetInstallmentActor(string id)

{

var childName = "installment-" + id;

var actor = Context.Child(childName);

if (!actor.Equals(Nobody.Instance)) return actor;

var props = Context.DI().Props<InstallmentBillingActor>();

actor = Context.ActorOf(props, childName);

return actor;

}

Due to Akka’s amazing supervision capabilities, the monitor is able to identify any exception scenarios and handle them accordingly.

Upon successful completion, the Installment Billing Actor sends a “Installment Processed” message to the monitor. Upon receiving this message, the Monitor logs the outcome and terminates the child by passing it a ‘Poison Pill’ message.

 private void TerminateInstallmentActor(RunInstallmentResultMessage message)
        {
            _logger.Debug("Terminating Installment Actor with Id: {OrderTransactionId}", message.OrderTransactionId);
            var actor = GetInstallmentActor(message.OrderTransactionId);
            _installmentIds.Remove(message.OrderTransactionId);
            actor.Tell(PoisonPill.Instance);
        }

private void TerminateInstallmentActor(RunInstallmentResultMessage message)

{

_logger.Debug("Terminating Installment Actor with Id: {OrderTransactionId}", message.OrderTransactionId);

var actor = GetInstallmentActor(message.OrderTransactionId);

_installmentIds.Remove(message.OrderTransactionId);

actor.Tell(PoisonPill.Instance);

}

Using this approach, HyperTrends was able to process over 10K installments in no time (mere seconds).

Here is a Sequence Diagram to summarize the process:

So there you have it. Akka makes it extremely easy to process such scenarios using lightweight actors. Feel free to give it a try and let us know what you think!

Leave a Reply Cancel reply