Azure Function benchmark for .Net 4.7 vs .Net Core beta

Azure Function benchmark for .Net 4.7 vs .Net Core beta, how much request per seconds we can squeeze?

There was some benchmark done by my colleague Roberto about running application on app service plan vs app service plan for docker.

That inspired me for some other approach to run the same code sample but on azure function running .net 4.7 vs azure function running .net core

The size of machine for app service plan is S1, testing approach is ab benchmark tool running under windows hosted in azure, do not hit network limits or CPU the machine size is DS5_V2

The setup is pre compiled Azure Function running .Net 4.7

public static class Function1
{
    private const int MaxPage = 101;

    private static Dictionary<int, string> Values { get; set; } = new Dictionary<int, string>();

    private static string GetBodyOfSize(int i)
    {
        if (Values.ContainsKey(i))
            return Values[i];

        var s = new String('X', i);
        //Values.TryAdd(i, s);
        Values.Add(i, s);
        return s;
    }

    [FunctionName("Function1")]
    public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)]HttpRequestMessage req, TraceWriter log)
    {
        var s = req.GetQueryNameValuePairs()
                .FirstOrDefault(q => string.Compare(q.Key, "s", true) == 0)
                .Value;

        if (string.IsNullOrEmpty(s))
        {
            // return simple hello, world
            var now = DateTime.UtcNow;
            return req.CreateResponse(HttpStatusCode.OK, $"Hello World, from ASP.NET Fun App Net 4.7! {now.ToString("yyyy-MM-dd HH:mm:ss.FFF")}");
        }

        if (int.TryParse(s, out int i))
        {
            if (i > 0 && i < MaxPage)
            {
                var body = GetBodyOfSize(i * 1000);
                return req.CreateResponse(HttpStatusCode.OK, body);
            }
            else
            {
                return req.CreateResponse(HttpStatusCode.BadRequest, $"Size must be an integer between 1 and {MaxPage}");
            }
        }
        else
        {
            return req.CreateResponse(HttpStatusCode.BadRequest, $"Size must be an integer between 1 and {MaxPage}");
        }
    }
}

vs Azure Function .Net Core having kestrel under hood in beta version

#r "Newtonsoft.Json"

using System.Net;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
using Newtonsoft.Json;

  private const int MaxPage = 101;

        private static Dictionary<int, string> Values { get; set; } = new Dictionary<int, string>();

        private static string GetBodyOfSize(int i)
        {
            if (Values.ContainsKey(i))
                return Values[i];

            var s = new String('X', i);
            Values.TryAdd(i, s);
            //Values.Add(i, s);
            return s;
        }

        public static IActionResult Run(HttpRequest req, TraceWriter log)
        {
            var s = req.Query["s"];
          if (string.IsNullOrEmpty(s))
            {
                var now = DateTime.UtcNow;
                return  (ActionResult)new OkObjectResult($"Hello World, from ASP.NET Core and Net Core 2.0! {now.ToString("yyyy-MM-dd HH:mm:ss.FFF")}");
            }

            if (int.TryParse(s, out int i))
            {
                if (i > 0 && i < MaxPage)
                {
                    var body = GetBodyOfSize(i * 1000);
                    return  (ActionResult)new OkObjectResult( body);
                }
                else
                {
                    return  (ActionResult)new OkObjectResult( $"Size must be an integer between 1 and {MaxPage}");
                }
            }
            else
            {
                return  (ActionResult)new OkObjectResult( $"Size must be an integer between 1 and {MaxPage}");
            }
}

Cmd for testing ab -n 5000 -c 150 -l http://…… ?s=”” s=1, s=10 , s=100

ab results :

Host	Scenario	RPS (mean)	95% within ms
azure function .net 4.7	Hello World	229	999
azure function .net 4.7	1KB	227	999
azure function .net 4.7	10KB	206	1106
azure function .net 4.7	100KB	127	3798
azure function .net core beta	Hello World	72	2696
azure function .net core beta	1KB	70	3500
azure function .net core beta	10KB	67	3656
azure function .net core beta	100KB	51	5691
azure function .net 4.7 (csx) not precompiled	Hello World	226	1094
azure function .net 4.7 (csx) not precompiled	1KB	242	922
azure function .net 4.7 (csx) not precompiled	10KB	200	1268
azure function .net 4.7 (csx) not precompiled	100KB	126	3128

From results we can see that .net core on azure functions is still in beta and the performance is not amazing but if we compare azure function ruining pre compiled azure function on .net 4.7

the performance look even nice.

When comparing .net 4.7 pre-compiled vs .net 4.7 not precompiled (csx) the result is very similar but the first call are slower.

When we look on .net core beta on Azure function it do not look good but this is just beta, so for final version I assume it can be much better.