ElectronCGI – A solution to cross-platform GUIs for .Net Core

.Net Core brought the ability to run (and develop) .Net in multiple platforms. However, it is still not possible to build graphical user interfaces (GUIs) using .Net that run in non-windows environments.

For multi platform GUIs Electron is probably the most popular choice. Electron applications are usually built exclusively using JavaScript or, in the case of using .Net with Electron, by employing tricks like having a full webserver (.Net) and having the Electron application be a “regular” website that makes requests to server. This mimics exactly the workings of a normal website, the only difference is that the website is running inside Electron.

Requiring a full webserver to be able to run a desktop application just doesn’t feel right. Plus, having to deal with all the intricacies of a web framework when the end goal is a desktop application feels wasteful.

ElectronCGI is an alternative to this, where no webserver is required. The way it works is by starting a .Net process in Node and using its stdin and stdout streams as communication channels. This enables full duplex communication between two processes written in different languages with very little overhead.

Also, ElectronCGI makes this super easy: create a connection, and then just listen for, and send requests. That’s all there is to it.

Even though ElectronCGI only requires Node (and .Net) I’ve named it ElectronCGI because I imagined that its primary use case would be for enabling writing GUIs in Electron whose behavior was defined in .Net. ElectronCGI is not limited to this though. It can be useful for any scenario where you want or need some functionality form .Net in Node (or the other way around).

In the last couple of months I’ve been working on improving ElectronCGI, namely having intellisense in JavaScript when electron-cgi is imported/required. Also improving async support in .Net and finally enabling full duplex communication between the two processes (i.e. having .Net code start a request that is fulfilled in Node). These improvements is what this blog post is about.

Full two-way communication between Node and .Net

ElectronCGI – Cross Platform .Net Core GUIs with Electron is the original blog post where I introduced ElectronCGI. There were no changes in the syntax between the last version and the one we will be discussing in this blog post (0.5).

If you want to know the motivation behind ElectronCGI and read a more comprehensive discussion on how to get started, that’s the place to go. Alternatively you can also watch this video where I show how you can setup ElectronCGI and also showcase some applications built with it.

Before I continue with an example that demonstrates how you can have two way communication between Node and .Net using ElectronCGI let me describe some terminology.

ElectronCGI provides you with the ability to create a connection between a Node and a .Net project. The connection is initiated in Node and the .Net project “listens” for for connections.

Node.js

Here’s how creating a connection looks like in Node:

const { ConnectionBuilder } = require('electron-cgi');

const connection = new ConnectionBuilder()
    .connectTo(pathToDotNetExcecutable, commandLineArgument1, commandLineArgument2, ...)
    .build();

When connectingTo a .Net process you can supply the path to an executable, or you can also use the dotnet command with the appropriate arguments. This has the advantage of not requiring you to publish the dotnet executable every time you make changes to it (if you use dotnet command and there are changes, there’s an automatic compilation step that is triggered before the executable is run).

Here’s how that would look like:

const { ConnectionBuilder } = require('electron-cgi');

const connection = new ConnectionBuilder()
     .connectTo('dotnet', 'run', '--project', 'PathToDotNetCoreProject')
    .build();

After having a connection in Node you can send requests to the .Net process:

connection.send('loadOrder', orderId, order => {
    //do something with order
});

The connection’s send method required 3 arguments. A request type (e.g. loadOrder), one request argument (e.g. orderId) and a callback function that is invoked after the request (loadOrder) is handled in the .Net connected process.

You can omit the callback function if you are not expecting a response from .Net (e.g. connection.send('start') or connection.send('select', 'theSelection')). But if you are expecting a response form .Net but don’t need to send an argument there’s currently no support for that. You can send null or undefined for example and ignore the argument in .Net. For example

connection.send('start', null, () => { console.log('started'); });

New in this version is that you can also register handlers for request types initiated in .Net, for example:

connection.on('posts', listOfPosts => {
    //...do something with listOfPosts                 
});

.Net

Here’s how listening for incoming connections looks like in .Net:

    var connection = new ConnectionBuilder().Build();

    //register handlers for request types

    connection.Listen();

To register a handler for a request type you can use one the overloads of connection.On for example:

connection.OnAsync<int, Order>("loadOrder", async orderId => {
    var order = await OrdersDb.GetOrder(orderId);
    return order;
});

You can also send requests from .Net to Node.js directly, for example:

connection.Send("theAnswer", 42);

Or, if you want to run some code after Node.js handles que request:

connection.SendAsync("getData", whichData, async theData => {
    await Db.Save(theData);
});

Basically all the request capabilities from Node are now available in .Net as well.

Async improvements

The last release (0.3) was all about adding proper async support in .Net.

In the previous versions, even though you could register async request handlers (with the .OnAsync connections methods) the handlers would run in sequence. That means that if you had 2 requests, one that would take 2 seconds and another that would take 1 second, you would get the response for the second request after 3 seconds. That’s the time that both requests would take to complete.

After and in version 0.3, if a request comes in and another is still being handled they both run in parallel. In the example above you would get a response for the second request after 1 second.

The impact that this change had is easier to visualise than to imagine. The next two gifs show what making 200 concurrent requests that take a random amount of time to complete (from 0 to 200ms) looks like in one version and the other.

In the example the requests are sent from Node.js with a number from 0 to 199 and when a response is received the corresponding square in the grid turns red.

Versions before 0.3:

Versions 0.3 and after:

Concurrency was enabled by using TPL Dataflow. I wrote two blog posts about Dataflow: TPL Dataflow in .Net Core, in Depth – Part 1 and TPL Dataflow in .Net Core, in Depth – Part 2 during the process of incorporating it in ElectronCGI.

Changes on what goes “in the wire”

In version 0.5 we added the ability to have full duplex communication between Node.js and .Net. As a consequence, the format of the messages that are exchanged between the two processes has changed.

Previously a message sent from Node.js to .Net had the following format (.Net’s stdin):

{"id": "guid for request", "type": "theRequestType", args: JSON.stringify(theArgsPassedInSend || {})}

And a response from .Net to Node.js (.Net’s stdout):

{"id": "guid from request", "result": "the return value from the handler or this property omitted if no value was returned"}

This old format assumed that only requests will be written to .Net’s stdin stream and responses on its stdout’s.

In this new version both types of messages can be sent in either channel (stdin or stdout).

These are the new message formats:

{"type":"REQUEST","request":{"type":"theRequestType","id":"guid for request","args":JSON.stringify(theArgsPassedInSend || {}}}

{"type": "RESPONSE", "response": {"id":"guid from request","result":"the return value from the handler or this property omitted if no value was returned"}}

The consequence of this change is that if you don’t upgrade both the Node.js npm package and the .Net Nuget package you’ll get communication errors.

Better logging

When you create a connection in a .Net project you have an option to enable logging. You should definitely do this since at this point in time in the project it’s the only way to have visibility if something goes wrong in the .Net project.

Here’s how you can turn it on:

var connection = new ConnectionBuilder().WithLogging(minimumLogLevel: LogLevel.Trace, logFilePath: "the-log-file.txt").Build();

minimumLogLevel and logFilePath are optional with default values of LogLevel.Debug and electron-cgi.log respectively.

Here’s how an uncaught exception in a request handler in .Net looks like in the logs:

ElectronCgi.DotNet.HandlerFailedException: Request handler for request of type 'start' failed.
    ---> System.Exception: An error occurred: Error details
    at RedditScanner.Program.<>c__DisplayClass0_0.<<Main>b__1>d.MoveNext() in /home/blinkingcaret/electron-cgi-duplex-reddit-showcase/reddit-new-posts-scanner-dotnet/Program.cs:line 28
    --- End of stack trace from previous location where exception was thrown ---
    at ElectronCgi.DotNet.RequestHandler`1.HandleRequestAsync(Guid requestId, Object arguments) in /home/blinkingcaret/published/electron-cgi-dotnet/ElectronCgi.DotNet/RequestHandler.cs:line 22
    at ElectronCgi.DotNet.RequestExecutor.<>c__DisplayClass5_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/blinkingcaret/published/electron-cgi-dotnet/ElectronCgi.DotNet/RequestExecutor.cs:line 35
    --- End of inner exception stack trace ---
    at ElectronCgi.DotNet.RequestExecutedChannelMessage.Send(IChannel channel) in /home/blinkingcaret/published/electron-cgi-dotnet/ElectronCgi.DotNet/RequestExecutedChannelMessage.cs:line 16
    at ElectronCgi.DotNet.MessageDispatcher.<>c__DisplayClass3_0.<<StartAsync>b__0>d.MoveNext() in /home/blinkingcaret/published/electron-cgi-dotnet/ElectronCgi.DotNet/MessageDispatcher.cs:line 27

If you turn on LogLevel.Trace the messages that are exchanged between the processes will also be logged.

Better debugging

When you are building an application while taking advantage of ElectronCGI’s capabilities it can be difficult to understand when things are not going the way we expect, especially if the problem is in the .Net side of things.

In previous versions, apart from the log file and the connection closing, there was no feedback when things didn’t work as expected.

This was especially annoying when there was, for example a compile error in .Net code, or an incorrect path when calling dotnet run. The connection would simply close and no other feedback was available.

Going forward ElectronCGI redirects the .Net stderr stream to the Node process’ stdout stream and inverts the background color to highlight that messages came from .Net’s stderr. Here’s what you’ll see now if you, for example, mistype the path for the .Net project when creating a connection in Node.js:

In the image above is what you see in the console when you run the node project that creates a connection by using dotnet run --project pathThatDoesNotExist and what you would see if you ran dotnet run --projet pathThatDoesNotExist directly.

The part in red (highlighted with the yellow box) is what you can see in the Node project’s console output. Not terribly helpful in this case (it would be better if MSBUILD’s output was also written to stderr, but there’s no easy way to change that).

Other mistakes have more helpful error messages, for example, if you mistype --project, e.g.: dotnet -project path:

This also means that in the .Net code you can use Console.Error.WriteLine and have messages be displayed in the Node process’ console. Might be useful during development.

It is also possible to use Visual Studio’s debugger and attach it to the .Net application while it’s running. After doing this you can add breakpoints and have a debugging experience identical to any other .Net application.

Here’s a video of me doing that with the example application described later in the post:

Just be sure you pick the right process in the .Net Attach dropdown.

Intellisense

Another addition that will improve the development experience when using ElectronCGI is the availability of intellisense when you include the electron-cgi npm package.

To get this you only need to update to the latest npm package.

Full example

For this version I decided to create an example that focused on async and duplex (i.e. the ability to initiate requests from either Node or .Net).

I decided to keep the example as simple as possible, so instead of using Electron the application is just a Node.js and .Net Core application that connect to reddit to periodically fetch new posts for a subreddit that the user can select when the application starts.

If you want to see examples with Electron, there’s the calculator demo and a video of encelados-viewer which is an application that allows a user to query a postgres database with a UI built using Angular. This app isn’t on github but you can see a video of it running here. The only reason why it’s not on github is because it relies on the postgres db, and I haven’t had time to create a stripped down version of it.

Ok, so the example for this version is an application that allows you to select a subreddit and on every second updates a list of the latest posts from that subreddit.

There are 4 request types that are used in the example. 3 sent from Node.js to .Net and one from .Net to Node. From Node, the request types are: start, stop and select-subreddit. From .Net to Node.js there’s only the show-posts request type.

One interesting thing in this example is that the start request will execute in .Net until a stop request is received from Node.

While that stop request doesn’t arrive, the .Net process sends show-posts requests to Node with a list of posts.

While start is running, if a select-subreddit requests arrives form Node it will update the selected subbreddit maintained in .Net. From that moment on the show-posts request will contain the posts from the new subreddit (this doesn’t happen in the sample app though, the select-subredit request is only sent once, before start).

Here’s a diagram that summarizes the example:

You can find the full source code for this example in github here.

Most of the interesting parts are in the .Net side, here’s a short description of how each request type works.

First the select-subreddit (all of this code is in the Main method):

var selectedSubreddit = "";
connection.On("select-subreddit", (string newSubreddit) =>
{
    selectedSubreddit = newSubreddit;
});

Here we use a closure to update a shared variable named selectedSubreddit that’s going to hold a subreddit name. This illustrates that we can maintain state in the .Net side.

Next is the start request type:

CancellationTokenSource cancelationTokenSource = new CancellationTokenSource();
connection.OnAsync("start", async () =>
{
    Console.Error.WriteLine("started"); //this will show up in the node console
    await Task.Run(async () =>
    {
        while (true)
        {
            if (cancelationTokenSource.Token.IsCancellationRequested)
                return;
            try
            {
                var posts = await redditClient.GetLatestPostsFromSubreddit(selectedSubreddit);
                connection.Send("show-posts", posts);
                await Task.Delay(1000);
            }
            catch (Exception ex)
            {
                Console.Error.WriteLine($"Failed to get posts for {selectedSubreddit} (maybe it does not exist?)"); //this will show up in the node console
                Console.Error.WriteLine(ex.Message);
                return;                            
            }
        }
    }, cancelationTokenSource.Token);
});

Here we are using async to await for a task that we created inside the start request handler. That task will, at 1 second intervals, query reddit for new posts on the selected subreddit. Every time new posts are read a request named show-posts is dispatched from .Net to Node with the list of the latest posts. This illustrates the duplex part (i.e. both Node.js and .Net can initiate requests to each other).

This will go on until the Cancel method is invoked in the CancellationTokenSource. When this happens the start request handler completes. So in theory the start request may run indefinitely.

Here’s how performing the start request in Node.js looks like:

connection.send('start', null, () => {
    //this will run when start finishes in .Net
    connection.close();
});

This means that the connection will be maintained until the start request completes.

Finally, the stop request handler in .Net is what invokes the CancelationTokenSource‘s Cancel method. Here it is:

connection.On("stop", () =>
{
    Console.Error.WriteLine("Stop");
    if (cancelationTokenSource != null)
    {
        cancelationTokenSource.Cancel(); //this will cause the start request handler to complete
    }
});

In Node.js, the show-posts request handler will display the latests posts in the console:

connection.on('show-posts', posts => {
    console.clear();    
    if (posts.length === 0){
        console.log('No results...');
    }
    posts.forEach(post => {
        console.log(`${post.title} ${post.upvoteCount}↑ (${post.url})\n`);
    });
});

What’s next

One really important thing for the future of electron-cgi is getting feedback. The easiest way to give feedback is through the project’s github issues (both for electron-cgi and ElectronCgi.DotNet).

What I’ll probably work on next is on error handling. Particularly having the errors propagate between Node.js and .Net and vice-versa.

Right now, if there’s an exception on a handler in .Net it gets written to a log file and the process terminates, which is not ideal.

An alternative is to have that error surface on the other process, for example:

connection.send('getOrder', orderId, (err, order) => {
    if (err) {
        //there was an error
    }else {
        //use order
    }
});

Something that might be interesting is to have the send method in Node.js return a promise that resolves or rejects in case the request is successful or fails, respectively.

And finally, more documentation. Some not necessarily related to electron-cgi in particular, but on using Electron with electron-cgi and other technologies like React, Angular, Vue or Blazor.

There a few hoops to jump through in order to get a good development experience while writing a UI using React or Angular while having non-UI logic running in .Net Core with electron-cgi. That’s also something I plan to write about in the future.