physical limitation: we can always add new servers to withstand a greater load. However, to work on this architecture, applications have to be designed anticipating this possibility, because as each request can be processed by a different server, you cannot use the local resources of the latter (such as the memory or the file system) to store data persistently or to store the state of the application itself.

As a side note, there is an alternative called session affinity, which consists in making the load balancer always assign the requests coming from a certain user to the same server, whereby state information could be stored in the servers. Also known as sticky sessions, this technique is better than vertical scale-up but worse than the horizontal scale-out with a stateless approach.

Scalability in SignalR

We have explained that in horizontally scaled web environments we should not use local server resources such as the memory or disk. Things get even more complicated when it comes to SignalR applications.

When we looked at state maintenance techniques at the server in Chapter 5, “Hubs,” we described the problems of having multiple SignalR servers serving in parallel and we gave some guidelines for solving them. But, as we also explained, SignalR poses a real challenge in horizontally distributed scenarios, especially because each server is aware of only the clients that are connected to it directly. The problem this presents is illustrated in the “minimalist” example shown in Figure 8-4: user A is connected to the SignalR service, and the load balancer allocates it to server #1. Then comes user B and it is allocated to server #2.

FIGURE 8-4  Possible scenario in a horizontally distributed environment with SignalR.

Although both users would theoretically be connecting to the same persistent connection or hub—there would be only one access URL—the balancer would have delegated its processing to a particular server, a different one in each case. And because each one would be physically connected

Deploying and scaling SignalR  Chapter 8

155

www.it-ebooks.info

to a different server, a broadcast message issued by server #1 would be received only by user A, and the same applies to the other server.

In addition, this allocation of clients to servers could change every time a new physical connection opens, generating situations that would be difficult to manage due to roaming. For example, imagine that user A is connected to server #1 and then experiences a time-out due to network problems, reconnects, and is allocated to server #2. From the perspective of server #1, user A has left the application, whereas from server #2, user A would be considered a new user. Moreover, during the time taken for the reconnection process, new messages that merit being sent to the user could have entered the system.

This same effect can take place with parallel connections made by the client to send information to the server in push one-way transports such as server-sent events or long polling. Each one could be processed by a different server, yet the main push connection would remain active and fixed from the start. As you can guess, these problems are not easy to tackle.

To address these scenarios, SignalR comes with a powerful out-of-the-box mechanism allowing the deployment of its server components in horizontally distributed environments through backplanes. As shown in Figure 8-5, a backplane is a component that acts as a messaging system between SignalR nodes, similar to a bus for internal communication between systems. When enabled on a server, all messages generated from it will be sent through the backplane, where the remaining servers will be listening to forward them to their clients.

FIGURE 8-5  Operation of SignalR with a backplane.

There exist different backplanes, each using a different technology to manage publications and subscriptions internally used by the messaging system. Currently, SignalR officially offers backplanes

156 Chapter 8  Deploying and scaling SignalR

www.it-ebooks.info

for Windows Azure Service Bus, SQL Server, and Redis, but the developer community has provided some additional ones. We will describe each one in detail later on.

Although employing a backplane enables the use of SignalR in a server farm and the distribution of user load among its servers, doing so also has its limitations. The main limitation is that it will inevitably lead to an increase in the time required to send each message and hence to a significant decrease in the number of messages that can move through the system—that is, the through-

put. Paradoxically, the same solution used to enable horizontal scalability of SignalR services could become a bottleneck in certain scaled out architectures. In fact, scalability is not synonymous with increased performance in this case.

Therefore, we should not think of backplanes as the ultimate solution to the scalability problems of all kinds of SignalR applications, but only as the mechanism that allows using this framework with balanced server farms, in the event that this were to be the deployment architecture finally chosen for our system.

The use of backplanes is recommended in applications where there is a server broadcasting the same message to several clients or groups of clients. This is because, in SignalR, messages are processed in such a way that sending the same message to many clients would not saturate the bus. Their use is also appropriate when there is a very low frequency of submissions, or where their amount is not directly related to the increase in the number of clients. In all these cases, a backplane could offer good performance and, potentially, at the same time, nearly unlimited scalability.

However, as soon as personalized information needs to be sent to each client or immediacy is critical, we have to resort to other solutions.

When designing a scalability strategy that offers appropriate performance for the type of application in question, there are as many solutions as types of systems and development teams trying to implement them. What would be the best way to design a scalable SignalR application? The best answer we can give is “it depends.”

In general, the formula for scaling our systems always consists in distributing the load across multiple servers, but if we want to maintain a good performance in SignalR-based real-time services, we have to design and implement our own solutions. And in this case, they will almost always involve searching for techniques for partitioning or distributing users following logical criteria that are specific to the domain of the application that we are developing, to group them into the same server. If all the users with whom we want to communicate as a group are found on the same server, submissions will be direct and we will avoid intermediaries that might introduce latency, such as when using backplanes. We will also achieve efficiency, and we will be able to scale out as much as the grouping strategy that we have chosen allows us, although usually at the cost of increased development effort.

The simplest and most basic approach could arise if we knew in advance and with certainty what users we are dealing with and their distribution, because we would be able to define a static architecture for them. We would just have to prepare an infrastructure of servers configured and adapted to the initial needs of the system, as shown in Figure 8-6, and adapt it according to the evolution of demand.

Deploying and scaling SignalR  Chapter 8

157

www.it-ebooks.info

FIGURE 8-6  Horizontal partitions of a social network for developers.

In this example, we could create new chat services as new technologies emerged in the future and host those services on servers with spare capacity or add new servers. However, to make this option possible, due to the very nature of the service, we should ensure that none of the partitions will grow beyond what has been assumed when planning the size of the infrastructure.

We can also take a more dynamic and flexible approach, which would be advisable in most scenarios. Conceptually, this would mean creating a “smart” balanced system, sensitive to the domain of the application, which could consist of a front service able to redirect clients to the appropriate server, based on certain criteria.

For example, in a collaborative text editing service, a user-grouping criterion could be the document itself. If a user entered the application and decided to open a document, a SignalR server could be assigned to them on the fly—the least busy server in a set of available servers. This mapping would be stored in a centralized persistence system, and the user would be redirected to said server so that she could start working on the document. If some other user accessed the same document later, the application would know that there is already a real-time server assigned to it and would redirect this new user to it. In this case, the criterion for partitioning or distributing users would be the document.

We could find another use in an internal messaging system on a multitenant ERP distributed as

SaaS4. In this case, the criterion for allocating users could be the tenant itself—that is, the company using the services—because each of them would be completely independent of the others and, therefore, real-time services could be provided in separate servers.

Note that a similar approach is currently used in many applications. For example, there are many online games where users have a usable area to find opponents or to perform other activities, but at the time of starting the game, an existing server is assigned and all users associated with the same game are redirected to it. In this case, the partitioning criterion would be the game.

4 SaaS: Software as a Service.

158 Chapter 8  Deploying and scaling SignalR

www.it-ebooks.info