You may want to learn more about Origin Application, Transcode Application and Edge Application.
Cluster Configuration - Storm Streaming Server
In this guide, we will demonstrate how to configure a basic Storm Streaming Server cluster. To create such a cluster, you technically need only a single Storm server instance, where you can deploy a complete set of applications: origin, transcode, and edge. This serves as an excellent starting point for later expanding the cluster with additional servers.
However, for the purposes of this guide, we will assume that you have three server instances running on separate machines, with each instance hosting only a single application.
Server structure
We will have a total of three servers, which we will name Alfa, Beta, and Gamma, respectively for easier identification.
| Server name | Server IP | Hosted Applications |
|---|---|---|
| Alfa | 192.168.10.2 |
ClusterManager origin |
| Beta | 192.168.10.3 | transcode |
| Gamma | 192.168.10.4 | edge |
Table 1. Server structure table
As shown above, the Alfa server will host both the origin application and an active ClusterManager module. The remaining two servers will have this module deactivated.
ClusterManager (Alfa Server)
First, we need to activate the ClusterManager module in the configuration of the Alfa server. This module is located in the main block of the preference.xml file and should look as follows:
<ClusterManagerUnit enabled="true">
<ClusterAuthentication>
<Identity>Mother</Identity>
<IPWhiteList></IPWhiteList>
<Secret>X9fT7mLq2Z</Secret>
</ClusterAuthentication>
<ClusterTranscoding enabled="true" active="true">
<MinimalSourceResolution>1280x720</MinimalSourceResolution>
<StreamGraceTimeout>300</StreamGraceTimeout>
<StreamEvaluationInterval>30</StreamEvaluationInterval>
<StreamWeights enabled="false" />
</ClusterTranscoding>
</ClusterManagerUnit>
Field explanation:
| Identity | Identifier for the mother, so it is possible to recognize which instance we are dealing with. We recommend using naming like "Primary", "Secondary". |
| IPWhiteList | Access to the ClusterManager can be restricted to a list of defined IP addresses. If this field is empty, no restrictions apply. If we want to define an IP list that the server checks before each connection, simply enter the IP addresses separated by commas, e.g., "192.168.0.2, 192.168.0.3" (whitespace doesn't matter). |
| Secret | This is the password that all applications logging into the ClusterManager must present. |
| ClusterTranscoding:enabled | Defines whether transcoding within the cluster using transcode applications is enabled. |
| ClusterTranscoding:active | Defines whether transcoding using dedicated applications should start immediately after the cluster is launched. It is also possible to pause and resume transcoding from the control panel. |
| MinimalSourceResolution | The minimum resolution above which a stream qualifies for transcoding. |
| StreamGraceTimeout | The ClusterManager module dynamically assigns and revokes transcoding tasks for individual streams. This parameter defines a grace period (in seconds) during which a stream, once assigned for transcoding, is protected from having its transcoding revoked. |
| StreamEvaluationInterval | This is the time interval, measured in seconds, after which the ClusterManager re-evaluates all streams to determine whether transcoding should be assigned or revoked. |
| StreamWeights:enable |
Defines whether the ClusterManager should use the number of viewers of a given stream when assigning transcoding, or rely on a weighting system (if the value is set to true). If the weighting system is enabled, two additional parameters must be provided:
Example:
Where {streamKey} will be replaced with a proper streamKey. Expected answer (JSON format):
Available weight strategies:
|
Table 2. ClusterManager field explanation table
Origin (Alfa Server)
Below is an example configuration for an origin-type application. The most important block here is StreamingClusterSettings, which defines the connection with the ClusterManager. It is possible to define more than one connection, but each must be assigned a priority (higher the number, higher the priority).
Sample configuration:
<Application name="origin" type="origin">
<StreamingClusterSettings>
<SupervisorList>
<ClusterManager host="192.168.10.2" port="443" isSSL="true">
<Priority>1</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
</SupervisorList>
<PeerAccessSettings host="192.168.10.2" port="443" isSSL="true"/>
</StreamingClusterSettings>
<!-- other blocks below -->
</Application>
Field explanation:
| ClusterManager:host | Host or IP address where the ClusterManager controller is located. |
| ClusterManager:port | Port on which the WebSocket protocol of the ClusterManager controller is available (usually 8080 or 443). |
| ClusterManager:isSSL | Whether to use SSL communication with the ClusterManager. |
| Priority | Importance of this connection compared to parallel connections. |
| Secret | Password with which the application will log into the ClusterManager (must be the same). |
| PeerAccessSettings:host | Public host or IP address of this application, so other applications know where to connect. |
| PeerAccessSettings:port | Public port of this application, so other applications know where to connect (indicate a port with an active RTMP protocol). |
| PeerAccessSettings:isSSL | Whether the connection should use SSL. |
Table 3. Origin / Transcode / Edge field explanation table
You can learn more about specifics configurations in: Origin Application Guide.
Transcode (Beta Server)
<Application name="transcode" type="transcode">
<StreamingClusterSettings>
<SupervisorList>
<ClusterManager host="192.168.10.2" port="443" isSSL="true">
<Priority>1</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
</SupervisorList>
<PeerAccessSettings host="192.168.10.2" port="443" isSSL="true"/>
</StreamingClusterSettings>
<!-- other blocks below -->
</Application>
You can learn more about specifics configurations in: Transcode Application Guide.
Edge (Gamma Server)
Below is an example configuration for an edge-type application - it's very similar to Origin application with just a few changes.
Sample configuration:
<Application name="edge" type="edge">
<StreamingClusterSettings>
<SupervisorList>
<ClusterManager host="192.168.10.2" port="443" isSSL="true">
<Priority>1</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
</SupervisorList>
<CopyOnPublish>true</CopyOnPublish>
</StreamingClusterSettings>
<!-- other blocks below -->
</Application>
Field explanation:
| CopyOnPublish | Whenever stream from an origin application should be immediately copied to this edge application, otherwise it'll wait for a first request from a viewer. |
Table 4. Edge field explanation table
You can learn more about specifics configurations in: Edge Application Guide.
Redundancy
As mentioned earlier, a single server instance can only support a single ClusterManager controller. However, for each edge and origin application, it is possible to define connections with multiple such controllers located on different servers. This ensures that in case of a failure of the main instance, the cluster will continue to operate without interruption. When the main ClusterManager regains functionality, all applications will reconnect to it.
Sample configuration:
<Application name="edge" type="edge">
<StreamingClusterSettings>
<SupervisorList>
<ClusterManager host="192.168.0.2" port="8080" isSSL="false">
<Priority>2</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
<ClusterManager host="192.168.0.10" port="8080" isSSL="false">
<Priority>1</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
</SupervisorList>
<CopyOnPublish>true</CopyOnPublish>
</StreamingClusterSettings>
<!-- other blocks below -->
</Application>
Support Needed?
Create a free ticket and our support team will provide you necessary assistance.