Multicast In the Cloud: The Missing Link to Removing Barriers You Didn’t Know You Had

Multicast technology is the underpinning for high-performance networks spanning sectors and industries. From legacy networks that have used multicast applications for more than a decade, including power and energy simulation systems, emergency management and audio dispatch systems, to newer uses by the finance, media and broadcasting worlds, this advanced feature improves performance, scale and keeps networks flexible under the tremendous weight added by data.

Multicast Network Challenges: Not Making the Connection

Despite the functionality multicast provides, this workhorse feature stops in its tracks as soon as its moved to the cloud, where the robust on-premises hardware and equipment needed to support multicast unavailable. Cloud migration projects come to a halt when this roadblock is discovered, spending time and money to create expensive workarounds. Worse, other developers don’t realize multicast in the cloud is available at all, assuming it remains a pipedream.

This piece reviews the ways that swXtch.io’s cloudSwXtch overlay network removes these barriers, known and unknown to engineers, bringing virtual bare-metal performance and true multicast to on-prem-to-cloud and cloud-to-cloud environments across sectors.  

What Is Multicast?

Multicast is two things:  

1. It’s a protocol or standards-defined way of creating a kind of User Datagram Protocol (or UDP packet) that’s replicable inside of a network infrastructure.  

2. It’s the theory of “fanning out,” or duplicating data packets to go to multiple endpoints at once.  

Let’s look at point number one. Multicast is an Institute of Electrical and Electronics Engineers (IEEE) standard, created in the mid ’80s, when engineers began realizing that if they wanted to use more high-performance or complex systems relying on different subsystems that required data at the same time, they’d need to be able to reproduce UDP packets and shoot them to those different end points simultaneously.

Multicast additionally relies on a standard called IGMP, Internet Group Management Protocol, which allows many applications, or a multicast group, to share a single IP address and receive the same data at once. (For instance, IGMP is what might allow multiple people to receive messages sent to a single email address. It’s also used in gaming, when multiple gamers are using the network at once to play together.)  

How Does Multicast Work?

To join the multicast group, an application subscribes to the data via an IGMP join. Whenever a packet from that multicast group comes to the switch, it makes a copy of the packet and sends it to them. Simultaneously, it also sends the copy to the other, say, 10 members or end points that are also part of the multicast group.  If the end points aren't interested in those packets anymore, they leave the group, using an IGMP leave message. By virtue of leaves and joins, you create communities of packet consumption.  

On to point number two and fan out. Envision multicast as a hub and a spoke: the hub is the switch, and the spokes go out to the end points. If you have two switches on opposite ends of a network, they can push a multicast stream between them, but the end point switch won't take on the work of replicating or creating multiple copies of a unicast packet. Instead, it’s fanned out at the edge of the network, or what we call our Protocol Fanout feature, allowing you to send only one stream across several different hubs or switches and not start replication until it’s closer to the consumers that want it. This is also known as delayed replication.  

For example, if you have a producer working on a complicated network with four switches that needs to create 10 separate streams, you don't want those streams winding across multiple switches to get to 10 consumers at the end. Instead, delayed replication allows you to use only one stream between all the switches, fanning them out to the 10 consumers at the end. By delaying the replication, you’ve greatly simplified the network.  

This magic bullet becomes powerless in a cloud setting, however, where cloud network vendors have made strategic decisions not to enable multicast to their core networks.  

Multicast in the Cloud and Lowering Egress Costs

cloudSwXtch creates a single multicast network across all networks, with an emphasis on tackling congestion. It connects on-prem networks to clouds, clouds to clouds, or clouds to edge networks. This means engineers can create a multicast data stream inside of on-prem networks, and, using cloudSwXtch, push it into a cloud and deliver that stream as if it were originating in the cloud and connecting it seamlessly to the on-prem network.

This is important for developers to keep costs down, because when they push data out of a cloud, cloud networks may charge them for that egress, since clouds are best served when they keep your data in their estate.  

You may also incur costs from cloud networks if you need to stream to multiple places. If you can use multicast or protocol fanout to push one stream of data out of the cloud to somewhere else, say your on-prem network or another cloud, you can save money in egress costs by fanning the streams out at the edge.    

True IGMP-Compliant Multicasting in the Cloud for Financial Services

Multicast is required for financial services when it comes to dependencies like trading, pricing, and exchange-type systems, because their multicast endpoints need to get data simultaneously.

The phrase “multicasting in the cloud,” however, will likely get a side-eye from a wary financial services developer. That’s because products that purport to offer “multicast” are really offering simulated multicast solutions, which don't have the performance to support these financial services applications.

Financial service workloads, however, have hundreds of different processing end points that need the data at the same time. Each one of those end points might also be a producer of data, however, requiring the need for hundreds of processes pushing multicast data out to all the others as well as consuming it. This level of complexity typically isn’t supported by a simulated multicast solution.

swXtch.io has built a generalized application in cloudSwXtch, analogous to physical networks that scale with high performance, offering true, IGMP-compliant multicast.  

This is why bringing multicast to the cloud has been called the “holy grail” for financial services developers, who need to migrate their high-performance systems to the cloud without modification.

Once multicast is supported in the cloud via a tool like cloudSwXtch, it unlocks the ability for engineers to move their exchanges, trading systems and, and data distribution systems into the cloud.  

Multicasting in the Cloud for Media and Broadcasting

Like the way financial services have built up multicast-dependent network systems, media has too.  

The standard SMPTE 2110, published in 2017 by the Society of Motion Picture and Television Engineers, specifically requires multicast and is a new way of looking at content streams and moving them to different processing end points that need to interact with them.  

SMPTE 2110 is used in components like playout servers, multiviewers, master control systems or production switchers -- all the essential tools that are part of an on-prem broadcast network or contribute to the production of a video signal into a program. They allow for functionality like combining audio and video or switching to the right camera during a live broadcast. If media engineers want to move that capability to the cloud, they need to be able to support the same standards and methods.

Beyond SMPTE 2110, broadcast has traditionally used multicast because it's a more efficient way to move data and huge video streams. Engineers use multicast to avoid network congestion, and to make more efficient distribution to people at the edge of the network. Some IPTV solutions even use multicast as a way to switch TV channels. Picture that you have a TV with a set-top box from your cable provider. When you go to change the channel, the software in the box is doing an IGMP leave and joining to a different multicast group. Multicast traverses that whole network in order to allow you to get the channel that you want.  

Because there is much legacy use of multicast in the broadcast industry, engineers have been without the ability to move to the cloud for so long they have assumed it wasn’t an option. This wrong assumption has meant engineers have had to architect unnecessary solutions. Expensive decisions that are hard to undo are being made today simply because there is a lack of knowledge multicast is possible.    

Multicast Monitoring

swXtch's multicast monitoring offerings, demonstrated in partnership with TAG Video Systems, showcase the capabilities of their cloudSwXtch virtual overlay network. This technology enhances the monitoring of media workflows as they move through the cloud to endpoints for audience consumption. It includes reliable QC and live signal monitoring with PTP synchronization and supports "by-exception" alerts for specific issues. By connecting to cloudSwXtch's API, TAG Video Systems can deploy diverse monitoring applications in various network configurations, creating a comprehensive toolset for issue resolution and network performance visibility. This collaboration enables broadcasters and media companies to monitor uncompressed formats in the cloud with ultra-low latency, making large-scale cloud monitoring a reality and supporting true cloud diversity, ultimately enhancing media operations in the digital landscape.

A Virtual Switch in the Cloud

While cloudSwXtch cuts down on cost and times, it also makes systems more configurable and maintenance more manageable.  

This is key for engineers who have had to create complicated systems in the cloud, with five or six things talking to each other. Moving to the cloud might mean configuring each one of those virtual machines to talk to every other virtual machine. The provisioning required can become prohibitively complicated.  

Using a virtual switch with no code changes required like cloudSwXtch with multicast and other protocols, makes changes in configurations easy. When using a virtual switch that pushes the data out to everybody at the same time and accepts it from them, engineers can simply add or subtract an end point or a workload easily without reconfiguring every other one.  

If you can add a new feature without changing anything else, you reduce risk, you reduce complexity, and you have a more flexible system.  

Supporting Cloud Infrastructure

Multicast communication is a network communication method that allows data to be efficiently transmitted from one sender to multiple receivers. While multicast is widely used in traditional on-premises networks, its support in cloud computing platforms like AWS (Amazon Web Services), Azure (Microsoft), Google Cloud, and Oracle Cloud is limited and differs across these providers.

Multicast is not typically supported in these major cloud platforms, as it poses challenges in terms of network isolation, scalability, and security. Cloud providers usually favor unicast-based or application-specific alternatives to achieve similar communication patterns while maintaining the flexibility and security of their cloud environments. If multicast-like communication is a critical requirement for your application, you may need to explore third-party solutions or hybrid cloud configurations.

cloudSwXtch supports cloud providers by offering advanced capabilities that enable companies to migrate demanding and mission-critical workflows to the cloud. Here's how cloudSwXtch supports cloud providers:

• Multicast Capabilities: cloudSwXtch provides multicast capabilities, allowing for efficient data distribution to multiple endpoints across cloud networks. This is particularly useful for scenarios where data needs to be simultaneously delivered to numerous consumers.
• Ground-to-Cloud Bridging: cloudSwXtch offers ground-to-cloud bridging, allowing seamless communication and data exchange between on-premises environments and cloud resources. This capability facilitates a smooth transition to the cloud for mission-critical workloads.
• Protocol Translation and Fanout: The platform supports protocol translation and fanout, ensuring that data can be efficiently transformed and distributed across cloud networks. This is crucial for handling diverse data formats and ensuring compatibility.
• Network Path Redundancy: cloudSwXtch supports network path redundancy, ensuring that data remains accessible even in the event of network disruptions or failures. Redundancy is critical for maintaining the reliability and availability of cloud services.
• Integration with Major Public Clouds: cloudSwXtch integrates with several major public cloud service providers, making it a versatile solution for cloud networking. This ensures that companies can leverage the cloud provider of their choice while benefiting from swXtch's advanced capabilities.
• Enhanced Visibility and Control: The platform offers enhanced visibility into different network paths and endpoints, enabling cloud providers to monitor and manage their networks more effectively. This increased visibility is crucial for troubleshooting and maintaining network performance.

The support cloudSwXtch has for cloud providers is focused on enabling the migration of mission-critical workloads to the cloud while providing essential capabilities like multicast, bridging, redundancy, and enhanced control. This empowers cloud providers to offer more robust and flexible solutions to their customers while maintaining high standards of performance and reliability in the cloud environment.

How cloudSwXtch Works to Enable Multicasting in the Cloud

To recap, multicast is both a format for creating packets as well as the theory of fanning out or moving a packet to a place and then replicating it as many times as necessary to the end consumer. cloudSwXtch, however, breaks apart these two ideas to provide solutions for the many customers who use protocols like SRT or unicast, which are point-to-point protocols and cannot typically be fanned out.  

Let’s say you have a multicast group, or multicast stream of data, and a subscriber to that group that can't support multicast. For example, a CDN, or content delivery network, that takes video data and gets it to several consumers at the edge of the network. If you want to push a multicast data stream to your CDN partner, there will be no way to do it. cloudSwXtch, however, can still fan out a multicast stream to the packet protocols that can’t support it, making it so the network understands this notion of fanning out as just a function, and a packet protocol as just a protocol-based way of packaging, keeping them separate.  

We call this our Protocol Fanout feature. cloudSwXtch does this by taking packets that aren't multicast, like SRT packets or unicast UDP packets, and fanning them out as if they were multicast. This way, users will get both the benefit of better network efficiency as well as the simultaneity of arrival that comes with using multicast natively. cloudSwXtch implements an IGMP-like control syntax to implement the leaves and joins for non-multicast packets.  

cloudSwXtch will also use this feature to support other protocols in the future like TCP, NDI, and QUIC, acting as a sort of Swiss Army Knife in its ability to take many different network protocols in and fan them out to subscribers that want them with the same performance and distribution benefits that multicast has.

This allows swXtch.io to service different vendors across sectors who might have different protocols they support. Once engineers can take in different protocols and fan them out without regard to what systems can accommodate them, they become free to focus instead on building best of breed solutions.

Multicast technology, a linchpin of high-performance networks across industries, faces challenges during migration to the cloud. In response, cloudSwXtch overlay network removes barriers, offering authentic multicast capabilities. The separation of multicast's packet creation and replication theory allows non-multicast protocols to leverage its efficiency. This flexibility extends to cloud monitoring, real-time issue detection, and reduced egress costs. Financial services, reliant on true multicast for high-performance systems, can now transition to the cloud seamlessly. Similarly, media and broadcasting benefit from genuine multicast support, essential for complying with standards like SMPTE 2110. cloudSwXtch empowers engineers to focus on innovation while simplifying complex configurations. It's a call to embrace multicast's power in the cloud era, breaking free from unicast constraints and unlocking a new realm of possibilities. Explore cloudSwXtch today and revolutionize your cloud journey!