Understanding the basics of bare-metal switches
A comprehensive collection of articles, videos and more, hand-picked by our editors
Startup Cumulus Networks came out of stealth mode Wednesday with a Linux-based network operating system that is...
the yin to Facebook's Open Compute switch project's yang.
Cumulus Linux is a version of the open source operating system built to run bare-metal switch hardware, including the open-specification switches that the Open Compute project is planning. Facebook announced Cumulus as a vendor partner in the project several weeks ago, although Cumulus was still in stealth mode at the time.
The software startup wants to replicate for the networking industry what Linux did for the server industry when it helped break up the stranglehold that vertically integrated computing companies like Sun Microsystems had before x86 –based servers became mainstream, according to J.R. Rivers, founder and CEO of Cumulus.
Like Facebook, Rivers believes Cumulus can change the operational and economic models of networking by separating hardware from software. And he believes that Linux is the operating system that can enable that revolution.
"Wherever humanly possible, we use a Linux kernel construct to hold state and maintain configuration," Rivers said. "And we focus on hardware accelerating that Linux kernel construct. We have a device driver that pushes it into the silicon."
Cumulus has hitched itself to Linux as the basis of its software for three reasons:
- Linux is mature. "Linux has a very full-featured networking stack," Rivers said. "Someone can go and reinvent the wheel, but you create an opportunity for bugs, as opposed to leveraging all the debugging and evolution that has already occurred."
- Most networking vendors already use Linux for early implementation of their technology before building it into their proprietary platforms. Rivers said Cumulus can move faster with Linux as its primary platform.
- There is a huge variety of vendor-agnostic tools for managing and operating Linux platforms already available. "The toolset available to sysadmins for deployment, management, monitoring and orchestration is substantially more advanced than the toolsets that are available to network admins," Rivers said. "Network admins get tool chains [from] specific vendors that are designed to lock you into their hardware. It's all about, 'How do I force people to buy our hardware?'"
To some extent, Linux is already a mainstream network operating system, said Eric Hanselman, research director for New York-based 451 Research.
"There is already a whole set of implementations that are headed down that route," he said. "If you look at all the network appliances in the market today, most of them are running some form of Linux under the hood."
How the Cumulus Networks model works
Customers that embrace Cumulus's model will find that the deployment and configuration of switches and servers will converge.
"Cumulus is identifying that network devices should run the same way that servers do, with the same kind of tools and the same operational ideas of how you deploy and manage them," Hanselman said. "You can take a piece of bare-metal hardware and plug it into the network, load up the software --PXE [Preboot Execution Environment] booting it just like servers. It turns out the majority of OpenFlow switches out there PXE boot. It treats [servers and switches] as the same automatable objects and uses the same kinds of tools you would use to deal with any physical instance in that environment, rather than have network gear with its own management and deployment environment."
The Cumulus approach will be particularly attractive to companies whose data centers are constantly expanding. "If you [have] a hyper-scale data center with lots of new infrastructure, this has an advantage because as you push out a rack full of gear, you can use the same management [tools] to push out compute and switches," Hanselman said. "If your network infrastructure is going to be, then this is going to have less impact."
Network engineers using a Cumulus-based environment will find that some of the more mundane tasks of their jobs fall to the system administrators so that they can focus on more advanced networking tasks.
"The racking-and-stacking piece of a network administrator's job starts to be interchangeable with the kinds of tasks that a server admin would perform. [But Cumulus] is still using BGP," Hanselman said. "Their default stack is Quagga. There is real networking stuff that has to be done that your average server administrator is not going to be able to wrangle, like BGP path metrics."
Cumulus' approach to switching will enable some of the programmability SDN promises without the need for a central controller, said Andre Kindness, senior analyst at Cambridge, Mass.-based Forrester Research. With an open, Linux-based platform, network engineers will be able to directly program networks with existing management platforms.
"The OpenFlow method is going to take a while," Kindness said. "I'm surprised we haven't seen something like this already. I always wondered why you had to have a controller in the middle. Why not just program straight to the switches through Linux?"
Supply chain plays key role in Cumulus Networks' strategy
Cumulus is building a supply chain of original design manufacturers (ODMs), distributors, value-added resellers (VARs) and system integrators to build and deliver bare-metal switches with Cumulus' software on-board.
Early Cumulus partners include ODMs Accton and Quanta, merchant silicon vendor Broadcom, cloud-focused system integrators Cloudscaling and Metacloud, OpenStack software developer Piston Cloud, Linux management vendor CFEngine, and software-defined networking (SDN) startup PlumGrid.
The company is focusing on recruiting supply chain partners and equipping channel partners with the expertise needed to provide direct hardware and software support to customers. To enable this, Cumulus has hired Jason Martin, VMware's former vice president of worldwide professional services, to run its customer support organization. "He's the one who started and ran the same organization for VMware for eight years," Rivers said.
Cumulus is targeting a very narrow but high-volume market with its networking model: hyper-scale data centers, Web 2.0 companies and very large enterprises, particularly those in financial services, experiencing network constraints in their data centers. The company will likely find that the rest of the market isn't quite ready for its buffet-style approach to switch hardware and software, according to analysts.
"It's hard to think you're going to have enterprise data center [teams], the way they are currently set up, buy a Linux-based operating system for network switches and then go to an ODM for hardware," said Brad Casemore, research director for Framingham, Mass.-based IDC. "I don't think they are at all ready for that. I think Cumulus is looking at the usual suspects -- companies that look a lot like Google and that are on the board of the Open Networking Foundation."
Casemore said Cumulus will appeal to companies that have highly virtualized compute environments and are embracing cloud computing and cloud bursting because these companies are looking for ways to automate their network.
"That's a voyage that the average enterprise is just starting, and some haven't even begun it," he said.
Although Cumulus is establishing a sales channel, the average IT department will remain uncomfortable with networking gear from ODMs. For a company like Cumulus to really take off in the general enterprise market, networking original equipment manufacturers (OEMs) will have to get involved, Casemore said. Many OEMs might eventually move in that direction, however, given that application-specific integrated circuits (ASICs) are going away in the top-of-rack switching market.
One data center-focused OEM, Mellanox, has already cracked open its vertically integrated switching platform to allow open source network operating systems to run on it.
"I think a lot of OEMs will [try this]," Casemore said. "Right now, they see no need to do it. They will say they have a lot of business from existing customers and it's a good business. Why erode it? But I think you can see a lot of networking vendors are changing. Cisco is morphing into more of an IT company, bringing a lot of value from the compute and management and orchestration side."
The sales channel that Cumulus has established will charge customers around $6,000 for a top-of-rack, bare-metal switch with 48x10 Gigabit Ethernet (Gbe) and 4x40 GbE ports, according to Rivers. The annual software subscription for the Cumulus Linux software and overall support costs will come to about $1,500. He compared this to the $48,000 list price of an equivalent Cisco Nexus 3064 with a Smartnet contract, which typically costs 8% to 12% of list price.
"For the cost of Cisco support, you can go off and buy new hardware and change over to our model," he said.
Let us know what you think about the story; email: Shamus McGillicuddy, news director.