Understanding the basics of bare-metal switches
A comprehensive collection of articles, videos and more, hand-picked by our editors
Facebook's Open Compute Project (OCP) wants to make your network hardware as simple and as inexpensive as possible. Toss aside the expensive application-specific integrated circuits for merchant silicon in an open source switch that is flexible enough to use any OS that fits the problem you're trying to solve.
The closer you tie applications to the network, the harder it becomes to get things done. When two big pieces have to move in sync, that's hard to solve.
director of technical operations, Facebook
The OCP began developing the concept of OS-agnostic network hardware in May, when it started working on a spec for an open source top-of-rack (ToR) switch. Since then, the organization has made headway on at least one set of ToR specs, said Najam Ahmad, Facebook director of technical operations, who spoke on an SDN keynote panel at Interop New York last week.
And the OCP won't stop there. Once this model of the "desegregation" of network hardware and software takes shape, the OCP will develop specs for switches at every layer of the network.
What does all this mean for SDN and network programmability?
With an open source, OS-agnostic switch, you can use a proprietary OS (think Cisco IOS) or choose a Linux implementation for super-programmability. That programmability means you won't be locked into making only the changes that your command-line interface (CLI) allows. You can make changes on the fly and program for network automation.
"CLI is dead," Ahmad said during his Interop session to a room packed full of engineers who use the interface every day. At Facebook, when a network monitoring tool reports a failure, it can be fixed in code overnight, he explained.
SearchSDN sat with Ahmad at Interop New York last week to get an update on the OCP's work in networking.
What is the status of the OCP's top-of-rack open source switch?
Najam Ahmad: Since May, we've had three engineering workshops where a bunch of people have participated from all aspects of the ecosystem -- from semiconductor people like Broadcom, to systems manufacturers, to people who build boxes, software development shops and companies building SDN stuff. Plus there are a bunch of users. We have had a lot of interest from financial industry, so we did [the last] workshop at Goldman Sachs. We've gone through building a charter on what we want to build, and then we are working on a specification.
There are several companies that have proposed switch-design contributions to the project, so we are going through those contributions and validating them against what users want. We are actually quite further along in at least one specification where there is already a prototype, which we showed at the last workshop. We actually passed bits through it.
How are the requirements these users are seeking different from what they can find in a proprietary black box?
Ahmad: If you look at the hardware piece, it's not that different. But what everybody is after is that desegregation. If you look at organizations like Goldman Sachs or Fidelity, they have huge Linux deployments [in their compute]. None of those developments can be leveraged across the network. The same thing goes for us [Facebook]. We are trying to leverage our development environment to work across the infrastructure.
Does that mean they don't want to use a proprietary OS anymore?
Ahmad: Particular vendor proprietary operating systems only expose things through a CLI, and at the scale and the speed that CLIs just don't scale anymore. The proprietary switch doesn't do certain things today in software that we can do if we go write our own software.
But proprietary switches do take care of some very specific user needs without requiring extensive time programming. Are there some enterprise shops that won't be interested in this kind of openness, and therefore work?
Ahmad: I am sure there are tons of them out there, which I think is an opportunity for vendors to build ecosystems around those needs. In the end, you still have the much more agile environment and platform you can build on. We are already seeing that in other parts of the OCP where there is an ecosystem of companies like Hive that are building OCP-based servers and providing that to [users] who can directly participate and leverage this design [even if] they don't have the resources. There is the same opportunity on the network side, where traditional vendors can take these ideas and concepts and package them in a way for other enterprises who don't have the skill sets or resources.
Will there be a process of looking at software specifications for these switches?
Ahmad: We are discussing that. We haven't made any calls. The idea is to have as many options as possible in that open environment. What we probably will end up doing is testing certain implementations and saying we have tested them on this box.
Are vendors like Cisco and HP Networking looking to test their operating systems on the box?
Ahmad: We are having conversations with them.
Since Open Compute was first announced, we've seen interesting alternatives in network switches emerge with the option to run Linux. Do we still need an OCP switch, considering?
Ahmad: Switches come and go. I am more after the concept and a model. You have this phone, and you will have another two years from now. But if you bought this phone and it didn't allow you to add apps, you wouldn't want it. That's why BlackBerry is dying. The concept is that the next network box I get will still run any app.
Will you use this model to develop other switches too?
Ahmad: We are looking to build a larger spine switch next. So the switch itself is not important; it's the model that's important. As the project progresses and we solve the model, we'll apply the same model in other switches as well.
If you are completely readdressing the way you build a switch, why are we talking top-of-rack, aggregation, core? Why not just toss it all out and rethink the way we architect networks completely?
Ahmad: If you start talking about the topologies and architecture, you will get into a rat hole you won't be able to come out of. Prescribing an architecture is a far bigger problem, and I think it's the wrong way to solve the problem. There is no one architecture that solves everybody's problem. The idea is to find a component that can actually fit into multiple architectures.
If you want a traditional BGP [Border Gateway Protocol] or OSPF [Open Shortest Path First]-based network, you can run a traditional operating system and you could build a traditional architecture. If you wanted to switch to OpenFlow, you could reboot it and bring it up as OpenFlow. We wanted to build a component that could fit multiple architectures. The reason we picked a top-of-rack is because there's no dispute about what that box is. [That's important] since we are after the model and not the switch. It's the easiest thing to get going.
Throughout Interop, we heard about application-driven or application-aware networking, but you said something different -- that there is no need for the application to know all about the network and vice versa. What do you mean?
Ahmad: My personal opinion is that that's the wrong way to solve the problem. The closer you tie applications to the network, the harder it becomes to get things done. When two big pieces have to move in sync, that's hard to solve. What you need to do is find the right abstraction layer so each layer can move independently but still be connected.
More on open source networking
Cumulus: Linux OS for bare-metal switches
Juniper's Contrail controller goes open source
OpenDaylight controller emerges, ready for contributions
Big Switch unveils open source OpenFlow switch software
What do you mean by abstraction layer?
Ahmad: The way applications copy large amounts of data today is to copy some code. The network is not aware; it just sees a flood of bits coming at it. Then it can either handle it, or drop it, or cause all sorts of chaos. If the developer was smart enough to figure out when the network had less traffic or the best time to [move] it, or things of that nature, then they can put process around those [occurrences].
In the abstraction we built (in Facebook's Bot Traffic Manager application), there is a scheduling mechanism to support the application where you can say, 'I want to move 4 petabytes of data from this point to this point and I need it done by 4 p.m. today.' The network says, 'Oh cool, we'll find a spot, and we will fit it in on any pipe that we have capacity and get it done by 4 p.m. If we have to shape it, if we have to control it, we can handle that.' We don't need to know anything about the app other than the app needs this done. That's an abstraction layer.
But in SDN environments, vendors are adding policy to that same scenario to decide which apps to optimize, for example. In that case, wouldn't you want the network to know more about the application and the application to know more about the network?
Ahmad: You can still do that. When an app comes to the scheduler and says, 'I need this moved by 4 p.m. today,' the network can say, 'Sorry, it can't be done' or 'We need more time, come back later tonight.' It also depends on where you want that control to be. If you're saying that the application should go over some part of the network, why do you even have to tell the application? The network can do it on its own. Why do we have to teach the application what's happening on the network? Networks are complex topologies. If you have to teach the application about that topology, it's a losing proposition.
I know you were involved in discussion with both the ONF and OpenDaylight. Will work from both organizations be incorporated or used as reference in the switch specs?
Ahmad: Both the ONF and OpenDaylight are OCP members. The ONF was one of the organizations that instigated getting this project started. The ONF is defining technology, but they are not building boxes. So they came to OCP and said, 'Can you build a box that can run OpenFlow?' They are very much part of this conversation. And the same thing with OpenDaylight.