Facebook building a 100-gigabit switch to keep up with video, VR traffic

19.11.2015
Facebook is more than doubling the speed of its Wedge open-source network switch, which is good news both for Facebook users and for anyone who may want to build a 100-gigabit switch.

The Wedge sits at the top of a rack of servers to connect them to Facebook's network. It was announced in June 2014, and thousands are already deployed in the company. Plus, the social network made the design of the Wedge open source so other manufacturers could build switches like it.

However, less than two years later, the 40-Gigabit Ethernet ports in the original Wedge are proving no match for the fast-growing traffic on Facebook's network. To keep up, the company is now developing a version of the switch with 100-Gigabit Ethernet interfaces. It gave more details in a blog post but hasn't said when the new switches will be ready.

Much of the traffic in Facebook's data centers is just among the company's own switches. But it's growing along with much more bandwidth-intensive content on the social network, including big video files and virtual-reality content like 360-degree videos that Facebook recently introduced.

"Whenever there is capacity, people will build stuff to consume it," said Jay Parikh, vice president of global engineering and infrastructure, at the Structure conference in San Francisco on Thursday. Facebook says it has more powerful servers, and more of them, than it did when the first Wedge was introduced. As many as four servers can be hooked up to each port of the Wedge.

The Wedge 100 will have 32 100G ports, the same maximum number as on today's Wedge but all at the higher speed. Like the current switch, it has a non-blocking design, meaning that it can fully feed all of its ports at the same time if necessary. Thus the total capacity of the Wedge 100 is 3.2Tbps.

Facebook designs its own switches and writes its own software for them because it wants the flexibility to meet its own rapidly developing needs. In addition to the Wedge, earlier this year it introduced the 6-pack, a modular switch for connecting server racks to each other. The 6-pack is based on the Wedge and can have as many as 128 40-gigabit ports. Facebook has also developed its own data-center network design.

What Facebook invents in network hardware, it makes available for others to use. At least one vendor, Accton, already sells a customizable top-of-rack switch based on the 40-gigabit Wedge. The design has been accepted by the Open Compute Project, which Facebook founded in 2011 to share open-source hardware designs. It also plans to share the Wedge 100's design.

The company uses an internally developed OS, called FBOSS, on its own switches. But it has also joined up with Big Switch Networks to offer an open-source network operating system called Open Network Linux for others to use.

The advent of so-called "white box" networking gear based on open hardware and software could trickle down from giant IT shops like Facebook to provide enterprises with an alternative to traditional systems from companies like Cisco Systems. But Cisco says its overall architecture costs customers less in the long run.

Stephen Lawson