Network Equipment: Transceivers

April 17, 2019 yman

So what’s becoming a new hot topic in the industry nowadays? SD-WAN? IBN? Maybe service meshes? Maybe… but how about not to confuse OSPF with OSFP? Try to say OSFP three times out loud! I even can’t write this on first try.
So, it was quite a while from my previous post and that time I choose to write not usual type of post with configs and code, but to deal with some inventory stuff. Today we gonna look at different types of network transceivers. This is probably not an interesting thing to read, but a handy thing to keep bookmarked.

NOTE: not all pictures here looks neat and some of them may break some copyright or something.

Why we need them

So why we need transceivers in the first place? Why not to plug optical cables into switch directly? And you can, or could in the past. Have a look at Catalyst WS-C3550-24-FX there [1]! That question can grow in your head even stronger, if looking into invoice for new DC rack, you find that set of transceivers can cost more that switch itself! The thing is they provide flexibility. Wanna change multi-mode fiber to single-mode? Just change transceiver, not the switch. In some cases you can change link speed by changing transceiver and not the switch (multi rate ports). You can swap 500 meters connection to 20 km one. So despite they high price, most of the time people find it’s easier to change some or all transceivers and not switches on both sides.
And keep in mind we talking about hot-pluggable transceivers. Because there are also not pluggable ones, SFF for example [2].

Too many of them

In previous years there were many different transceivers. Most of them failed competition to SFP family, but you still can stump on many of them scattered all over campuses and data centers here and there. 100G era brings more new monsters on a market, different in size, form and mounting. And now with 200G/400G products coming, there are new competing standards!
I thought it would be nice to have a little handy guide on transceivers to which anyone can refer when lost in all that mess. So here it is. But keep in mind it is by no means full list of all transceivers ever existed. Especially, it is focused on Ethernet ones (transceivers exclusively for InfiniBand existed, for example). All transceivers will be divided by maximum data rate provided.

Inventory

Gigabit Ethernet era

GBIC

GBIC or GigaBit Interface Converter is one of the first pluggable transceivers ever exist with first specification dates back to 1995! Today it has version 5.5 of year 2000 in document named SFF-8053. This transceivers are quite big and GBIC-T variant (with 8P8C connector for twisted pair cables) is even bigger. They are outdated, but you can still stumble upon some of this in some Cisco Catalyst 3550 sitting in a dark dusty corner of someone wired closet. You even can still buy them if you need!

SFP

Small Form-factor Pluggable transceivers probably most ubiquitous on the market. They support different types of optics (and of course twisted pair cables), connectors and Ethernet standards. And while DCs looking forward for 200/400G equipment, this little things filling every campus and enterprise all over the world. They can even provide 100 Mb/s connections, if you need it. And with size as small as gas lighter they are pretty handy!
Specification laid out in INF-8074 paper, version 1.0 of 2001. You can find that type of transceivers fit in so many devices (especially in ‘enterprise’ section of any vendor site).

Rise of ten gigs

XENPAK

Monstrous transceivers with thumbscrews on front side! They look high-tek, but can be rarely found as today. It was first transceivers aimed at emerging 10G optical standard. Specification covered by INF-8474, with version 3.0 came out in 2002.

XPAK

XENPAK was not alone in 10G realm conquest. XPAK was a competing standard that came out a little bit later. It has two different options: “low profile” (a little smaller than XENPAK) and “tall profile” (higher version). But hey! Neither one has screws! As XENPAK this things are pretty rare today. Standard document is INF-8475 with current version 2.2 from 2002.

Low profile XPAK module

How many X-something will be there, huh? Now we get to something not so rare. Not so rare, not common, mind you! X2 transceivers looks like a little upgrade from previous PAKs. And you can still found them in Catalyst 4500 switches out there (but both switches and transceivers are legacy now). They covered in INF-8476, latest version 2.0b of 2005.

XFP

This thigs finally looks like handy pluggable transceivers. They are like fat brothers of SFP. Probably that give them some recognition, so you can find them in actual products on the market! Juniper MX80 routers use them, for example. They covered in INF-8077, current version is 4.5 of 2005.

SFP+

And now to something easy to find. SFP+ is an evolution of SFP transceivers, which allowed to bring 10G in the same housing. So you can easily mistake them for usual SFP modules. Same body as SFP allowed that transceivers to become really popular, just as they little brothers. Even late appearance on market (compared to all that X transceivers) didn’t prevent them from conquering it. Standard paper is SFF-8431, current version is 4.1 of 2009.

Intermission 1: On module standards and governing bodies

All transceivers described here isn’t just piece of metal of course. It is complex mechanism with different pins, housing, optical and electrical interfaces and software inside. Most of them covered by set of papers by different standard bodies. Here I’m focused on hardware specifically, so I try to provide either paper defined hardware (module housing) or “main” paper that define such a transceivers (QSFP family for example has common hardware document but distinct definition papers).
Most of the standards born under MSA – a multi-source agreement, which formed by different hardware vendors. Vendors put they money and power into development of this type of transceivers to later use them. Some transceivers (such as CFP or OSFP) still developed inside MSAs. Most of current and older papers ends up in SNIA (Storage Networking Industry Association), so where you see INF or SFF nomenclature you can go on SNIA site [3] and find this papers there.

Twenty five gigs to the server

As data centers crawl into 100G for uplinks there are also rising need to push more bandwidth into servers. Different data consuming applications emerge or evolve from previous generations and 10G is not enough for them. So here comes…

SFP28 (SFP+ 28)

Once again an update to SFP transceivers, which now allowed 25G data rates, again in a same body. Came on the market somewhere around first half of 2010s, they are not really new thing. But while hyper-scalers using them in DCs for a while, smaller players just start to adopt them. So this piece of hardware is probably will be a big thing in connecting servers.
They covered in SFF-8402 with current version 1.1 of 2014.

Intermission 2: On PHY technologies

Up until now we dealt with transceivers, which put one lane (data channel) into single fiber to reach proposed data rates. For example, SFP module with 1000BASE-SX PHY use dual LC patch-cord to get 1 Gig tx and 1 Gig rx. Same thing with SFP+ module and 10GBASE-SR. Of course there are some exceptions like 1000BASE-BX10 and 10GBASE-LX4, but they either legacy or not so common, or both.
To get more bandwidth and move into 100G era (40G often considered part of 100G evolution) transceivers must either use more than 2 fibers or employ some –tricks– additional technologies.
First method is available by using MPO fiber connectors, which can house 8 to 24 fibers to bring them to transceiver or patch-panel, so you can get many fibers in one cable. 40GBASE-SR, 100GBASE-SR4, 200GBASE-SR4 and 100GBASE-PSM4 doing exactly that.
Second method based on WDM (wavelength division multiplexing), which allow to modulate different lanes with different wavelength and MUX them into single cable. This allowed to use dual LC patch-cords to get to 100G and beyond. 40GBASE-LR4, 100GBASE-LR4, 100GBASE-CWDM4 and 200GBASE-LR4, for example, doing such a thing.
Of course as current standards allows to push up to 50G (and 100G is around the corner) into one lane we will meat “basic” transceivers further.
In reality, there are much more standards that formulated by IEEE or vendors MSAs. So much more that to acquaint with all of them one must spent quite some time reading through many boring papers. Keep in mind that apart of different PHY technologies, which defines how to put some data rate into cable there are things like encoding (NRZ, PAM4, etc..), error-correction and others. But to get a nice overview of that things I highly recommend you to visit Wikipedia pages on Ethernet standards. They are awesome! Here’s one on 100G [4].
And if you want to dive deep into fiber strands, connectors, CWDM, OADM and all that optics stuff here is an amazing presentation on NANOG by Richard Steenbergen [5].

More uplink with forty gigs

To get beyond 10G and give switch interconnects more bandwidth without LAG, forward came 40G and 100G transceivers. At first both were just aggregates of 10G in form of 4×10 or 10×10. But as optics evolve to push 25G and 50G into single fiber 40G transceivers began fading. Users that used 40G in the past already evolved to a next step and needs more. Users which just now start thinking about moving from 10G up, can find 100G more interesting.

QSFP+ (QSFP10)

QSFP+ combines four lanes of 10G to produce 40G data rates. This transceivers still in use in many current data centers as switch uplinks, so you can find them just everywhere. But they start to give way to 100G. They are slightly bigger than SFPs and about the same size as XFP.
They covered in SFF-8635, which has version 0.6 from 2015.

Fifty gigs

This things still pretty rare. The reason I believe is exactly in how much bandwidth they provide. While servers conquering 25G, switches already use 100G for interconnection. So it will probably takes sometime for this modules to become popular. And of course you can get 50G out of 100G QSFP28 modules directly or by splitting one 100G to two 50G.

SFP56

This SFP size transceivers use one lane of 50G. I can’t find paper that covers such a transceivers, but they definitely exist [6]. Probably early products already available, while specification finalized.

Intermission 3: On 28s and 56s

While you reading this, question may arise. Why SFP28 and SFP56? The thing is that real data rates isn’t nicely rounded. Another thing is that same modules can be used in another applications like Infinband or Fibre Channel, that can have other bandwidth requirements. So 1G SFP providing 1.25G rate in reality, and SFP28 can do… 28G! But Ethernet switches uses them according to Ethernet standards and rates.

Hundred gigs

100G came out in 3 phases and there are 4th phase coming. It started with 10x10G transceivers, which was huge and expensive. Then forth came 4x25G, which are most common now. Third phase rolled out modules with 2x50G lanes. And latest developments producing 1x100G transceivers.

CFP

Huge modules that somewhat reminds of XENPAK. This thing allowed for 100G operation in late 2000s, but became legacy in presence of more compact transceivers. At first they allowed 10x10G operations and now support 4x25G. They specification is CFP-HW-Spec by CFP MSA, latest version of which is 1.4 of 2010. By the way, C here is for Latin numeral C, which means hundred, just like X used in all that 10G transceivers.

QSFP28 (QSFP+ 28)

Another form of SFP (and QSFP) evolution. It has same body as QSFP10, but allowed 4x25G and 2x50G operation. This is most common 100G modules in the wild now. Standard is SFF-8665, version 1.9 of 2015.

CFP4

This is a smaller brother of CFP module, which provide competition to QSFP28. It is covered by CFP4-HW-Spec by CFP MSA, latest version 1.1 of 2015. This transceivers use 4x25G and also allow 4x10G operation. They are about same size as QSFP.

SFP-DD

SFP-DD is one of the latest development in a field of 100G. This modules backward-compatible with SFPs, but allow 100G rates with 2x50G lanes. Backward-compatible means that you can insert SFP28 into SFP-DD port and it will work, but electrical pin configuration is different for them. Standard emerges in 2017 and it looks like there are no transceivers available to buy as of now. They developed by SFP-DD MSA and current specification has version 2.1 of 2018. And by the way, DD stands for double density.

SFP112

Like SFP56 I can’t find any standard papers. There’s little mention of this modules on the ‘net and no products on the market. But name pop-ups here and there, probably because of 400G support of hundred gigs over single lane. So this transceivers aimed at 1x100G operation.

Intermission 4: On splitting

While I’m not diving into cables and connectors in this post, one thing must be mentioned here. Transceivers which use multiple lanes can provide split or breakout possibility. Which means you can connect four 10G transceivers to one 40G, for example, with a special breakout cable [7]. Some hardware really treats this modules as distinct connections (even in non-breakout mode), which they really are. Some switches will even allow you to do opposite thing and aggregate four 10G ports as one 40G! With such thing you can “transform” your 24x100G switch into 96x25G with a simple command!

Two hundred gigs

While 400G QSFP-DD still seek it’s away to the market, QSFP56 is probably fastest transceivers of QSFP breed you can find. But keep in mind you can use CFP2 there too (which will pop ups a little later in 400G section).

QSFP56

Another QSFP compatible transceivers bringing in 4x50G. Standard is nowhere to be found, but modules already introduced by major vendors [8].

Four hundred gigs

CFP2

Bigger than CFP4, but smaller than original CFP, this modules provide different combination of rates and lanes such as 10x10G, 4x25G, 8x25G and ultimately 8x50G. Standard is CFP2-HW-Spec by CFP MSA with latest version 1.0 of 2013. This is probably the first modules, which allowed 200G and later step up into 400G.

CFP8

Slightly smaller version of CFP2, which aimed exactly on 400G with 16x25G or 8x50G variants. This is probably first (and possibly the only as of now) modules which bring 400G on the table. They covered in CFP8-HW-Spec by CFP MSA, latest version is 1.0 of 2017.

QSFP-DD

Latest and greatest of the QSFP breed, this modules provide 8x50G lanes. Developed by QSFP-DD MSA it has version 4.0 of 2018. Most prominent member of MSA probably Cisco. This modules provides backward compatibility with QSFPs in a same sense as SFP-DD with SFPs.

OSFP

‘O’ here stands for octal, as like previous ones this modules provide 8x50G. 400G fight will probably will be around OSFP vs QSFP-DD. And not surprisingly most prominent member of OSFP MSA is Arista. Current specification is 1.12 of 2017. This modules is not compatible with QSFP, but can be used with adapter, which probably will be a disadvantage for them. And they somewhat larger.

A little more pictures for size comparison

CFP modules

QSFP-DD and QSFP28

QSFP-DD and QSFP modules size comparison

400G modules

Conclusion

As you can see there are many different modules already in the wild, and more of them coming. It’s easy to confuse one for another even if you deal with them frequently. But if you sit in some NOC room and never goes in DC things can be even more confusing. I hope you find that list helpful and want two mention some more interesting links. In this [9] video you can see different transceivers comparison on the table. Here [10] Cisco engineer talks about transceiver evolution and 400G especially, quite interesting. And finally in this presentation [11] Juniper engineer goes deep into different optics related things and transceiver form-factors.

DVJourney

endless quest for knowledge