Top 10 Reasons NOT to use SONiC
Over the past few years we’ve spoken to hundreds of companies about the promise of the SONiC Network Operating System. The network architects we spoke with exist along a continuum of two different extremes, let’s call them “Over my dead body” and “Show me the features”.
Let’s not combat network religion. If you have a Cisco tattoo, a heart that pumps Junos blood, or a license plate that reads EOS4LIFE, that’s great! Being a huge fan of your preferred vendor is good for your business (and theirs).
But for the engineers who have previously said “I won’t use SONiC until it has Feature X”, let’s dispel some FUD…
Ahh, the most common objection when people consider SONiC. Since MLAG/MCLAG/vPC is vendor proprietary with dependencies on certain ASICs, getting a standardized protocol for switch redundancy has proven problematic. However in the past 5 years, each ASIC vendor implemented it in their own SONiC distribution, so while the feature is distribution-specific, it is indeed available on every platform. So we can stop saying that SONiC doesn’t do switch redundancy!
However, an even more exciting feature is now available in SONiC. ESI provides a functionally compatible method for switch-server redundancy, implemented in EVPN deployments, which is basically 90% of the SONiC deployments in the world. Unlike MLAG, ESI can provide redundancy for more than 2 switches, and eliminates the requirement for an inter-switch link. More redundancy with less required cables? YES PLEASE!
No Lossless Fabric Protocols
The huge tidal wave of requests for RoCEv2 peaked with the increased interest in GenAI which started in 2023. This feature is essentially a requirement to make Ethernet support long lived traffic flows between GPUs with lossless communication. Guess what? In Verity it is one click and in some SONiC distributions it’s literally two commands and it’s available NOW.
One more thing. SONiC now offers NVMe, Dynamic Load Balancing (DLB), Adaptive Load Balancing (ALB) and a number of other whizz-bang features that help smooth out traffic flows across your trusted ECMP links. If we had these features back in 2010, we’d all probably would have more hair on our heads.
Limited Hardware Options
Platforms are now available from ALL of the top vendors, Dell, Cisco, Nvidia, Edgecore, Celestica, Arista, HP, Accton, Wistron, Ufispace, and more. The speeds range from 1G to 800G and offer all the common copper and optical transceivers. By the time you read this Starbucks may have released a SONiC-compatible Frappuccino!
Poor Programmability
SONiC supports virtually every programming method, including CLI, API, gNMI, direct DB access, and more. Obviously, most people are gravitating towards gNMI and CLI, but like most things with open source software, you can HAVE IT YOUR WAY.
Difficult Orchestration
Wow, where to begin? Due to its inherent openness and the sheer number of hardware options, there are a lot of GREAT network orchestration and automation solutions, from purpose-built platforms optimized specifically for SONiC networks, to ad hoc devops tools like Terraform, Opentofu, Ansible, Chef, Puppet and more. Depending on your level of expertise you can buy a platform that comes with 24/7 support, or you can build something on your own. Personally, we like a platform with an easy button to build an underlay/overlay and do things like ZTP and NOS upgrades with one click.
OK this is indeed a big one! ZTP has never been easy, but SONiC uses ONIE which allows you to migrate away from a legacy NOS to a more modern architecture. A SONiC device can go from the cardboard box to being part of a Clos fabric in about 15 minutes, unattended. Most of the SONiC automation platforms provide some form of ZTP support.
Some of the engineers we’ve spoken to don’t believe that the cost savings from using SONiC are worth it, as they need to develop new skills and pursue additional training. SONiC isn’t really that difficult, if you know Linux and have used FRR/quagga before you’ll be fine. You can also deploy extremely complex networks with an automation platform, ideally something that uses “intent”, so you can define high level requirements and let the software manage the actual device configs.
The truth is that when coupling SONiC with a good automation solution, you’re able to do a lot more for a lot less. SONiC hardware options provide the same ASIC performance characteristics but at about half the price. In addition, if you are using SONiC you can consider multiple hardware vendors and let them bid on your needs. This translates to lower CAPEX and shorter lead times. Don’t believe us? Check out the Network Build CAPEX Calculator or the Verity OPEX Calculator to get a good estimate of what you can expect to save by using SONiC.
This is the cherry on top! You can now get support on all the hardware platforms direct from the manufacturers, including support for firmware, SAI, and the enterprise SONiC distribution you select. Automation platforms for SONiC have been in development for almost 10 years now and are rock solid. By using a software system to manage your network, you gain an additional level of support from the software vendor. So not only are you covered on hardware and device configurations, but you can also find established companies that provide support for your own SONiC customizations, 24/7.
And in Closing…
Just like that, we’ve debunked the most common complaints about SONiC. You may have your own concerns or know of an important nerd-knob that is not yet available. We’d love to discuss your needs and how SONiC, Open Hardware, and Verity can provide a cost effective and high performance network with very little operational overhead.
SONiC has come a LONG WAY! The community has thousands of contributors, hardware platforms from every major enterprise hardware vendor, all of the popular features you need and demand, and no signs of slowing down. That’s why we believe that The Time is NOW for your SONiC network deployment.
Thanks for reading!
– BE Networks
