Many of the most significant advances in artificial intelligence, machine learning, and data science (AI/ML/DS) have been spearheaded by California’s K-20 research and education community. To maintain its leadership position in an accelerating world, that community will need an ever-increasing CalREN backbone bandwidth, and CENIC has provided that. Over a decade ago, the CENIC community had a 10 Gbps CalREN backbone available to them. Today, a greatly expanded CENIC community now enjoys a 400 Gbps backbone, and that forty-fold increase in backbone bandwidth was achieved by CENIC at the merest fraction of equipment, space, and power used in 2010.
How can this be possible – and can both the backbone and end-to-end bandwidth continue to increase sustainably at the pace the CENIC member institutions need?
The answer to this question is a decisive yes, and attendees learned how from CENIC Network Architect Fady Bekhit and Lead Network Engineer Alaa Shaheen at CENIC’s biennial conference “The Right Connection” this past March in Monterey. (Bekhit and Shaheen’s slides are also available for download in PDF format.)
Accepting 400 Gbps Handoffs with New Juniper Line Cards
If the benefits of a 400 Gbps backbone are to be enjoyed from end to end, then each step along the path from one lab, classroom, device, display, or other resource to another – customer handoffs to routers, signal aggregation from routers to transponders, data signals from transponders to the CalREN backbone, and the CalREN backbone itself, as shown below – must also be individually capable of transferring data at 400 Gbps.
To this end during the 2023-24 fiscal year, CENIC has been installing Juniper LC9600 400 Gbps line cards on its Juniper MX10008 aggregation routers that accept customer handoffs and pass them along to transponders before they enter the CalREN backbone. Previously, 100 Gbps line cards were used, meaning that four ports were required to achieve a 400 Gbps throughput.
Not only has the throughput per line card improved, but the recently installed LC9600 line cards also accept coherent pluggable optics of the sort that CENIC already validated over CalREN both with and without integrated amplifiers. These cards also support conventional gray optics and are backward-compatible with 100 Gbps handoffs – a perfect future-facing and legacy-compatible solution.
The CalREN Backbone at 400 Gbps and Beyond
During the 2022-23 fiscal year, CENIC upgraded its backbone bandwidth to 400G by deploying sixteen Cisco NCS1004 transponders, with three southern backbone routes running natively at 400 Gbps and others achieving 400 Gbps by inverse-multiplexing of 200G links.
However, the 2024-25 vision is the deployment of what’s called IP-over-DWDM (IPoDWDM) on a more compact and power-efficient backbone infrastructure. This infrastructure will take advantage of the previously mentioned coherent pluggable optics to obviate transponders where possible, an upgraded line system, and low-loss fiber that can carry a data signal for longer distances without regeneration and also broadens the frequencies of light that can be used and hence the potential throughput.
ZRP Bright Coherent Pluggable Optics
The pluggables that CENIC has chosen to use are the Juniper ZRP Bright optics due to their high launch power, compatibility with legacy technology, maturity as a technology, and our experience validating their performance over CalREN. Where these new optics are employed, transponders will no longer be necessary nor will the maintenance, upgrade, rack space, and power they require.
An Upgraded Line System
Instead of the current Cisco NCS2000, the line system will be migrated to the more future-facing Cisco NCS1010 running in a router-like configuration thanks to the IOS-XR network operating system. The NCS1010 accommodates coherent pluggable optics as well as enabling expansion to fiber optic cables capable of carrying data beyond the conventional wavelengths of light. (A short discussion of fiber-optic wavelength bands used to carry data can be found in this article on the CENIC blog.) It also enables more monitoring and network automation by extending automation into the line system itself.
Innovative Ultra-Low-Loss Fiber
Lastly, new innovations in fiber-optic cables themselves have changed what is possible and necessary in terms of throughput and equipment. Lumen and Corning have partnered to create a new ultra-low-loss (ULL) fiber that carries a signal further than previous without regeneration sites with an improved signal-to-noise ratio.
An 800 Gbps Backbone with Less Power and Equipment
With these innovations in place at select node sites, an 800 Gbps backbone becomes possible and is CENIC’s vision for the coming years.
Not only will this vision ultimately double the CalREN backbone capacity, but it will also do so with a 45% reduction in total cost of ownership (TCO) and a 40% reduction in capital expenditures. Further, the power, space, and carbon footprint reductions are projected at 54%, 77%, and 55% respectively.
A proof-of-concept for IPoDWDM between the Emeryville and Sunnyvale backbone node sites has already been completed, and as always, we will keep readers up to date on how CENIC expertise is put to work for the benefit of our member community now and into the future.
Related blog posts
