When a CalREN connection goes down, network engineers in CENIC’s Network Operations Center receive an alarm and immediately start troubleshooting. “Our NOC is not a typical NOC,” Manager Stanley Han said. “Most NOCs have Tier I technicians who respond to phone calls and emails and do initial triaging. But everyone in CENIC’s NOC is a network engineer, and they are the ones who answer phones calls and emails as well as log in and troubleshoot them.”
When Pepperdine University’s connection was burned by wildfires, NOC engineers didn’t waste any time contacting AT&T. The company was able to raise the bandwidth limit on one of its circuits with the university free of charge, helping to keep Pepperdine’s emergency communications up and running. “AT&T understood the importance of maintaining the Internet connection, and everyone jumped through hoops to get it done very, very quickly,” Han said. In addition to assisting Pepperdine, NOC engineers reached out to other institutions on the network which may have been affected by this fall’s deadly wildfires in California. “When [CENIC President and CEO] Louis Fox mentioned that there were some telecommunications companies offering to deploy free temporary solutions, we made it a priority to reach out to all the sites affected to see if there was anything we could do,” Han said. (Read more about CENIC’s agile response to wildfire emergencies.)
The NOC employs just 21 network engineers, answering calls and emails 24 hours a day, 7 days a week. Over three months this fall, NOC engineers triaged about 24,000 alarms. They also handled more than 3,600 service requests, including 136 tickets for new circuit deployments and 289 maintenance tickets. Less than 1% of the requests NOC receives are escalated to CENIC’s Core Engineering department.
NOC staff from left: Stanley Han, Alaa Shaheen, Eugene Ho, Mark Lozada (back), Phat Tran, Ola Gamal, Viet Do (back), Jose Gonzalez (back), Gary Sun, Travis Sevigny, and Ozzie Reyes. Not pictured: Thomas Araneta, Bryan Barnett, James Bellon, Kenny Calalang, Josh Dickman, George Dolidze, Ron Esteban, Michael Gong, Dan Hegarty, Lawrenze Morales, Steve Shibley, and Mark Swank.
Having worked at CENIC for 11 years, Han said he could go on “for days” about what he likes about working in the NOC. He started as a NOC engineer and has been promoted three times, first to associate core engineer, then lead operations engineer, and finally to his current role as NOC manager. Han said the positive work culture is what keeps him at CENIC. He looks forward to coming to the office every day because he likes his co-workers and he likes working with his team.
“New hires always comment that they’ve never worked at a place where everyone was so willing to help everyone else,” he said. “We all understand that we each have our own strengths and weaknesses, and we’re only stronger when we’re open to working with other people.” Han credits knowledge-sharing and teamwork for keeping the NOC running smoothly.
“You learn a lot more working here than you would at a lot of other companies, just because the first day you step into NOC you have access to everything, and it’s a huge network,” he said.
The Network Operations Center is available to assist CENIC members 24 hours per day, 7 days per week, including holidays. Reach the NOC at 714-220-3494 (714-220-FIXIT) and noc@cenic.org. Learn more >
Related blog posts
