Stable, predictive, cheaper, more efficient with network automation
NTT Com’s Michael Wheeler has seen two decades of improvements through network automation – since long before anyone heard of SDN or NFV. He tells Alan Burkitt-Gray of the benefits
Automation has been central to NTT Communications’ strategy since long before it was part of NTT. “It’s one of the core secret sauces that’s behind much of our business success,” says Michael Wheeler.
The automation story started, he recalls, in the Verio days back in the late 1990s, when its engineers wanted to reduce cost and create efficiencies, and they didn’t want to hang around for all the manual tests that then had to be done.
NTT, the Japanese telecoms giant, completed its purchase of Verio in August 2000, for a price of $5.5 billion. Wheeler joined Verio in 1997 and has been with the group as it transitioned to NTT Communications ever since, apart from a two-year interval in 2000-02.
Now, for the past nine years and 21 years after joining Verio, Wheeler is officially global IP network EVP at NTT Communications – still, fundamentally, Verio as it was. And many of the people who pushed ahead with the company’s automation project back then are still there, he says.
“It was 1997 when we first did it, thanks to the global engineering team,” he says. “They are still part of the development team. They were more than software engineers: they were part of running the network. That was the original driver.”
And why? Then, as now, “customers get a more stable, more predictive environment”, he says. They wanted “standard programmatic management of the network”, an automatic way to configure the network. This was, he points out, long before anyone had thought of, or even dreamed up the terms for, software- defined networks (SDNs) or network functions virtualisation (NFV). But the arguments applied now were the arguments that the Verio engineers applied back in 1997. “From an operational perspective automation is a critical component of how to run a network effectively,” he says.
And there was a little impatience in their rationale. Engineers need to make frequent changes to the networks, and when they do that they spend a lot of time checking and double-checking. Engineers were “tired of waiting”, says Wheeler, but really they wanted to drive up efficiency and reduce time. “They were asking: ‘How can we do this differently?’,” he recalls.
But Verio was ahead of its time. “There were no standards then,” says Wheeler. “We weren’t pigeon-holed. We wanted to reduce cost and create efficiencies.”
And that work has continued from the Verio days right through to NTT Communications today, he says. “We’ve taken the mindset from then and we have learned some things since then,” he says. “It’s a cultural thing as much as an operational thing. There are different tools available now.”
And NTT Com has been able to build on this work by providing tools for customers for use in the configuration and management of their own network. It’s not self-provisioning, he says, but it’s a vital part of the service. “When you have a business that’s on our scale, it’s definitely continued to be a big part of our offer,” says Wheeler.
Outside NTT Com, which provides IP network services, other parts of the NTT group have “similar mindsets”, he says. Software-defined wide area network (SD-WAN) specialist Virtela has automation of systems in the private network business, he notes. NTT announced its acquisition of Virtela in 2013 for a reported $525 million.
And NTT Com’s approach has helped to focus the minds of equipment providers Cisco and Juniper Networks, says Wheeler. “We’ve been very transparent with our ideas,” he says. “Both of them have shown a willingness to share information. It’s been helpful for them and for their broad set of customers. It’s helped them to work out what’s important and why.”
Cisco, he says, was “very aware of how we wanted to interact with the software”, he says. “We don’t dictate our approach but we do it at a programmatic level.” Others have built their own automation technology, he says, but “the concept, the philosophy has become part of the industry. Every company ends up implementing it in different ways.”
And the advantages? He’s clear. “The benefit for the customer is the stability and reliability of the network.” This is particularly the case when changes are made to the configuration. “When you implement changes across the network, the level of error is very low, it’s dramatically reduced,” says Wheeler.
Changes are common, of course, and this is where automation is really “more impactful for customers”, he notes. “You are able to do things about customer migrations and movements. For example, when we are doing a major migration of a router platform, we might get 200 customers on a router.” With automation, “we’re able to reduce the maintenance time to a couple of hours”.
By automating the preliminary work, errors can be checked for, and the configuration can be right. “That’s the physical part of the transition. Automation means maintenance windows are shorter and more reliable. You preload all of those changes.”
How often are changes needed? “With a network our size, hardware and software changes regularly,” says Wheeler. Changes happen two or three weeks of every month. A network doesn’t stay in a static state.” Customers see a fair amount of the impact of automation, and the benefits.
It’s been an industry-wide change, but Wheeler is firmly of the belief that NTT Com has been “a positive contributor” to the industry’s transformation. “We’ve had of lot of discussions with competitors,” he says. “People have taken the idea and adapted to the reality. We’re contributing in a very positive way.”
What have been the topics of these discussions? Wheeler replies: “Our conversations have been about routing security and IPv6. We’ve been big evangelists of IPv6 for 10 years. We’ve created a positive attitude to the need for better network security.”
But he turns back to the economic and operational benefit of automation. “The other big benefit of automation is cost. Compare the number of people running our network with the other four or five leading networks globally,” he says. “We have a smaller number of people. If you take a cost and business perspective, others are larger with respect to staff numbers. For us, it means an agile, flatter organisation.”
It’s “hard to quantify”, he says, conceding that some of NTT Com’s rivals are “great, but compare the head count.” Numbers? “For our entire operational people, fewer than 130-150 globally. It’s a pretty small organisation. For the IP network we’re smaller than Level 3 or Telia,” he claims. “The benefit is that it allows us to be cost effective.”
He admits it’s hard to prove, especially as on the IP side NTT has not bought any company since the Verio deal 18 years ago. “Most of our acquisitions have been on colocation and services.” He contrasts the position with other companies – Level 3, which bought Global Crossing and then itself got bought by CenturyLink; or Zayo, where acquisition is “the whole strategy”, he says.
“In all honesty if we did an acquisition of an IP network then the automation would allow it to be a lot more efficient,” he says.
And he says other carriers accept that “what we are doing makes a lot of sense”, warning that it’s important to differentiate between NTT Com’s full automation and many others’ partial automation. “They are two different things,” he says.
Where’s it going? Automation isn’t complete yet. “There are still a number of ways for automation to continue to expand,” he says. “There are benefits we haven’t got yet. We still want to make improvements.”
In particular, he says: “We are going to continue to enhance the core control.” The company wants migration tools, tools to allow migration of points of presence and tools to allow route security.
“We are creating more and more tools. We are creating better and better capability for customers to use our network in an automated way. These toolkits will be available because of automation. It means customers won’t need an engineer to look at the network.” This is important when a network moves to new hardware. “It reduces the time needed to ensure it’s working correctly.” And that is what you want from a network.