When storms hit, the Weather Company needs the cloud
When major storms hit, the people at the Weather Channel want to focus on the science behind the weather and getting information out to those who need it most.
That's why the Weather Company has gone all-in with the cloud.
"For us, the days the infrastructure is the most tasked are the days you need to be focused on the business and not on the back end," said Landon Williams, vice president of infrastructure architecture and services at The Weather Company, the parent company behind The Weather Channel, weather.com and Weather Underground. "We have to make sure we perform the best when the weather is at its worst. Our whole ecosystem has to handle it."
Three and a half years ago, the Weather Company began a migration to the cloud because it needed to be able to effortlessly scale up to handle huge increases in traffic when big storms hit. Some storms caused site traffic to increase by 20 times, according to Williams.
Jagmeet Chawla, chief architect for the Weather Company, agreed that the company had to figure out a way to handle massive increases in traffic.
Generally, they deal with about 125 million unique visitors a month. However, if a big storm strikes, they could get that number in a single day. For some storms, the sites have gotten as many as 30 million visits in an hour.
Aside from the need to scale, the Weather Company also wanted to focus its technical staff on work that will add value to the business, instead of simply keeping the IT trains running.
"When you look at our size, we're 1,600 people," Williams said. "We're a big company in reach but not a big company in people. We are a weather intelligence, analytics company. We're a data science company. We're very good at that. I think we should get out of the data center business and focus on the decision making, the weather storytelling side of business. Running a Dell server really well doesn't make our business better."
Williams wanted to hand over the server-running tasks to a cloud vendor so his workers could focus on getting the apps and services that the business needs to run and stay ahead of its competitors.
Williams and his team also had to deal with the fact that after two acquisitions, the Weather Company owned 13 data centers spread across the country. Those data centers ran the gamut from top-tier managed facilities to closets with eight- or nine-year-old servers stuffed inside.
To get to the cloud, the Weather Company decided to start out using Amazon Web Services (AWS).
Today, 80% of the company's services and apps are running on the cloud, and the final 20% are expected to migrate to the cloud over the next 18 months.
And the company is saving money.
Williams noted that three years ago, it was about 10% cheaper for the company to run its own data centers. However, with an approximately 10% price cut that AWS made in April 2014, it was suddenly 5% cheaper to run on the AWS cloud. With subsequent price cuts, it's now 15% cheaper, according to Williams. It's also easier for the tech team with the cloud.
"It's night-and-day different," Williams said. "We don't scramble and run around and think about what we'll turn off if [traffic] goes too high. What was the high-water mark last time What will we do differently this time We just don't have to do that anymore. It's the difference between not sleeping and sleeping."
While Williams decided to start with AWS, he wanted to make sure the Weather Company's systems could handle multiple cloud vendors. It was an insurance plan.
"We've always architected to be able to move on to another vendor without significant changes," he explained. "What if Amazon ever had a systemwide failure What if Amazon stops innovating or they change their business model and we don't want to be there We wanted the resiliency and confidence that we could handle as many fault scenarios as we could think of."
Because of its scale, the Weather Company needed to make sure its systems never crashed, so it was worth it to work with more than one vendor.
That's why it's also started working with IBM-owned Softlayer Technologies.
Today, about 70% of the Weather Company's system is on AWS, another 10% is with Softlayer and 20% is still on the company's legacy data centers.
Some apps and services run on one platform but not the other. However, the company's most critical systems, such as its SUN (Storage Utility Network) data platform, run on both.
If Amazon had a major outage, the Weather Company's traffic would automatically shift to Softlayer. If Softlayer goes down, its work would shift to AWS.
"We don't force traffic in one direction," said Williams. "It's all automated where traffic should go based on rules and logic."
For instance, when a big storm, like Hurricane Joaquin, which battered the Carolinas earlier this month, is coming, the Weather Company might contact AWS and Softlayer and warn them that they likely will have to deal with some scaling of their systems.
"That's about it, and then we get ready to watch the performance of all the systems," Williams said.
Robert Mahowald, an analyst with IDC, said that's a solid plan when uptime is critical for a company. "In the non-cloud world, we've talked about strategic safety -- not putting all your eggs in one basket," he added. "If uptime is super important, you put a mirror image on two different providers so you're protected if something goes down."
The Weather Company had already begun its move to the cloud when Hurricane Sandy struck in October 2012.
According to Williams, without the cloud, the Weather Company would have struggled to handle the bombardment of web traffic it received.
"We would have made it through Sandy," said Williams. "We would have had to downgrade the site to more static and limited content, and disable a lot of the other site features. We would have changed our normal day-to-day communications to something smaller but able to serve that scale.
"We would have not been able to have as valuable a conversation with our users," he added. "We would not have been able to give them the insights and the valuable information they needed."
While migrating to the cloud has offered many benefits for the Weather Company, it hasn't always been easy. One obstacle was a change in the tech culture.
The company has about 400 tech workers, and all of them had to update their skills and take on new jobs.
When a company moves to the cloud, IT workers who had been storage specialists, for example, have to update their skills and move to new roles because the cloud service takes over the storage work. Suddenly, workers have to be retrained to handle DevOps, mobility, big data and automation.
That's not an easy change, and the company had to address resistance and unhappiness among some of the staff.
However, fewer than 10 people left the company's IT department because of the change and many workers were happy to update their skills and try new jobs.
The company also stopped referring to the IT department as the, well, IT department. Now, they're not IT workers. They are tech workers. That change is meant to erase the wall between tech and business, bringing all those employees on to one page with the common goal of focusing on the business and not simply keeping email and databases running.
The renaming, retraining and the new jobs that workers had to assume were all part of a big culture shift for the tech department.
Another obstacle was figuring out what to do with the company's legacy systems.
While the Weather Company has 80% of its services and apps running on the cloud, it's that last 20% that will be the toughest to deal with.
"There are still legacy systems you have that are hard to re-architect or are fundamentally built to not be compliant with what the cloud needs," Williams said. "I want to get off some of these legacy systems or upgrade these legacy systems, but they have architectures built 15 years ago for an [on-premises] data centers, and they're not built to move to the cloud."
He added that if the old app is not a core need with a solid ROI, it's difficult to get the company behind the expense and effort of either rebuilding the app or tweaking it enough to work on the cloud.
"The harder, more complex and ingrained parts of your business are still hard to move to the cloud," Williams said. "I think the enterprise space still has a hard time with this.
IDC's Mahowald agreed that the legacy systems are the hardest to deal with.
"I'd say all migrations face this," Mahowald said. "It's those custom applications you built over time or a lot of SAP or Oracle ERP… It's really unwieldy to re-platform it. And if they do that, the performance might not be the same [on the cloud] and their users won't be happy."
At this point in the migration, it may come down to old apps that only a few people are using. Is it worth it to re-engineer them to work on the cloud
"Of the 200 apps running in the data center, maybe 70 of them might not have been used in the last two months," Mahowald said. "Then maybe you shut them down and see who complains. If you get a phone call to the help desk, then you have a conversation with the business unit using it. Is it worth the money it would take to move it to the cloud If it is, then they are paying for it."