Why 'data scientist' is this year's hottest job

With more than 1,700 active job openings on the site earlier this month and a median base salary of $116,840, data scientist also garnered Glassdoor's top "job score" ranking and "career opportunity" score.

Last year, data scientist ranked No. 9; occupying the top spot then was physician assistant. This year's No. 11 spot, meanwhile, goes to the related position of analytics manager, which wasn't even on the list last year.

It's become painfully clear that data scientists and analytics talent in general are in chronically short supply, even as universities have begun establishing degree programs to increase their ranks. Back in 2011 already, McKinsey estimated that by 2018 the U.S. alone could face a shortage of 1.5 million managers and analysts "with the know-how to use the analysis of big data to make effective decisions."

The importance of data scientists is tied to two key issues, said Charles King, principal analyst with Pund-IT. First is the increasing desire among businesses to gain greater value from their data.

That's related also to the rapidly growing prevalence of analytics capabilities in a wide variety of enterprise software.

Second is the fact that the vast majority of information that businesses generate and collect is unstructured or semi-structured data "that can't be effectively analyzed with traditional relational databases or tools," King said.

In fact, some estimates put that percentage as high as 80 percent, he noted.

"In essence, data scientists are trained to manage and analyze large, often highly complex data sets, and to develop the necessary tools for maximizing the benefits of that information for their employers," he explained. "The job is anything but simple and typically requires intensive training."

What, exactly, the data scientist role involves is a matter of some debate, however.

"Some see the role as an architect of data platforms, designing a workable environment for leveraging data," said Nik Rouda, a senior analyst with Enterprise Strategy Group.

Others see it as "technical integration of different systems -- almost a data plumber," while yet another view regards the role as "more a bridge between data analysis and the business needs -- someone who is fluent in both worlds," Rouda explained.

However it's defined, "there just isn’t enough talent to meet the demand," he added.

That's particularly true given that many of the tools data scientists use today are rapidly evolving, meaning they're both a bit unfamiliar and have rough edges.

"Data scientists are therefore held out as the hope for a better future in big data," Rouda said, "even if their day jobs are mostly planning, implementing or coding."