The future of data science: Trends in a rapidly-growing industry

This article was contributed by Michael Burke, lead of the PR Intelligence Lab at BYU.

Worldwide digitization accelerated as the pandemic sent many inside and online, and this digitization continues for the data science industry. As society has transformed, increased demands must be met at a rapid pace. Customers are online more than ever before, causing a steep increase in data. Now, data science is one of the top careers for recent college graduates as the need for strategic, data-driven decision-making has increased at exponential rates across industries.  

Industry and company-wide commitments to data science and digital transformation are not small — in fact, it’s practically a gold rush for talent. Look at Fortune 250 fintech, FIS, as a prime example. In the last year, they have committed $150 million to innovation ventures, internally built an entire real-time payments engine to move B2B transactions instantly and launched an Impact Labs incubator in Denver that has already yielded a product called GoCart, which transforms the shopping cart experience into a one-click payment. 

Finance, healthcare, and other large “old” industries are not usually known for being the fastest to make digital transformations, so when you see these kinds of moves being made, you know that data science is making an impact.

And that impact will only continue. Data is the backbone of businesses across industries, and some estimates indicate that the total amount of data created, captured, and consumed will likely reach 149 zettabytes by 2024. This staggering number proves not only how much the field will grow but also why it is so important to identify and stay ahead of the trends. Two big trends right now have to do with those that are joining the data science field, and the conflicts in the type of data that larger and smaller companies are prioritizing.

Applicants in the data science industry

We are seeing an increase in people who went to school with a math background (or who might have started out in data engineering) wanting to make the switch to data science. This should be comforting news to anyone who got a degree in math (or a STEM-related field) and now wants to make a change in their career to be part of this rapidly growing industry. This may also be good news in that it means that people with a solid understanding of how to interpret statistics properly will be available. While software advances may allow users to more easily create charts, they won’t necessarily be able to understand all the nuances and implications of them. More mathematicians in data science means more grounded decision-making.

“10 years ago, you would have had to go down a very specific track and make a structured career decision to end upin data science,” says Michael Tarselli, chief scientific officer at TetraScience. “Nowadays, scientists are coming out of school and saying, ‘You know what? I can do an end around on this. I can do a one-month data science boot camp and school myself up quickly on Python, recursive logic, or neural networks’ and then boom, they are a leading candidate for us.”

We are also seeing a lot of applicants who have a data science education or an education compatible with data science but who went into other fields. This is largely because, when these job seekers entered the market, the data science field wasn’t booming the way it is now. This means we have data scientists with broad experience who are only now joining the industry, and the value of this domain expertise cannot be understated. To interpret data relating to, say, a fintech company, you have to understand the language of finance–and this knowledge just isn’t something one gains overnight. Luckily, with the right educational background or certifications (which are now more accessible than ever), many people who have a wealth of domain-specific expertise can consider making the career switch to data science.

The data science divide

The other big trend is the divide that we are seeing between what large companies are looking for in data scientists and what smaller companies and startups are looking for—and that divide is growing. 

Larger companies already have a lot of the infrastructure in place for managing their data and cleaning it up. They’re looking for data scientists and researchers to come in and go very deep, with a focused and narrow breadth of scope. Large companies are looking for scientists to level all of their focus on specific data science problems. 

Startups and smaller companies, on the other hand, are likely lacking in data infrastructure and determining how to put one in place and then how to use the data it pulls. They’re looking for “jacks of all trades” who can start getting insights out into production and work on more of the stack. 

As many people are starting their careers, they need to consider what type (and size) of company they want to work for. Smaller companies give you room to grow and focus on a wider array of issues, and larger companies remain more focused and specified. This gap continues to grow today. 

Relationship trends between company and job candidate

As the field continues to grow at a rapid pace, the hiring process becomes more competitive on both sides of the spectrum. Companies have a good amount of people to choose from, and applicants know that companies need data scientists now more than ever. The hiring process is contingent upon where any given company is on its data maturity journey. 

Smaller companies are looking for people with more chops who can approach things from a full-stack perspective. Bigger companies are looking for specified knowledge and academic machine learning researchers to build comprehensive models. But no matter where they are in that continuum, companies and job seekers alike are trying to find the right fit. Those early-on in their career may be looking for mentorship, growth, and an understanding of how data science fits into a real-world scenario. Whereas, those who are more senior in their positions want to focus on difficult yet achievable problems. 

One thing you might hear from a more senior data scientist is that they were brought in to do data science but the company wasn’t ready for it, so they were saddled with other responsibilities (like data engineering). These data scientists will be brought in to work on projects that sound interesting — but end up being a letdown when the company reveals that they aren’t ready to execute. 

Some of this may be due to deficiencies in the hiring process. As most HR departments lack experience hiring data scientists, they often fall into the trap of putting out overly broad job descriptions, and testing applicants for non-relevant skills. Whatever the reasons may be, this leaves data scientists in an interesting spot in their careers (especially if they are unable to do the work they set out to do), and it might inspire them to seek new opportunities. Overall, the trend leans toward favoring data science specialists over generalists, so data science professionals must consider how they can fortify and specify their strengths in order to excel. 

In general, the goal of every business leader should be to unlock human potential — and this is especially true in fields that rely on continued education, innovation, and passion. Few industries are experiencing an explosion of need and innovation quite like this pandemic-prompted industry. Businesses will continue to bolster their digital and eCommerce efforts through data, and we have the opportunity to help steer the economy through our educated efforts. Glassdoor even ranked data science the #2 profession for 2021. As we continue to grow as an industry, those in the field will be met with more and more opportunities for growth — opportunities that business leaders must strive to enable.

Michael Burke is a data scientist with an MS in Data Analytics. He has written extensively on topics related to machine learning, and has won industry awards for his study of factors affecting SEO. He leads the PR Intelligence Lab at BYU’s School of Communications.