How SambaNova Systems is tackling dataflow-as-a-service

SambaNova Systems, winner of VentureBeat’s 2021 Innovation in Edge award, is a significant contender in the global edge computing market. The startup raised $676 million in April 2021 and is moving from its origins as an AI-specific chip company to one that provides comprehensive dataflow-as-a-service to its clients.

Research firm MarketsandMarkets forecasts an impressive 34% compounded annual growth rate for the market and anticipates its value reaching $15.7 billion by 2025.

Genesis in software-driven hardware

Traditional central processing units (CPUs) and graphics processing units (GPUs) are based on transactional processing, which needs accuracies to the nth degree for computations. As a result, related chip substrates are made for endless caches and stores of data.

AI is more about training models with datasets and handling neural networks. Traditional chip substrates are insufficient for AI, SambaNova VP Marshall Choy told VentureBeat. “AI is really probabilistic computing rather than deterministic. What you actually need is greater flexibility and performance to run these workloads,” Choy said. “Traditional chip companies start with a chip and hope that software will automatically be generated by the ecosystem,” Choy added. “That’s a very hopeful but unrealistic approach.” SambaNova’s software-driven chip design and solutions development turns that theory on its head, with a focus on AI computing needs and eliminating redundancies.

Stanford University professors Kunle Olukotun and Chris Ré, who are also cofounders of SambaNova, helped shape the startup’s initial focus on developing a software-defined chip led by a software stack. The main attraction is the Cardinal SN10 reconfigurable dataflow unit (RDU), which facilitates continuous learning at the edge, saving days, if not weeks, spent reconfiguring and retraining models when new datasets are introduced. Eight Cardinal chips, packaged with an AMD processor and 12 terabytes of DDR4 memory constitute SambaNova’s DataScale SN10-8R, the startup’s primary hardware offering.

Dataflow-as-a-service

SambaNova’s move into dataflow-as-a-service stemmed from customer demands, although the startup does not share its client list. “Customers told us they wanted to focus on business outcomes and objectives, not on integrating infrastructure and building large data science teams to deal with AI model selection, optimization, tuning, and maintenance,” Choy said.

And so SambaNova has reinvented itself as a company that also offers machine learning services. It offloads the complexity of machine learning by augmenting customer expertise and attends to model selection, training (with customer datasets), maintenance, and other data services so customers can focus on outcomes and business value, Choy said.

An implementation for the manufacturing industry involves defect detection. SambaNova has trained AI models with the highest resolution images at scale to deliver an overall higher model quality, Choy said. “We provide best-in-class accuracy while eliminating the need for additional, labor-intensive hand-labeling of images without downsizing image resolution,” he explained. The end result is a model that is low-effort for users and offers high image fidelity so it can recognize defects easily, he said.

The company prepared early for the global chip shortage that has adversely affected the hardware industry, Choy said. SambaNova continues to rely on internal supply chain experts to navigate the crisis.

SambaNova is currently focused on amping up its dataflow-as-a-service offering, well as the DataScale system.

“We’re at a transformation in computing with AI,” Choy said. “This is not a hardware problem. This is not a software problem. It’s a complete technology stack problem to be solved for. We’re building complete technology stacks to deliver services and products to solve customers’ pressing needs.”