SiMa in Hindi means edge, and that is exactly what this startup has brought to the embedded edge market. By integrating machine learning on a system-on-chip (SoC), Sima.ai has set the course for something truly innovative. Founder and CEO Krishna Rangasayee believes that the embedded edge market can support multiple players. Read on to know more…
Q. Why do we need machine learning for a system-on-chip?
A. I think for the last 30-40 years, SoCs have been fundamentally computing for a wide range of applications, including the embedded market. What’s changing now is that I think the computing needs are going through the roof. So the bandwidth and the computing needs are far outpacing the capacity for traditional computing to keep up. To stay ahead of computing, the industry needs to move from classic computing, and classic computer vision computing into a machine learning basis to keep up with the demands of bandwidth, performance and power. So there are new classes of applications, such as L3 autonomous cars or L4 autonomous cars — traditional technologies are incapable of solving the problem anymore. So you have to have a machine learning approach to solve these problems. And that’s one example. But there are many examples where ML is not a nicety, it’s a necessity to solve the problem.
Q. Is this product a completely new innovation or an upgrade over something you’ve been working on previously?
A. I would say it’s a new product category. I don’t think this industry has a machine learning SoC. I think many other companies have pushed their core competency into machine learning systems. They’ve either pushed GPU technology or CPU technology or FPGA technology. We don’t have any legacy. We have the opportunity to come back and understand the customer’s pain points, build exactly what we think the customers need, and not worry about who we were before. The problem with big public companies is that they cannot switch the technology basis completely. But as a private company, I could do anything that I think is the right thing for us to do. So I would say that the concept of bringing machine learning is not a new technological perspective. How we are bringing it and how we have architected this so that people can use a heterogeneous compute platform — having an ARM processor subsystem, a computer vision pipeline, a customized machine learning accelerator that is purpose-built for computer vision problems and embedded — that approach is unique. I think SiMa.ai is creating a new product category.
Q. You’ve mentioned that this MLSoC has diverse applications, but every specific application has different requirements. So do you have different designs or models of this chip for different applications or just this one?
A. So that’s a great question. I think that’s one of our biggest problems. I think if I were to take a step back, every choice we make is, how generic we want to be. If you’re generic, you’re not optimal for one thing. You can cover a lot, but you’re not optimal. If you’re very specific, you’re fantastic at one thing, but then you don’t scale for the others. So this balance between generality and specificity is something we have grappled with quite well. We are building a single system today, that is going to serve multiple markets. We have given it quite a bit of thought and said, how could we scale from robotics, to unmanned aircraft to industry 4.0? We’ve picked the right IO structures, the right makeup of the cost structure, and the right power and performance. I would say if somebody were to build a custom silicon with just one problem, there is a pretty good chance that they would be good at doing that better than us. But if somebody were to look at something generic to solve for everybody, I think we probably are way better than any other system. So this is not an easy problem to solve. That’s part of all of our collective knowledge and experience that’s helped us. I would say that with the experience we have had with customers, compared to the alternatives, we are better than them consistently all the time.
Q. What is a machine learning accelerator?
A. A machine learning accelerator is a very targeted function that accelerates all machine learning application needs. You could do it on a CPU, which is a sequential machine, or a DSP processor which is parallel that can add more efficiency. Sometimes DSP processors are 100x better than the CPU. A purpose-built ML accelerator is built around the construction of matrix multiplications. So it’s a specialized architecture solved for accelerating machine learning problems and workloads. If you think of a hierarchy, CPUs are 1x while DSP vector engines could be 100x. ML could be 100 times the DSP Engine. So it’s a very high performance, very high capability ML accelerator. It’s good at ML acceleration only, it is not general purpose. So CPUs are the most general purpose, DSPs are performance oriented with some amount of general purpose, while ML accelerators only do ML acceleration. So they’re like a custom function, and in ML, they are amazing. They’re not meant for generic applications. We provide all three — an ARM processor subsystem for CPUs, a DSP vector engine for image processing applications, and for DSP vector processing, we provide an ML accelerator. So the MLA is our proprietary ML accelerator. We have built a very high-performance ML accelerator but at a very, very low power.
Q. So would you call the ML Accelerator the USP of your product?
A. That’d be a fair thing to say, but I truly think that our USP is how we have integrated everything to solve customer problems. So no doubt, the ML accelerator is the largest innovation, but I think the bigger innovation is how we have integrated all of this so that there’s a holistic customer experience, that’s software-centric.
Q. Traditionally SoCs utilise very low power. But you’ve constantly mentioned that your SOC has even lower power. I want to know, how does this low-power mode operate? How have you scaled further down and what features are you compromising on in the low-power mode?
A. Wonderful question. So embedded SOCs are typically constrained by the embedded markets’ power requirements. So SOCs built for the embedded market are typically 5W, 10W, and 20W. Machine Learning (ML) breaks that mould. If you look at high-performance ML, they’re mostly been cloud-centric. They have high performance, but they have very high power. So 75W, 100W, and 200W are what they dissipate. Our problem is to bring that kind of ML performance but at 5W. What we have had to innovate is not necessarily in the SOC is low power, but the ML being very low power, so that it fits into the overall power and thermal constraint that customers have. We have had to build a very unique ML architecture that fits in the power envelope of embedded. This is where I think we have done the best differentiation compared to anybody else. Today in the industry, if you want very high-performance ML, there’s a lot of power. If you want very low power, you get very low ML, there is no good solution for high-performance ML with low power. And that’s, I think, our sweet spot and what we have done well.
Q. Alright, but you’ve still not answered what features you compromise on in the low power mode.
A. No doubt we have different modes of operation. By default, we support a class of performance of 50 Tops at 5W. If people only want to dissipate 3W, we cap the performance, so that it fits the 3W envelope. That’s a very clever innovation that we have done. We can optimize for performance, or we can optimize for power. There are trade-offs in terms of how much utilization you get, and how much performance you get, and we give customers that choice. And one inherent thing about our company is that we have gone out of our way not to be smart, and tell customers what to do. We are more about giving them choices so that they can make intelligent choices because they know their applications better than us every day.
Q. Do you customize the MLSoC for every client that you have?
A. So another really good question. Our silicon is the same. We do not make changes to silicon, how customers use our silicon is different. So how our robotics customer uses silicon and our software is quite different from how an unmanned aircraft uses it, or how an automotive product or how a medical product uses it. So differentiation comes in two folds. Firstly, how they use our silicon from a software perspective, and what features they use. Secondly, how they will optimize the ML software solution for their application. So we enable innovation and those two factors. One is the utilization of silicon. The second one is the construction of the software, and how it’s compiling different problems in embedded, medical, automotive problems and drones. So the end product is customized, but the customization is not full stack. Silicon is the same for everybody. How they use the silicon, and what software they put on it gives them differentiation.
Q. How sustainable is it for you to continue with the customization once you move on to mass production, and you commercialize the product fully?
A. I have done this for a very long time and we believe that we could retain the same architecture and remain differentiated and customised for the customer, forever. So I think that is not a limitation for us. One of the benefits of keeping it programmable is that customers can continue to improve their algorithms. So they may have a better approach in six months and want to change their previous approach. They can do it very easily with the same silicate, same software – they can customize. We have this concept of a future-proof approach. We know that customers learn a lot. Customers want to change a lot, but they don’t want to keep changing the silicon, the software and the boards every day. We want to keep the same board, and the same silicon, but allow them to differentiate through software upgrades.
Q. What are the strategies that you’re using for thermal cooling?
A. Our biggest focus is to make sure that people don’t have to be innovative in thermal cooling. With our chip, very few customers will need thermal cooling for our chips, because we are in a very different power category. We use traditional approaches such as heat sinks for thermal cooling if at all people need it. We support them in our boards, and they come with cooling elements. We give guidelines to our customers on what to do for thermal cooling. Some companies have become super innovative with thermal cooling because power is a big problem. With us, that’s not the problem. So I don’t think we need to be too innovative on thermal cooling. We just need to provide guidelines and allow customers to figure out what makes sense.
Q. How do you avoid interference on the chip?
A. We have a Network-on-a-Chip (NoC) infrastructure. We have partnered with a company called Arteris Inc, which is a world-class company NoC. We have a very elegant system where there are many subsystems. Think of it as a highway that connects different cities, so that people and traffic move from one to the other. We manage the bandwidth, the data flow and the traffic that flows along it. The interference either from a functional perspective or even a physical perspective is all managed in a very careful way. There’s a very complicated program. There’s a lot of data movement and if not done right, people could create a lot of interference in many different ways. So we have given this a lot of thought, and that’s one of our secrets — how do we do data management and memory management in such a way that there is no interference in data, and it does not hurt the bandwidth or the physical concept of signal integrity?
Q. When you’re marketing your product to a particular company, who is typically the decision maker that will want to use this product, and who is the user?
A. Very good question. So it depends on the company. Sometimes it’s a CTO or VP of R&D, sometimes it’s a product manager. So they end up being “the decision makers” for endorsing a new technology. The implementers will be the R&D team.
Q. While marketing the product, you’ve constantly used the term 10x. How does SiMa.ai justify the 10x?
A. Everybody hides behind benchmarks. There is no company I know on the planet that says they are number two, in benchmarks. Everybody has very clever ways of how they say in this circumstance, this benchmark, I’m number one. What truly matters to us is that in customer applications, customers see the benefit. That’s 10x. So in all the customer interactions we have, we consistently deliver that 10x capability for them. In my mind, that is the only thing that matters. ML is so new, standardization is so new, and benchmarking is so much an art-craft, that people have spent too much time on positioning, marketing and benchmarking. I honestly think as the industry matures, the only thing that matters is whether the customer sees it. I think people optimize so much for benchmarking that they don’t allow customers to see the benefit. Take all these companies that have amazing benchmarks. If you give them a real customer application, you will see that their performance is radically different. So we have brought ourselves down saying we don’t want to chase the benchmarking world. We want to put our heads down and do a good job helping customers. We are delighted with what customers see day in and day out. We will also at some point be publishing our benchmarks so that I think customers can get some yardstick for what we could do, but our priority is customers.
Q. How do you propose to explain the benefits of your product to your target audience?
A. So really good questions. I think we have understood the space well. We have a lot of key system architecture expertise within our company. We call them system architects. We show them proxy examples. We tell them that we don’t know their application, but here’s a relevant proxy of the application. We show them what they’d get with others and what they’d get with us. That gives people enough confidence that we can solve their problems. We improve our capability year after year. So if we have learned a lot about robotics, then we can bring that collective knowledge to our robotics customers. If we have learned a lot about the automotive sector, we bring it forward. We preserve our customers’ proprietary information, but the collective learning and by showing proxy examples of image segmentation, image recognition, or semantic segmentation, we can show them what we can do. This gives our customers a lot of confidence that this case can do something quite different.
Q. How do you plan to reach your customers? Do you plan to enrol any mid-channel partners?
A. Great question. Absolutely. For the first 20 to 50 customers, we’ll probably end up being in direct contact with them or enable demand creation directly with the customers. Even at my previous company, we had a very sophisticated system of representatives and distributors globally. As we scale to 100, 1000 and 10,000 customers, we will need the right representatives and global and local distribution partners. We will be working on that in the next few months and we’ll start building our channel strategy.
Q. How many chips have you sold till now?
A. We haven’t publicly disclosed where we are. We have told everybody that we will be in production in the March timeframe. We have a fantastic team that’s got our first-time-right silicon. We are on track. We will be in production with our silicon and boards in March ‘23.
Q. What is SiMa.ai’s targeted revenue?
A. We haven’t gone public with that. One of the benefits is that we are a private company. So I would say we are already getting into revenue. We have already made the initial revenue, but we will be getting into mainstream revenue in 2023 and 2024.
Q. Apart from Fidelity, has SiMa.ai sought funding from any other partners?
A. Fidelity is our lead investor. We have many investors that have invested in us. One company that is not only our seed investor, and has a very large investor offer, is a company called Amplify Partners. We have MSD, which is Michael Dell’s family fund. We have a great partnership with Dell Technologies Capital (DTC) and Wing Venture Capital. We have a fantastic investor Adage Capital from Boston. We have had many, many stellar investors, and one thing in common with all of them is they have a deep ML background and they have also invested in semiconductors. One person, I would highlight is Lip-Bu Tan. He’s personally invested in our company and he’s also on our board. Lip-Bu is probably the industry’s most prolific investor. He has invested in more than 125 IPOs. We’re very lucky to have such great people like him around us. Moshe, the former CEO at Xilinx, who is also on TSMC’s board is our chairman. So all these great people helped me navigate the company. We just announced that Harry Kroger joined us. Harry headed the automotive electronics at Bosch and we are delighted that he’s taking us into automotive. All these are tough businesses and unless you have people that have a great pedigree, great background, great experience and an illogical belief system that they’ll figure out how to make it a success it’s pretty hard. My day job is to make sure that I’m surrounded by the best and good things happen.
Q. Apart from funding challenges, do you expect any other challenges in the forthcoming future and how do you intend to deal with them?
A. It’s a startup, we work with challenges every day. Challenges are always going to be there. When you solve a few challenges, you will get new challenges. I put my emphasis on how capable are we in anticipating these challenges, how ready can we be when the challenges arise and whether we have the talent and the intelligence to solve them. So in my mind, that’s the only thing I can control. I can control our readiness, I can control our ability to execute. We’ve worked through so many challenges in the last four years. I have very high confidence now that with the talent we have, and the maturity of the company. we will work through the challenges. So I think we’ve been very good. Funding is always a challenge, but it’s not been a very big challenge for us. Writing and creating the right architecture is a challenge, and we have done a good job. Coming up with a value proposition, that’s so differentiated is hard, and we have done a good job. You could have all these strategies, but if you don’t execute nothing matters. I think this team of 140 people has done a phenomenal job in executing and delivering this product with such complexity and software. With COVID, it’s a pretty hard thing to do, but they’ve gone through a lot together as a team, and we believe in each other quite a bit. I have very high confidence that we will manage through whatever challenges we get ahead of us.
Q. Does SiMa.ai plan to take on any other partners apart from TSMC?
A. For now, I think we’re very happy with TSMC. We are too small a company to be working with multiple partners. I think our success is about really continuing to work with what we believe in and TSMC has been fantastic with us. So for the foreseeable future, we plan to stay with them.
Q. Two days ago, Nvidia collaborated with Foxconn to start mass manufacturing. Nvidia is an established company and Foxconn is a giant contract manufacturer. You are a startup which has collaborated with a giant such as TSMC. How do you expect SiMa.ai to sustain its individuality in this course?
A. There are so many amazing companies, and I’m sure they’ll continue to do amazing things. What I would tell you is that I think they are purpose-built and purpose designed for solving the embedded market. There’s a massive market and the opportunity is phenomenal. I do not doubt that there’ll be a lot of other good companies competing for the space as well. The good thing about the embedded market is that it can sustain multiple players. It is not a market where there’s just one player. There will be multiple players. Some of these companies will do a great job and continue to win heavily there, but what we’re doing is very unique. Based on my industry experience and the track record of where we are with the customers today, I feel that we’re also going to be a successful company. We worry a lot about competition, but in reality, if we do our job, I think we’ll be fine.
Q. What is your next milestone?
A. Scaling the company and revenue.
Q. Any other upcoming project or innovation that you’re working on?
A. I think in our industry, the gift of doing a good job is you have to do it again better. So we are working on Gen 2 (Generation 2). We are building an amazing Gen 2 as well. Our focus in many parts is to make sure that Gen 1 is successful. We are very confident it’s going to be a great product. I think we’re going to delight a lot of customers. We’re raising the bar on Gen 2 and we are pushing ourselves beyond our limits to build an even better job. While that’s super exciting, the job at hand is to make sure Gen 1 is successful.
