“We want to make it very easy to expand from a single chip to a large number of chips”

- Advertisement -

Utilising RISC-V Architecture, Tenstorrent is building AI processors that can automatically and performantly compile and execute its customers’ machine learning models and scale the execution across multiple chips and servers seamlessly. CEO Jim Keller tells Yashasvini Razdan from Electronics For You that the company’s processors can communicate with each other, giving them an advantage in code writing and cost efficiency. Read on...

Q. How would you explain your technology?

A. We are developing a groundbreaking artificial intelligence (AI) computer. AI computers perform vast amounts of dense mathematical computations and perform numerous operations. Computation is similar to how the brain works. The brain consists of approximately 10,000 groups of cells that communicate with each other locally and broadly. AI problems can be thought of as highly intensive mathematics. Computation occurs in the brain when multiple cells exchange information with each other, continuously enabling its functionality.

- Advertisement -

We are constructing a computer capable of executing extensive mathematics while simultaneously communicating the results of each calculation step. This form of representation is trainable and produces intriguing results with AI software. To summarise, we are building hardware equipped with many independent processors. We are also developing software that can distribute the company’s AI program across these processors, which can natively communicate with each other. This provides us with advantages in code writing and cost efficiency for the computer and the system at large.

Q. Could you explain your basic business model?

A. We are a design company specialising in computers, AI, CPUs, and so on. Different users utilise computers in various ways, such as cloud computing, servers, chips, templates, and more. These are just a few consumer methods. Different customers have different preferences for purchasing computers at different stages. Our goal is to make our technology available to people in the manner they prefer to buy it.

Q. What are the benefits of using Tenstorrent AI processors?

A. We are significantly more cost-effective compared to some of our competitors, and we have demonstrated this to multiple individuals. Our hardware is highly optimized for this type of calculation, surpassing traditional GPUs. We are in the process of open-sourcing certain compilers that people are interested in accessing. Several components of the software stack have been proprietary for many researchers and AI programmers. By using open source, we aim to provide access to those components. Furthermore, our chips have native communication capabilities with a large number of other chips, eliminating the need for additional networks.

Q. What challenges does your hardware architecture address?

A. We are currently entering the market with a few customers. Some customers struggle to translate certain types of AI programs into high-performance software tools. Converting algorithms to programs that run efficiently requires considerable effort. Our mission is to simplify the transition from a new model to a fast program.

Another challenge is that certain models require multiple computers to run, making it difficult to expand from a single chip. We are dedicated to making it effortless to scale from a single chip to a large number of chips. While we have made some progress in this area, we continue to work on solving this challenging problem.

Q. What is the advantage of having multiple chips, instead of a single processor?

A. A single server chip is big and very expensive. It’s a single piece of silicon with multiple functions on that one piece. If we divide that chip into four pieces, each piece, called a chiplet, is smaller. You can now put the memory interface on one piece, the network interface on another piece, the IO interface on another and so on. So now each piece is simpler to build.

We’ve talked to a whole bunch of companies. Some want to get a speciality in network chiplets, or memory chiplets. We want to specialise in AI processor chiplets. If a company wishes to build a product, they can buy some chiplets as finished pieces and only customise the part that they need to, rather than having to buy all the pieces, as it’s just too hard and expensive. Chiplets can be used to break one big problem into smaller problems, and smaller companies can independently solve those problems.

Q. Are there any limitations to power with so many chips running?

A. Power management is a significant challenge when designing chips. We implement a power management system to monitor consumption and temperature, adjusting the clock and voltage to optimize performance. We have carefully chosen the size of our chip to strike the right balance.

The brain consists of clusters of cells, and the same principle applies to AI processors. If the clusters are too large, communication becomes difficult; if they’re too small, frequent communication with other components becomes necessary. We actively manage the trade-off between computation power and networking capabilities, ensuring efficient power utilization. We closely monitor power consumption and temperature to maintain limits when building a rack of computers. This enables effective control of computation across the system.

Q. What is the difference between processors built using RISC-V architecture and those built by ARM or Intel?

A. RISC-V is an open source and open standard architecture, allowing anyone to build their own RISC-V processor. On the other hand, ARM, Intel, and AMD employ proprietary architectures, controlling their production. RISC-V International serves as a coordinating body, facilitating collaboration through meetings and conferences.

We utilise the RISC-V architecture in our AI processors, with modifications to optimise their performance. We have added instructions and communication capabilities between processors, enabling fast inter-processor communication, a feature not present in ARM or Intel processors. RISC-V is instrumental in building our product, and it offers the flexibility for customisation and innovation, allowing users to build their desired products without restrictions.

Q. How is debugging impacted when using open source?

A. In the case of proprietary software, only the owning company has access to the underlying code, making it challenging for external experts to identify and fix issues. However, with open-source software, transparency allows for more effective debugging.

While using open-source software, if something doesn’t work, experts can examine the code, identify the problem, and fix it themselves. This level of visibility and accessibility significantly improves the debugging process compared to proprietary software. With open source, users have greater control and the ability to resolve issues promptly.

Q. What software programming tools are available for developers to leverage an AI processor’s capabilities?

A. Developers commonly use programming languages such as PyTorch, TensorFlow, ONNX, and JAX to write to AI models. We support these languages. Additionally, we have developed a program called TBM, which helps programmers’ interface with our hardware. We have also created our own software stack, which we have demonstrated to a small number of customers, and we plan to release it as open-source software.

Q. How are you different from your competitors, such as Groq and SambaNova Systems?

A. They’re all AI startups, and they’re picking their particular area of expertise. Groq is focused on inference processing. SambaNova is building models for customers. We want to supply computing technology for inference and training, with various ways to use that technology.

AI is in its infancy and things are changing fast. Our competitors have to map their software to the hardware manually. Manually mapping orders isn’t scalable in the long term.

The other big difference between us and them is that we’re integrating CPUs into our AI chips. So the processor can take advantage of Tenstorrent cores and CPU cores, and we’re the only ones doing that right now. What that means, ultimately, is that while the AI workloads are running, calculations which are better done by the CPU have to leave and go to the server CPU and then come back and it stops everything. With us, we have CPUs on our dies, and so speeds up everything, making it more efficient.

Q. What are the challenges of using processors in AI workloads, and how do you address them?

A. The primary challenge in AI is the rapid pace of change. The field has witnessed advancements such as inference and training, language models versus image models, and the emergence of large-scale models consisting of thousands of processors. Generative models involving multiple lookups based on previous results have also gained prominence. Each technique introduces new software requirements, necessitating a highly programmable and adaptable computer system.

Additionally, a wide range of memory capacity and power budgets is required for AI workloads. The competition in AI is constantly evolving, and the operating range encompasses diverse needs. To address these challenges, we ensure our AI processors are programmable and support various software constructs. We aim to accommodate the varied requirements of different industries, whether the AI models are deployed in edge computers, data centres, or robots.

Q. How do you ensure that your AI processors support the needs of different industries?

A. Consider a manufacturer producing screws or nails. They manufacture various sizes without specifying whether they are intended for a house, car, tractor, or fence. Similarly, we offer AI computation in different sizes: small, medium, or even the option to combine a large number of chips to create powerful machines. The choice of models depends on the customer’s requirements.

While certain variations exist, such as generative text image models differing from language models, they still share common elements. We examine the range of software constructs and ensure our processors support a wide spectrum of AI models. Whetaher the AI models are integrated into edge computers, data centres, or robots, we prioritize meeting the diverse needs of different industries.

Q. How good is it for an AI processor startup business to have a diverse product range?

A. Well, we’re a startup, so when somebody says they want to buy our product, we say yes. If you grow a lot, you might find the effort for robotics is much higher than in a data centre. You must put resources there. Many of these chip manufacturing giants make a lot of money on certain things, but there’s a whole bunch of people nobody’s supporting.

I’ve spoken to people making robots who cannot afford to put a $10,000 AI computer in a $10,000 robot. These guys require very high computational power, else they can’t make the product. We talked to a big company building a data centre and they want to use AI and grow it but it’s too expensive.

So if we can help them solve those problems, then you know, we’ll have a happy customer and we will be able to grow our business.

Q. Could you give us some insight on your partnerships with industry and academia, in AI and deep learning?

A. We have just started a joint venture with a company called Bodhi. So they want to build server products that include both AI and computation. We work closely with the foundries and the CAD companies because we build chips.

We’ve talked to quite a number of researchers, and one recently built Tenstorrent ML software stock, because researchers want the ability to have an open-source software method, which they could program right down to the hardware, and they don’t have access to that currently, with other people. So they want to make novel new models, which may be successful, but they have no way to do that.

We’ve talked to people outside of AI as well, that want to use a similar kind of computation, but they can’t write the code they want in PyTorch or TensorFlow. We want to be able to build up the software methods so people can program well.

Q. What are the future developments and advancements in AI processors?

A. AI processors will continue to get bigger. We will continue to optimise how the mathematics operations work for more efficiency. We will make them easier to program, so more people can participate and build high-performance AI models.

We’re going to see people building AI into more products. People have talked to us about licensing our AI and RISC-V technology to go off and make their product that does what they want. A big chip manufacturer may sell a standard AI card that’s quite good and very expensive, but they won’t let you change it. We think that flexibility is really important.


 

- Advertisement -

Most Popular Articles

Yashasvini Razdan
Yashasvini Razdan
Yashasvini Razdan is a journalist at EFY. She has the rare ability to write both on tech and business aspects of electronics, thanks to an insatiable thirst to know all about technology. Driven by curiosity, she collects hard facts and wields the power of her pen to simplify and disseminate information.

Infineon, Stellantis Partner To Advance Next-Gen Vehicle Power

0
With a dedicated Joint Power Lab and secured supply agreements, the partnership is set to drive transformative advancements in EV performance, cost-effectiveness, and the...

Audi Launches New EV Brand In China, Drops 4-Ring Logo

0
On Thursday, Audi introduced a new electric vehicle brand in China, branding the cars solely with the name 'AUDI' instead of its traditional four-ring...
IIT Bhubaneswar

IIT Bhubaneswar Partners MOSart Labs To Boost Engineering Skills In Chip Industry

0
The aim is to help students with essential skills in areas such as very-large-scale integration (VLSI), semiconductor technology, analog and digital design, validation, and...
Jaguar concept EV

JLR Highlights Its Biggest Reinvention With The Concept Car Jaguar Design Vision

0
The four-door GT is anticipated to be Jaguar's most powerful model to date. Its targeted range is approximately 430 miles (700 kilometers), and its...

ISE Labs Expands With New OSAT Facility In Mexico

0
Turbocharging its growth in North America’s tech hub, ISE Labs is set to build an OSAT semiconductor facility in Jalisco, Mexico, eyeing to create...

“Adoption Of IoT-Enabled SMT Machines Is Still In Early Phases” – Pradeep Tandon, Director...

0
While electronics are becoming smarter through the Internet of Things, is electronics manufacturing following suit? Pradeep Tandon, Director for India, Middle East, and SAARC...
Dr Yashodhan P. Gokhale, Vice President for Battery Technology, JSW Energy

Charge Forward: High-Voltage Batteries And MSMEs Can Fuel India’s EV Revolution

0
A nuanced explanation of low- and high-voltage EV batteries by Dr Gokhale, Vice President for Battery Technology at JSW Energy, illuminates their advancements and...
Sanjay Gupta, CDO & Global Head of Engineering, L&T Semiconductor Technologies

Can MSMEs Contribute And Profit From The Semicon Boom?

0
Semiconductor manufacturing often requires significant investment, typically feasible only for governments or large corporations. But are there opportunities for MSMEs as well? We spoke...
N.P. Ramesh, COO and Co-Founder, Orb Energy

“We Ensured Customers’ Monthly EMI Would Be Equal To Or Less Than Their Electricity...

0
In a conversation with Nitisha from EFY, N.P. Ramesh of Orb Energy outlines the early financing challenges customers faced. He explains how Orb Energy’s...

“Government Policy Should Equitably Address All Electric Vehicles: Extend Subsidies To Retrofits Or Remove...

0
Aryaman Raghuvanshi from EFY engages with Pramod Narendra, Head of Brand Management and Retail Network at Starya Mobility, to explore the overlooked potential of...

Wireless Light-Based Communication With Velmenni’s Technology

0
Velmenni is at the forefront of wireless communication innovation with its pioneering light-based technologies, Li-Fi and LC Link. Founded by Deepak Solanki in October 2014...

Harnessing AI Technology For Efficient Pest Management

0
AI-Genix, an agri-tech startup, is transforming pest management for farmers through advanced AI-enabled systems that precisely target harmful pests. In an industry where pesticides and...

One Charger To Charge All Your Devices

0
Now, there is no need to carry different chargers for different products. This Bhubaneswar startup has launched a universal 65W charger using GaN technology...

Karnataka Government Initiative Promises ₹5M To Tech Startups

0
Boosting innovation and entrepreneurship across the state’s vibrant ecosystem, Karnataka launched Elevate 2024 and KAN, offering grants up to ₹5 million for startups. On Tuesday,...

CastNX Tech Startup Secures ₹232M For Power Electronics Growth

0
Eyeing to boost R&D and produce 100,000 variable frequency drives by 2026, Indian power electronics startup CastNX has secured ₹232.2 million from Plutus Investments. CastNX...

Industry's Buzz

Learn From Leaders

Startups