A plan– to change the world using AI. With the power of open source and an ambitious plan to make AI chips accessible to a lot more people, Tenstorrent has made its foray into India to leverage the talent here and grow the opportunities in the AI chip design space. Interestingly, it also comes at a time when the country’s brightest minds are toiling hard to make chip manufacturing a reality in the country after several failed attempts. Jim Keller, CTO, Tenstorrent elaborates on why it chose India to expand to, its business strategy in India, and the future they envision for AI.
Q. A lot of big player companies have been getting into the AI chip space recently. How do you look to make a mark among them?
A. Whenever there’s a new thing, there always seems to be 30-40 new companies jumping on it. When graphic processors first took off, I think there were 35 GPU companies. So there are a lot of companies developing technologies but they’re also developing teams that create things and add value.
The other thing is, in a new space, there is a lot of exploration. You can see this in AI. Researchers are writing new papers all the time, there’s a huge number of papers that talk about new technologies and stuff. But companies like ours that build a hardware and stuff run a lot slower than a researcher. So a programmer can have a new idea, test the stuff, spend 3-4 months doing experiments and then write a paper and then get it published and we read all those papers and keep an eye on it – but to build a chip, it takes a year to design it and over a year to bring it to market. So we run on a different cadence. If you look across all AI researchers or AI companies, people are exploring different things.
And also, with regard to the big companies, a lot of them do things that might not follow the right approach when it comes to building AI programs. These things develop over time and it makes sense since they are a for-profit company and they are trying to accelerate their things and they build it on their infrastructure.
We think what we’re doing is relatively novel. We’re building processors with arrays of small processors that talk to each other directly, and then we are going to build systems that take thousands of chips and hook them together and we’re hooking them together in a fairly novel way compared to anybody else that we know and we think that has value. We have some demonstrations of it so we’re also quite keen on it. But we also think it is interestingly different that is going to attract something.
We like what we’re doing and that’s why we’re doing it. If we didn’t like it, we’d be doing something else and we think that it’s relatively novel and for somewhat puzzling reasons. It’s pretty obvious that you wanna scale up things in certain ways.
But there are some interesting things going on with the software stacks, you can challenge existing things that cause some limitations and then you have to take your bet because you think there is a better way to scale this software and then do something different and then see what happens.
Q. You’ve recently inaugurated your Bangalore office. When do you expect to start operations there?
A. Pretty soon. All the corporate and paperwork are already done so we’re gonna be there in a couple of weeks. We already had 10 accepts for the starting core team. We’re looking at a new office space. We’re starting in a temporary office space and then we’re gonna look at a permanent office that is bigger. We got some pretty good people early on who pinged us and we said what we are going to do and people got excited about it.
So we can expect the team to start doing real work around May or June.
Q. Not a lot of startups look at India when expanding to other countries. What attracted you to India?
A. I’ve had a really good experience working with engineers in India. I worked at Intel and AMD and we had big teams there. My experience in India has been really great, so 2 people in my team were very keen on opening an office in Bangalore. They spent a lot of time researching what is the talent pool available and if this is a good time to do it.
I found that India was a great place to have low-cost resources and have a test team or software feature team, but most importantly I found that the Indian teams really wanna own stuff. When you give them a part of the design and they own it, the enthusiasm for getting it done is really high and I like working with teams that are really engaged. There are some engineers that just do the work and go home. You don’t really advance technology with that attitude. But there’s other people who love what they do and are so involved that you have to tell them to take time off – people like that are what move us all forward.
Lots of engineers go to big companies– which is good because you get a lot of good experience– but what they find is that they tend to work on incremental tweaks to existing things for years. Right out of college or a couple of years, that’s probably a good thing. You get good skills and get to know how things are done – but if you do that too often, you end up doing the same things over and over. There’s a saying ‘do you know the difference of having 7 years of experience and 1 year of experience 7 times?’ A lot of time I see people who have been in a company for 10 years – what have they done? The same thing 10 times.
The thing we offer is– we’re building new processes from scratch and we’re exploring new methods to do it, and so if you join us, you’re not gonna come in and tweak somebody else’s stuff. It’s a daunting thing too. It is very daunting to edit a paper versus having a clean piece of paper in front of you and having to put something on it. It is not for everybody. But it is also an opportunity if you wanna do something new. So I’m quite keen on what we are doing and it is going to be interesting.
Q. How much of a presence is open source going to have in the way you build your products?
A. The AI world is really interesting because virtually all the top models are published by top researchers and when a new model comes out that works, everybody copies it and there are a lot of papers. Most of the time, it seems like most papers are tweaks and iterations and improvements of existing models. But sometimes, people will come up with a new one and pretty quickly others will copy that. So AI is basically an open-source development now.
We’ve downloaded the RISC 5 verification tools, we support the RISC 5 reference models that many people use. We are involved in it already and we are using some of our RTL in RISC 5 and we’re gonna upgrade some of the commonly used RISQ 5 processors so other people using it can benefit. Our plan with our AI open stack is that we’re gonna open source the top part of that so people can see it. We wanna open source as much as we can so that programmers can program their computers the way they want to and do experiments.
AI in general is still a research-y place and having a platform where people who want to add a new kind of model can add it that way is important.
Then there is another thing – there is this program called GitHub Pilot which uses AI to write software so our guys started playing with that and said it is ridiculously easy to write software with it so that is an open source project to write more open source code and it has positive feedback which I think is going to be really interesting. That’s gonna connect AI models and methods with AI as well, and it’s gonna be interesting to see what happens.
RISC 5 processors are going into more and more devices and there’s a number of people offering RISC 5 processors and severs. We’re building a highend RISC 5 processor but we’re also working with RISC 5 ecosystem and software, and we’re interested in people who are going to build interesting RISC 5 products and then solve the gaps – for example, RISC 5 doesn’t have the software maturity, the operating system maturity, but it is actually moving along really fast and we think that – the interesting thing about RISQ 5 is that it is an open innovation platform, and there is a space to modify to solve different problems whereas ARM or x86 cannot be changed.
Those architectures are owned by companies who have a vested interest in owning them and not changing them, so we’re quite interested in working with RISQ 5 and it is going to show up in so many places in the next couple of years, similar to what happened with ARM – it was a little thing and all of a sudden it was the dominant player in mobile films and I think RISC 5 is going to have a similar trajectory.
We’re building this new RISC 5 CPU partly because we think that having AI processor and a regular processor integrated together and tightly coupled is going to make that hardware better and we also think that RISC 5 is going to take over more of the CPU market, and we wanna participate with that.
Q. Can you provide some details about your Bangalore operations? What kind of work are you expecting to get done over here?
A. We are building a CPU and an AI software stack from scratch and then we have an AI processor we could iterate on.
We are looking to work with some great minds who can put practical knowledge into the way they build stuff. Everybody who joins a company changes what you do a little bit, so I like to watch it grow organically. You make a plan and you hire people, and people don’t fit your plan periodically. People are always a little different, they are passionate about this or that, and then if you’re flexible and you have a small team, then you can really let people do stuff that’s really cool and then you get surprised sometimes.
But our job descriptions look like standard job descriptions. We have hired some open stack software engineers already who are working on our cloud computer programs, We have hired people with good verification skills. We have hired a guy who is leading fairly major new processors in another company. But I’m always curious about what happens as you hire more people because some people come with good programming knowledge, some people really love finding problems in software, because if they find problems then the customer won’t.
But we’re generally looking for a broad range of computer science and chip design people and as we hire more people, people fit together in interesting ways and then we’ll see what happens.
Q. We can expect a lot of R&D to happen in India?
A. Yes. But R&D depends on the people doing it – I have seen people talk big about setting up R&D and they all have people with PhDs and they all walk around with clipboards and talking about stuff– but they do nothing.
R&D comes from inside a person, somebody who figures out there is a problem and is bugged by it and then goes ahead to fix it, that is R&D. Not the shiny building with a fancy title.
Q. Is there an opportunity to partner with Tenstorrent for Indian companies?
A. We will be interested in that. We’re already talking to probably too many companies.
The AI software world is turning into lots of layers– we’re building AI hardware and then a compiler that lets you run models and then on top of that there is ML, optimization, orchestration layers, there are all kinds of different layers. There are companies that are doing data solutions, we’ve talked to a bunch of companies who have real-time requirements for image processing, so there Is so many interesting different pieces of it. We’re talking to most of them now to understand what they’re doing, what they need, what is their gap and what we could do about it.
We’re very interested in people doing something interesting– we’re not just partnering to partner. We’re not partnering to have a sales channel. I am always interested to partner with people who have a real problem that they can’t solve but they’re technically competent and they know where they lack and when we talk to them it’s gonna be really interesting.
Q. What is your future vision for Tenstorrent and how do you see India fitting into all of this?
A. We have products we like. In a chip company, you always have a roadmap so we have the first product that we’re going to do the introduction with, the second one is in the lab, the third one, we’re ready to tap out and we’re working on architecting the next one. In terms of our future – we believe that we’re gonna start to have real customers run real problems on our hardware.
For the India team, we started with OpenStack and CPU development and they’re gonna work on that and participate in our company.
I’m sure there are going to be many AI startups and new chips in India. Partnerships are really great when somebody has an interesting problem has their expertise and we have our expertise and we work together to see what happens. I have talked to people about what RISC 5 servers for India, and what about India built AI chips. We have talked to a few but we’re just getting started on this.
There is also something about having a local presence because then someone from our company will start to talk to someone from another company and then you start to find the opportunities and do it that way, so being there is actually important in developing the next pile of opportunities.
Q. How do you see the future of AI panning out?
A. It is a really interesting time to be in computer design because it is changing so much. RISC 5 opens up so many opportunities. Everyone thought CPUs are all done and nobody can do anything more about it but now there are 10-15 startups who are using RISC 5 and are all doing new stuff. This is really a time for innovation and young engineers who work in a new area that’s growing are really important.
You don’t wanna be stuck doing things for the same old design. It is the real-time to say what are the new methods and we know that not every AI startup is going to be successful and the people involved in the new stuff are actually gonna be better engineers and they are gonna opportunities created by the fact that there is a whole bunch of opportunities in front of them and that’s gonna be important for their development and it;s also fun to do that stuff. It’s really great when people work on stuff and they go home and are really excited about it.
It is literally gonna rewrite how we build, verify chips and how we build software and it’s happening really fast.
