Richard Socher on theCUBE + NYSE Wired: Mixture of Experts

Summary

TheCUBE's John Furrier talks with Richard Socher at theCUBE's NYC studio about how neural networks went from experiment to engineering, and why the next step is replacing the manual work of doing research with learned systems. Socher says the biggest shift for most people will be managing multiple agents, and that Recursive thinks in agent count rather than headcount.

They cover his book The Eureka Machine, why biology is becoming a programmable engineering science, and why it needs far more data before AI can build a virtual cell. They also discuss compute as the biggest constraint, the debate over SaaS companies and AI, and what advice Socher has for companies that want specialized intelligence without becoming frontier labs.

Key highlights

• From craft to engineering. Fields move from trial and error to engineering as their principles are understood, and each time a manual process is replaced by a learned one, capabilities grow.
• Agent count, not headcount. Socher says most people will become managers of multiple agents, and Recursive asks how much compute and how many agents it can delegate work to.
• Swarm intelligence. Using the example of dropping one person in a forest and asking them to build a toaster, he argues human intelligence is collective and builds on past inventions.
• Compute is the constraint. He expects compute to stay the biggest constraint for some time, so humanity will have to decide which problems deserve the most resources.
• Biology becomes programmable. He expects biology to shift from a natural science to an engineering science, but says it needs much more data, such as perturbation studies and organoids, to build a virtual cell.
• Profluent and Eli Lilly. He says Eli Lilly signed a contract worth more than $2 billion across multiple milestones with Profluent, whose protein generation work grew out of the ProGen research.
• Building blocks for companies. For companies that want specialized intelligence, he suggests open-source models, a search layer like You.com, and your own proprietary data.
• Why the book. Socher wrote it to counter negativity about AI and to highlight benefits in energy, materials, batteries, and cures for disease.

Watch the interview

https://www.youtube.com/watch?v=hf691EqfmX8

Transcript

Lightly edited for readability: timestamps follow the source transcript, speaker labels are not included, and obvious transcription errors and name spellings are corrected.

[0:00] Palo Alto studio connection: Silicon Valley and Wall Street. I'm John Furrier, here with Dave Vellante, my co-host.

[0:13] Hello, I'm John Furrier with theCUBE. We are here at theCUBE's NYC studio. Of course, we have our Palo Alto studio connecting Silicon Valley to Wall Street, bringing that deep tech coverage here as part of our NYC Wired program and community. We've got a great guest, co-founder and CEO of Recursive and You.com, AIX Ventures, neural network professor, author of the book The Eureka Machine. What haven't you done? Welcome to theCUBE. We could go an hour. We should get you back in Palo Alto.

[0:38] Thanks for having me. Yeah, excited to be here.

[0:41] You know, it's public, it just came public. You were an early seed investor in Hugging Face, which is a great success. They really did a lot of pioneering work on getting open source out there. So, hats off to them. But also, you were doing the neural network class at Stanford. You were a professor. They were in your class. Neural networks come to today. Everyone is raging about neural networks. It's been around for a long, long time, as have ontologies, as in knowledge graphs and knowledge layers. But with AI, the computer science of data, graphs, neural networks is really compatible with some of the software and the foundation models, the mathematics and the AI infrastructure. I mean, it's math. Feed the math with more math. GPUs. It's mathematics-based. It's a beautiful thing. How has that impacted the decades of people in computer science, and now that's being commercialized? How would you describe this major shift?

[1:38] Boy, yeah, there's a lot to unpack there. I think the whole field has shifted, right? A lot of times in the history of science, fields shift from first trying out things and not knowing how things really work, and eventually, when you figure out enough of the foundational principles, they become more and more of an engineering science. AI, computer vision, natural language processing certainly went through that transition, from "we're trying out a lot of things, there are a lot of manual processes, manual experts." But the writing was on the wall even way back in the day, when IBM said, every time I fire a linguist my accuracy goes up, there's a famous quote, and they used more statistics. Well, statistics have now been replaced by even more powerful models, namely neural networks. And so neural networks need a lot of raw data with the overall reward and the goals to predict. They need a lot of compute. But they also needed a lot of fine-tuning and improvements on the algorithms. The foundations were there. We laid some of them, some of them were laid in the '50s in the early days, Rosenblatt's perceptrons and so on. But then the really important work of scaling laws and trying to start scaling things up started in the 2010s. And now we're seeing the fruits of that, not just chatbots but agents and everything else.

[2:51] You know, I was doing a theCUBE interview, I think it was an AMD event in San Francisco last month, and Dave Vellante and I were talking about the future, and like the UIs change, obviously, it's prompt-based, but the machines are key, and I said something I want to get your reaction to, because we riffed on it: the winners will be the ones who can adapt and build infrastructure and software the way we think and work and play, humans. So if you look at neural networks, a lot of that kind of concept of, hey, software and the environment should be aligned with how I think and how I work or how I play. A lot of the biology references are coming in, right? You're starting to see, you know, the brain of the company is your moat. How would you frame that in today's world? Because this is not an IT thing. It's not a mechanism. It's like biology in a way. It's like these are living things when you get these networks. What is your reaction to that?

[3:47] I think we can be inspired by biology, but we can also not be hindered or held back by pushing that analogy too far. It's kind of similar to birds. We can be inspired by flight, but we don't want to fly planes like birds with flapping wings. And so similarly there are some inspirations of neural networks in our brains, but we don't, for instance, model any of the chemistry that happens in the human brain. And certainly we learn very differently, and in some cases we can do things much faster, more scalable. Obviously models can now translate more languages than any single human can. They can have a very spiky intelligence that spikes around certain skills but maybe is not as smooth in general as human intelligence still is. But overall it alludes to learning algorithms that could still do better. So one thing that we're very inspired by at Recursive, for instance, is the open-endedness of both biological and technological evolution: finding interestingly different ideas, recombining them. And the intelligence of humanity is also not just an individual intelligence, it's a swarm intelligence. There are multiple people and we often stand on the shoulders of giants on past inventions. If you take one human, you drop them into a forest, they wouldn't be able to build a toaster, right? It's pretty complex to build all the infrastructure to make a toaster work. But together with past inventions, in this history and evolution of culture and technology, we actually get to build amazing civilizations.

[5:26] I like the swarm. I like the swarm intelligence. There's also another term that's been bantered around maybe a couple decades ago, collective intelligence. We saw that in personalization and predictive analytics. What's the difference? Because swarm seems more actionable, like swarming on something is situational, but then you also have all this AI. How would you talk about that in terms of first principles, because you're seeing a generation of engineers commercializing the technology that once was maybe not even a path to commercialization, all that manual labor. Swarm intelligence sounds like a very community-oriented thing, that's open source. Sounds like most go-to-market strategies for companies these days is to swarm with their... so there's a lot going on there, almost as if first principles have shifted.

[6:14] Yeah, there's definitely a shift in how we think about working with AI. I think the biggest shift for most people will be that they're going to be managers of multiple agents. At Recursive, we don't think about headcount. We think about agent count. How much compute, how many agents do you get to delegate different work to? And then how much work can that community of scientists do to do research in AI? And that's how you get to recursive self-improvement loops, where it's AI that can code and is code and can do, hence, AI research.

[6:48] I like the Recursive name because it also has a play on words with recursion, which is a great technique in the AI field. Talk about some of the things that are kind of poised to break out as we get to faster commercialization, new discoveries. What are some of the computer science principles that you think are going to be turbocharged into the spotlight?

[7:11] I think the biggest shift that we've seen every time we replace a manual process with a learned process is that then things will improve and capabilities will increase. And so right now we're thinking about the most fundamental process of doing science, and that is the ideation, implementation and validation of ideas. And when we can automate that, we can build what I call the Eureka Machine in my book too, which is the ultimate invention-generating system that will basically come up with all kinds of innovations for us afterwards.

[7:45] I want to get into it. So, well, one more question before we get into the book. I just wrote a post on this. In all big markets, the constraints are where the money goes. Well, first the engineers go, the entrepreneurs go, and then the money follows. How would you describe the constraints in today's technology market, as the infrastructure is going to power and accelerate a lot of new capabilities, agents, robotics going to be another, how society operates potentially? What are the constraints that need to be pressure-pointed?

[8:17] One of the biggest constraints will continue to be compute for quite some time. We as humanity will almost have to decide which problems do we want to solve and hence give the most compute and resources to. And then of course, with compute and the right models and simulations and verifiers comes more data. But you can more and more generate data also in synthetic ways with more compute.

[8:46] So let's get into the book, Eureka. I think there's going to be some energy things too, and obviously security. I want to get into your book because it's about why AI is key to unlocking a new era of scientific discoveries. Peter Thiel wrote a book, Zero to One, that a lot of people have read. There's one point in the book where he talks about how most people solve one of three problems: the easy problems, the very difficult but attainable problems, and then the impossible. And he goes on to say people's anxiety, they try to do something impossible and they can never get there. We are seeing breakthroughs with AI that were once impossible. So I have to ask you, that obviously reframes the conversation, and then what's impossible? So that's the laws of physics, but take me through your thoughts on that and then we'll get into the book: problems that couldn't be solved that are now solvable. What are some of those?

[9:40] There are so many, and I talk about everything from physics, chemistry, biology, medicine, neuroscience, economics, astrophysics and so on in the book. I'll maybe choose one that is particularly exciting, and that is preclinical biology. I think biology will move from a natural science to an engineering science. It will become programmable. We don't just study what nature does but how can we improve it? How can we modify it? How can we live longer, healthier lives? How can we cure many different diseases? In many ways, when you think about humanity and some of our biggest ailments, like bacteria and viruses, bacteria were pretty incredibly resolved as a major core reason of death for a lot of people with antibiotics. Viruses are much, much harder, but I think with AI we can and will eventually cure many viruses the way we've basically gotten rid of the plague and all the other bacterial infections that plagued humanity. So I think likewise cancer, which is kind of a trope, but I think is correctly identified as one of the many things that AI will help us solve, and then eventually it will get to longevity: instead of sick care, we'll do actual healthcare.

[10:53] You know, one of the constraints in health and biology has been compute. You mentioned that earlier, again, a constrained resource. Now they have supercomputing and you don't have to wait for it, or like the time-sharing high-performance computing system, they're affordable.

[11:08] Biology is a good example where we also actually need more data. Like in natural language processing, we have the whole internet and so we could solve, to a large degree, most of the really hard problems in natural language processing just by having enough data on the internet. We don't have enough data for biology. They're incredibly complex systems. But if we can get what's called perturbation studies, when you use, for instance, companies like Parallel Bio that create small organoids in petri dishes that you can analyze and skip animal trials, when you combine that with Tahoe Therapeutics, who do these perturbation studies where you turn off one gene and you look at how a cell interacts. If you do enough of those, eventually you will be able to build a virtual model of a cell. And once you have a simulation, now AI can create billions and billions of experiments and learn and find new cures.

[11:56] The virtualization angle is actually interesting because, you know, I was talking to someone here on theCUBE that was a spin-out from Google Moonshot. They're building a frontier model for biology, right? You know, materials. Very exciting. I mean, you can't crawl the internet for that.

[12:11] That's right.

[12:12] There's also real-world data, too. There's also computer vision, as well as just getting sampling. How far along are we, if you had to peg the progress bar on getting more data that's not crawlable or searchable? Not even game time yet. How would you peg the progress? Good. Just getting started, figuring it out.

[12:33] We're in the early days in biology. I mean, you can now download hundreds of millions of protein sequences, for instance, and one of our companies, Profluent, that came from research, ProGen, protein generation, that I did with Ali Madani, the first author, back in Salesforce days, was creating large language models on generating proteins. That started in 2018. Just a few months ago, Eli Lilly signed a $2 billion-plus contract with them across multiple milestones to actually bring that into real clinical practice. And so these things in biology take longer than pure software, because you have to have human trials and there is reasonable regulation for that.

[13:10] I'll try to pivot from biology into a SaaSpocalypse question, because you mentioned Salesforce, they had their earnings. I wrote a post on this. Like, hey, you know, they have moats, yeah, maybe the user interface goes away. They did a deal with Anthropic, which they invested in. We as a species, we're survivors. So if you're a SaaS company, this is the number one question I get, and bad software always dies. That's what I said once. Okay, you have bad software, it's just going to die. But if you have a competitive person, competitive organization, you can move with the times. What's your take on the whole SaaS apocalypse, and what advice would you give folks, knowing that they can adapt, and what does that adaptation look like?

[13:49] Yeah, it's really interesting. In AI, people are sometimes pulling to the fringes, like it will kill us all, or it will cure all diseases next year, and the truth is always in between and it's complicated, and you have to stay competitive. I felt like the market kind of put all the pressure on just Salesforce to resolve all of the AI questions in software, and it made no sense to them. Salesforce, I mean, I used to be the chief scientist there. We built, starting with Einstein, a ton of really exciting technology. We invented prompt engineering, which was cited multiple times in the early GPT papers. That organization is building agents and they've got Slack as the ultimate nervous system.

[14:28] Slack is a great way to interact with agents.

[14:29] So it made no sense, and it makes sense that now the stock's coming [unclear].

[14:33] All right. So is there a Eureka Machine for corporations? Because I've been saying on theCUBE, because there's no other way to describe it, every company should have their own brain, right? Almost like a living organization for their people that are working there.

[14:46] That's right. And they should contribute to that collective swarm intelligence when they need to, as more and more companies will want to use open-source models and then work with companies like Recursive to make them as efficient as possible, have the fastest, most efficient inference, and then optimize those models for their own use cases and know that because it's open source no one will ever be able to take it away from them.

[15:09] What's your advice to folks that want to work with Recursive, that want to have specialty intelligence? Because a lot of people are scratching their heads, my friends too are saying, hey, I don't really want to be a frontier player. I'm not going to be big. I'm not going to have a zillion GPUs, but I'm going to have some compute. I might have some engineered systems that look like a supercomputer for me, but I want specialized intelligence. I have my own data. What do I do with it? How would they interface? What should they do? How do they take the next steps, in your opinion?

[15:38] Yeah, I think there are a few building blocks that just don't make sense for every company to build themselves. One is, take a model and get really high-quality tokens from that model. Then, take a search engine of the web, you don't want to replicate all of that. That's what You.com provides for companies. It turns out when you give different LLMs, different large language models and chatbots, access to search, they all become more similar, because they have access to the same knowledge now through the search engine. And then you take your own proprietary data and infuse that into that system that you know has all the public data, but also you can update your own tokens, update your own model, and then have much cheaper inference, and you also don't have to worry that anyone learns from your company internally.

[16:21] Okay. I'll ask you, because this is something I publicly talked about, but you know, we've been doing theCUBE for 17 years, very open-source concept, free access, good community. I was just reading a story that AI is going to start building media and videos. So, you're like the Boston Consulting Group here. What's the Eureka Machine? What do we do? I mean, hell, if we're going to be replaced, what would be your advice? Because I need a Eureka Machine. I think everyone will control the data. I mean, what do we do?

[16:50] Anyone will benefit from a Eureka Machine to solve the hardest problems in their space. And I don't want to get into it too much, but it's kind of like the machine is similar to having 50,000 PhDs in AI work on your problem 24/7 until they make enough inventions to solve a particular problem for you. How a company then uses that in their organization is up to them.

[17:13] So what's your advice? Build a Eureka Machine first before the competition. [laughter]

[17:16] Of course. It's hard to build it from scratch, just like it doesn't make sense to build a web index or pre-train your own large language model, but you can then have your own LLM, have your own chatbot, your own AI, your own agents, and have them efficiently run.

[17:32] Yeah. All right. So, final question for you, Eureka Machine, what do you hope people get out of this? What was the motivation? What's the takeaway for the book?

[17:40] I think there's a lot of negativity around AI, and you know, I've seen it since 2003 when I joined the field and started studying it and then started doing research, and it's a little bit sad to see so much negativity in a field that will have so much positive impact. And I feel like we're not focused enough on all the positive impacts, in physics for better fusion reactors, cheaper energy, sustainable energy, and better materials, new batteries, new cures for a variety of diseases. There are so many positive applications. So I hope people read this book and realize, just like science has progressed humanity forward, AI will help in doing that too.

[18:20] Well, it's a great mission. We totally support it. Thanks for coming on theCUBE. I'm John Furrier. Yes, there are so many positive benefits of AI. It's been politicized. I mean, they're protesting data centers in areas where they need job creation. I don't know what's going on, but there's a lot of... again, society, research, but also biology. The next cure is out there. So again, don't, people are freaking out. Don't do it. We're doing our part to bring you the data here in theCUBE. Thanks for watching.