How Rhoda AI Trains Robots With Internet Video | Jagdeep Singh
Automated Podcast 59:07
841 views · 21 likes Watch on YouTube ↗
Rhoda AI trains its robot model on hundreds of millions of internet videos before teaching it specific tasks. CEO Jagdeep Singh says that foundation has helped the company learn jobs with just 10–20 hours of robot data.
In this episode of Automated, Brian Heater speaks with Jagdeep, co-founder and CEO of Rhoda AI, about how video pretraining could help physical AI handle the variability of factories, warehouses and other customer environments.
Jagdeep explains why he believes vision-language-action models face a data ceiling. Teleoperation can provide useful examples, but collecting enough diverse robot experience is difficult. Rhoda’s Direct Video Action approach starts with a different source: videos that capture how objects move and how the physical world behaves.
They discuss why broader footage can be more useful than a tightly curated collection of people performing one job. Pretraining builds a foundation of physical knowledge. Post-training teaches the task. Deployment then exposes the edge cases that can help improve the model further.
Jagdeep shares what happened when a container-folding task that worked in the lab met damaged boxes and rusted equipment in a factory. He says Rhoda relearned the task with roughly three hours of additional data, showing why adapting to actual working conditions is essential.
Brian and Jagdeep also explore Rhoda’s hardware plans. The company is developing mobile manipulators, custom actuators, tactile sensing and dexterous hands around requirements for payload capacity, reliability, safety and dexterity. Jagdeep explains the choice of wheels, the possible advantages of three-finger hands, and how hardware control affects the AI model.
The conversation also covers customer data ownership, learning across deployments, human demonstration following, and the economics of remote intervention.
For Jagdeep, the distinction between a compelling demo and a useful product comes down to repeatability. A customer needs a robot that succeeds consistently and works quickly enough to meet production requirements. He explains why success rate and cycle time guide Rhoda’s approach to deployment.
KEY MOMENTS
00:00 The robot data problem and Rhoda’s approach
01:43 Introducing Jagdeep Singh and Rhoda AI
02:48 How a serial founder chooses the next company
04:09 What changed to make physical AI possible
08:25 An outsider’s perspective on robot training data
12:18 Four things Jagdeep looks for in a company
16:53 Moving from early pilots to customer ROI
19:15 Learning from edge cases in deployment
21:23 Building one model that learns across customers
22:33 Customer data ownership and model training
23:49 Why less task-specific video can work better
28:14 Three stages of training a robot model
29:42 Taking container folding from the lab to a factory
30:10 Adapting with roughly three hours of factory data
32:25 Four requirements driving Rhoda’s hardware
35:16 What Rhoda builds and what it buys
37:18 Three-finger hands and the reliability tradeoff
39:17 Developing actuators and tactile sensing
41:16 How hardware control affects AI performance
42:53 Why Rhoda’s robots use wheels
44:26 Emergency stops in production environments
45:24 Deploying the model before the custom hardware
49:04 Deciding when a product is ready to show
50:23 Success rate, cycle time and customer expectations
52:02 The gap between demonstrations and deployment
54:03 Defining generalist robotic manipulation
55:59 Video pretraining and human demonstration following
58:20 Why task-specific post-training uses teleoperation
Connect with Jagdeep Singh
https://www.linkedin.com/in/jagdeep-singh-3b2a9222
Learn more about Rhoda AI
https://www.rhoda.ai/
Explore Rhoda AI’s research
https://www.rhoda.ai/research/direct-video-action
We’d love to hear from you. Have thoughts or guest suggestions?
Reach us at podcast@automate.org.
You can find the transcript and more episodes of Automated at automated.fm.
Unlock full access to Automated and explore everything automation.
Subscribe today and leave a review on YouTube, Apple Podcasts, Spotify, and the Automated Newsletter.
https://www.youtube.com/@automatedpodcast
https://podcasts.apple.com/us/podcast/automated-with-brian-heater/id1837762221
https://open.spotify.com/show/60olq6brlBEIJWggx2fMR6
https://www.automate.org/automation/automated-newsletter
You can also find us on:
LinkedIn https://www.linkedin.com/showcase/automated-podcast-by-a3/
Instagram https://www.instagram.com/automatedpod/
In this episode of Automated, Brian Heater speaks with Jagdeep, co-founder and CEO of Rhoda AI, about how video pretraining could help physical AI handle the variability of factories, warehouses and other customer environments.
Jagdeep explains why he believes vision-language-action models face a data ceiling. Teleoperation can provide useful examples, but collecting enough diverse robot experience is difficult. Rhoda’s Direct Video Action approach starts with a different source: videos that capture how objects move and how the physical world behaves.
They discuss why broader footage can be more useful than a tightly curated collection of people performing one job. Pretraining builds a foundation of physical knowledge. Post-training teaches the task. Deployment then exposes the edge cases that can help improve the model further.
Jagdeep shares what happened when a container-folding task that worked in the lab met damaged boxes and rusted equipment in a factory. He says Rhoda relearned the task with roughly three hours of additional data, showing why adapting to actual working conditions is essential.
Brian and Jagdeep also explore Rhoda’s hardware plans. The company is developing mobile manipulators, custom actuators, tactile sensing and dexterous hands around requirements for payload capacity, reliability, safety and dexterity. Jagdeep explains the choice of wheels, the possible advantages of three-finger hands, and how hardware control affects the AI model.
The conversation also covers customer data ownership, learning across deployments, human demonstration following, and the economics of remote intervention.
For Jagdeep, the distinction between a compelling demo and a useful product comes down to repeatability. A customer needs a robot that succeeds consistently and works quickly enough to meet production requirements. He explains why success rate and cycle time guide Rhoda’s approach to deployment.
KEY MOMENTS
00:00 The robot data problem and Rhoda’s approach
01:43 Introducing Jagdeep Singh and Rhoda AI
02:48 How a serial founder chooses the next company
04:09 What changed to make physical AI possible
08:25 An outsider’s perspective on robot training data
12:18 Four things Jagdeep looks for in a company
16:53 Moving from early pilots to customer ROI
19:15 Learning from edge cases in deployment
21:23 Building one model that learns across customers
22:33 Customer data ownership and model training
23:49 Why less task-specific video can work better
28:14 Three stages of training a robot model
29:42 Taking container folding from the lab to a factory
30:10 Adapting with roughly three hours of factory data
32:25 Four requirements driving Rhoda’s hardware
35:16 What Rhoda builds and what it buys
37:18 Three-finger hands and the reliability tradeoff
39:17 Developing actuators and tactile sensing
41:16 How hardware control affects AI performance
42:53 Why Rhoda’s robots use wheels
44:26 Emergency stops in production environments
45:24 Deploying the model before the custom hardware
49:04 Deciding when a product is ready to show
50:23 Success rate, cycle time and customer expectations
52:02 The gap between demonstrations and deployment
54:03 Defining generalist robotic manipulation
55:59 Video pretraining and human demonstration following
58:20 Why task-specific post-training uses teleoperation
Connect with Jagdeep Singh
https://www.linkedin.com/in/jagdeep-singh-3b2a9222
Learn more about Rhoda AI
https://www.rhoda.ai/
Explore Rhoda AI’s research
https://www.rhoda.ai/research/direct-video-action
We’d love to hear from you. Have thoughts or guest suggestions?
Reach us at podcast@automate.org.
You can find the transcript and more episodes of Automated at automated.fm.
Unlock full access to Automated and explore everything automation.
Subscribe today and leave a review on YouTube, Apple Podcasts, Spotify, and the Automated Newsletter.
https://www.youtube.com/@automatedpodcast
https://podcasts.apple.com/us/podcast/automated-with-brian-heater/id1837762221
https://open.spotify.com/show/60olq6brlBEIJWggx2fMR6
https://www.automate.org/automation/automated-newsletter
You can also find us on:
LinkedIn https://www.linkedin.com/showcase/automated-podcast-by-a3/
Instagram https://www.instagram.com/automatedpod/
Category (YouTube): Science & Technology
Playback is via YouTube's official embedded player. Data from YouTube; Exumo is not affiliated with YouTube.