Tony Zhao on Training Home Robots With Human Data
Automated Podcast 54:25
1,371 views · 21 likes Watch on YouTube ↗
How can a home robot learn household chores without collecting all of its training data from expensive robot fleets?
Sunday Robotics co-founder and CEO Tony Zhao joins Brian Heater to explain how Memo learns from human data captured through the Skill Capture Glove, and why this approach could help physical AI scale.
Tony traces the idea back to ALOHA and the Universal Manipulation Interface, two research efforts that changed how he thought about robot learning. More data and compute were making difficult manipulation possible, but robotics still lacked an internet-scale source of physical experience. A wearable device that closely matches the robot hand offered another path.
The conversation explores how smarter AI can compensate for simpler, lower-cost hardware, why manufacturing and model development have to be designed together, and how Memo could move from an early-adopter product toward the price of a gaming PC or phone as its capabilities grow.
Brian and Tony also discuss a surprising product decision: speed is not the first priority for a home robot. A machine can take hours to finish dishes, laundry, or tidying while no one is waiting on it. Reliability, safety, and the ability to complete more kinds of work matter more.
They also examine the gap between viral robot demos and repeatable deployment. Tony argues that the real test is not whether a robot can perform one impressive task on camera. It is whether the same system can work in unseen homes, with unseen objects, without a team of engineers standing nearby.
Finally, Tony explains why Sunday is starting in homes while much of the robotics industry begins in factories. His thesis is that the variability of the home creates research-market fit: the product only becomes useful by developing the same broad, adaptable intelligence the research is trying to achieve.
KEY MOMENTS
00:00 Why robot training data does not have to come from robots
01:06 Welcome to Automated
03:18 Telling his parents he was leaving Stanford
05:54 Going all in with no academic fallback
07:21 Bringing language-model scaling laws into robotics
09:07 What ALOHA revealed about dexterity and generalization
10:30 The UMI research that created Sunday's why-now
12:53 Why the glove and robot hand need to match
15:15 The General Magic warning about building too much too soon
16:56 How smarter AI can compensate for simpler hardware
20:09 Pricing a general-purpose home robot
23:36 Why early customer trust matters
25:25 Why Memo does not need to work at human speed
30:10 The gap between a robot demo and real deployment
33:54 Combining technical approaches and owning the full stack
37:44 The painful dependency tree behind a robotics startup
39:06 Meeting co-founder Cheng Chi on Twitter
42:21 Why Sunday started by building the glove
45:22 Research-market fit and the case for the home
49:43 Comparing deployment across homes and factories
51:44 How home robots could eventually move into industry
Connect with Tony Zhao
https://www.linkedin.com/in/tony-z-zhao
Learn more about Sunday Robotics
https://www.sunday.ai/
Apply for the Memo beta program
https://www.sunday.ai/beta-program
Explore ALOHA
https://tonyzhaozh.github.io/aloha/
Explore the Universal Manipulation Interface
https://umi-gripper.github.io/
Learn more about ACT-2
https://www.sunday.ai/blog/act-2-preview
We’d love to hear from you. Have thoughts or guest suggestions?
Reach us at podcast@automate.org
You can find the transcript and more episodes of Automated at https://automated.fm
Unlock full access to Automated and explore everything automation.
Subscribe today and leave a review on YouTube, Apple Podcasts, Spotify, and the Automated Newsletter.
https://www.youtube.com/@automatedpodcast
https://podcasts.apple.com/us/podcast/automated-with-brian-heater/id1837762221
https://open.spotify.com/show/60olq6brlBEIJWggx2fMR6
https://www.automate.org/automation/automated-newsletter
You can also find us on:
LinkedIn
https://www.linkedin.com/showcase/automated-podcast-by-a3/
Instagram
https://www.instagram.com/automatedpod/
Sunday Robotics co-founder and CEO Tony Zhao joins Brian Heater to explain how Memo learns from human data captured through the Skill Capture Glove, and why this approach could help physical AI scale.
Tony traces the idea back to ALOHA and the Universal Manipulation Interface, two research efforts that changed how he thought about robot learning. More data and compute were making difficult manipulation possible, but robotics still lacked an internet-scale source of physical experience. A wearable device that closely matches the robot hand offered another path.
The conversation explores how smarter AI can compensate for simpler, lower-cost hardware, why manufacturing and model development have to be designed together, and how Memo could move from an early-adopter product toward the price of a gaming PC or phone as its capabilities grow.
Brian and Tony also discuss a surprising product decision: speed is not the first priority for a home robot. A machine can take hours to finish dishes, laundry, or tidying while no one is waiting on it. Reliability, safety, and the ability to complete more kinds of work matter more.
They also examine the gap between viral robot demos and repeatable deployment. Tony argues that the real test is not whether a robot can perform one impressive task on camera. It is whether the same system can work in unseen homes, with unseen objects, without a team of engineers standing nearby.
Finally, Tony explains why Sunday is starting in homes while much of the robotics industry begins in factories. His thesis is that the variability of the home creates research-market fit: the product only becomes useful by developing the same broad, adaptable intelligence the research is trying to achieve.
KEY MOMENTS
00:00 Why robot training data does not have to come from robots
01:06 Welcome to Automated
03:18 Telling his parents he was leaving Stanford
05:54 Going all in with no academic fallback
07:21 Bringing language-model scaling laws into robotics
09:07 What ALOHA revealed about dexterity and generalization
10:30 The UMI research that created Sunday's why-now
12:53 Why the glove and robot hand need to match
15:15 The General Magic warning about building too much too soon
16:56 How smarter AI can compensate for simpler hardware
20:09 Pricing a general-purpose home robot
23:36 Why early customer trust matters
25:25 Why Memo does not need to work at human speed
30:10 The gap between a robot demo and real deployment
33:54 Combining technical approaches and owning the full stack
37:44 The painful dependency tree behind a robotics startup
39:06 Meeting co-founder Cheng Chi on Twitter
42:21 Why Sunday started by building the glove
45:22 Research-market fit and the case for the home
49:43 Comparing deployment across homes and factories
51:44 How home robots could eventually move into industry
Connect with Tony Zhao
https://www.linkedin.com/in/tony-z-zhao
Learn more about Sunday Robotics
https://www.sunday.ai/
Apply for the Memo beta program
https://www.sunday.ai/beta-program
Explore ALOHA
https://tonyzhaozh.github.io/aloha/
Explore the Universal Manipulation Interface
https://umi-gripper.github.io/
Learn more about ACT-2
https://www.sunday.ai/blog/act-2-preview
We’d love to hear from you. Have thoughts or guest suggestions?
Reach us at podcast@automate.org
You can find the transcript and more episodes of Automated at https://automated.fm
Unlock full access to Automated and explore everything automation.
Subscribe today and leave a review on YouTube, Apple Podcasts, Spotify, and the Automated Newsletter.
https://www.youtube.com/@automatedpodcast
https://podcasts.apple.com/us/podcast/automated-with-brian-heater/id1837762221
https://open.spotify.com/show/60olq6brlBEIJWggx2fMR6
https://www.automate.org/automation/automated-newsletter
You can also find us on:
https://www.linkedin.com/showcase/automated-podcast-by-a3/
https://www.instagram.com/automatedpod/
Category (YouTube): Science & Technology
Playback is via YouTube's official embedded player. Data from YouTube; Exumo is not affiliated with YouTube.