Skip to content

Physical AI Playbook

Last updated: 2026-09 · owner: Youngjin · volatility: medium

L0 TL;DR: A guide to deciding which technology fits a customer's robotics task and moving through AWS experiments, evaluation, and operations. Start with business fit, total cost, and execution requirements.

Unofficial reference

This personal reference is not official AWS documentation or an AWS position. Check linked primary sources and dates for technology, licenses, pricing, and Regions. Sample code and service GA do not guarantee customer-site outcomes.

Start by role

I am… Path Output
Customer decision maker Start/ROI → Executive Brief Task, alternatives, budget, pilot approval conditions
Customer engineer Execution → relevant pillar → Operations Results, cost, deployment/recovery evidence
AWS staff Conversation guide → Decisions Discovery, fit assessment, partner handover

Technical reference — five pillars

Pillar Coverage
P1 Data Collection, rights, formats, quality, training pipeline
P2 Training Model selection, licensing, sizing, evaluation
P3 Simulation Environments, tools, parallel execution, costs
P4 Sim-to-Real Physical transfer, edge deployment, validation, safety
P5 Orchestration Business planning, robot skills, permissions, fleet connectivity

Labels and verification scope

GA/Preview/Research describe release status. Check source comparison, reproduction, site validation, intended use, and support separately. [1] official documents/papers, [2] recorded reproduction, [3] vendor announcements, [4] unverified are source types, not AWS endorsement. See Evidence and Maintenance.

The questions below are discovery examples, not measured inquiry-frequency rankings. Start with a question, then check the pilot card before making a proposal.

Top 20 Frequently Asked Questions

# Question Where to Source
1 "How do I run Isaac Sim / Isaac Lab on AWS?" pillar-3 seed ⚠️
2 "How should I set up the infrastructure for VLA model training (fine-tuning)?" pillar-2 seed ⚠️
3 "I can't get GPUs — should I use On-Demand, Capacity Blocks, or an alternative?" decisions seed ⚠️
4 "How do you actually overcome the sim-to-real1 gap? Are there proven methods?" pillar-4 seed ⚠️
5 "It's real-time robot control (30–100 Hz) — can I put inference in the cloud?" decisions seed ⚠️
6 "Should I fine-tune a foundation model (GR00T/π0, etc.) or train my own?" decisions seed ⚠️
7 "How do I collect robot learning data and where should I store it? (teleoperation / synthetic data)" pillar-1 seed ⚠️
8 "How locked in am I to the NVIDIA full stack? What about open-source alternatives?" decisions seed ⚠️
9 "How do I connect edge deployment (Jetson, etc.) with AWS?" pillar-4 seed ⚠️
10 "Does an architecture where an LLM agent2 directs robots/equipment actually work?" pillar-5 seed ⚠️
11 "How much will all this GPU compute cost? How do I budget for it?" start AWS Embodied AI blog
12 "How do I connect my existing ROS 23 stack / rosbag4 data to AWS?" pillar-1 AWS ROS 2 on Isaac blog
13 "How do I scale training across multiple nodes? AWS Batch vs SageMaker HyperPod?" pillar-2 Isaac Lab on SageMaker
14 "How do I validate/benchmark whether a policy actually works before real deployment?" pillar-4 NVIDIA policy evaluation
15 "Our robot/factory data is sensitive — is cloud training OK for compliance? On-prem/hybrid?" decisions AWS AI sovereignty
16 "How do I version, reproduce, and recover checkpoints for trained policies?" pillar-2 Isaac Lab on SageMaker
17 "Can I use Isaac Sim / open models in a commercial product? When do I need NVIDIA AI Enterprise?" pillar-3 NVIDIA Isaac Sim
18 "How do I optimize policy inference for real-time (low latency)? TensorRT / quantization5 / action chunking6?" pillar-4 NVIDIA Jetson Edge AI
19 "How do I build an equipment/factory digital twin7 and connect it to robot simulation? TwinMaker / Omniverse?" pillar-3 AWS Physical AI blog
20 "We have no ML experts — where do we start? How to design a minimal PoC?" start AWS Physical AI blog

Page list

Get Started

Technical Pillars

Executives

News & Resources

Reference

owner: Youngjin · updated: 2026-09 · volatility: medium

  1. sim-to-real — transferring a policy trained in simulation to a real robot, or the methodology for doing so. The physical and visual differences between simulation and reality (the domain gap) mean a naive transfer collapses performance. 🎥 NVIDIA sim-to-real robotics showcase ↩

  2. LLM agent — software in which a large language model plans on its own, selects and calls tools (APIs, robot skills), and carries out multi-step tasks. Unlike simple Q&A, the key point is that it "acts." ↩

  3. ROS 2 (Robot Operating System 2) — the de facto standard open-source middleware for robot software. A distributed architecture in which sensor and control nodes communicate over topics; the shared foundation of industrial and research robot stacks. ↩

  4. ROS bag (rosbag2) — the standard log format in which the robot operating system ROS 2 records topics (sensor/command streams) wholesale. The de facto default form of robot companies' raw data, but it cannot be used for training as-is and requires conversion. ↩

  5. Quantization — a lightweighting technique that converts model weights and operations to lower precision, e.g. FP16→INT8/FP4, cutting memory and compute. A key means of meeting latency budgets on edge devices, managed as a trade-off against accuracy loss. ↩

  6. Action chunking — Generating several future actions per inference. Execution rate differs from response to fresh observations; validate horizons, transitions, and latency per model. ↩

  7. Digital twin — A physically faithful virtual replica of a real factory, warehouse, or robot. Enables policy training, validation, and scenario experiments without touching the real environment. ↩