I build and train autonomous AI — agents, products, robots.
Bestiary
Legged robots · Solo build
Robots I designed, then taught to walk — and a harness that catches the ones that cheat.
- MuJoCo · Isaac Lab
- SAC · PPO · Transformers
I re‑derived the reward for the morphology, then built the harness that catches the cheaters.
Wheels and legs want different rewards — the standard quadruped recipe quietly collapses on a wheel‑legged body. Custom robots designed, built and trained in simulation, with degenerate policies killed before they reach a highlight reel.
github.com/leoi137/Bestiary
ntp_spot_s0
Spot · next-token imitation
Causal transformer · PyTorch · 12 DoF · 20K steps
A pretrained walking policy recorded to tape, then a transformer trained from scratch to predict the next entry — no reward, no simulator in the loop, no exploration. It drives the robot closed-loop. An adversarial pass then cut the claim down to what the experiment can actually separate.
Model
Causal transformer · 8 pre-LN blocks (PyTorch) · 25,315,388 parameters
NTPModel(
(obs_in): Linear(in_features=48, out_features=512, bias=True)
(act_in): Linear(in_features=12, out_features=512, bias=True)
(pos): Embedding(64, 512)
(blocks): ModuleList(
(0-7): 8 x Block(
(ln1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
(qkv): Linear(in_features=512, out_features=1536, bias=True)
(proj): Linear(in_features=512, out_features=512, bias=True)
(ln2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
(mlp): Sequential(
(0): Linear(in_features=512, out_features=2048, bias=True)
(1): GELU(approximate='none')
(2): Linear(in_features=2048, out_features=512, bias=True)
)
)
)
(ln_f): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
(act_head): Linear(in_features=512, out_features=12, bias=True)
(obs_head): Linear(in_features=512, out_features=48, bias=True)
)
Two input projections because the sequence has two modalities: the rollout is written as an interleaved diary o₀, a₀, o₁, a₁, … and 64 learned positions cover 32 timesteps of it — 0.64 s at 50 Hz. Causal attention, so a position can only read its own past. `obs_head` is the second job: predicting what the world does next costs one linear layer and is what makes this a dynamics model as well as a policy.
Observation
Box (48,) — the teacher’s own observation, unaltered
- [0:3] base_lin_vel (3)
- torso velocity in its own frame — m/s
- [3:6] base_ang_vel (3)
- roll, pitch, yaw rate — rad/s
- [6:9] projected_gravity (3)
- gravity direction in the body frame
- [9:12] command (3)
- v_x, v_y, ω_z — the joystick
- [12:24] joint_pos (12)
- angle minus default pose — rad
- [24:36] joint_vel (12)
- joint rates minus default — rad/s
- [36:48] actions (12)
- last step’s raw network output
Blind — 48 proprioceptive numbers and no camera. Note the last block: the previous action sits verbatim in [36:48], so "copy what you did last step" is a non-learning baseline the model has to beat from inside its own input. It scores 0.2257, which is the number the result is measured against.
Action
Box (12,) — raw joint-position offsets
The teacher’s uncut network output, stored exactly as it crossed the interface. The controller applies target = default_pose + 0.2 × action (rad); the 0.2 and the default pose live in the episode metadata, so a training pipeline that bakes either one in has broken the dataset contract rather than simplified it.
Loss
- MSE(action) ×1.0
- the act head at observation positions — the policy
- MSE(next obs) ×0.5
- the obs head at action positions — the dynamics model
Splits: Holdout is decided at record time by seed (seed % 10 == 0) and never opened by training, including for normalization statistics. Validation is whole episodes, so val loss measures cross-episode generalization. Falls are quarantined rather than deleted — imitating a fall teaches falling, and the tape is still evidence.
There is no reward function. Its role is played by the loss, and the teacher’s reward already shaped the data — this model never sees one. No environment interaction and no exploration either: every timestep in the tape already carries its own correct answer, which is why 11.5 minutes on one GPU is the whole training budget.
Result
- Closed-loop
- 12/12 survived · 7.223 m vs the teacher’s 7.215 m
- Best val loss
- 0.0013 — 169× the copy-last-action baseline
- What survives refutation
- walks from tapes, single seed. Not “matches the teacher” — that metric is 99.7% determined by which command script ran
Spider-12 · commanded
Isaac Lab · six velocity commands
PPO · Isaac Lab · 12 DoF · 2.06B steps
The steering the forward-only run below it does not have. Same brain to the parameter, a command-tracking reward instead of one term, and then a fine-tune that widened the speed envelope 2.5× from the policy’s own checkpoint.
Model
PPO · rsl_rl ActorCritic · 286,616 actor parameters (235 → 512 → 256 → 128 → 12)
# actor — identical in shape to the forward-only run's
MLPModel(
(obs_normalizer): Identity()
(distribution): GaussianDistribution()
(mlp): MLP(
(0): Linear(in_features=235, out_features=512, bias=True)
(1): ELU(alpha=1.0)
(2): Linear(in_features=512, out_features=256, bias=True)
(3): ELU(alpha=1.0)
(4): Linear(in_features=256, out_features=128, bias=True)
(5): ELU(alpha=1.0)
(6): Linear(in_features=128, out_features=12, bias=True)
)
)
Not a bigger network — the same one. The observation is 235 wide and a one-way door, so a fine-tune that widened it would orphan the checkpoint it loads; the whole run is three numbers moved in a config. Steering was never a capacity problem, it was a reward problem.
Observation
Box (235,) — eight blocks, unchanged from the forward run
- [0:3] base_lin_vel (3)
- torso velocity in its own frame — m/s
- [3:6] base_ang_vel (3)
- roll, pitch, yaw rate — rad/s
- [6:9] projected_gravity (3)
- which way is down: the inner ear
- [9:12] velocity_commands (3)
- the joystick — and this reward finally reads it
- [12:24] joint_pos (12)
- angle minus rest angle — rad
- [24:36] joint_vel (12)
- joint rates — rad/s
- [36:48] actions (12)
- last step’s 12 outputs
- [48:235] height_scan (187)
- 17 × 11 downward rays over a 2.56 × 1.6 m patch
The height scan is the cost of not being allowed to change this list. Its 1.28 m of forward reach was sized as 2.13 s of lookahead at 0.6 m/s; at 1.5 m/s the same rays are 0.85 s. This policy moves 2.5× faster over the same ground while seeing 2.5× less far ahead in time, and that is a real limit of the fine-tune rather than of the reward.
Action
Box (12,) — joint position targets, unclipped
q_cmd = 0.5·a + q_default, then an implicit PD at 200 Hz: τ = 30.0·(q_cmd − q) − 5.5857·q̇, capped at ±40 N·m. One unit of action is 15 N·m, so |a| ≥ 2.667 saturates the drive before physics is consulted.
Reward
- track_lin_vel_xy_exp +1.0
- income — exp kernel on velocity error, std 0.3
- track_ang_vel_z_exp +0.5
- income — exp kernel on yaw-rate error, std 0.4
- action_rate_l2 −0.01
- the penalty a three-rung ladder measured as the winner
- feet_air_time +0.125
- threshold 0.5 s, on the tibias — pays long swings, taxes a shuffle
- lin_vel_z_l2 −2.0
- squared vertical torso speed — the largest weight in the table
Termination: Torso contact with the ground. Truncation at 1000 steps — 20 s at the 50 Hz policy rate.
The fine-tune widened the command box 2.5× and deliberately left the kernel widths alone, so std/v_max fell 0.5 → 0.2 and tracking at speed is graded relatively harder. Restoring the ratio would have been the obvious edit and the wrong one: it pays the machine for keeping its old speed. Two of the eleven terms the upstream recipe ships were deleted for being unmeasured rather than for being wrong, and two more had weight exactly 0.0 — a zero-weight term in a config is a term someone eventually switches on by accident.
Result
- Envelope
- ±1.5 m/s forward · ±0.6 lateral · ±1.5 rad/s yaw
- Verified
- commanded +1.50 m/s, held +1.44…+1.52 over five seconds — including up a hill crest
- Survival
- 933 of 1000 steps — unchanged at 2.5× the speed
- Cost
- 10.7 h · 20,999 iterations · 4,096 envs
- Paid for it
- ground contacts 0.078 → 0.121 · single seed, and a fine-tune, so neither is a finding
Spider-v0
Custom 12-DoF spider
SAC · MuJoCo · 12 DoF · 3.75M steps
The repo’s own environment: four legs of three joints, Ant’s reward recipe re-derived for a body Ant never had. Reward-hacked twice before it walked.
Model
SAC · MlpPolicy (Stable-Baselines3) · 494,364 parameters
SACPolicy(
(actor): Actor(
(features_extractor): FlattenExtractor(
(flatten): Flatten(start_dim=1, end_dim=-1)
)
(latent_pi): Sequential(
(0): Linear(in_features=113, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
)
(mu): Linear(in_features=256, out_features=12, bias=True)
(log_std): Linear(in_features=256, out_features=12, bias=True)
)
(critic): ContinuousCritic(
(qf0): Sequential(
(0): Linear(in_features=125, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
(4): Linear(in_features=256, out_features=1, bias=True)
)
(qf1): Sequential( ... identical to qf0 ... )
)
(critic_target): ContinuousCritic( ... frozen copy of critic ... )
)
Critic input is 125 = 113 observation + 12 action — SAC’s Q takes both. Twin Q-networks and their frozen targets are the other 3/4 of the weights; only the actor ships.
Observation
Box(-inf, inf, (113,))
- [0:17] qpos, less world x,y (17)
- torso height, orientation quaternion, 12 joint angles
- [17:35] qvel (18)
- torso linear + angular velocity, 12 joint rates
- [35:113] cfrc_ext (78)
- contact force/torque on each of the 13 non-world bodies
World x,y are excluded on purpose — walking looks identical anywhere on the plane, so absolute position only invites overfitting to where instead of how.
Action
Box(-1, 1, (12,))
One float per motor, per-leg order [hip, lift, knee] × 4. Sign is rotation direction, magnitude is a fraction of peak torque at gear 40.
Reward
- forward_velocity +1.0
- torso dx/dt, m/s
- healthy +1.0
- alive bonus, paid every step
- ctrl_cost −0.1 · ‖a‖²
- torque bill — Ant’s 0.5 retuned for 12 motors
- contact_cost −5e−4 · ‖clip(cfrc)‖²
- don’t slam the ground
Termination: Torso z leaves [0.2, 3.0], tilt past ~78°, or any state goes non-finite. Truncation at 1000 steps.
The ctrl weight is the whole fix. At Ant’s constants this body learned suicide-by-jumping — dying paid better than walking. 0.1 makes being alive net ~+0.6/step even while flailing, so no reachable termination can out-earn living. Closing that opened a second hole: a cartwheeling gait, correct per the spec, which never said "on your feet". The upright termination is chapter three.
Result
- Best eval
- 7,392
- Gait
- ~6.5 m/s four-legged bound
- Reward
- default (Ant-style + upright termination)
Spider-12
Isaac Lab · forward velocity only
PPO · Isaac Lab · 12 DoF · 147M steps
The same machine ported to GPU and trained with every shaping term deleted. One reward, so whatever it does is attributable to the stack rather than to a reward table.
Model
PPO · rsl_rl ActorCritic · 571,801 parameters (actor 286,616 · critic 285,185)
# actor — 286,616 parameters, loaded strict=True
MLPModel(
(obs_normalizer): Identity()
(distribution): GaussianDistribution()
(mlp): MLP(
(0): Linear(in_features=235, out_features=512, bias=True)
(1): ELU(alpha=1.0)
(2): Linear(in_features=512, out_features=256, bias=True)
(3): ELU(alpha=1.0)
(4): Linear(in_features=256, out_features=128, bias=True)
(5): ELU(alpha=1.0)
(6): Linear(in_features=128, out_features=12, bias=True)
)
)
# critic — 285,185 parameters, loaded strict=True
MLPModel(
(obs_normalizer): Identity()
(mlp): MLP(
(0): Linear(in_features=235, out_features=512, bias=True)
(1): ELU(alpha=1.0)
(2): Linear(in_features=512, out_features=256, bias=True)
(3): ELU(alpha=1.0)
(4): Linear(in_features=256, out_features=128, bias=True)
(5): ELU(alpha=1.0)
(6): Linear(in_features=128, out_features=1, bias=True)
)
)
obs_normalizer is Identity — raw numbers go straight in, joint rates in rad/s sitting next to a unit gravity vector. The Gaussian’s spread is 12 learned scalars that never see the observation; after 1500 iterations they sit at 0.357–0.614. The critic is 49.9% of the weights and none of it ships.
Observation
Box (235,) — eight blocks, declaration order
- [0:3] base_lin_vel (3)
- torso velocity in its own frame — m/s
- [3:6] base_ang_vel (3)
- roll, pitch, yaw rate — rad/s
- [6:9] projected_gravity (3)
- which way is down: the inner ear
- [9:12] velocity_commands (3)
- the joystick — unread by this reward
- [12:24] joint_pos (12)
- angle minus rest angle — rad
- [24:36] joint_vel (12)
- joint rates — rad/s
- [36:48] actions (12)
- last step’s 12 outputs
- [48:235] height_scan (187)
- 17 × 11 downward rays over a 2.56 × 1.6 m patch
The height scan’s footprint is chosen so the ±0.76 m foot centres fall inside it — the policy must never place a foot on ground it cannot see.
Action
Box (12,) — joint position targets, unclipped
q_cmd = 0.5·a + q_default, then an implicit PD at 200 Hz: τ = 30.0·(q_cmd − q) − 5.5857·q̇, capped at ±40 N·m. One unit of action is 15 N·m, so |a| ≥ 2.667 saturates the drive before physics is consulted.
Reward
- forward_velocity +1.0
- v_x in the torso’s own frame, m/s
Termination: Fall resets stay — they are episode machinery, not reward. Nothing else pays or charges.
That is the entire table. RewardManager multiplies by step_dt = 0.02 s, so the per-step reward is metres travelled and the episode return is a distance, not a score: 7.40 means 7.40 m of ground covered in 20 seconds. Body frame, not world frame — "go where your nose points", which is the only version that is learnable on a curriculum that respawns you anywhere.
Result
- Wall clock
- 39 min · 1,500 iterations
- Gait
- 4–6 m/s bounding
- Known limit
- reads no command — holds no heading, drifts as it runs
foot_contact_v1
Ant-v5 · foot-contact shaping
SAC · MuJoCo · 8 DoF · 3.75M steps
Stock Ant never says "use all four legs", so SAC found a two-legged hop that maximised it. One extra reward term buys back a gait that looks quadrupedal.
Model
SAC · MlpPolicy (Stable-Baselines3) · 477,972 parameters
SACPolicy(
(actor): Actor(
(features_extractor): FlattenExtractor(
(flatten): Flatten(start_dim=1, end_dim=-1)
)
(latent_pi): Sequential(
(0): Linear(in_features=105, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
)
(mu): Linear(in_features=256, out_features=8, bias=True)
(log_std): Linear(in_features=256, out_features=8, bias=True)
)
(critic): ContinuousCritic(
(qf0): Sequential(
(0): Linear(in_features=113, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
(4): Linear(in_features=256, out_features=1, bias=True)
)
(qf1): Sequential( ... identical to qf0 ... )
)
(critic_target): ContinuousCritic( ... frozen copy of critic ... )
)
Identical architecture to every other SAC run here — lr 3e-4, buffer 1e6, batch 256. Holding the agent fixed is what makes the shaped and unshaped Ant runs comparable at all.
Observation
Box(-inf, inf, (105,))
- [0:13] qpos[2:] (13)
- torso height, orientation quaternion, 8 joint angles
- [13:27] qvel (14)
- torso linear + angular velocity, 8 joint rates
- [27:105] cfrc_ext[1:] (78)
- contact force/torque on each of the 13 non-world bodies
The wrapper reads cfrc_ext directly rather than off the observation, resolving the four ankle bodies by geom name so an XML edit cannot silently repoint it.
Action
Box(-1, 1, (8,))
Two hinge motors per leg — hip and ankle — as a fraction of peak torque at gear 150.
Reward
- forward_velocity +1.0
- torso dx/dt, m/s
- healthy +1.0
- alive bonus
- ctrl_cost −0.5 · ‖a‖²
- stock Ant weight
- contact_cost −5e−4 · ‖clip(cfrc)‖²
- stock Ant weight
- foot_contact −1.0 × idle legs
- the added term — see below
Termination: Torso z leaves [0.2, 1.0]. Truncation at 1000 steps.
Over a rolling 50-step window, count the ankles whose contact force ever crossed 1.0 N and charge 1.0 per leg that never did. A rolling window, not an instantaneous one: legs are supposed to be airborne during their swing phase, so instantaneous imbalance is just walking. A leg that has not touched ground in fifty steps is the actual bug. The baseline still scores higher in raw reward because it never pays the penalty — the shaped run trades velocity for a gait.
Result
- Best eval
- 5,647 (shaped)
- Gait
- uses all four legs
- Control
- baseline_2leg, same agent, 6,657 unshaped on two legs
walker_baseline
Walker2d-v5 · stock reward
SAC · MuJoCo · 6 DoF · 3.75M steps
A planar biped on the stock reward, no shaping. It is here as a control: a biped cannot move forward on a degenerate gait, so there is no local optimum to shape away.
Model
SAC · MlpPolicy (Stable-Baselines3) · 362,256 parameters
SACPolicy(
(actor): Actor(
(features_extractor): FlattenExtractor(
(flatten): Flatten(start_dim=1, end_dim=-1)
)
(latent_pi): Sequential(
(0): Linear(in_features=17, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
)
(mu): Linear(in_features=256, out_features=6, bias=True)
(log_std): Linear(in_features=256, out_features=6, bias=True)
)
(critic): ContinuousCritic(
(qf0): Sequential(
(0): Linear(in_features=23, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
(4): Linear(in_features=256, out_features=1, bias=True)
)
(qf1): Sequential( ... identical to qf0 ... )
)
(critic_target): ContinuousCritic( ... frozen copy of critic ... )
)
The smallest network on the plate, and the same hyperparameters as the largest. 17 inputs against Humanoid’s 348 — the width of the world is the morphology’s, not the algorithm’s.
Observation
Box(-inf, inf, (17,))
- [0:8] qpos[1:] (8)
- torso height and angle, 6 joint angles
- [8:17] qvel (9)
- torso velocities and 6 joint rates, clipped to ±10
No contact-force block at all. A planar walker feels the ground only through its own posture.
Action
Box(-1, 1, (6,))
Thigh, leg and foot torques on each of the two legs, as a fraction of peak.
Reward
- forward_velocity +1.0
- torso dx/dt, m/s
- healthy +1.0
- alive bonus
- ctrl_cost −0.001 · ‖a‖²
- near-free effort
Termination: Torso z leaves [0.8, 2.0] or torso angle leaves [−1, 1] rad. Truncation at 1000 steps.
The 1e−3 control weight is the Hopper/Walker mechanism the Spider reward later borrows: effort is so nearly free that being alive is net-positive for any policy, which is what makes a termination-seeking exploit unprofitable without having to prove the termination is unreachable.
Result
- Best eval
- 5,944
- Gait
- upright two-legged walk
- Shaping
- none — and none needed
humanoid_baseline
Humanoid-v5 · stock reward
SAC · MuJoCo · 17 DoF · 4M steps
The widest observation on the plate — 348 numbers — on the same hyperparameters as the 17-input walker. It needs no shaping, only more steps.
Model
SAC · MlpPolicy (Stable-Baselines3) · 802,854 parameters
SACPolicy(
(actor): Actor(
(features_extractor): FlattenExtractor(
(flatten): Flatten(start_dim=1, end_dim=-1)
)
(latent_pi): Sequential(
(0): Linear(in_features=348, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
)
(mu): Linear(in_features=256, out_features=17, bias=True)
(log_std): Linear(in_features=256, out_features=17, bias=True)
)
(critic): ContinuousCritic(
(qf0): Sequential(
(0): Linear(in_features=365, out_features=256, bias=True)
(1): ReLU()
(2): Linear(in_features=256, out_features=256, bias=True)
(3): ReLU()
(4): Linear(in_features=256, out_features=1, bias=True)
)
(qf1): Sequential( ... identical to qf0 ... )
)
(critic_target): ContinuousCritic( ... frozen copy of critic ... )
)
Twice the parameters of the Ant policy, and every extra one is in the first layer: 348 inputs × 256 units. Widening an observation is the expensive edit, not deepening a network.
Observation
Box(-inf, inf, (348,))
- [0:22] qpos[2:] (22)
- torso height, orientation, 17 joint angles
- [22:45] qvel (23)
- root and joint velocities
- [45:175] cinert[1:] (130)
- centre-of-mass inertia, 10 per body
- [175:253] cvel[1:] (78)
- centre-of-mass velocity, 6 per body
- [253:270] qfrc_actuator[6:] (17)
- actuator force at each driven joint
- [270:348] cfrc_ext[1:] (78)
- external contact force/torque per body
Four of the six blocks are per-body physics the other benchmarks leave out. Humanoid is the env that hands the policy the mass matrix.
Action
Box(-0.4, 0.4, (17,))
Seventeen joint torques. Note the range: unlike every other env here, Humanoid’s control box is not ±1.
Reward
- forward_velocity +1.25
- centre-of-mass dx/dt, m/s
- healthy +5.0
- alive bonus — the largest on the plate
- contact_cost −5e−7 · ‖clip(cfrc)‖²
- effectively off
Termination: Torso z leaves [1.0, 2.0]. Truncation at 1000 steps.
An alive bonus of 5.0 against a forward term worth ~1 m/s is the other way to make living unconditionally profitable — the same guarantee Walker buys with a near-zero control cost, bought here on the income side instead.
Result
- Best eval
- 6,458
- Gait
- upright 3D bipedal walk
- Shaping
- none — just more steps
Hound-v0
Custom 16-DoF wheel-legged dog
Structure only · 16 DoF · Not trained
Four legs of four joints, with a driven wheel where the foot would be. Built, dimensioned and checked — but no policy has been trained on it yet, so there is nothing to show you but the machine.
No trained policy. This machine is structure only — there is no network, no reward table and no result on this card, because there is no run behind them yet.
Structure
- Degrees of freedom
- 16 — 4 legs × (abduction, hip, knee, wheel)
- Mass
- 17.0 kg
- Standing height
- 0.363 m on a 0.387 m wheelbase
- Wheel
- 0.085 m radius, direct-drive hub at the ankle
- Peak torque
- 23.7 N·m abduction and hip · 40 N·m knee · 3.0 N·m wheel
- Dimensions from
- Unitree Go2 link lengths and masses, via MuJoCo Menagerie
The wheel torque is the number worth reading twice. 3.0 N·m looks timid next to 40 at the knee, and it is not timidity: a wheel carrying 4.25 kgf breaks traction at 3.19 N·m, so a bigger hub motor would buy wheelspin and nothing else. The leg motors are sized by what they must hold; the wheel motors by what the ground will accept.
Where it stands
Done: Both models, the environment and its checks — robots.hound.check runs 38 assertions on the mechanics alone, and the env has the same obs, reward and terrain guards every trained run here is pinned against.
Open: Training. Runs are going, and none of them count until they clear the repo’s ≥3-seed bar — which is exactly why there is a card here with no numbers on it rather than a number with no seeds behind it.
Drawings
The machine · four legs × (abduction + hip + knee + wheel)
One leg, to scale — and why the fourth joint is not just a fifth link
Three measurements: the wheel angle can’t be an observation, more motor buys nothing, and a wheeled leg needs a stiffness floor