![](https://csdnimg.cn/release/download_crawler_static/19054050/bg1.jpg)
Blind Bipedal Stair Traversal
via Sim-to-Real Reinforcement Learning
Jonah Siekmann
∗†
, Kevin Green
∗
, John Warila
∗
, Alan Fern
∗
, Jonathan Hurst
∗†
*CoRIS Institute, Oregon State University
{greenkev, warilaj, afern, jhurst}@oregonstate.edu
†Agility Robotics
{jonah.siekmann}@agilityrobotics.com
Abstract—Accurate and precise terrain estimation is a difficult
problem for robot locomotion in real-world environments. Thus,
it is useful to have systems that do not depend on accurate
estimation to the point of fragility. In this paper, we explore
the limits of such an approach by investigating the problem
of traversing stair-like terrain without any external perception
or terrain models on a bipedal robot. For such blind bipedal
platforms, the problem appears difficult (even for humans) due
to the surprise elevation changes. Our main contribution is to
show that sim-to-real reinforcement learning (RL) can achieve
robust locomotion over stair-like terrain on the bipedal robot
Cassie using only proprioceptive feedback. Importantly, this
only requires modifying an existing flat-terrain training RL
framework to include stair-like terrain randomization, without
any changes in reward function. To our knowledge, this is the first
controller for a bipedal, human-scale robot capable of reliably
traversing a variety of real-world stairs and other stair-like
disturbances using only proprioception.
I. INTRODUCTION
In order to be useful in the real world, bipedal and humanoid
robots need to be able to climb and descend stairs and stair-
like terrain, such as raised platforms or sudden vertical drops,
which are common features of human-centric environments.
The ability to robustly navigate these environments is crucial
to getting robots to work with and alongside humans safely.
Achieving this level of robustness on a bipedal platform is no
easy task; while other platforms such as quadrupedal robots
benefit from inherent stability due to multiple points of contact
with the ground at a given time and the ability to stop and
stand like a table, bipedal robots such as Cassie rely entirely
on dynamic stability (essentially always existing in a state of
falling). On stair-like environments, this is especially apparent
due to the difficulty of recovery from missteps with only two
legs.
By contrast, robots with quadrupedal morphologies have
been able to use proprioception alone to negotiate stairs [1, 2],
and hexapedal robots have even been able to use open-loop
control to ascend and descend stairs [3]. While planar bipedal
robots have been shown to be able to reject disturbances like
large unexpected dropsteps [4], the vast majority of approaches
seeking to enable such robots to negotiate stairs in the real
world require either accurate vision systems [5, 6, 7] or
operation in a carefully controlled laboratory environment
[8, 9, 10], meaning the robot is localized through a known
Domain Randomization
Hardware Execution
Sim-to-Real
Learned Controller
Proprioceptive Data
PD Targets
Fig. 1: In this work, we investigate the limits of blind bipedal locomo-
tion. We present a training pipeline which produces policies capable
of blindly ascending and descending stairs in the real world. These
policies learn proprioceptive reflexes to reject significant disturbances
in ground height, resulting in highly robust behavior to many real-
world environments.
start location or the stairs are designed in tandem with robot
morphology.
However, robots must be able to operate outside of con-
trolled laboratory conditions and handle the massive variety
of conditions in the real world. This goal is not compatible
with a complete reliance on exteroceptive sensors such as
RGB and depth cameras for accurate terrain estimation, which
introduce fragility to real world conditions [11]. For instance,
cameras may be unreliable if exposed to occlusion, fog, or
varying lighting conditions. Further, integrating a state-of-
the-art computer vision system into a high-speed controller
is technically difficult, especially on a computationally lim-
ited platform like a mobile robot. For practical purposes,
underlying controllers should be as robust as possible while
relying on as little information about the world as possible.
Ideally, a bipedal robot should be able to traverse as much of
the entire breadth of human environments as possible using
proprioception, while relying on exteroceptive sensing for
further efficiency and high-level planning (and being robust
to mistaken perception). This begs the question: how robust
arXiv:2105.08328v1 [cs.RO] 18 May 2021