[Question] UR5 not picking up the object (cylinder)

[Ozzey]

Question

I modified Isaac-Lift-Cube-Franka-V0 task and replaced Franka with a UR5 with Robotique gripper. The results were good and the manipulator was able to lift the cube without any problems after training for 1500 epochs.

Next, I wanted to replace the cube with a cylinder. Despite training for 3000 epochs, the manipulator never lifts the cylinder and just keeps hovering over it. During the entire duration of training, of training, the cylinder is never lifted up.

Here is the code:

@configclass
class UR5CubeLiftEnvCfg(LiftEnvCfg):
    def __post_init__(self):

        super().__post_init__()

        self.scene.robot = UR5_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")

        self.actions.arm_action = mdp.JointPositionActionCfg(
            asset_name="robot",
            joint_names=[
                "shoulder_pan_joint",
                "shoulder_lift_joint",
                "elbow_joint",
                "wrist_1_joint",
                "wrist_2_joint",
                "wrist_3_joint",
            ],
        )

        self.actions.gripper_action = mdp.BinaryJointPositionActionCfg(
            asset_name="robot",
            joint_names=[
                "robotiq_85_left_knuckle_joint",
                "robotiq_85_right_knuckle_joint"
            ],
            open_command_expr={
                "robotiq_85_left_knuckle_joint": 0.0,
                "robotiq_85_right_knuckle_joint": 0.0
            },
            close_command_expr={
                "robotiq_85_left_knuckle_joint": math.radians(41.0),
                "robotiq_85_right_knuckle_joint": math.radians(41.0)
            },
        )

        self.commands.object_pose.body_name = "gripper_link" 
        self.commands.object_pose.debug_vis = False

        self.scene.object = RigidObjectCfg(
            prim_path="{ENV_REGEX_NS}/Object",
            debug_vis=False,
            init_state=RigidObjectCfg.InitialStateCfg(
                pos=[0.5, 0, 0.01],   # initial position of the object
                rot=[1.0, 0.0, 0.0, 0.0], 
            ),
            spawn=UsdFileCfg(
                usd_path=f"/home/jacob/Desktop/Projects/ur5_isaaclab-master/assets/cylinder.usd",
                # usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/DexCube/dex_cube_instanceable.usd",
                scale=(0.08, 0.08, 0.08),
                rigid_props=RigidBodyPropertiesCfg(
                    solver_position_iteration_count=16,
                    solver_velocity_iteration_count=1,
                    max_angular_velocity=1000.0,
                    max_linear_velocity=1000.0,
                    max_depenetration_velocity=5.0,
                    disable_gravity=False,
                ),
            ),
        )

        marker_cfg = FRAME_MARKER_CFG.copy()
        marker_cfg.markers["frame"].scale = (0.1, 0.1, 0.1)
        marker_cfg.prim_path = "/Visuals/FrameTransformer"
        self.scene.ee_frame = FrameTransformerCfg(
            prim_path="{ENV_REGEX_NS}/Robot/world",
            debug_vis=False,
            visualizer_cfg=marker_cfg,
            target_frames=[
                FrameTransformerCfg.FrameCfg(
                    prim_path="{ENV_REGEX_NS}/Robot/gripper_link",
                    name="end_effector",
                    offset=OffsetCfg(pos=[0.0, 0, 0.0]),
                ),
            ],
        )

        object_marker_cfg = FRAME_MARKER_CFG.copy()
        object_marker_cfg.markers["frame"].scale = (0.1, 0.1, 0.1)
        object_marker_cfg.prim_path = "/Visuals/ObjectMarker"
        self.scene.object_frame = FrameTransformerCfg(
            prim_path="{ENV_REGEX_NS}/Object",  
            debug_vis=False,
            visualizer_cfg=object_marker_cfg,
            target_frames=[
                FrameTransformerCfg.FrameCfg(
                    prim_path="{ENV_REGEX_NS}/Object",
                    name="object_frame",
                    offset=OffsetCfg(pos=[0.0, 0.0, 0.0]),
                ),
            ],
        )

        world_marker_cfg = FRAME_MARKER_CFG.copy()
        world_marker_cfg.markers["frame"].scale = (0.2, 0.2, 0.2)  
        world_marker_cfg.prim_path = "/Visuals/WorldMarker"
        self.scene.world_frame = FrameTransformerCfg(
            prim_path="{ENV_REGEX_NS}/Robot/world",
            debug_vis=False,
            visualizer_cfg=world_marker_cfg,
            target_frames=[
                FrameTransformerCfg.FrameCfg(
                    prim_path="{ENV_REGEX_NS}/Robot/world",
                    name="world_frame",
                    offset=OffsetCfg(pos=[0.0, 0.0, 0.0]),
                ),
            ],
        )

The only thing I changed from the cube task is that, the minimum heigh is increased (to compensate for the height of the cylinder)

class RewardsCfg:
    """Reward terms for the MDP."""

    reaching_object = RewTerm(func=mdp.object_ee_distance, params={"std": 0.12}, weight=7.0)  ## .25

    lifting_object = RewTerm(func=mdp.object_is_lifted, params={"minimal_height": 0.09}, weight=15.0)

    object_goal_tracking = RewTerm(
        func=mdp.object_goal_distance,
        params={"std": 0.3, "minimal_height": 0.09, "command_name": "object_pose"},
        weight=16.0,
    )

    object_goal_tracking_fine_grained = RewTerm(
        func=mdp.object_goal_distance,
        params={"std": 0.05, "minimal_height": 0.09, "command_name": "object_pose"},
        weight=5.0,
    )

    # action penalty
    action_rate = RewTerm(func=mdp.action_rate_l2, weight=-1e-5)  #7

    # print("Reaching Object: ", reaching_object)
    # print("Lifting Object: ", lifting_object)

    joint_vel = RewTerm(
        func=mdp.joint_vel_l2,
        weight=-1e-5,
        params={"asset_cfg": SceneEntityCfg("robot")},
    )

And I have added a terminal condition in case the cylinder is knocked over:

@configclass
class TerminationsCfg:
    """Termination terms for the MDP."""

    time_out = DoneTerm(func=mdp.time_out, time_out=True)

    object_dropping = DoneTerm(
        func=mdp.root_height_below_minimum, params={"minimum_height": 0.025, "asset_cfg": SceneEntityCfg("object")}
    )

I checked the collision mesh and everything seemed fine.

Here is the video of the trained model:

2025-08-15.14-31-05.mp4

[RandomOakForest]

Thank you for posting this. I'll move your question to our Discussions section for follow up. Here is a summary of ideas to consider.

When adapting the Isaac-Lift-Cube-Franka-V0 task for a UR5 with a Robotique gripper and replacing the cube with a cylinder, the issue of the manipulator hovering over the cylinder and never lifting it—despite extended training—suggests there are underlying simulation or RL pipeline considerations specific to the geometry and physical properties of the cylinder.

Key Troubleshooting Steps

1. Grasping & Contact Surface

• Cylinder Shape: Cylinders, unlike cubes, present curved surfaces. If your gripper’s contact points were tuned for a cube, they may not establish robust grasp on the cylinder. Check the gripper fingers’ distance, the friction parameters, and consider:
  • Increasing the gripper's closing force or friction on the fingertips in the simulation config.
  • Adjusting the cylinder’s material properties (friction, restitution) to ensure it isn’t too slippery for stable grasp.
  • Verifying the collision mesh is convex and accurately represents the physical boundaries (you’ve already checked the mesh, good).

2. Reward Structure Adaptations

• Reward Shaping: The RL agent may not get sufficiently dense or shaped rewards to discover a lift strategy for a cylinder, which usually requires more precision and coordination due to the geometry.
  • Ensure the reward includes not just lift, but also partial rewards for stable grasp or alignment.
  • Add a reward component for maintaining the cylinder in an upright orientation throughout grasp and lift.

Example Reward (pseudo Python):

# Add reward for stable upright orientation
upright_bonus = float(abs(object_orientation.z) > 0.95)
reward += upright_bonus * 0.2

3. Terminal Conditions

• Falling/Knock Over: The terminal condition when knocked over may be triggered prematurely, especially if the gripper interacts with the cylinder side and slightly tilts it. Consider revising the definition of “knocks over” for the more sensitive cylinder geometry.

Example:

# Terminal if orientation from Z axis drops below threshold
terminal = abs(object_orientation.z) < 0.5

4. Observation & Action Space Check

• Object Features: Ensure your observation space includes enough information about the cylinder’s orientation (quaternion, Euler angles), in contrast to a cube, where this may have been less critical.
• Action Granularity: For curved objects, finer control granularity may help. Check if action resolutions are sufficient for precise grasp maneuvers.

5. Curriculum Learning for Complex Geometries

• Progression: Start with "easier" cylinder tasks—e.g., pick up partly lifted or horizontally positioned cylinders—and gradually increase difficulty (upright, vertical pick, dense clutter) using a curriculum.

6. Testing with Manual Actions

• Manually control the UR5 using scripted actions to grasp and lift the cylinder. This helps identify if the gripper and physics parameters allow a feasible lift before RL training.

Recommended Steps

1. Tune grasp and friction properties for the UR5 gripper and the cylinder.
2. Adapt the reward function to provide denser feedback for cylinder manipulation, not just lift.
3. Review terminal conditions and ensure they aren’t penalizing small tilts or contacts prematurely.
4. Enhance observations to include cylinder orientation and spatial features.
5. Test curriculum learning for gradual complexity increase.
6. Manually verify feasibility by direct control before RL training.