'isaacsim.sensors' Module Not Found Error in Headless Mode

[RodsCoimbra]

Hi,
I'm running into an issue when trying to train in headless mode using a PhysX LiDAR sensor. I get an error saying that the module isaacsim.sensors was not found. I also tried using the --enable_cameras flag just in case, but it didn’t make a difference.
Running the same script without --headless works fine, and the module is detected, however it has a big impact on the performance.
Is this expected behavior? Do I really need to run without --headless for LiDAR to work, or is there a way to make it work properly in headless mode?
Thanks!

[Error] [omni.ext._impl.custom_importer] Failed to import python module isaaclab_tasks. Error: No module named 'isaacsim.sensors'. Traceback:
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/lab/lib/python3.10/site-packages/omni/kernel/py/omni/ext/_impl/custom_importer.py", line 76, in import_module
    return importlib.import_module(name)
  File "/home/ubuntu/miniconda3/envs/lab/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ubuntu/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/__init__.py", line 30, in <module>
    import_packages(__name__, _BLACKLIST_PKGS)
  File "/home/ubuntu/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/utils/importer.py", line 40, in import_packages
    for _ in _walk_packages(package.__path__, package.__name__ + ".", blacklist_pkgs=blacklist_pkgs):
  File "/home/ubuntu/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/utils/importer.py", line 87, in _walk_packages
    yield from _walk_packages(path, info.name + ".", onerror, blacklist_pkgs)
  File "/home/ubuntu/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/utils/importer.py", line 75, in _walk_packages
    __import__(info.name)
  File "/home/ubuntu/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/direct/tiago/__init__.py", line 4, in <module>
    from .tiago_env import TiagoEnv, TiagoEnvCfg
  File "/home/ubuntu/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/direct/tiago/tiago_env.py", line 22, in <module>
    from .lidar import LidarSensor
  File "/home/ubuntu/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/direct/tiago/lidar.py", line 2, in <module>
    from isaacsim.sensors.physx import _range_sensor
ModuleNotFoundError: No module named 'isaacsim.sensors'

[Toni-SM]

Hi @RodsCoimbra

You can enable it by adding the following line to the Isaac Sim Extensions [depencencies] in the Isaac Lab apps/isaaclab.python.headless.kit experience file.

IsaacLab/apps/isaaclab.python.kit

Line 36 in f0c1ca6

"isaacsim.sensors.physx" = {}

[RodsCoimbra]

Thanks for the response!
However, when I tried that, the values don't seem to update. The output just shows the data as if there are no obstacles or even the ground and all the values are at the maximum range. Do you know why that might be?

[RodsCoimbra]

I found this solution in #2082.

Disabling fabric can solve this problem, although I don't know why. https://isaac-sim.github.io/IsaacLab/main/source/api/lab/isaaclab.sim.html#isaaclab.sim.SimulationCfg.use_fabric

However, even after setting use_fabric = False in my SimulationCfg, it still didn't work.

[kldst]

Hi, I think you should set "enable_scene_query_support = True" in SimulationCfg. And I override the step function in my environment class. Inside it, I call self.render() every step, and then return the result of super().step(actions). I also set render_interval = 1, which means the environment will render on every step.

def step(self, actions):
        self.render()
        return super().step(actions)

[RandomOakForest]

Following up, some sensors are CPU only. Have you tried running this using only CPUs?

[RodsCoimbra]

I was under the impression that this sensor runs only on the CPU, since it's based on CPU-based physics simulations. If that's not the case, how to explicitly configure it to run on the CPU?
For context, I'm using Isaac Lab and running the sensor as part of a Direct-based task during training. I create the sensor in the setup_scene function with code like the following:

def create_lidar(self, min_range, max_range, horizontal_fov, vertical_fov, horizontal_resolution, vertical_resolution):
        for env_id in range(self.num_envs): 
            robot_sensor_path = f"/rtx_lidar"
            robot_parent_path = f"/World/envs/env_{env_id}/Robot/torso_lift_link/Unitree_L1"
            result, _ = omni.kit.commands.execute(
                "RangeSensorCreateLidar",
                path=robot_sensor_path,
                parent=robot_parent_path,
                min_range=min_range,
                max_range=max_range,
                draw_points=False,
                draw_lines=False,
                horizontal_fov=horizontal_fov,
                vertical_fov=vertical_fov,
                horizontal_resolution=horizontal_resolution,
                vertical_resolution=vertical_resolution,
                rotation_rate=0.0,
                high_lod=True,
                yaw_offset=0.0,
                enable_semantics=False
            )
            if not result:
                raise RuntimeError(f"Failed to create lidar sensor in the environment {env_id}.")

Interestingly, the sensor works fine when I don't use the --headless flag. Does that mean it's running on the GPU in that case? Or is it still using the CPU, but something about headless mode prevents it from initializing properly?