Skip to content

gcsfuse_mount_spec parsing and supporting mount_options from command line #1290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

deepikarajani24
Copy link

This PR improves the command-line parsing for --gcsfuse_mount_spec to allow for multiple key-value pairs and robustly handle custom GCS FUSE mount_options.

Usage

  1. To specify custom mount_options:
    The entire argument must be wrapped in single quotes ('...') to protect the inner double quotes from the shell.
--gcsfuse_mount_spec='gcs_path=gs://my-bucket/data,mount_path=/data,mount_options="implicit-dirs,metadata-cache-ttl-secs=-1"'
  1. To use default mount_options:
    No special quoting is needed.
--gcsfuse_mount_spec=gcs_path=gs://my-bucket/data,mount_path=/data

Testing
Unit Tests: Added parameterized unit tests to validate the new parsing logic for various formats (complex, basic, and empty mount_options).

End-to-End Test: Confirmed the full functionality by successfully launching a Fuji 7B training job that used a custom GCS FUSE mount defined with this flag.

The existing mount flags hardcoded in the code are not changed and I do see errors with those but haven't changed them. Instead passing the mount options I need in the command.

@deepikarajani24 deepikarajani24 requested a review from a team as a code owner July 11, 2025 00:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant