Shared clusters require .ipython files to be copied to /etc/skel/.ipython/profile_default/startup

[hengrumay]

Test (shared clusters approach) and update documentation from feedback:

On Tue, Jul 9, 2024 at 12:55 PM Eschman, Daniel <[email protected]> wrote:
All,

Thanks again for taking the time to meet yesterday. I was able to get this up and running after a few more modifications. I believe the notebook documenting this sample works only for clusters in single user access mode.

To get it working on my cluster configured for shared access mode, I had to use the init script to copy the python startup script to /etc/skel/.ipython/profile_default/startup This is because the cluster running in shared access mode dynamically creates new users for each session with a random GUID attached to their uid.

By copying the python startup script to a directory under /etc/skel, it ensures that the OS creates those same files in each new user’s $HOME directory and that they are found when the ipython kernel starts for a given notebook.

I was also seeing the same behavior I had previously noted when using a symlink instead of copying the python startup script. The symlink returns an error when a notebook is created that the file does not exist but copying works just fine.

Hopefully that explanation is helpful for building out the documentation a bit further – this trick is big for us!

Best,
Dan