Skip to content

Fixing formatting #4874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fixing formatting #4874

wants to merge 1 commit into from

Conversation

rcrowe-google
Copy link

The bullets weren't bullets

What does this PR do?

Fixes # (issue)

Checklist

  • This PR fixes a minor issue (e.g.: typo or small bug) or improves the docs (you can dismiss the other
    checks if that's the case).
  • This change is discussed in a Github issue/
    discussion (please add a
    link).
  • The documentation and docstrings adhere to the
    documentation guidelines.
  • This change includes necessary high-coverage tests.
    (No quality testing = no merge!)

The bullets weren't bullets
* It changes the checkpoint structure, as each layer will have unique RNG state.
* It changes how ``nnx.split_rngs`` interacts with transforms like ``nnx.vmap`` and ``nnx.scan``,
as the resulting RNG state will now not be stored in scalar form.
* It changes how ``nnx.split_rngs`` interacts with transforms like ``nnx.vmap`` and ``nnx.scan``, as the resulting RNG state will now not be stored in scalar form.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
* It changes how ``nnx.split_rngs`` interacts with transforms like ``nnx.vmap`` and ``nnx.scan``, as the resulting RNG state will now not be stored in scalar form.
* It changes how ``nnx.split_rngs`` interacts with transforms like ``nnx.vmap`` and ``nnx.scan``, as the resulting PRNG state will now not be stored in scalar form.

@@ -11,9 +11,9 @@ Using Rngs in NNX Transforms
NNX layers that use RNGs like Dropout or MultiHeadAttention now hold a ``fork``-ed copy of the ``Rngs``
object given at construction time instead of a shared reference to the original ``Rngs`` object. This has
two consequences:

* It changes the checkpoint structure, as each layer will have unique RNG state.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
* It changes the checkpoint structure, as each layer will have unique RNG state.
* It changes the checkpoint structure, as each layer will have a unique PRNG state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants