Augmentations should specify allowed partial declarations.

[lrhn]

Augmentations can add missing pieces to existing declarations. If those missing pieces are required, then the initial declaration is not valid by itself.
The current specification would make that declaration a compile-time error.

Augmentations makes some such declarations still be compile-time errors that apply before augmentations are even applied, and other declarations be only partially declared, and filled in by an augmentation, in which case the property does not apply to the individual declarations, but to the entire declaration+augmentations stack, the "effective declaration".

If that allows a declaration to omit something that would otherwise be required by the grammar, we need to specify the allowed partial syntax too. (#3687 is an example of the augmentation declaration having a different syntax than the main declaration, but not of the main declaration being allowed to omit something it can't omit today).

I'm not absolutely sure there are any cases where augmentations allow omitting something that is otherwise grammatically required, except maybe bodies of functions (which are grammatically allowed to be omitted, because of abstract declarations, but just not in every position - top-level functions cannot grammatically end in ;).

It still means we need to go thrugh every "it's a compile-time error if declaration D ...." and decide whether that applies to the initial declaration syntax, or to the post-augmentation-application effective declaration. (And the latter case would imply that it's a semantic restriction, not a syntactic restriction.)

Some prose-introduced syntax errors are irrecoverable, which probably means being so invalid that applying an augmentation wouldn't make sense anyway. Other errors are not, and should be changed to only apply to the result of augmentation.

Examples:

• It's definitely and eagerly a compile-time error if a static method has a covariant parameter or a this.x or super.x parameter. We can't augment that, because it has no meaning. Anything that would add a meaning to the declaration that cannot possibly apply to that kind of declaration, should be an eager error.

• It's possibly not a compile-time error if a static method has a ; body ("no body"), if a later applied augmentation adds a body.

• It's possibly not a compile-time error if a non-abstract class has an abstract member declaration, if a later augmentation adds a body (but that was never an eager error, because the class could inherit the implementation). Generally, anything related to satisfying an interface is going to be checked late.

• It's a compile-time error to have a super.foo invocation in an instance function, if the superclass doesn't have a foo member. It's probably allowed if an augmentation adds a superclass that does have a foo. (Which means the check should be written in a way that doesn't point to the surrounding class declaration, but to the semantic superclass of the surrounding class. It probably already is written that way, but we need to check. Everywhere, at least where we say "it's a compile-time error".)

[davidmorgan]

@johnniwinther this is the issue I'm aware of for the topic of "augmentations mean partial code can now be allowed".

We talked about something not in this issue, though, which is that the meaning of the partial code can be not yet known when macros ask about it. So for example

abstract class Foo {
  void doSomething();
}

if a macro asks about doSomething we have to choose whether we tell the macro it's abstract, not abstract, or that we don't know; and we have to choose what we do if the macro should ask again later; or otherwise, whether to inform it.

We discussed that indicating that you expect something to be augmented would resolve the ambiguity

abstract class Foo {
  void doSomething() augmented;
}

but that this does not look super attractive.

@lrhn @jakemac53 I couldn't find any other issues/discussion around this, do you know of any please?

It does seem like we need to work through the issue as per Lasse's opening description and all the consequences that follow, including for macro introspection.

[lrhn]

It's tricky. All we can safely say that the method does not appear to have a body.
If it's inside a non-abstract class or will likely get a body, ... or the class has or will get a noSuchMethod implementation.

If we run macros in parallel, then two macros can both see a declaration with no body and generate a body that does not call augmented().
The latter to be applied wins.
The fact that there is a "latter" suggests that macros are not truly parallel, and it would be better to let the latter macro see the results of the former. Even if we interleave phases, the macros of the individual phases should be serialized in a consistent way.
(And then macros could generate new macro augmentations too.)

Or we can tell macro authors to always call augmented if they would if there was a body being augmented. (And we would allow calling if there is no augmented body, as a no-op.)

Are there any other properties that a macro can change significantly (other than mere existence in phase 1)?
Overriding fields with setters and getters doesn't change the signature, but it does change whether a final private instance variable is promotable.
Macro code shouldn't look at a private final instance variable and generate code that assumes it's promotable. (But code is generated on phase 3, and the augmenting getter would be added in phase 2?)

Adding elements to enums changes exhaustiveness, but again they're added in a phase before code containing switches.

It may just be adding bodies where no body existed before, that's changing something in phase 3.

[jakemac53]

This is why the current API uses has* instead of is* for these getters (for example hasAbstract) - it indicates only the presence of the keyword, and not concretely whether or not it is abstract.

For functions, you also get hasBody, which tells you whether or not you can call augmented() safely.

[jakemac53]

If we run macros in parallel, then two macros can both see a declaration with no body and generate a body that does not call augmented().

No two macros which could generate a body for a function can run in parallel based on the ordering specification today.

[johnniwinther]

We build outlines before phase 3 so we need to have a way to tell whether a method is concrete without having to provide a body. Currently the only way to do this is to use external but that is wild west territory where only backends roam.

[lrhn]

Using external is almost certainly wrong. It may work, but it's sending all the wrong signals, and even when it works, it's not better than => throw Unimplemented("Not augmented"); in any way.

The reason it works is that we are currently very lenient about external members, more than we should be, so that if a declaration is external and has no actual implementation supplied, it will just throw at runtime if called.

The intent of external should be that the declaration is concrete and that an implementation is provided by the compiler in some way. And we should make it a compile-time error if one isn't!
It shouldn't just be a "late" method where we dynamically check if it's defined or not. We don't need more dynamic checks at runtime.

[jakemac53]

We build outlines before phase 3 so we need to have a way to tell whether a method is concrete without having to provide a body. Currently the only way to do this is to use external but that is wild west territory where only backends roam.

Hmm, this does sound fundamentally limiting. I think you might have to run phase 3.

Or, we will have to make macros explicitly signal ahead of time which methods they will be filling in.

[davidmorgan]

I discovered the other issue with discussion on this :) it's #3580 about adding an "augmented" modifier.

[davidmorgan]

I had a thought on this one :)

In cases where hasBody is a meaningful part of the declaration, should it be the case that macros that add a body are required to do so in Phase 2? It's no longer just a definition.

Maybe that helps the CFE, not sure whether it also comes with problematic downsides for macros.

[jakemac53]

Fwiw, I think that the CFE might want to run phase 3 regardless during outline creation, because there might be errors it is missing otherwise?