Should an is T test make T? a type of interest?

[stereotype441]

Currently flow analysis has the rule that an is T or as T test causes both T and NonNull(T) to become types of interest. This allows the user to do things like this:

f(num? n) {
  if (n is int?) {
    // `n` is promoted to `int?` now
    if (n == null) return;
    // `n` is promoted to `int` now
  } else {
    n = 0; // `n` is promoted to `int` now
  }
  // `n` is promoted to `int` on both code paths, so it remains promoted to `int`.
}

If we didn't consider NonNull(T) to be a type of interest, then the statement n = 0 would merely promote n to int?.

In dart-lang/sdk#60646 (comment), @lrhn suggested that maybe an is T test should also make T? a type of interest. This would allow things like this:

g(num? n) {
  if (n is! int) return;
  // `n` is promoted to `int` now
  n = /* some expression with static type `int?` */;
  // `int?` is a type of interest, therefore `n` is now promoted to `int?`.
}

(Today, int? is not a type of interest, so the assignment to n in g loses the promotion completely, and n gets demoted back to num?).

CC @dart-lang/language-team

[eernstg]

T being a type of interest means that an assignment like x = e where e has static type T can give x a new value and a new type.

The reason why we don't always give a promotable variable which is being assigned a new value the type of the assigned expression (except that we wouldn't ever do it for dynamic) is presumably that this would be confusing and hence error prone. E.g., the type of the right hand side of the assignment may be non-obvious, and it may be really tricky to follow a series of promotions and demotions involving types that are really not of interest (e.g., _SomeClassImpl). So we basically require that a type must be mentioned explicitly in order to be available (that is, "of interest") for promotion via assignment.

The reason why T being of interest implies that NonNull(T) is of interest is probably that promotions in general yield a decreasing type: The variable has a more general type for a while, and then it gets a more special type because it turns out to be justified by the actual value. So that's the natural direction of a promotion, and it fits well with a situation where a parameter has a nullable type, but must be treated differently when it is null and when it is non-null, taking us from S? to S for some type S.

Hence, making T? of interest because T is of interest is the "unnatural direction" for promotion. The fact that it is only relevant for promotion via assignment means that we essentially expect to have a variable whose type is the non-nullable type T, and then we're going to assign a possibly-null value to that variable (in the example we're assigning an expression of type int? to a variable whose type was previously promoted to int). The question comes to mind—how often do we do this?

We could certainly do it, and it seems useful in the example, but it is at least a lot more marginal than the direction "T? -> T" that we already have.

[lrhn]

The reason we don't always promote is indeed that it may be confusing when the promoted type gets captured.

num x;
// ...
if (case1) {
  x = 0.0; /
  // something something
  if (case1a) {
    var finalValue = 37;
    x = 42;
    return [x]..add(finalValue);
  }
// ...

If x = 0.0 promotes to double, the x = 42; means x = 42.0;, which is a different program, and the [x] is a List<double> not a List<num>, and ..add(finalValue) is an error.

That's a risk. If we promote and the user doesn't expect it, they get subtle changes in behavior from what they expect.
If we don't promote and they expect it, they just get annoyed.

So, assignment only promotes if the user has shown some interest in whether the variable has that particular type.

Nullability is not like a type, it's more of an "either something of that type or not".
If you have shown yourself interested in whether a nullable variable contains an int, you probably want to know that if you assign an int? to it.

int toInt(Object? o) {
  if (o == null) return 0;
  if (o is! int) {
    if (o is double) {
      o = o.toInt();
    } else {
      o = int.tryParse("$o");
    }
  }
  return o ?? -1;
}

I feel like the o = tryParse(o) here should promote from Object? to int?, because int is a type of interest,
so the user wants to know if it's an int, even if it can also be nothing.

we essentially expect to have a variable whose type is the non-nullable type T, and then we're going to assign a possibly-null value to that variable

The variable doesn't have to be promoted to the type of interest. It's also of interest on the non-promoted branch. That's actually where it's usually useful: if (x is! int) x = 0; promotes to int on the non-promoted branch, so that both branches end up with the same type.

Basically: Special-case nullability, treat it as not a type, but as a variation. All variations of int are of interest if int is of interest.

(When we intersect promotion chains, do we recognize fx int and int? as being related, and merge them to int?? If we do, then I think recognizing int? as a type of interest when the other branch promotes to int is consistent.)

[leafpetersen]

If I understand this correctly, I think this would introduce a new situation in which we can promote a variable to a type which is not a subtype of its original declared type. There's nothing per se wrong with that, but it's a bit odd, and I'm not sure I love it. I think it means that you can end up in a situation where you have to demote the variable to something more general than its original declared type:

void test1(Object o) {
  if (o is int) { // int? is a type of interest now
    o = o == 3 ? null : 0;
  }
  // What is the type of o on the join?  Do we use UP to find a common super type?
  o = null;  // valid because o has type Object? now?
}

In general, this feels like kind of a tarpit to me. Is this right, or do the above complications not arise for some reason?

[leafpetersen]

Related discusion here, for reference.

Also, I'm pretty sure that at some point we considered a generalization of this in which an assignment of something of type T to a variable of current type S demoted to UP(T, S), possibly subject to UP(T, S) being a subtype of a type of interest? This seems more consistent to me, to be honest. That is, the following also seems like a useful variant of what is being proposed here:

g(Object n) {
  if (n is! int) return;
  // `n` is promoted to `int` now
  n = 3.4
  // UP(double int) is a type of interest, therefore `n` is now promoted to `num`.
}

[stereotype441]

@leafpetersen

If I understand this correctly, I think this would introduce a new situation in which we can promote a variable to a type which is not a subtype of its original declared type. There's nothing per se wrong with that, but it's a bit odd, and I'm not sure I love it. I think it means that you can end up in a situation where you have to demote the variable to something more general than its original declared type:

void test1(Object o) {
  if (o is int) { // int? is a type of interest now
    o = o == 3 ? null : 0;
  }
  // What is the type of o on the join?  Do we use UP to find a common super type?
  o = null;  // valid because o has type Object? now?
}

In general, this feels like kind of a tarpit to me. Is this right, or do the above complications not arise for some reason?

Agreed that would be a tarpit. Fortunately there's a subtlety in the "promote to type of interest" logic that prevents this kind of complication from arising; unfortunately that subtlety is not documented in flow-analysis.md. I'm going to try to make a PR to rectify that this week (gosh, is it Thursday already?), but for now you can look at my informal description from issue 60646.

In short, the "promote to type of interest" logic first cancels any promotions that are no longer valid; then, when considering candidate types of interest that it might promote to, it filters out any candidate types that are not a subtype of the current type. This preserves the invariant (also maintained by other parts of flow analysis) that every type in the chain of promotions is a subtype of the previous ones (and the first type in the chain of promotions is a subtype of the declared type). Thus, by transitivity, the promoted type is always a subtype of the declared type.

So in your example above, even if we say that o is int makes int? a type of interest, that will have no effect on this particular example, because int? is not a subtype of the declared type of o; therefore it will never be a subtype of the current type of o, so it will never be accepted as a "type of interest" promotion.

[leafpetersen]

So in your example above, even if we say that o is int makes int? a type of interest, that will have no effect on this particular example, because int? is not a subtype of the declared type of o; therefore it will never be a subtype of the current type of o, so it will never be accepted as a "type of interest" promotion.

I couldn't entirely follow the paragraph that started with "in short", because I'm not sure entirely what exactly things like "promotions that are no longer valid", "candidate types", "current type" etc mean. Sorry, I don't know how much of this is me not being super dialed in on this vs the paragraph just being informal. :). Working from the paragraph quoted above though, I don't think the logic there is enough (which is not to say that the full logic doesn't handle it. Expanding my example:

void test1(Object? o) {
  if (o is Object) {
    if (o is int) { // int? is a type of interest now
      o = o == 3 ? null : 0;
    }
    // What is the type of o on the join?  Do we use UP to find a common super type? Or demote to Object?
    o = null;  // valid because o has type Object? now?
  }
}

Here, the logic "int? is not a subtype of the declared type of o" doesn't apply. So what happens here, and why?

[lrhn]

Whether we filter types of interest at the type-check point or the assignment point, we should never actually use a type of interest which is not a subtype of the declared type of a variable. We can say that x is T makes T, T? and NonNull(T) types of interest only if when are subtypes of the declared type of x. That's easier than remembering them, and then at x = o, only actually promoting to types of interests that are actually subtypes of the declared type of x.

It's not new to have potential types of interest that are not subtypes of the declared type of the variable.

void foo(Object o) {
  if (o is! int?) { // Makes `int?` and `int` potentially types of interest, but `int?` isn't valid.
     o = 42; // OK, promotes to int.
     o = (42 as int?); // Not OK.
  }

About:

g(Object n) {
  if (n is! int) return;
  // `n` is promoted to `int` now
  n = 3.4
  // UP(double int) is a type of interest, therefore `n` is now promoted to `num`.
}

That would make every supertype of int a type of interest, fx including Comparable<num> here.
If you have a taller class hierarchy, that could be a lot.

void somethingWidget(Widget widget) {
  if (widget is ClipRect) return; // Don't do anything, they're special.
  widget = Transform(child: widget, ...);
  // What is the type of `widget` here? 

The type is SingleChildRenderObjectWidget, which both Transform and ClipRect happen to implement.

Is that a problem? Probably usually not, but the reason we don't just assignment promote every assignment also applies here.

   var listOfWidget = [widget];
   workWithListOfWidget(listOfWidget); // Error. Cannot add MultiChildRenderObjectWidget to list.

I think nullability is special, and it's OK to special case it. I don't think it necessarily generalizes.

[stereotype441]

@leafpetersen

So in your example above, even if we say that o is int makes int? a type of interest, that will have no effect on this particular example, because int? is not a subtype of the declared type of o; therefore it will never be a subtype of the current type of o, so it will never be accepted as a "type of interest" promotion.

I couldn't entirely follow the paragraph that started with "in short", because I'm not sure entirely what exactly things like "promotions that are no longer valid", "candidate types", "current type" etc mean. Sorry, I don't know how much of this is me not being super dialed in on this vs the paragraph just being informal. :). Working from the paragraph quoted above though, I don't think the logic there is enough (which is not to say that the full logic doesn't handle it. Expanding my example:

void test1(Object? o) {
  if (o is Object) {
    if (o is int) { // int? is a type of interest now
      o = o == 3 ? null : 0;
    }
    // What is the type of o on the join?  Do we use UP to find a common super type? Or demote to Object?
    o = null;  // valid because o has type Object? now?
  }
}

Here, the logic "int? is not a subtype of the declared type of o" doesn't apply. So what happens here, and why?

At the site of o = o == 3 ? null : 0, first UP is used to compute the static type of o == 3 ? null : 0. That type is int?. In my informal description, I refer to this type as the written type. Since neither of the two promoted types (Object and int) is a supertype of the written type, those promotions are discarded, so the type of o is back to Object?.

Then, promotion to types of interest is considered. Today, the types of interest are Object and int. Neither of these is the same type as the written type, and neither of them is a supertype of the written type, so they are both rejected, and no further promotion is performed. So after o = o == 3 ? null : 0, o has been fully demoted.

At the end of the if, a flow model that promotes to Object and then to int is joined with a flow model that does no promotion. The rule for join is that only promotions that exist on both control flow paths are kept, so the result of the join is that o is fully demoted.

At the point of the assignment o = null, o does indeed have type Object?. But that's not the reason why the assignment o = null is valid. That assignment would be valid even if o had still been promoted (it would have demoted in that case).

If we accept the suggestion that a test on T causes T? to be a type of interest, then here's how that will change:

At the site of o = o == 3 ? null : 0, the static type of o == 3 ? null : 0 will still be int?, and the demotion step will still demote o back to Object?.

But then, when promotion to types of interest is considered, the types of interest will be Object, Object?, int, and int?. Since one of these types is the same type as the written type, the type of o will be promoted to that type. So after o = o == 3 ? null : 0, o's promotion chain will contain the single type int?.

Then, at the end of the if, a flow model that promotes to Object and then to int will be joined with a flow model that promotes to int?. Since there are no promoted types common to both control flow paths, the result of the join will be that o is fully demoted.

[leafpetersen]

@stereotype441 thanks, that helps.

Stepping up a level, I don't really have strong feelings here, but I admit to having a hard time formulating a crisp story for why we should do this. I don't find anything here particularly compelling either way. I can certainly see cases where it is useful to promote to the nullable version of a type of interest. I can also see cases where it is useful to promote to a different super type of a type of interest (e.g. promote to num given that double is a type of interest). Taken to the limit, we could just say that a variable has the type of whatever the last thing that was assigned to it was. Where do we draw the line and why? And if we're going to move that line, what's the principled reason for this particular choice of how to do so?

[lrhn]

My reasoning here is that nullability is different from other supertypea and subtypes.

We already accepted that when we made NonNull(T) a type of interest when you check for T. We don't make int a type of interest when you check for num, we do this because nullability is different.
When you do is int?, you assume that you actually do care whether it's an int by itself, the question you ask is "is this an int, or the absence of an int?". If you later assign an int, we promote. And we promote to int, not to int? which was the typed you tested for.

That logic also applies when you start out asking is int, and later assign "an int or the absence of an int", you still care about the int-ness of that.

[stereotype441]

@lrhn

Whether we filter types of interest at the type-check point or the assignment point, we should never actually use a type of interest which is not a subtype of the declared type of a variable.

Agreed! This is addressed in #4370 (spec) and https://dart-review.googlesource.com/c/sdk/+/427920 (implementation).

We can say that x is T makes T, T? and NonNull(T) types of interest only if when are subtypes of the declared type of x. That's easier than remembering them, and then at x = o, only actually promoting to types of interests that are actually subtypes of the declared type of x.

I agree that, as an implementation optimization, it makes sense not to bother remembering a type of interest that's never going to have an effect.

As for what we say in the spec, what I found easiest to do in #4370 was to say that each variable has a set of tested types (the set of types that has appeared on the right hand side of is and as expressions applied to the variable), and then define the set of "types of interest" to be those types, plus NonNull applied to all of those types, and then to say that we only consider a type of interest T for promotion if written_type <: T <: declared_type.

About:

g(Object n) {
  if (n is! int) return;
  // `n` is promoted to `int` now
  n = 3.4
  // UP(double int) is a type of interest, therefore `n` is now promoted to `num`.
}

That would make every supertype of int a type of interest, fx including Comparable<num> here. If you have a taller class hierarchy, that could be a lot.

I agree. One of the design principles I remember coming up in the early discussions about flow analysis was that we wanted to make sure we didn't promote to types that would be "surprising" to users. That was where the idea of "types of interest" came from. I feel like if we made UP(written_type, tested_type) a type of interest, we would be creeping too far into "surprising" territory.

[stereotype441]

@leafpetersen

@stereotype441 thanks, that helps.

Stepping up a level, I don't really have strong feelings here, but I admit to having a hard time formulating a crisp story for why we should do this. I don't find anything here particularly compelling either way. I can certainly see cases where it is useful to promote to the nullable version of a type of interest. I can also see cases where it is useful to promote to a different super type of a type of interest (e.g. promote to num given that double is a type of interest). Taken to the limit, we could just say that a variable has the type of whatever the last thing that was assigned to it was. Where do we draw the line and why? And if we're going to move that line, what's the principled reason for this particular choice of how to do so?

FWIW, I think @lrhn has a point that if we're going to make a NonNull(T) a type of interest when you check for T, it makes sense to also make T? a type of interest. That to me is enough to count as a principled reason. But so far I haven't seen a compelling real-world example where it matters (i.e., some code in the wild where someone is doing a type test of T, and later an assignment of T?, and then having to do extra work because a type-of-interest promotion didn't occur). I would probably want to dig around google3 looking for some concrete examples before I dove into working on this.

[leafpetersen]

FWIW, I think [@lrhn](https://github.com/lrhn) has a point that if we're going to make a NonNull(T) a type of interest when you check for T, it makes sense to also make T? a type of interest. That to me is enough to count as a principled reason.

I don't really buy this, FWIW. I can see an argument for saying that is int? makes int a type of interest. int is written right there. The general principle I would probably go for is to say that any time you test against a union type, all of the arms of the union are types of interest. I realize that we don't do that for FutureOr, but maybe we should, if we're going to change something? On the other hand, saying that int? is a type of interest because you asked about int just feels weird to me. I've gone to the trouble of establishing that something is a non-null int, but the system just lets me assign a nullable int to it, even though I've never expressed any interest in nullable things? Feels weird to me. But as I say, not a hill I want to die in if we think it's useful.

But so far I haven't seen a compelling real-world example where it matters (i.e., some code in the wild where someone is doing a type test of T, and later an assignment of T?, and then having to do extra work because a type-of-interest promotion didn't occur). I would probably want to dig around google3 looking for some concrete examples before I dove into working on this.

Yeah, this would be a lot more compelling of an argument if we had some actual use cases to work from.

[lrhn]

I think nullability is different from other union types. Well, the other union type. And maybe it isn't really, but it's just more common.

If I do x is int?, we make int a type of interest, but not Null. Why? Because Null is special. We don't want to promote to Null. That has been the stated policy so far, which is why int? x = ...; if (x == null) { ... } else { ... } promotes to int in the else branch, but not to Null in the then-branch. The argument, as I understand it, is that promoting to Null might give a surprising context type for a later operation, which is most likely intended to store another non-Null value anyway. (No need to store to a variable which has the null value unless you want to store something that is potentially not null.)

We don't say something similar for FutureOr

If you have: FutureOr<int> f = ...; if (f is Future<int>) { ... } else { ... } then we promote to Future<int> in the then-branch and int in the else-branch. Both are useful types and values. I don't know if we make int a type of interest in the else branch, it feels a little iffy to promote to something that is not a type of interest.
(Checking, we do not. We have a few weird cases, though. Having a FutureOr<int> and doing is Future<int> or is int does promote to the other one on the false branch, but seemingly forgets to make the tested type a type of interest on the true branch, and we make Future<int> a type of interest on the false branch of is int, but not int on the false branch of Future<int>. @stereotype441, see gist.
We do not make FutureOr<int> a type of interest event if both int and Future<int> is, or vice versa.

For nullability, we don't make Null a type of interest when checking an int? for is int or == null.
(We do actually promote to Null on the false branch of is int, or on an is Null, but not on the true branch of == null or false branch of != null. And we never make Null a type of interest, even for is Null.

Of the two union types and two component types, the one that stands out here is Null.
We don't (want to) promote to it, and we don't make it a type of interest. We try to pretend the nullability isn't really there.
When we have is int? make int a type of interest, I see it more like pretending the ? isn't there and making int a type of interest, and then doing so independent of nullability and adding the ? back too.
So when you do is int, I'd also make it make int a type of interest, independent of nullability, so that the types of interest are NonNull(T) and NonNull(T)?, because we'll ignore the ? anyway.