Nested type arguments doesn't work with polymorphic types

[Raniz85]

When using @JsonSubTypes/@JsonTypeInfo on an interface with a generic parameter (de)serialising a subclass that wraps it's own generic parameter in another generic class or interface doesn't work.

It's a bit complex to explain with words so here's a short example:

public class GenericPolymorphismTest {
    @JsonTypeInfo(property = "type", use = JsonTypeInfo.Id.NAME)
    @JsonSubTypes({
            @JsonSubTypes.Type(value = AddToList.class, name = "addToList")
    })
    public interface Operation<T> {
        T perform(T operand);
    }

    public static class AddToList<T> implements Operation<List<T>> {
        T value;
        @Override
        public List<T> perform(List<T> operand) {
            operand.add(value);
            return operand;
        }
    }

    public static class Container {
        Operation<List<UUID>> listOperation;
    }

    private static final ObjectMapper objectMapper = new ObjectMapper();
    static {
        objectMapper.setVisibility(new VisibilityChecker.Std(JsonAutoDetect.Visibility.ANY));
    }

    public static void main(String[] args) throws JsonProcessingException {
        AddToList<UUID> addToList = new AddToList<>();
        addToList.value = UUID.randomUUID();
        Container container = new Container();
        container.listOperation = addToList;

        System.out.println(objectMapper.writeValueAsString(container));
    }
}

This crashes with the following exception:

Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Class java.util.UUID not subtype of [collection type; class java.util.List, contains [simple type, class java.util.UUID]] (through reference chain: GenericPolymorphismTest$Container["listOperation"]->GenericPolymorphismTest$AddToList["value"])
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:388)
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:348)
	at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:343)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:698)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:581)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:706)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:690)
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292)
	at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3681)
	at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3057)
	at GenericPolymorphismTest.main(GenericPolymorphismTest.java:57)
Caused by: java.lang.IllegalArgumentException: Class java.util.UUID not subtype of [collection type; class java.util.List, contains [simple type, class java.util.UUID]]
	at com.fasterxml.jackson.databind.type.TypeFactory.constructSpecializedType(TypeFactory.java:359)
	at com.fasterxml.jackson.databind.cfg.MapperConfig.constructSpecializedType(MapperConfig.java:306)
	at com.fasterxml.jackson.databind.DatabindContext.constructSpecializedType(DatabindContext.java:149)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter._findAndAddDynamic(BeanPropertyWriter.java:870)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:682)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:690)
	... 8 more

If I try work around it by making value into a List<T> deserialisation works but serialisation gives me a List<String> instead of List<UUID> so using a list after perform will throw ClassCastExceptions.

[cowtowncoder]

And this is with late(st) version, like 2.8.8?

[Raniz85]

This is with 2.8.8

[wlp1979]

I am able to recreate this bug without using polymorphism. Using the following code, the GoodOuter can be serialized correctly, but BadOuter fails. I've tested this with both 2.8.4 and 2.8.10.

import java.util.ArrayList;
import java.util.List;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

public class JacksonBug {
	public static class Data<T> {
		private T data;

		public Data(T data) {
			this.data = data;
		}

		public T getData() {
			return data;
		}

		public static <T> Data<List<T>> of(List<T> data) {
			return new DataList<>(data);
		}
	}

	public static class DataList<T> extends Data<List<T>> {

		public DataList(List<T> data) {
			super(data);
		}
	}

	public static class Inner {
		private int index;

		public Inner(int index) {
			this.index = index;
		}

		public int getIndex() {
			return index;
		}
	}

	public static class BadOuter {
		private Data<List<Inner>> inner;

		public BadOuter(Data<List<Inner>> inner) {
			this.inner = inner;
		}

		public Data<List<Inner>> getInner() {
			return inner;
		}
	}

	public static class GoodOuter {
		private DataList<Inner> inner;

		public GoodOuter(DataList<Inner> inner) {
			this.inner = inner;
		}

		public DataList<Inner> getInner() {
			return inner;
		}
	}

	public static void main(String[] args) throws JsonProcessingException {
		List<Inner> inners = new ArrayList<>();
		for (int i = 0; i < 10; i++) {
			inners.add(new Inner(i));
		}
		GoodOuter goodOuter = new GoodOuter(new DataList<>(inners));
		BadOuter badOuter = new BadOuter(Data.of(inners));
		ObjectMapper objectMapper = new ObjectMapper();
		System.out.println("GoodOuter: " + objectMapper.writeValueAsString(goodOuter));
		System.out.println("BadOuter: " + objectMapper.writeValueAsString(badOuter));
	}
}

[cowtowncoder]

@wlp1979 Thank you for sharing.

[cowtowncoder]

Yes, I can reproduce this. Looks like code path may be slightly different, but at least there's something to dig into.

[epollan]

@cowtowncoder I isolated this down to a change introduced in 2.7.4. Serialization of both of these object graphs succeeds starting back as far as 2.5.3 and works through 2.7.3. I'll look at the changes to databind in 2.7.4 to see if there's something obvious. I'm a long-time Jackson user, but have never had cause to modify the codebase, so it may take me a bit ;)

[epollan]

Scanning through the diff, this looks fishy:

diff --git a/src/main/java/com/fasterxml/jackson/databind/type/TypeFactory.java b/src/main/java/com/fasterxml/jackson/databind/type/TypeFactory.java
index bf01aaef5..88def8c54 100644
--- a/src/main/java/com/fasterxml/jackson/databind/type/TypeFactory.java
+++ b/src/main/java/com/fasterxml/jackson/databind/type/TypeFactory.java
@@ -364,19 +364,22 @@ public final class TypeFactory
                 }
             }
             // (3) Sub-class does not take type parameters -- just resolve subtype
-            if (subclass.getTypeParameters().length == 0) {
+            int typeParamCount = subclass.getTypeParameters().length;
+            if (typeParamCount == 0) {
                 newType = _fromClass(null, subclass, TypeBindings.emptyBindings());     
                 break;
             }
-
+            
             // If not, we'll need to do more thorough forward+backwards resolution. Sigh.
             // !!! TODO (as of 28-Jan-2016, at least)
-
+            
             // 20-Oct-2015, tatu: Container, Map-types somewhat special. There is
             //    a way to fully resolve and merge hierarchies; but that gets expensive
             //    so let's, for now, try to create close-enough approximation that
             //    is not 100% same, structurally, but has equivalent information for
             //    our specific neeeds.
+            // 29-Mar-2016, tatu: See [databind#1173]  (and test `TypeResolverTest`)
+            //  for a case where this code does get invoked: not ideal
             if (baseType.isInterface()) {
                 newType = baseType.refine(subclass, TypeBindings.emptyBindings(), null,
                         new JavaType[] { baseType });
@@ -388,7 +391,20 @@ public final class TypeFactory
             if (newType == null) {
                 // But otherwise gets bit tricky, as we need to partially resolve the type hierarchy
                 // (hopefully passing null Class for root is ok)
-                newType = _fromClass(null, subclass, TypeBindings.emptyBindings());        
+                TypeBindings tb = null;
+
+                // 14-Apr-2016, tatu: One possible short-cut; if type parameter counts
+                //   match, chances are they ought to match. Let's take our chances...
+                if (baseType.containedTypeCount() == typeParamCount) {
+                    if (typeParamCount == 1) {
+                        tb = TypeBindings.create(subclass, baseType.containedType(0));
+                    } else if (typeParamCount == 2) {
+                        tb = TypeBindings.create(subclass, baseType.containedType(0),
+                                baseType.containedType(1));
+                    }
+                }
+                newType = _fromClass(null, subclass,
+                        (tb == null) ? TypeBindings.emptyBindings() : tb);
             }
         } while (false);

[cowtowncoder]

@epollan Excellent detective work. This is bit concerning, mostly since I do not think that undoing change would work here. Code would rather have to do better resolution of type parameterization, ideally "pushing" type variables through hierarchy and get binding that way.

But it does make sense that this would cause false match of bindings.

[cowtowncoder]

Yes, I think what goes wrong here is resolution of

Data<List<Inner>>

with sub-type of

 DataList

which short-circuits with assumption that matching number of type variables means that simple substitution works; not true in this case.

Proper resolution would resolve original type, retaining type variable, and then sort of back-tracking...
and the reason this is not done is that it's rather difficult thing to do. java-classmate does handle this, but it is unencumbered by API level complexity Jackson's type system has, and code can not be directly used.

This may be very difficult to solve, esp. for 2.9.x (I won't even attempt changing any older versions since risks for change will be non-trivial)

[cowtowncoder]

And this essentially goes wrong in the fallback case, in which subtype (one being specialized as) is given existing TypeBindings of base type.

Removing handling and only using base type as-is would fix this case, but breaks one of existing tests (#1186), in which (conversely) existing handling works ok.

[epollan]

Looks like the problem is in TypeFactory._bindingsForSubtype. Specifically, the baseCount == typeParamCount branch is being entered, with the variable's JavaType (Data<List<Inner>>) containing one immediate binding (Data<T> reifies T to List<Inner>), and the Class<?> (DataList<Inner>) also has one type parameter.

[epollan]

@cowtowncoder is there a README/doc that describes how to get the jackson code compiling locally? The tip of master references a parent pom of com.fasterxml.jackson:jackson-base:3.0.0-SNAPSHOT, which leads to:

[ERROR]     Non-resolvable parent POM: Could not find artifact com.fasterxml.jackson:jackson-base:pom:3.0.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 5, column 11 -> [Help 2]

I pinned the parent to the latest released version I found on maven central (2.9.2). I then get an error from the enforcer plugin complaining about jacoco:

[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-java) @ jackson-databind ---
[WARNING] Rule 2: org.apache.maven.plugins.enforcer.RequirePluginVersions failed with message:
Some plugins are missing valid versions:(LATEST RELEASE SNAPSHOT are not allowed )
org.jacoco:jacoco-maven-plugin. 	The version currently in use is 0.7.9
[ERROR] Best Practice is to always define plugin versions!

I pinned jacoco to 0.7.9, and now get java 8 compilation errors:

[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ jackson-databind ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 415 source files to /Users/epollan/src/jackson-databind/target/classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING : 
[INFO] -------------------------------------------------------------
[WARNING] bootstrap class path not set in conjunction with -source 1.7
[INFO] 1 warning
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /Users/epollan/src/jackson-databind/src/main/java/com/fasterxml/jackson/databind/ext/jdk8/Jdk8StreamSerializer.java:[74,28] lambda expressions are not supported in -source 1.7
  (use -source 8 or higher to enable lambda expressions)
[ERROR] /Users/epollan/src/jackson-databind/src/main/java/com/fasterxml/jackson/databind/ext/jdk8/LongStreamSerializer.java:[37,30] lambda expressions are not supported in -source 1.7
  (use -source 8 or higher to enable lambda expressions)
[ERROR] /Users/epollan/src/jackson-databind/src/main/java/com/fasterxml/jackson/databind/ext/jdk8/IntStreamSerializer.java:[39,30] lambda expressions are not supported in -source 1.7
  (use -source 8 or higher to enable lambda expressions)
[ERROR] /Users/epollan/src/jackson-databind/src/main/java/com/fasterxml/jackson/databind/ext/jdk8/DoubleStreamSerializer.java:[39,30] lambda expressions are not supported in -source 1.7
  (use -source 8 or higher to enable lambda expressions)

I suspect there's some basic local config I need to do to get the jackson-xxx modules building locally...

[cowtowncoder]

@epollan I would recommend using branch 2.9 actually. I can push snapshots of components for 3.0.0 (that is, master), but that is far away from being released.
There should be no special set up needed in general, but jackson-databind does have some dependencies. With 2.9 it is easier to somewhat reduce that (parent poms can be release versions, not snapshots).