Deserialization Not Working Right with Generic Types and Builders

[gilbode]

When trying to deserialize a generic type using a builder it is deserializing the generic type as a LinkedHashMap instead of the proper type, exact same code deserialized using @JsonCreator works fine. There seems to be something missing in the builder code.
Test case below demonstrates the problem:

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.instanceOf;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;

import org.junit.Test;

import java.util.List;

public class JacksonDeserTest {
  public static class MyPOJO {
    public String x;
    public String y;

    @JsonCreator
    public MyPOJO(@JsonProperty("x") String x, @JsonProperty("y") String y) {
      this.x = x;
      this.y = y;
    }
  }

  @JsonDeserialize(builder = MyGenericPOJO.Builder.class)
  public static class MyGenericPOJO<T> {
    private List<T> data;

    private MyGenericPOJO(List<T> data) {
      this.data = data;
    }

    public List<T> getData() {
      return data;
    }

    public static class Builder<T> {
      private List<T> data;

      public Builder<T> withData(List<T> data) {
        this.data = data;
        return this;
      }

      public MyGenericPOJO<T> build() {
        return new MyGenericPOJO<T>(data);
      }
    }
  }

  public static class MyGenericPOJOWithCreator<T> {
    private List<T> data;

    private MyGenericPOJOWithCreator(List<T> data) {
      this.data = data;
    }

    @JsonCreator
    public static <T> MyGenericPOJOWithCreator<T> create(@JsonProperty("data") List<T> data) {
      return new MyGenericPOJOWithCreator.Builder<T>().withData(data).build();
    }

    public List<T> getData() {
      return data;
    }

    public static class Builder<T> {
      private List<T> data;

      public Builder<T> withData(List<T> data) {
        this.data = data;
        return this;
      }

      public MyGenericPOJOWithCreator<T> build() {
        return new MyGenericPOJOWithCreator<T>(data);
      }
    }
  }

  @Test
  public void testWithBuilder() throws Exception {
    final ObjectMapper mapper = new ObjectMapper();
    final String json = "{ \"data\": [ { \"x\": \"x\", \"y\": \"y\" } ] }";
    final MyGenericPOJO<MyPOJO> deserialized =
        mapper.readValue(json, new TypeReference<MyGenericPOJO<MyPOJO>>() {});
    assertThat(deserialized.data, hasSize(1));
    assertThat(deserialized.data.get(0), instanceOf(MyPOJO.class));

  }

  @Test
  public void testWithCreator() throws Exception {
    final ObjectMapper mapper = new ObjectMapper();
    final String json = "{ \"data\": [ { \"x\": \"x\", \"y\": \"y\" } ] }";
    final MyGenericPOJOWithCreator<MyPOJO> deserialized =
        mapper.readValue(json, new TypeReference<MyGenericPOJOWithCreator<MyPOJO>>() {});
    assertThat(deserialized.data, hasSize(1));
    assertThat(deserialized.data.get(0), instanceOf(MyPOJO.class));

  }
}

[cowtowncoder]

Getting a LinkedHashMap does indeed suggest that generic type is dropped, and java.lang.Object is assumed as the type.
Thank you for the test case: I hope this can be supported as expected.

[jaroslawZawila]

I do face the same problem what is the status of this issue ?? is there any work around??

[cowtowncoder]

@jaroslawZawila I don't think anyone has been working on this issue.

[jaroslawZawila]

Thanks for your response. Could you point me in the direction where do you think is a problem with that?? I may look at this as I really need it to work :D

[cowtowncoder]

@jaroslawZawila I would first verify that this still occurs with 2.7 (that is, 2.7.0-SNAPSHOT from master branch, using local build or snapshot). If it does not occur, you might want to wait for 2.7.0-rc1 which should be released quite soon (I am hoping for next weekend). If it still occurs, the way generics are resolved probably needs more work; you'd have to follow the build process and compare that to standard handling of Bean(De)Serializer. I suspect this is related to either using raw (type erased) type where generic type is needed; or generic type being resolved using wrong context. Latter becomes bit involved, but basically one has to get type from accessor and resolve it against exact class in which it was declared. With 2.7 this is properly automated so that calling 'getType()' of accessor (like AnnotatedField or AnnotatedMethod) should do it all; but with older versions, calls had to do more work-arounds.

So yeah, it can be bit involved. I hope to get this working for 2.7 regardless, but there is a significant backlog of issues to work on unfortunately. So I appreciate any help you can give!

[jaroslawZawila]

I have tried using 2.7.0-SNAPSHOT unfortunately the issue still exists. I will investigate a bit more.

[cowtowncoder]

I can reproduce this with 2.7.0-rc1; added the failing unit test.

[vjkoskela]

I did some checking and the sample test case provided by @gilbode and committed in 53fb51f by @cowtowncoder fails for the heads of all the 2.x branches (2.1 through 2.7 -- inclusive). It also fails on master. So this does not appear to be a regression -- not sure if that was clear, at least based on a-k-g's issue #1210.

I've been debugging this in the 2.7 branch and in BeanDeserializerFactory's createBuilderBasedDeserializer method the valueType contains the bindings as expected, they are not in beanDesc but looking at the BasicBeanDescription that does not seem unexpected. However, the builderType created in this method does not discover/inherit the bindings from the valueType. As a crude first attempt I tried to build the builder type with the value type's bindings:

JavaType builderType = ctxt.getTypeFactory().constructType(builderClass, valueType.getBindings())

However, builderClass is not a ParameterizedType so TypeFactory refuses to perform this construction. Temporarily working around this by exposing _fromClass on TypeFactory publicly yields:

JavaType builderType = ctxt.getTypeFactory()._fromClass(null, builderClass, valueType.getBindings());

And this fixes the problem! -- and all existing tests continue to pass -- So aside from coming up with a cleaner way in TypeFactory to allow this combined type creation (builder base class and value type bindings) there is the question of what assumptions this introduces between the type bindings of the value type and builder.

Consider:

1. Rename the T parameter to U in the builder only. This results in the same problem where the generic member's type cannot be determined correctly. Below is the pojo, builder and test:

@JsonDeserialize(builder = MyGenericPOJO_B.Builder.class)
    public static class MyGenericPOJO_B<T> {
        private List<T> data;

        private MyGenericPOJO_B(List<T> d) {
            data = d;
        }

        public List<T> getData() {
            return data;
        }

        public static class Builder<U> {
            private List<U> data;

            public Builder<U> withData(List<U> d) {
                data = d;
                return this;
            }

            public MyGenericPOJO_B<U> build() {
                return new MyGenericPOJO_B<U>(data);
            }
        }
    }

    public void testWithBuilder_B() throws Exception {
        final ObjectMapper mapper = new ObjectMapper();
        final String json = "{ \"data\": [ { \"x\": \"x\", \"y\": \"y\" } ] }";
        final MyGenericPOJO_B<MyPOJO> deserialized =
                mapper.readValue(json, new TypeReference<MyGenericPOJO_B<MyPOJO>>() {});
        assertEquals(1, deserialized.data.size());
        Object ob = deserialized.data.get(0);
        assertNotNull(ob);
        assertEquals(MyPOJO.class, ob.getClass());
    }

2. Two parameters if the order of types are swapped across the value type and builder type. No surprise here either, this fails to deserialize correctly. Below is the pojo, builder and test:

    public static class MyGenericPOJOWithCreator<T, U> {
      private List<T> data;
      private Map<String, U> map;

      private MyGenericPOJOWithCreator(List<T> d, Map<String, U> m) {
          data = d;
          map = m;
      }

      @JsonCreator
      public static <T, U> MyGenericPOJOWithCreator<T, U> create(
              @JsonProperty("data") List<T> data,
              @JsonProperty("map") Map<String, U> map) {
          return new MyGenericPOJOWithCreator.Builder<T, U>().withData(data).withMap(map).build();
      }

      public List<T> getData() {
          return data;
      }

      public Map<String, U> getMap() {
          return map;
      }

      public static class Builder<U, T> {
          private List<U> data;
          private Map<String, T> map;

          public Builder<U, T> withData(List<U> d) {
              data = d;
              return this;
          }

          public Builder<U, T> withMap(Map<String, T> m) {
              map = m;
              return this;
          }

          public MyGenericPOJOWithCreator<U, T> build() {
              return new MyGenericPOJOWithCreator<U, T>(data, map);
          }
      }
    }

    public void testWithBuilder() throws Exception {
      final ObjectMapper mapper = new ObjectMapper();
      final String json = "{ \"data\": [ { \"x\": \"x\", \"y\": \"y\" } ], \"map\": { \"a\": 1, \"b\": 2 } }";
      final MyGenericPOJO<MyPOJO, Integer> deserialized =
          mapper.readValue(json, new TypeReference<MyGenericPOJO<MyPOJO, Integer>>() {});
      assertEquals(1, deserialized.data.size());
      Object ob = deserialized.data.get(0);
      assertNotNull(ob);
      assertEquals(MyPOJO.class, ob.getClass());
    }

The exception shows it tries to deserialize the integer value for the data object:

com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.Integer out of START_OBJECT token
 at [Source: { "data": [ { "x": "x", "y": "y" } ], "map": { "a": 1, "b": 2 } }; line: 1, column: 13] (through reference chain: com.fasterxml.jackson.databind.Builder["data"]->java.util.ArrayList[0])

The type reference for the value type does not imply anything about the builder's generic type bindings. One option that we could consider a BuilderTypeReference to pass such a binding into the deserialization. Another option would be to add annotations to the builder and/or value expressing how the bindings map from the value class to the builder class. However, such measures should only be necessary when the bindings don't map 1:1 in name and order across the value type and builder type. The common case for us at least is that the parameter mapping exists so simply supporting transfer of bindings does not seem like an unreasonable start.

If someone can give some guidance on my strategy and how TypeFactory should be extended to supporting merging a value type's bindings with the builder class into a builder type I can submit a PR for this. Otherwise, if I'm barking up the wrong tree here any pointers would be welcome!