Find toJson method even from DataSourceImpl against Embulk v0.10.2 or earlier, whose DataSource interace does not have "toJson"

[dmikurube]

An issue was reported about embulk-input-s3:0.5.1 using embulk-util-config:0.2.0, when running it on Embulk v0.9.23. It was not happening with embulk-input-s3:0.5.0 with embulk-util-config:0.1.4.

This issue does not happen when the user is running Embulk v0.10.3+.

It was because :

• Embulk v0.9.23's org.embulk.config.DataSource interface does not have toJson.
  • Then, the change in Fix against IllegalAccessException: Class org.embulk.util.config.Compat can not access a member of class org.embulk.util.config.DataSourceImpl with modifiers "public" #20 made it to skip calling toJson, and instead to fallback to invoking getObjectNode.
• But, the actual DataSource object was an instance of embulk-util-config's DataSourceImpl which does not have getObjectNode.
  • Then, it failed.

We still had to try looking for toJson even from the implementation class.

By another fix in #20, embulk-util-config's DataSourceImpl has already been public. Then, invoking the toJson Method should succeed if all the loaded plugins are also using embulk-util-config:0.2.1+ or embulk-core's DataSourceImpl.

---

Only a remaining case that this pull-request cannot save is :

• The user is using Embulk v0.10.2 or earlier (typically, v0.9.23), AND
• The user is using another plugin with embulk-util-config:0.1.4 or earlier.

To save this case, I prepared another fix #26. But still wondering whether we should merge this or not... (because it is too hacky).

---

Reproducing case on Embulk v0.9.23 (Copied from https://gist.github.com/Yasushi/9f24aa4f245284086d31a0862d2895c3)

in:
  type: s3
  auth_method: default
  bucket: {{ env.BUCKET }}
  path_prefix: {{ env.TARGET }}
  decoders:
    - type: gzip
  parser:
    type: fluentd_out_file
    delimiter: "\t"
    columns:
    - {name: time, type: timestamp, format: '%Y-%m-%dT%H:%M:%S%:z'}
    - {name: tag, type: string}
    - {name: record, type: json}

out:
  type: stdout

2021-02-03 01:09:44.865 +0000: Embulk v0.9.23
2021-02-03 01:09:46.367 +0000 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected.
2021-02-03 01:09:50.890 +0000 [INFO] (main): Gem's home and path are set by default: "/root/.embulk/lib/gems"
2021-02-03 01:09:53.582 +0000 [INFO] (main): Started Embulk v0.9.23
2021-02-03 01:09:53.762 +0000 [INFO] (0001:transaction): Loaded plugin embulk-input-s3 (0.5.1)
2021-02-03 01:09:53.861 +0000 [INFO] (0001:transaction): Loaded plugin embulk-parser-fluentd_out_file (0.1.0)
2021-02-03 01:09:54.518 +0000 [WARN] (0001:transaction): Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
2021-02-03 01:09:54.519 +0000 [WARN] (0001:transaction): Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
2021-02-03 01:09:54.519 +0000 [WARN] (0001:transaction): Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
2021-02-03 01:09:54.520 +0000 [WARN] (0001:transaction): Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
2021-02-03 01:09:54.521 +0000 [WARN] (0001:transaction): Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
2021-02-03 01:09:54.546 +0000 [INFO] (0001:transaction): Start listing file with prefix [xxxxxxxxxxxxxxxxxxx]
2021-02-03 01:09:55.417 +0000 [INFO] (0001:transaction): Found total [1] files
org.embulk.exec.PartialExecutionException: java.lang.ClassCastException: DataSource specified is not org.embulk.config.DataSourceImpl.
	at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:340)
	at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:566)
	at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:35)
	at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
	at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
	at org.embulk.spi.Exec.doWith(Exec.java:22)
	at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
	at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:242)
	at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:291)
	at org.embulk.EmbulkRunner.run(EmbulkRunner.java:155)
	at org.embulk.cli.EmbulkRun.runSubcommand(EmbulkRun.java:431)
	at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:90)
	at org.embulk.cli.Main.main(Main.java:64)
	Suppressed: java.lang.NullPointerException
		at org.embulk.exec.BulkLoader.doCleanup(BulkLoader.java:463)
		at org.embulk.exec.BulkLoader$3.run(BulkLoader.java:397)
		at org.embulk.exec.BulkLoader$3.run(BulkLoader.java:394)
		at org.embulk.spi.Exec.doWith(Exec.java:22)
		at org.embulk.exec.BulkLoader.cleanup(BulkLoader.java:394)
		at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:245)
		... 5 more
Caused by: java.lang.ClassCastException: DataSource specified is not org.embulk.config.DataSourceImpl.
	at org.embulk.util.config.Compat.callGetObjectNodeAndRebuildIfAvailable(Compat.java:136)
	at org.embulk.util.config.Compat.rebuildObjectNode(Compat.java:94)
	at org.embulk.util.config.TaskMapper.map(TaskMapper.java:73)
	at org.embulk.input.s3.AbstractS3FileInputPlugin.resume(AbstractS3FileInputPlugin.java:162)
	at org.embulk.input.s3.AbstractS3FileInputPlugin.transaction(AbstractS3FileInputPlugin.java:153)
	at org.embulk.spi.FileInputRunner.transaction(FileInputRunner.java:62)
	at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:507)
	... 11 more

Error: java.lang.ClassCastException: DataSource specified is not org.embulk.config.DataSourceImpl.

[huylenq]

LGTM

[dmikurube]

Thanks!