PubSub simplification and helpers

This release removes the PubSubConfig and allows you to use BulletConfig in its place. The settings found in PubSubConfig can also now be found there.

The Publisher and Subscriber interfaces now have default methods to help with providing parts of a PubSubMessage to them, rather than having to wrap them yourself.

PubSubException can also wrap other exceptions now.

PubSub

This release (0.2.0+) adds in the interfaces and classes that will form the core of the PubSub arch shift in Bullet that will allow Bullet to support incrementally updating query results and remove Bullet's tie-in to Storm.

A PubSub or a Publisher/Subscriber is any system that lets you write messages to and read responses out of, such as Kafka. This is the piece that will tie Bullet's API to Bullet's backend (currently the backend is on Storm and we use Storm DRPC). The API will write queries through the PubSub to the backend and read responses back from the backend for the user. The backend will read the queries from the PubSub and write the results back to it. Both halves have a Publisher and a Subscriber.

The release of PubSub will enable Bullet:

1. To be implemented on any stream processor such as Spark Streaming, Flink, Heron etc
2. To support incremental updates (multiple results during the lifetime of a query)

The PubSub consists of three main abstractions: A PubSub abstract class and two interfaces, a Publisher and a Subscriber. A PubSub implementation instance takes in a configuration and returns Publishers and Subscriber instances (we both support multi-threading usage patterns and horizontally sharded usage patterns) wired up to do the right thing for where that particular PubSub is running: the backend or the API.

The config will contain the information that lets the PubSub know in which particular context it is running:

bullet.pubsub.context.name: QUERY_PROCESSING
bullet.pubsub.context.name: QUERY_SUBMISSION

QUERY_PROCESSING is the mode that the backend will use the PubSub and QUERY_SUBMISSION is the mode for the API.

The particular instance that API and backend will use is all driven through settings. You can implement your own and set:

bullet.pubsub.class.name: com.foo.bar.MyPubSub

to a class that extends our abstract PubSub class and returns instances of the Publisher interface and Subscriber interface.

Since Kafka is so popular, Bullet will release and support a bullet-kafka PubSub in the near future that implements our PubSub for Kafka to start with.

Simplifying Config

This simplifies how Config and BulletConfig are instantiated and work.

• Config can be created with a defaults file as before but needs to passed in the name of the defaults file
• BulletConfig stores the default file name for bullet-core
• Config can also be loaded with a single file for simplicity
• A new merge(Config) method is supported to combine multiple Configs

First release

This release includes the core Bullet logic as of Bullet Storm 0.4.3.

It removes the Storm settings from bullet_defaults.yaml and adds a couple of convenience methods to the BulletConfig.