Merge pull request #23 from Nasdaq/release-0.2.x

0.2.0 release

Author: ruchirvaninasdaq

README.md

@@ -6,7 +6,7 @@ Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of deliv
 ### Equities
 #### The Nasdaq Stock Market
 - [Nasdaq Basic](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NasdaqBasic-Cloud.pdf)
-- [Nasdaq Last Sale+](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlus-cloud.pdf) 
+- [Nasdaq Last Sale+](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlus-cloud.pdf)
 - [Nasdaq TotalView](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/Totalview-ITCH-cloud.pdf)
 - [Nasdaq Consolidated Quotes and Trades](https://github.com/Nasdaq/CloudDataService/raw/master/specs/CQT-cloud.pdf)
 #### Nasdaq BX
@@ -21,13 +21,11 @@ Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of deliv
 - [Global Index Data Service](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/GIDS_Cloud.pdf)
 ### Options
 #### Nasdaq U.S. Derivatives
-- [Nasdaq Smart Options](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NCDSSmartOptions.pdf) 
+- [Nasdaq Smart Options](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NCDSSmartOptions.pdf)
 ### Mutual Funds
-- [Nasdaq Fund Network](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NFNDS_NCDS.pdf) 
+- [Nasdaq Fund Network](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NFNDS_NCDS.pdf)
 ### News
-- [Financial News](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/MTNewswires-cloud.pdf)  
-
-
+- [Financial News](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/MTNewswires-cloud.pdf)
 
 # Items To Note
 
@@ -45,7 +43,7 @@ Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of deliv
 The SDK currently supports Python 3.9 and above
 
 ### Get the SDK
-The source code is currently hosted on GitHub at: https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python 
+The source code is currently hosted on GitHub at: https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python
 - Clone the repository: ```git clone https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python.git```
 - Move into the directory ```cd NasdaqCloudDataService-SDK-Python```
 - Install the library and its dependencies from local source with ```pip install -e .```
@@ -55,31 +53,21 @@ Optional: to use the Jupyter notebook provided,
 - To run the notebook, use the command ```jupyter notebook``` and the Notebook Dashboard will open in your browser
 - Select the file ```python_sdk_examples.ipynb```
 
-### Retrieving certificates
-
-Run `ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py` with arguments, which takes the path where the certificate should be installed.
-
-For example:
-```python3.9 ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py -opt INSTALLCERTS -path /my/trusted/store/ncdsinstallcerts```
-
 ### Stream configuration
 
-  Replace example stream properties in the file **kafka-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/kafka-config.json) with provided values during on-boarding.
+Replace example stream properties in the file **kafka-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/kafka-config.json) with provided values during on-boarding.
 
-  **Note**: Ensure that the full path to the ca.crt file is provided. If the certificate was installed in the directory 
-  `/my/trusted/store/ncdsinstallcerts`, then the full path would be `/my/trusted/store/ncdsinstallcerts/ca.crt`
+  Required kafka configuration
 
 ```properties
-"bootstrap.servers": "{streams_endpoint_url}:9094"
-"ssl.ca.location": "/path/to/dir/ca.crt"
+"bootstrap.servers": {streams_endpoint_url}:9094
 ```
 
   For optional consumer configurations see: https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
 
 ### Client Authentication configuration
 
-Replace example client authentication properties in the file **client-authentication-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/client-authentication-config.json) with valid credentials provided during on-boarding. 
-
+Replace example client authentication properties in the file **client-authentication-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/client-authentication-config.json) with valid credentials provided during on-boarding.
 
 ```properties
 oauth.token.endpoint.uri: "https://{auth_endpoint_url}/auth/realms/pro-realm/protocol/openid-connect/token"
@@ -97,7 +85,6 @@ oauth.client.secret: "client_secret"
   "        * METRICS - Display the Metrics for the topic\n" +
   "        * TOPICS - List of streams available on Nasdaq Cloud DataService\n" +
   "        * GETMSG - Get one example message for the given message name\n" +
-  "        * INSTALLCERTS - Install certificate to keystore\n" +
   "        * CONTSTREAM   - Retrieve continuous stream  \n" +
   "        * FILTERSTREAM  - Retrieve continuous stream filtered by symbols and/or msgtypes \n" +
   "        * HELP - help \n" +
@@ -108,7 +95,6 @@ oauth.client.secret: "client_secret"
 "-kafkaprops -- Provide Kafka Properties File path   --- For using different set of Kafka Properties \n" +
 "-n -- Provide number of messages to retrieve        --- REQUIRED for TOP \n" +
 "-msgName -- Provide name of message based on schema --- REQUIRED for GETMSG \n" +
-"-path -- Provide the path for key store             --- REQUIRED for INSTALLCERTS \n" +
 "-timestamp -- Provide timestamp in milliseconds     --- OPTIONAL for TOP, CONTSTREAM and FILTERSTREAM\n"
 ```
 
@@ -138,7 +124,6 @@ security_cfg = {
 }
 kafka_cfg = {
     "bootstrap.servers": "{streams_endpoint_url}:9094",
-    "ssl.ca.location": "ca.crt",
     "auto.offset.reset": "earliest"
 }
 ```

ncdssdk/src/main/python/ncdsclient/internal/utils/InstallCertificates.py

@@ -31,7 +31,6 @@ def nasdaq_specific_config(p):
             p["security.protocol"] = "SASL_SSL"
             p["sasl.mechanism"] = "OAUTHBEARER"
             p["ssl.endpoint.identification.algorithm"] = "https"
-            p["enable.ssl.certificate.verification"] = False
 
         return p
 

ncdssdk/src/main/python/ncdsclient/internal/utils/KafkaConfigLoader.py

@@ -3,7 +3,6 @@
 import sys
 import logging
 from ncdssdk.src.main.python.ncdsclient.NCDSClient import NCDSClient
-from ncdssdk.src.main.python.ncdsclient.internal.utils.InstallCertificates import InstallCertificates
 from ncdssdk_client.src.main.python.ncdsclient.utils.ValidateInput import ValidateInput
 from confluent_kafka import KafkaException
 from importlib import resources
@@ -22,7 +21,6 @@ def __init__(self, cmd):
         self.symbols = cmd['symbols']
         self.msgnames = cmd['msgnames']
         self.message_name = cmd['msgname']
-        self.certificate_path = cmd['path']
         self.num_top_messages = cmd['n']
         self.timestamp = cmd['timestamp']
 
@@ -79,15 +77,6 @@ def main(self):
                 ncds_client.get_sample_messages(
                     self.topic, self.message_name, True)
 
-            elif self.test_option == "INSTALLCERTS":
-                try:
-                    print("Installing Certificates")
-                    # TODO: This could return the name of the full path of the certificate
-                    InstallCertificates(self.certificate_path).install()
-                    print(f"Installed certificate at: {self.certificate_path}")
-                except Exception as e:
-                    logging.exception(f"Error in installing certificates {e}")
-
             elif self.test_option == "TOPICS":
                 ncds_client = NCDSClient(self.security_cfg, self.kafka_cfg)
                 self.topics = ncds_client.list_topics_for_client()

ncdssdk/src/tests/unit-tests/test_InstallCertificates.py

@@ -5,7 +5,6 @@ def print_help_message():
           "        * METRICS - Display the Metrics for the topic\n" +
           "        * TOPICS - List of streams available on Nasdaq Cloud DataService\n" +
           "        * GETMSG - Get one example message for the given message name\n" +
-          "        * INSTALLCERTS - Install certificate to keystore\n" +
           "        * CONTSTREAM   - Retrieve continuous stream  \n" +
           "        * FILTERSTREAM  - Retrieve continuous stream filtered by symbols and/or msgtypes \n" +
           "        * HELP - help \n" +
@@ -16,6 +15,5 @@ def print_help_message():
           "-kafkaprops -- Provide Kafka Properties File path   --- For using different set of Kafka Properties \n" +
           "-n -- Provide number of messages to retrieve        --- REQUIRED for TOP \n" +
           "-msgName -- Provide name of message based on schema --- REQUIRED for GETMSG \n" +
-          "-path -- Provide the path for key store             --- REQUIRED for INSTALLCERTS \n" +
           "-timestamp -- Provide timestamp in milliseconds     --- OPTIONAL for TOP, CONTSTREAM and FILTERSTREAM\n"
           )

ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py

@@ -41,13 +41,6 @@ def validate_user_input(self):
                 print_help_message()
                 sys.exit()
 
-        elif self.test_option == "INSTALLCERTS":
-            if not self.certificate_path:
-                print(
-                    "You must provide -path (Directory path) for getting example message")
-                print_help_message()
-                sys.exit()
-
         elif self.test_option == "CONTSTREAM":
             if not self.topic:
                 print("You must provide -topic")

ncdssdk_client/src/main/python/ncdsclient/utils/PrintHelpMessage.py

@@ -1,5 +1,4 @@
 {
     "bootstrap.servers": "bootstrap-server-endpoint",
-    "ssl.ca.location": "ncdscert.crt",
     "auto.offset.reset": "earliest"
 }