From 6f4b0e055eada598c05a968fa41e940554ae40a8 Mon Sep 17 00:00:00 2001 From: Ruchir Vani <32844383+ruchirvaninasdaq@users.noreply.github.com> Date: Mon, 1 Nov 2021 13:41:31 -0400 Subject: [PATCH 1/2] 0.2.0 release Removing certificate pinning Co-Authored-By: Spencer Sortman <47249767+ssortman@users.noreply.github.com> --- README.md | 70 ++---- .../internal/utils/InstallCertificates.py | 48 ---- .../internal/utils/KafkaConfigLoader.py | 1 - .../unit-tests/test_InstallCertificates.py | 32 --- .../src/main/python/ncdsclient/NCDSSession.py | 11 - .../ncdsclient/utils/PrintHelpMessage.py | 2 - .../python/ncdsclient/utils/ValidateInput.py | 7 - .../main/python/resources/kafka-config.json | 1 - python_sdk_examples.ipynb | 221 +++++++----------- setup.py | 4 +- 10 files changed, 103 insertions(+), 294 deletions(-) delete mode 100644 ncdssdk/src/main/python/ncdsclient/internal/utils/InstallCertificates.py delete mode 100644 ncdssdk/src/tests/unit-tests/test_InstallCertificates.py diff --git a/README.md b/README.md index 3431cd1..f036eb6 100644 --- a/README.md +++ b/README.md @@ -2,38 +2,12 @@ Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of delivery for realtime exchange data and other financial information. Data is made available through a suite of APIs, allowing for effortless integration of data from disparate sources, and a dramatic reduction in time to market for customer-designed applications. The API is highly scalable, and robust enough to support the delivery of real-time exchange data. -# Products Currently Available -### Equities -#### The Nasdaq Stock Market -- [Nasdaq Basic](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NasdaqBasic-Cloud.pdf) -- [Nasdaq Last Sale+](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlus-cloud.pdf) -- [Nasdaq TotalView](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/Totalview-ITCH-cloud.pdf) -- [Nasdaq Consolidated Quotes and Trades](https://github.com/Nasdaq/CloudDataService/raw/master/specs/CQT-cloud.pdf) -#### Nasdaq BX -- [BX BBO](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/BX_BBO_Cloud.pdf) -- [BX Last Sale](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/BLS_Cloud.pdf) -#### Nasdaq PSX -- [PSX BBO](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/PSX_BBO_Cloud.pdf) -- [PSX Last Sale](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/PLS_Cloud.pdf) -#### Nasdaq Canada -- [Nasdaq Canada Basic](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/Nasdaq-Basic-Canada-Cloud-Specification.pdf) -### Indexes & ETPs -- [Global Index Data Service](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/GIDS_Cloud.pdf) -### Options -#### Nasdaq U.S. Derivatives -- [Nasdaq Smart Options](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NCDSSmartOptions.pdf) -### Mutual Funds -- [Nasdaq Fund Network](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NFNDS_NCDS.pdf) -### News -- [Financial News](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/MTNewswires-cloud.pdf) - - - -# Items To Note + +### Items To Note * Connecting to the API requires credentials, which are provided by the Nasdaq Data Operations team during an on-boarding process * This sample code only connects to one topic (NLSCTA); during on-boarding process, you will receive a topic list that you're entitled to. -* See https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Java for our officially support Java-based SDK. +* See https://github.com/Nasdaq/CloudDataService for our officially support Java-based SDK. # Table of Contents @@ -45,46 +19,37 @@ Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of deliv The SDK currently supports Python 3.9 and above ### Get the SDK -The source code is currently hosted on GitHub at: https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python -- Clone the repository: ```git clone https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python.git``` -- Move into the directory ```cd NasdaqCloudDataService-SDK-Python``` -- Install the library and its dependencies from local source with ```pip install -e .``` +The source code is currently hosted on GitHub at: https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​ +- Clone the repository: ```git clone https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​.git``` +- Move into the directory ```cd ncds-python-sample-consumer``` +- Install the library and its dependencies from local source with ```pip install .``` Optional: to use the Jupyter notebook provided, -- Download Jupyter notebook using either pip ```pip3 install notebook``` or conda ```conda install -c conda-forge notebook``` +- Download Jupyter notebook using pip ```pip install notebook``` - To run the notebook, use the command ```jupyter notebook``` and the Notebook Dashboard will open in your browser - Select the file ```python_sdk_examples.ipynb``` -### Retrieving certificates - -Run `ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py` with arguments, which takes the path where the certificate should be installed. - -For example: -```python3.9 ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py -opt INSTALLCERTS -path /my/trusted/store/ncdsinstallcerts``` - ### Stream configuration - Replace example stream properties in the file **kafka-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/kafka-config.json) with provided values during on-boarding. + Replace example stream properties in the file **kafka-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​/blob/master/ncdssdk_client/src/main/python/resources/kafka-config.json) with provided values during on-boarding. - **Note**: Ensure that the full path to the ca.crt file is provided. If the certificate was installed in the directory - `/my/trusted/store/ncdsinstallcerts`, then the full path would be `/my/trusted/store/ncdsinstallcerts/ca.crt` + Required kafka configuration ```properties -"bootstrap.servers": "{streams_endpoint_url}:9094" -"ssl.ca.location": "/path/to/dir/ca.crt" +"bootstrap.servers": {streams_endpoint_url}:9094 ``` For optional consumer configurations see: https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md ### Client Authentication configuration -Replace example client authentication properties in the file **client-authentication-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/client-authentication-config.json) with valid credentials provided during on-boarding. +Replace example client authentication properties in the file **client-authentication-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​/blob/master/ncdssdk_client/src/main/python/resources/client-authentication-config.json) with valid credentials provided during on-boarding. ```properties -oauth.token.endpoint.uri: "https://{auth_endpoint_url}/auth/realms/pro-realm/protocol/openid-connect/token" -oauth.client.id: "client_id" -oauth.client.secret: "client_secret" +oauth.token.endpoint.uri: https://{auth_endpoint_url}/auth/realms/demo/protocol/openid-connect/token +oauth.client.id: client +oauth.client.secret: client-secret ``` ### Create NCDS Session Client @@ -97,7 +62,6 @@ oauth.client.secret: "client_secret" " * METRICS - Display the Metrics for the topic\n" + " * TOPICS - List of streams available on Nasdaq Cloud DataService\n" + " * GETMSG - Get one example message for the given message name\n" + - " * INSTALLCERTS - Install certificate to keystore\n" + " * CONTSTREAM - Retrieve continuous stream \n" + " * FILTERSTREAM - Retrieve continuous stream filtered by symbols and/or msgtypes \n" + " * HELP - help \n" + @@ -108,7 +72,6 @@ oauth.client.secret: "client_secret" "-kafkaprops -- Provide Kafka Properties File path --- For using different set of Kafka Properties \n" + "-n -- Provide number of messages to retrieve --- REQUIRED for TOP \n" + "-msgName -- Provide name of message based on schema --- REQUIRED for GETMSG \n" + -"-path -- Provide the path for key store --- REQUIRED for INSTALLCERTS \n" + "-timestamp -- Provide timestamp in milliseconds --- OPTIONAL for TOP, CONTSTREAM and FILTERSTREAM\n" ``` @@ -138,7 +101,6 @@ security_cfg = { } kafka_cfg = { "bootstrap.servers": "{streams_endpoint_url}:9094", - "ssl.ca.location": "ca.crt", "auto.offset.reset": "earliest" } ``` @@ -302,7 +264,7 @@ while True: print(f"No Records Found for the Topic: {topic}") for message in messages: - print(f"value :" + str(message.value())) + print(f"value :" + message.value()) ``` Example output: diff --git a/ncdssdk/src/main/python/ncdsclient/internal/utils/InstallCertificates.py b/ncdssdk/src/main/python/ncdsclient/internal/utils/InstallCertificates.py deleted file mode 100644 index df57f81..0000000 --- a/ncdssdk/src/main/python/ncdsclient/internal/utils/InstallCertificates.py +++ /dev/null @@ -1,48 +0,0 @@ -import os.path -import requests -from os import path - - -class InstallCertificates: - """ - Gets a one-time-url from kafka_cert_url and downloads the kafka stream certificate from that url. - - Attributes: - cert_dir (str): The name of the directory where the certificate should be installed - """ - - def __init__(self, cert_dir): - self.cert_dir = cert_dir - self.cert_path = os.path.join(cert_dir, "ca.crt") - self.kafka_cert_url = "https://tgbstk5716.execute-api.us-east-1.amazonaws.com/v1/get-certificate" - - def install(self): - return self.__install_cert() - - def __install_cert(self): - one_time_url = self.__get_one_time_url() - - response = requests.get(one_time_url) - - if not path.exists(self.cert_dir): - raise Exception(f"Path was not found: {self.cert_dir}") - - with open(self.cert_path, 'wb') as f: - for chunk in response.iter_content(chunk_size=128): - f.write(chunk) - - return os.path.abspath(self.cert_path) - - def __get_one_time_url(self): - response = requests.get( - self.kafka_cert_url, - headers={'Prama': 'no-cache', 'Cache-Control': 'no-cache'} - ) - - if response.status_code != 200: - raise Exception("Internal Server Error") - - json_response = response.json() - one_time_url = json_response['one_time_url'] - - return one_time_url diff --git a/ncdssdk/src/main/python/ncdsclient/internal/utils/KafkaConfigLoader.py b/ncdssdk/src/main/python/ncdsclient/internal/utils/KafkaConfigLoader.py index 41c519e..10de023 100644 --- a/ncdssdk/src/main/python/ncdsclient/internal/utils/KafkaConfigLoader.py +++ b/ncdssdk/src/main/python/ncdsclient/internal/utils/KafkaConfigLoader.py @@ -31,7 +31,6 @@ def nasdaq_specific_config(p): p["security.protocol"] = "SASL_SSL" p["sasl.mechanism"] = "OAUTHBEARER" p["ssl.endpoint.identification.algorithm"] = "https" - p["enable.ssl.certificate.verification"] = False return p diff --git a/ncdssdk/src/tests/unit-tests/test_InstallCertificates.py b/ncdssdk/src/tests/unit-tests/test_InstallCertificates.py deleted file mode 100644 index 461e476..0000000 --- a/ncdssdk/src/tests/unit-tests/test_InstallCertificates.py +++ /dev/null @@ -1,32 +0,0 @@ -from ncdssdk.src.main.python.ncdsclient.internal.utils.InstallCertificates import InstallCertificates -from os import path, makedirs -import shutil - - -def test_install_at_default(): - install_cert = InstallCertificates() - install_cert.install() - - assert path.exists('/tmp/ca.crt') - - -def test_install_at_existing_dir(): - p = './tmp/testDir' - if not path.exists(p): - makedirs(p) - - install_cert = InstallCertificates(p) - install_cert.install() - - assert path.exists(p + '/ca.crt') - - -def test_install_at_non_existing_dir_and_file(): - p = './tmp/testDir' - if path.exists(p): - shutil.rmtree(p) - - install_cert = InstallCertificates(p) - install_cert.install() - - assert path.exists(p + '/ca.crt') diff --git a/ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py b/ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py index 2560050..d99e840 100644 --- a/ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py +++ b/ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py @@ -3,7 +3,6 @@ import sys import logging from ncdssdk.src.main.python.ncdsclient.NCDSClient import NCDSClient -from ncdssdk.src.main.python.ncdsclient.internal.utils.InstallCertificates import InstallCertificates from ncdssdk_client.src.main.python.ncdsclient.utils.ValidateInput import ValidateInput from confluent_kafka import KafkaException from importlib import resources @@ -22,7 +21,6 @@ def __init__(self, cmd): self.symbols = cmd['symbols'] self.msgnames = cmd['msgnames'] self.message_name = cmd['msgname'] - self.certificate_path = cmd['path'] self.num_top_messages = cmd['n'] self.timestamp = cmd['timestamp'] @@ -79,15 +77,6 @@ def main(self): ncds_client.get_sample_messages( self.topic, self.message_name, True) - elif self.test_option == "INSTALLCERTS": - try: - print("Installing Certificates") - # TODO: This could return the name of the full path of the certificate - InstallCertificates(self.certificate_path).install() - print(f"Installed certificate at: {self.certificate_path}") - except Exception as e: - logging.exception(f"Error in installing certificates {e}") - elif self.test_option == "TOPICS": ncds_client = NCDSClient(self.security_cfg, self.kafka_cfg) self.topics = ncds_client.list_topics_for_client() diff --git a/ncdssdk_client/src/main/python/ncdsclient/utils/PrintHelpMessage.py b/ncdssdk_client/src/main/python/ncdsclient/utils/PrintHelpMessage.py index 95a09ff..5e9ee6c 100644 --- a/ncdssdk_client/src/main/python/ncdsclient/utils/PrintHelpMessage.py +++ b/ncdssdk_client/src/main/python/ncdsclient/utils/PrintHelpMessage.py @@ -5,7 +5,6 @@ def print_help_message(): " * METRICS - Display the Metrics for the topic\n" + " * TOPICS - List of streams available on Nasdaq Cloud DataService\n" + " * GETMSG - Get one example message for the given message name\n" + - " * INSTALLCERTS - Install certificate to keystore\n" + " * CONTSTREAM - Retrieve continuous stream \n" + " * FILTERSTREAM - Retrieve continuous stream filtered by symbols and/or msgtypes \n" + " * HELP - help \n" + @@ -16,6 +15,5 @@ def print_help_message(): "-kafkaprops -- Provide Kafka Properties File path --- For using different set of Kafka Properties \n" + "-n -- Provide number of messages to retrieve --- REQUIRED for TOP \n" + "-msgName -- Provide name of message based on schema --- REQUIRED for GETMSG \n" + - "-path -- Provide the path for key store --- REQUIRED for INSTALLCERTS \n" + "-timestamp -- Provide timestamp in milliseconds --- OPTIONAL for TOP, CONTSTREAM and FILTERSTREAM\n" ) diff --git a/ncdssdk_client/src/main/python/ncdsclient/utils/ValidateInput.py b/ncdssdk_client/src/main/python/ncdsclient/utils/ValidateInput.py index a12422b..c057cb8 100644 --- a/ncdssdk_client/src/main/python/ncdsclient/utils/ValidateInput.py +++ b/ncdssdk_client/src/main/python/ncdsclient/utils/ValidateInput.py @@ -41,13 +41,6 @@ def validate_user_input(self): print_help_message() sys.exit() - elif self.test_option == "INSTALLCERTS": - if not self.certificate_path: - print( - "You must provide -path (Directory path) for getting example message") - print_help_message() - sys.exit() - elif self.test_option == "CONTSTREAM": if not self.topic: print("You must provide -topic") diff --git a/ncdssdk_client/src/main/python/resources/kafka-config.json b/ncdssdk_client/src/main/python/resources/kafka-config.json index 55176ff..ddb536b 100644 --- a/ncdssdk_client/src/main/python/resources/kafka-config.json +++ b/ncdssdk_client/src/main/python/resources/kafka-config.json @@ -1,5 +1,4 @@ { "bootstrap.servers": "bootstrap-server-endpoint", - "ssl.ca.location": "ncdscert.crt", "auto.offset.reset": "earliest" } diff --git a/python_sdk_examples.ipynb b/python_sdk_examples.ipynb index 814e4a6..b4ee72c 100644 --- a/python_sdk_examples.ipynb +++ b/python_sdk_examples.ipynb @@ -2,57 +2,28 @@ "cells": [ { "cell_type": "markdown", - "id": "3492b987", - "metadata": {}, "source": [ "# Using the NCDS Python SDK\n", "\n", - "This is a jupyter notebook for the example code of the Python SDK. \n", - "\n", - "Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of delivery for realtime exchange data and other financial information. Data is made available through a suite of APIs, allowing for effortless integration of data from disparate sources, and a dramatic reduction in time to market for customer-designed applications. The API is highly scalable, and robust enough to support the delivery of real-time exchange data. \n", + "This is a jupyter notebook for the example code of the Python SDK. To get set-up with the SDK, follow the directions on the github page.\n", "\n", "**NOTE**: Be sure that your notebook is using Python 3.9 when running the examples" - ] - }, - { - "cell_type": "markdown", - "id": "baf91a24", - "metadata": {}, - "source": [ - "## Installation\n", - "\n", - "Install the dependencies and certificate necessary for running the notebook by running the following cell block." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a6c505ee", - "metadata": {}, - "outputs": [], - "source": [ - "! pip install .\n", - "! python ncdssdk_client/src/main/python/ncdsclient/NCDSSession.py -opt INSTALLCERTS -path ." - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "id": "1695a62a", - "metadata": {}, "source": [ - "## Items To Note\n", + "## Code Examples\n", "\n", - "* Connecting to the API requires credentials, which are provided by the Nasdaq Data Operations team during an on-boarding process\n", - "* This sample code only connects to one topic; during on-boarding process, you will receive a topic list that you're entitled to.\n", - "* See https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Java for our officially support Java-based SDK.\n" - ] + "### Getting list of data stream available\n", + "List all available data stream for the user" + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 1, - "id": "7ca38c3f", - "metadata": {}, - "outputs": [], "source": [ "security_cfg = {\n", " \"oauth.token.endpoint.uri\": \"https://{auth_endpoint_url}/auth/realms/pro-realm/protocol/openid-connect/token\",\n", @@ -64,39 +35,33 @@ " \"ssl.ca.location\": \"ca.crt\",\n", " \"auto.offset.reset\": \"earliest\"\n", "}" - ] - }, - { - "cell_type": "markdown", - "id": "caad7abe", - "metadata": {}, - "source": [ - "\n", - "## Code Examples\n", - "\n", - "### Getting list of data stream available\n", - "List all available data stream for the user" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "code", "execution_count": 2, - "id": "75589d27", - "metadata": {}, - "outputs": [], "source": [ "from ncdssdk import NCDSClient" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "code", "execution_count": 3, - "id": "e1438284", - "metadata": {}, + "source": [ + "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", + "topics = ncds_client.list_topics_for_client()\n", + "print(\"Data set topics:\")\n", + "for topic_entry in topics:\n", + " print(topic_entry)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Entitled data set topics:\n", "TOTALVIEW\n", @@ -116,63 +81,60 @@ ] } ], - "source": [ - "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", - "topics = ncds_client.list_topics_for_client()\n", - "print(\"Data set topics:\")\n", - "for topic_entry in topics:\n", - " print(topic_entry)" - ] + "metadata": {} }, { "cell_type": "markdown", - "id": "03d2bdae", - "metadata": {}, "source": [ "### Getting schema for the stream\n", "\n", "This method returns the schema for the stream in Apache Avro format (https://avro.apache.org/docs/current/spec.html)" - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 3, - "id": "271c5e0a", - "metadata": {}, + "source": [ + "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", + "topic = \"NLSCTA\"\n", + "schema = ncds_client.get_schema_for_topic(topic)\n", + "print(schema)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "latest record found\n", "[{\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqAdjClosingPrice\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"int\", \"name\": \"adjClosingPrice\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqAdjClosingPriceLong\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"long\", \"name\": \"adjClosingPrice\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqDirectoryMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"marketClass\"}, {\"type\": \"string\", \"name\": \"fsi\"}, {\"type\": [\"null\", \"int\"], \"name\": \"roundLotSize\"}, {\"type\": [\"null\", \"string\"], \"name\": \"roundLotOnly\"}, {\"type\": [\"null\", \"string\"], \"name\": \"issueClass\"}, {\"type\": [\"null\", \"string\"], \"name\": \"issueSubtype\"}, {\"type\": [\"null\", \"string\"], \"name\": \"authenticity\"}, {\"type\": [\"null\", \"string\"], \"name\": \"shortThreshold\"}, {\"type\": [\"null\", \"string\"], \"name\": \"ipo\"}, {\"type\": [\"null\", \"string\"], \"name\": \"luldTier\"}, {\"type\": [\"null\", \"string\"], \"name\": \"etf\"}, {\"type\": [\"null\", \"int\"], \"name\": \"etfFactor\"}, {\"type\": [\"null\", \"string\"], \"name\": \"inverseETF\"}, {\"type\": [\"null\", \"string\"], \"name\": \"compositeId\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqEndOfDayTradeSummary\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"int\", \"name\": \"consHigh\"}, {\"type\": \"int\", \"name\": \"consLow\"}, {\"type\": \"int\", \"name\": \"consClose\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}, {\"type\": \"int\", \"name\": \"consOpen\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqEndOfDayTradeSummaryETMF\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"int\", \"name\": \"consHigh\"}, {\"type\": \"int\", \"name\": \"consHighNAV\"}, {\"type\": \"int\", \"name\": \"consLow\"}, {\"type\": \"int\", \"name\": \"consLowNAV\"}, {\"type\": \"int\", \"name\": \"consClose\"}, {\"type\": \"int\", \"name\": \"consCloseNAV\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}, {\"type\": \"int\", \"name\": \"consOpen\"}, {\"type\": \"int\", \"name\": \"consOpenNAV\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqEndOfDayTradeSummaryLong\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"long\", \"name\": \"consHigh\"}, {\"type\": \"long\", \"name\": \"consLow\"}, {\"type\": \"long\", \"name\": \"consClose\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}, {\"type\": \"long\", \"name\": \"consOpen\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqIPOMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"refForNetChange\"}, {\"type\": \"int\", \"name\": \"refPrice\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqIpoQuoting\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"int\", \"name\": \"releaseTime\"}, {\"type\": \"string\", \"name\": \"releaseQualifier\"}, {\"type\": \"int\", \"name\": \"ipoPrice\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqMWCB\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"long\", \"name\": \"level1\"}, {\"type\": \"long\", \"name\": \"level2\"}, {\"type\": \"long\", \"name\": \"level3\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqMWCBStatus\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"level\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqMarketCenterActionMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"market\"}, {\"type\": \"string\", \"name\": \"action\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqShortSaleRestrictionIndicator\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"regSHOAction\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqSystemEventMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"event\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeCancel\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"origControlNumber\"}, {\"type\": \"int\", \"name\": \"origPrice\"}, {\"type\": \"int\", \"name\": \"origSize\"}, {\"type\": \"string\", \"name\": \"origSaleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeCancelETMFMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"origControlNumber\"}, {\"type\": \"int\", \"name\": \"origPrice\"}, {\"type\": \"int\", \"name\": \"origNavPrice\"}, {\"type\": \"int\", \"name\": \"origSize\"}, {\"type\": \"string\", \"name\": \"origSaleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeCancelLong\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"origControlNumber\"}, {\"type\": \"long\", \"name\": \"origPrice\"}, {\"type\": \"int\", \"name\": \"origSize\"}, {\"type\": \"string\", \"name\": \"origSaleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeCorrection\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"origControlNumber\"}, {\"type\": \"int\", \"name\": \"origPrice\"}, {\"type\": \"int\", \"name\": \"origSize\"}, {\"type\": \"string\", \"name\": \"origSaleCondition\"}, {\"type\": \"string\", \"name\": \"correctedControlNumber\"}, {\"type\": \"int\", \"name\": \"correctedPrice\"}, {\"type\": \"int\", \"name\": \"correctedSize\"}, {\"type\": \"string\", \"name\": \"correctedSaleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeCorrectionETMFMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"origControlNumber\"}, {\"type\": \"int\", \"name\": \"origPrice\"}, {\"type\": \"int\", \"name\": \"origNavPremium\"}, {\"type\": \"int\", \"name\": \"origSize\"}, {\"type\": \"string\", \"name\": \"origSaleCondition\"}, {\"type\": \"string\", \"name\": \"correctedControlNumber\"}, {\"type\": \"int\", \"name\": \"correctedPrice\"}, {\"type\": \"int\", \"name\": \"correctedNavPremium\"}, {\"type\": \"int\", \"name\": \"correctedSize\"}, {\"type\": \"string\", \"name\": \"correctedSaleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeCorrectionLong\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"origControlNumber\"}, {\"type\": \"long\", \"name\": \"origPrice\"}, {\"type\": \"int\", \"name\": \"origSize\"}, {\"type\": \"string\", \"name\": \"origSaleCondition\"}, {\"type\": \"string\", \"name\": \"correctedControlNumber\"}, {\"type\": \"long\", \"name\": \"correctedPrice\"}, {\"type\": \"int\", \"name\": \"correctedSize\"}, {\"type\": \"string\", \"name\": \"correctedSaleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeReportETMFMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"controlNumber\"}, {\"type\": \"int\", \"name\": \"price\"}, {\"type\": \"int\", \"name\": \"size\"}, {\"type\": \"int\", \"name\": \"navPremium\"}, {\"type\": \"string\", \"name\": \"saleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeReportLongMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"controlNumber\"}, {\"type\": \"long\", \"name\": \"price\"}, {\"type\": \"int\", \"name\": \"size\"}, {\"type\": \"string\", \"name\": \"saleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradeReportMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"marketCenter\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"securityClass\"}, {\"type\": \"string\", \"name\": \"controlNumber\"}, {\"type\": \"int\", \"name\": \"price\"}, {\"type\": \"int\", \"name\": \"size\"}, {\"type\": \"string\", \"name\": \"saleCondition\"}, {\"type\": \"long\", \"name\": \"cosolidatedVolume\"}]}, {\"type\": \"record\", \"version\": \"1\", \"name\": \"SeqTradingStateMessage\", \"namespace\": \"com.nasdaq.equities.trades.applications.nls.messaging.binary21\", \"fields\": [{\"type\": \"int\", \"name\": \"SoupPartition\"}, {\"type\": \"long\", \"name\": \"SoupSequence\"}, {\"type\": \"long\", \"name\": \"trackingID\"}, {\"type\": \"string\", \"name\": \"msgType\"}, {\"type\": \"string\", \"name\": \"filler\"}, {\"type\": \"string\", \"name\": \"symbol\"}, {\"type\": \"string\", \"name\": \"market\"}, {\"type\": \"string\", \"name\": \"tradingState\"}, {\"type\": \"string\", \"name\": \"reason\"}]}]\n" ] } ], - "source": [ - "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", - "topic = \"NLSCTA\"\n", - "schema = ncds_client.get_schema_for_topic(topic)\n", - "print(schema)" - ] + "metadata": {} }, { "cell_type": "markdown", - "id": "1edb0601", - "metadata": {}, "source": [ "### Get first 10 messages of the stream" - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 5, - "id": "09efdd2e", - "metadata": {}, + "source": [ + "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", + "topic = \"NLSCTA\"\n", + "records = ncds_client.top_messages(topic)\n", + "for i in range(0, 10):\n", + " print(\"key: \", records[i].key())\n", + " print(\"value: \", str(records[i].value()))" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "latest record found\n", "key: 1\n", @@ -198,33 +160,32 @@ ] } ], - "source": [ - "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", - "topic = \"NLSCTA\"\n", - "records = ncds_client.top_messages(topic)\n", - "for i in range(0, 10):\n", - " print(\"key: \", records[i].key())\n", - " print(\"value: \", str(records[i].value()))" - ] + "metadata": {} }, { "cell_type": "markdown", - "id": "89c3d200", - "metadata": {}, "source": [ "### Get first 10 messages of the stream from given timestamp\n", "This returns the first 10 available messages of the stream given timestamp in milliseconds since the UNIX epoch." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 6, - "id": "f43d8fd0", - "metadata": {}, + "source": [ + "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", + "topic=\"NLSCTA\"\n", + "timestamp = 1590084446510\n", + "records = ncds_client.top_messages(topic, timestamp)\n", + "for i in range(0, 10):\n", + " print(\"key: \", records[i].key())\n", + " print(\"value: \", str(records[i].value()))" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "latest record found\n", "key: 1\n", @@ -250,63 +211,62 @@ ] } ], - "source": [ - "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", - "topic=\"NLSCTA\"\n", - "timestamp = 1590084446510\n", - "records = ncds_client.top_messages(topic, timestamp)\n", - "for i in range(0, 10):\n", - " print(\"key: \", records[i].key())\n", - " print(\"value: \", str(records[i].value()))" - ] + "metadata": {} }, { "cell_type": "markdown", - "id": "ed5625ee", - "metadata": {}, "source": [ "### Get example message from stream\n", "Print message to the console for given message name." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 5, - "id": "1554eb95", - "metadata": {}, + "source": [ + "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", + "topic = \"NLSCTA\"\n", + "print(ncds_client.get_sample_messages(topic, \"SeqDirectoryMessage\", all_messages=False))" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "latest record found\n", "{'SoupPartition': 0, 'SoupSequence': 500, 'trackingID': 11578737109589, 'msgType': 'R', 'symbol': 'AMN', 'marketClass': 'N', 'fsi': '', 'roundLotSize': 100, 'roundLotOnly': 'N', 'issueClass': 'C', 'issueSubtype': 'Z', 'authenticity': 'P', 'shortThreshold': 'N', 'ipo': '', 'luldTier': '2', 'etf': 'N', 'etfFactor': 0, 'inverseETF': 'N', 'compositeId': 'BBG000BCT197', 'schema_name': 'SeqDirectoryMessage'}\n" ] } ], - "source": [ - "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", - "topic = \"NLSCTA\"\n", - "print(ncds_client.get_sample_messages(topic, \"SeqDirectoryMessage\", all_messages=False))" - ] + "metadata": {} }, { "cell_type": "markdown", - "id": "0e600e73", - "metadata": {}, "source": [ "### Get continuous stream" - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 8, - "id": "04d70f89", - "metadata": {}, + "source": [ + "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", + "topic = \"NLSCTA\"\n", + "consumer = ncds_client.ncds_kafka_consumer(topic)\n", + "for i in range(0, 10):\n", + " messages = consumer.consume(num_messages=1, timeout=5)\n", + " if len(messages) == 0:\n", + " print(f\"No Records Found for the Topic: {topic}\")\n", + " \n", + " for message in messages:\n", + " print(f\"value :\" + message.value())" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "latest record found\n", "value :{\"SoupPartition\": 0, \"SoupSequence\": 1, \"trackingID\": 7233292771056, \"msgType\": \"S\", \"event\": \"O\", \"schema_name\": \"SeqSystemEventMessage\"}\n", @@ -322,18 +282,7 @@ ] } ], - "source": [ - "ncds_client = NCDSClient(security_cfg, kafka_cfg)\n", - "topic = \"NLSCTA\"\n", - "consumer = ncds_client.ncds_kafka_consumer(topic)\n", - "for i in range(0, 10):\n", - " messages = consumer.consume(num_messages=1, timeout=5)\n", - " if len(messages) == 0:\n", - " print(f\"No Records Found for the Topic: {topic}\")\n", - " \n", - " for message in messages:\n", - " print(f\"value :\" + str(message.value()))" - ] + "metadata": {} } ], "metadata": { @@ -352,9 +301,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.7" + "version": "3.9.5" } }, "nbformat": 4, "nbformat_minor": 5 -} +} \ No newline at end of file diff --git a/setup.py b/setup.py index e969f2e..092dd5e 100644 --- a/setup.py +++ b/setup.py @@ -8,7 +8,7 @@ setup( name='ncdssdk', - version='1.0.0', + version='0.2.0', description='A Python SDK for developing applications to access the NCDS API', long_description=long_description, long_description_content_type='text/markdown', @@ -37,4 +37,4 @@ project_urls={ # Optional 'Source': 'https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python', }, -) +) \ No newline at end of file From cf52b56c7eab1522434f44045e2d2d188e8d1445 Mon Sep 17 00:00:00 2001 From: Ruchir Vani <32844383+ruchirvaninasdaq@users.noreply.github.com> Date: Mon, 1 Nov 2021 13:54:31 -0400 Subject: [PATCH 2/2] Update README.md --- README.md | 53 ++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 38 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index f036eb6..012f26a 100644 --- a/README.md +++ b/README.md @@ -2,12 +2,36 @@ Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of delivery for realtime exchange data and other financial information. Data is made available through a suite of APIs, allowing for effortless integration of data from disparate sources, and a dramatic reduction in time to market for customer-designed applications. The API is highly scalable, and robust enough to support the delivery of real-time exchange data. - -### Items To Note +# Products Currently Available +### Equities +#### The Nasdaq Stock Market +- [Nasdaq Basic](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NasdaqBasic-Cloud.pdf) +- [Nasdaq Last Sale+](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NLSPlus-cloud.pdf) +- [Nasdaq TotalView](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/Totalview-ITCH-cloud.pdf) +- [Nasdaq Consolidated Quotes and Trades](https://github.com/Nasdaq/CloudDataService/raw/master/specs/CQT-cloud.pdf) +#### Nasdaq BX +- [BX BBO](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/BX_BBO_Cloud.pdf) +- [BX Last Sale](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/BLS_Cloud.pdf) +#### Nasdaq PSX +- [PSX BBO](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/PSX_BBO_Cloud.pdf) +- [PSX Last Sale](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/PLS_Cloud.pdf) +#### Nasdaq Canada +- [Nasdaq Canada Basic](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/Nasdaq-Basic-Canada-Cloud-Specification.pdf) +### Indexes & ETPs +- [Global Index Data Service](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/GIDS_Cloud.pdf) +### Options +#### Nasdaq U.S. Derivatives +- [Nasdaq Smart Options](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NCDSSmartOptions.pdf) +### Mutual Funds +- [Nasdaq Fund Network](http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NFNDS_NCDS.pdf) +### News +- [Financial News](http://nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/MTNewswires-cloud.pdf) + +# Items To Note * Connecting to the API requires credentials, which are provided by the Nasdaq Data Operations team during an on-boarding process * This sample code only connects to one topic (NLSCTA); during on-boarding process, you will receive a topic list that you're entitled to. -* See https://github.com/Nasdaq/CloudDataService for our officially support Java-based SDK. +* See https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Java for our officially support Java-based SDK. # Table of Contents @@ -19,19 +43,19 @@ Nasdaq Cloud Data Service (NCDS) provides a modern and efficient method of deliv The SDK currently supports Python 3.9 and above ### Get the SDK -The source code is currently hosted on GitHub at: https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​ -- Clone the repository: ```git clone https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​.git``` -- Move into the directory ```cd ncds-python-sample-consumer``` -- Install the library and its dependencies from local source with ```pip install .``` +The source code is currently hosted on GitHub at: https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python +- Clone the repository: ```git clone https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python.git``` +- Move into the directory ```cd NasdaqCloudDataService-SDK-Python``` +- Install the library and its dependencies from local source with ```pip install -e .``` Optional: to use the Jupyter notebook provided, -- Download Jupyter notebook using pip ```pip install notebook``` +- Download Jupyter notebook using either pip ```pip3 install notebook``` or conda ```conda install -c conda-forge notebook``` - To run the notebook, use the command ```jupyter notebook``` and the Notebook Dashboard will open in your browser - Select the file ```python_sdk_examples.ipynb``` ### Stream configuration - Replace example stream properties in the file **kafka-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​/blob/master/ncdssdk_client/src/main/python/resources/kafka-config.json) with provided values during on-boarding. +Replace example stream properties in the file **kafka-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/kafka-config.json) with provided values during on-boarding. Required kafka configuration @@ -43,13 +67,12 @@ Optional: to use the Jupyter notebook provided, ### Client Authentication configuration -Replace example client authentication properties in the file **client-authentication-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python​/blob/master/ncdssdk_client/src/main/python/resources/client-authentication-config.json) with valid credentials provided during on-boarding. - +Replace example client authentication properties in the file **client-authentication-config.json** (https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Python/blob/master/ncdssdk_client/src/main/python/resources/client-authentication-config.json) with valid credentials provided during on-boarding. ```properties -oauth.token.endpoint.uri: https://{auth_endpoint_url}/auth/realms/demo/protocol/openid-connect/token -oauth.client.id: client -oauth.client.secret: client-secret +oauth.token.endpoint.uri: "https://{auth_endpoint_url}/auth/realms/pro-realm/protocol/openid-connect/token" +oauth.client.id: "client_id" +oauth.client.secret: "client_secret" ``` ### Create NCDS Session Client @@ -264,7 +287,7 @@ while True: print(f"No Records Found for the Topic: {topic}") for message in messages: - print(f"value :" + message.value()) + print(f"value :" + str(message.value())) ``` Example output: