GSoC topic: iyp cli / python library

[romain-fontugne]

It would be nice to have a simple tools to query iyo from a terminal. For example:

> iyp 2497

Would print all we know from AS2497.
This tool coups also be available as a python library

[roopeshsn]

It would be nice to have a simple tools to query iyo from a terminal. For example:

> iyp 2497

Would print all we know from AS2497. This tool coups also be available as a python library

In case it's a cli, do you want to see all the information the same as in IYP Views (data that is displayed in tables) when executing a command like iyp 2497? Or would you like to execute cypher queries in the cli and return the data? @romain-fontugne

[romain-fontugne]

You can find all the information about the database/API here:
https://github.com/InternetHealthReport/internet-yellow-pages

But be aware that this requires a certain understanding of data collected in IYP, some networking knowledge, and neo4j querying language. The main difficulty is not to write the tool.

[MAVRICK-1]

You can find all the information about the database/API here:
https://github.com/InternetHealthReport/internet-yellow-pages

But be aware that this requires a certain understanding of data collected in IYP, some networking knowledge, and neo4j querying language. The main difficulty is not to write the tool.

Sir, I've sent you a code via Slack. Could you please review it for me?
My slack username is Rishi Mondal.

[romain-fontugne]

thanks, I got it and replied on slack

[romain-fontugne]

that said, discussions that starts with GSoC topic/GSoC idea are here to keep track of potential gsoc projects and to discuss which idea make sense or not. It doesn't mean this will be a gsoc project. The list of projects will be available after IHR is officially accepted for GSoC'24.

[MAVRICK-1]

that said, discussions that starts with GSoC topic/GSoC idea are here to keep track of potential gsoc projects and to discuss which idea make sense or not. It doesn't mean this will be a gsoc project. The list of projects will be available after IHR is officially accepted for GSoC'24.

Okay sir, I understood my apologies, I just made demo to see I am going into right track or not.

[MAVRICK-1]

@roopeshsn I also have the exact same question

[MAVRICK-1]

@romain-fontugne I've already developed a working prototype for this. Additionally, we can integrate this module with frameworks like Flask and Django. By using APIs to connect it to the frontend, we can directly query Neo4j from the frontend interface. Your views on this?

[romain-fontugne]

We are already working on integrating IYP and the frontend:
InternetHealthReport/ihr-website#719

[MAVRICK-1]

We are already working on integrating IYP and the frontend:
InternetHealthReport/ihr-website#719
Okay sir, but this has benifits like
Project Benefits:

• Enhances accessibility and usability of IYP data for users.
• Facilitates quick and convenient retrieval of information from the terminal.
• Promotes seamless integration into Python projects through a dedicated library.
• It can also be use for data analytics

As an active contributor to IYP and other IHR library, I have knowledge about the code based
,I can do it .

[romain-fontugne]

I meant we already have the frontend part. But I agree having a CLI tool would be nice

[MAVRICK-1]

@romain-fontugne

Proposal for Unified CLI Tool and Library for IHR Data Analysis & Exploration

The current frontend interface lacks the capability to comprehensively display specific data and is unsuitable for conducting data analytics and exploration . It is essential to develop a dedicated library tailored for data analysis purposes to overcome these limitations. Relying solely on the frontend for analysis tasks is inadequate, especially for individuals with limited knowledge beyond Neo4j. Hence , there is a need for an IYP library that provides Data of each node.

A specialized library for IYP (Internet Yellow Pages) data analysis and data explorationwould be advantageous, providing users with the tools necessary to conduct comprehensive data analytics tasks efficiently. Such a library would enable:

• Data selection and indexing
• Data manipulation
• Data cleaning
• Data visualization
• Data transformation
• Advanced operations
• Integration with other libraries

You can explore the prototype at https://github.com/MAVRICK-1/IYP_Lib. It functions as a dedicated library where CLI features can be integrated seamlessly.

Combining existing IHR libraries—abondance, route-origin-validator, and IYP—into a unified CLI tool and library is proposed. This consolidation would introduce new Python modules and streamline data retrieval processes, making them faster and more organized. For users, having a single, comprehensive CLI tool would greatly enhance usability and manageability.

For example, the CLI tool could be used as follows:

$ ihr rov 8.8.8.0/24 15169 --rpki_archive 2023/10/01
$ ihr iyp 2497
$ ihr hegemony 2500 --start "2018-09-15 00:00" --end "2018-09-15 23:59"

This also provides output in the form of JSON, CSV, or DataFrame as required by the user and will also print the data in the form of tables so that the user can see it. These commands illustrate the versatility of the tool, allowing users to retrieve specific data related to ROV (Route Origin Validation), IYP (Internet Yellow Pages), and Hegemony for analysis and exploration.

In conclusion, the development of a unified CLI tool and library for IHR data analysis would significantly enhance the capabilities of users in handling diverse datasets and analytical tasks effectively.

[MAVRICK-1]

The library enables users to access specific details from each node comprehensively. You can explore the prototype at https://github.com/MAVRICK-1/IYP_Lib. It functions as a dedicated library where CLI features can be integrated seamlessly.

[roopeshsn]

Hey, things won't work that way! I suggest you do not flood with messages. One single message consolidated with your thoughts is enough. Maintainers will reach you if they really want to convey something. Give some time and be patient.

…

On Sun, Feb 18, 2024, 10:05 PM Rishi Mondal ***@***.***> wrote: @romain-fontugne <https://github.com/romain-fontugne> I propose combining all the IHR libraries—abondance, route-origin-validator, and IYP—into a unified CLI tool and library. This consolidation would introduce new Python modules and streamline data retrieval processes, making them faster and more organized. For users, having a single, comprehensive CLI tool would greatly enhance usability and manageability. What your views on this? — Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQ34ASZDD5ILQY4JA6TSNTDYUIUWZAVCNFSM6AAAAAA2TRN6M2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DKMBYGI2TE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[AdamLucife]

@romain-fontugne Is this idea discarded?

[romain-fontugne]

no, it is not discarded but has a lower priority

[SAHELISAHAA]

@romain-fontugne For this library , the Data will be obtained through API?

[dpgiakatos]

The Neo4j Python library is generic, which means we typically need to provide a Cypher query to retrieve results. However, in this case, we're discussing getting results without the need to provide a Cypher query directly. For example, in Python, ASN(2497).get_country() should return "Japan."

[MAVRICK-1]

The Neo4j Python library is generic, which means we typically need to provide a Cypher query to retrieve results. However, in this case, we're discussing getting results without the need to provide a Cypher query directly. For example, in Python, ASN(2497).get_country() should return "Japan."

@dpgiakatos sir I mean we can create a function like this where ASN will be passed as parameter. Inside the function there will cypher query written, Basically users don't have to write cypher query for this. The have to pass only the argument.

[dpgiakatos]

@MAVRICK-1 please, let's aim to keep each thread focused and avoid repeating existing comments across multiple threads. This will help maintain clarity and streamline communication. Thank you for your cooperation!

[SAHELISAHAA]

@dpgiakatos I agreed with what @MAVRICK-1 that we can create a function for example get_country() and passed ASN as argument. And Using the Neo4j Library we easily retrieve the data. I guess our main aim is to retrieve data and export it in json formate right?

[dpgiakatos]

When creating a library or package, it's crucial to consider various use cases. It's challenging to understand why a library should be used if it only returns JSON data when specific information is needed. For instance, when querying information about a country, the desired outcome should be the country info as a string, not a JSON object that needs additional filtering. If all information is needed, it should be delivered in a human-readable format. Additionally, providing an option in the parameters or a separate function to retrieve JSON data could be beneficial.

Furthermore, the necessity of using the Neo4j Python library should be scrutinized. The library utilizes WebSocket as the communication method, raising questions about whether a WebSocket connection is truly needed over an HTTP request. Consideration should be given to providing both communication methods.

The proposed architecture should adhere to engineering principles for easy maintenance by the community. If a class inherits functions from other files or classes, it could enhance maintainability.

Writing a proposal is undoubtedly challenging, but it should not be confined to a specific use case. A well-rounded proposal should address various scenarios and considerations.

[dpgiakatos]

I believe it would be beneficial to have a Linux package instead of relying on Python scripts and third-party libraries executed through the terminal. This package could directly communicate with the IYP server. Basic packages for handling HTTP requests and WebSockets are typically included in all Linux distributions, so a Debian package that allows querying the IYP from the terminal would be preferable.