Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]Add iplocation function to PPL for IP address geolocation #3037

Open
YANG-DB opened this issue Sep 16, 2024 · 2 comments
Open

[FEATURE]Add iplocation function to PPL for IP address geolocation #3037

YANG-DB opened this issue Sep 16, 2024 · 2 comments
Labels
enhancement New feature or request PPL Piped processing language

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Sep 16, 2024

Description:
We propose adding an geoip function to OpenSearch's Piped Processing Language (PPL) and SQL to provide built-in IP address geolocation capabilities.
This feature would be similar to functionality used in OpenSearch's geospatial feature, enhancing PPL's ability to enrich log data with geographical information based on IP addresses.

Proposed Functionality:

  1. The 'geoip' function should take an IP address as input and return geographical information.
  2. It should support both IPv4 and IPv6 addresses.
  3. The function should return multiple fields including country, region, city, latitude, longitude, and others as available.
  4. It should allow users to specify which geolocation fields to include in the output.
  5. The function should use a regularly updated IP geolocation database for accuracy.

Example Usage:

... | eval geolocation = geoip(ip_field)

This would add a new field 'geolocation' with all available location information for the IP address in 'ip_field'.

... | eval country = geoip(ip_field, "country")
... | eval lat = geoip(ip_field, "lat"), lon = iplocation(ip_field, "lon")

This would add new fields with specific geolocation information.

... | eval location_info = geoip(ip_field, "country,region,city,lat,lon")

This would add a new field 'location_info' with multiple pieces of geolocation data.

Additional considerations

  • Allow for registering a DB table that allows resolving the IP to Geo
  • Adding a generic way to register the IP to Geo location resolving mechanism / service
    • Adding auth tokens for calling such service

Support for PPL iplocation function is required for both:

OpenSearch based PPL engine

Spark based PPL engine


Related resources

@YANG-DB YANG-DB added enhancement New feature or request untriaged PPL Piped processing language labels Sep 16, 2024
@dblock dblock removed the untriaged label Oct 7, 2024
@dblock
Copy link
Member

dblock commented Oct 7, 2024

[Catch All Triage - 1, 2, 3, 4]

@andy-k-improving
Copy link
Contributor

For the community awareness, I'm working on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request PPL Piped processing language
Projects
None yet
Development

No branches or pull requests

3 participants