Add IP Location APIs #3107

pquentin · 2024-11-08T09:49:30Z

I extracted the rest-api-spec JSON files from #3019. We'll need a review from the Data Management team before merging.

joegallo · 2024-11-08T22:21:12Z

I think we'll still need to make a change to Database.ts -- I'll reach out to you offline.

lcawl · 2024-11-13T00:57:54Z

Thanks for creating this PR! I was looking at how to ensure the work from elastic/elasticsearch#116611 got carried over to the new API docs site and this PR already does a lot of the heavy lifting (we can port the examples over in a subsequent PR).

specification/ingest/_types/Database.ts

pquentin · 2024-11-19T12:25:41Z

specification/ingest/_types/Database.ts

-  /** The configuration necessary to identify which IP geolocation provider to use to download the database, as well as any provider-specific configuration necessary for such downloading.
-   * At present, the only supported provider is maxmind, and the maxmind provider requires that an account_id (string) is configured.
-   */
-  maxmind: Maxmind


We do need the maxmind key which is gone with this approach.

Also it's OK to have some duplication, eg I'm happy to keep the previous approach but simply add an ipinfo field and type, with one less field.

Agreed, yeah, I'll take a swing at fixing that -- it wasn't an intentional change in this second version of things, I just wasn't thoughtful enough about how I did it.

What do you think about 31fd86a?

This looks great, thanks! Can you please confirm that the GeoIP APIs also accept ipinfo databases? I'm asking as DatabaseConfiguration is currently shared between the two APIs.

Yes, they do.

joegallo · 2024-12-05T19:43:38Z

@pquentin I've got two more things on this one, one that's hopefully minor, and then one that might be more-than-minor:

1.) Can you help me diagnose and understand the failure on ingest.put_ip_location_database? I'm seeing 3/4 for the 'Request' column, and I'm not sure what there is to be done about that, if anything.

2.) The PUT needs to be able to handle ipinfo database configurations, too, and I didn't realize until just now that that's not currently covered. To make things slightly more difficult, the PUT API will not handle web and local configurations -- those are effectively read-only.

pquentin · 2024-12-10T08:00:55Z

@pquentin I've got two more things on this one, one that's hopefully minor, and then one that might be more-than-minor: [...]

Sorry for the delay!

So 1. and 2. are the same actually! The failure is the one I mentioned in https://github.com/elastic/elasticsearch-specification/pull/3107/files#r1850637399. I can help you offline to get make validate api=ingest.put_ip_location_database branch=main to work if you're interested.

That said, I tried to fix those in my latest commit, please take a look.

Something else we need to do (possibly in a follow-up PR) is to apply the same changes to the existing GeoIP APIs.

outdated

specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts

…abaseResponse.ts Co-authored-by: Laura Trotta <[email protected]>

…abaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]>

l-trotta

LGTM!

and then make generate, etc

github-actions · 2024-12-18T05:03:24Z

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
`ingest.delete_geoip_database`	🟢	1/1	1/1
`ingest.delete_ip_location_database`	🟢	1/1	1/1
`ingest.delete_pipeline`	🟢	15/15	15/15
`ingest.geo_ip_stats`	🟢	1/1	1/1
`ingest.get_geoip_database`	🟢	6/6	6/6
`ingest.get_ip_location_database`	🟢	7/7	7/7
`ingest.get_pipeline`	🟢	22/22	22/22
`ingest.processor_grok`	🟢	1/1	1/1
`ingest.put_geoip_database`	🟢	3/3	3/3
`ingest.put_ip_location_database`	🟢	4/4	4/4
`ingest.put_pipeline`	🟢	60/60	60/60
`ingest.simulate`	🟢	10/10	10/10

You can validate these APIs yourself by using the make validate target.

joegallo

Ship it!

@Variants

* Add IP Location APIs * Rename these files * Reword this description * Handle ipinfo in addition to maxmind * A slightly more faithful alternative approach * Remove accidentally-added line * A more-correct implementation of the second approach * Another approach using @Variants container * Fix IP location APIs * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Laura Trotta <[email protected]> * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]> * Fix an import and then make generate, etc * Remove unused import --------- Co-authored-by: Joe Gallo <[email protected]> Co-authored-by: Laura Trotta <[email protected]> (cherry picked from commit dae081f)

@Variants

* Add IP Location APIs * Rename these files * Reword this description * Handle ipinfo in addition to maxmind * A slightly more faithful alternative approach * Remove accidentally-added line * A more-correct implementation of the second approach * Another approach using @Variants container * Fix IP location APIs * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Laura Trotta <[email protected]> * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]> * Fix an import and then make generate, etc * Remove unused import --------- Co-authored-by: Joe Gallo <[email protected]> Co-authored-by: Laura Trotta <[email protected]> (cherry picked from commit dae081f)

@Variants

* Add IP Location APIs * Rename these files * Reword this description * Handle ipinfo in addition to maxmind * A slightly more faithful alternative approach * Remove accidentally-added line * A more-correct implementation of the second approach * Another approach using @Variants container * Fix IP location APIs * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Laura Trotta <[email protected]> * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]> * Fix an import and then make generate, etc * Remove unused import --------- Co-authored-by: Joe Gallo <[email protected]> Co-authored-by: Laura Trotta <[email protected]> (cherry picked from commit dae081f)

@Variants

* Add IP Location APIs * Rename these files * Reword this description * Handle ipinfo in addition to maxmind * A slightly more faithful alternative approach * Remove accidentally-added line * A more-correct implementation of the second approach * Another approach using @Variants container * Fix IP location APIs * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Laura Trotta <[email protected]> * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]> * Fix an import and then make generate, etc * Remove unused import --------- Co-authored-by: Joe Gallo <[email protected]> Co-authored-by: Laura Trotta <[email protected]> (cherry picked from commit dae081f) Co-authored-by: Quentin Pradet <[email protected]>

@Variants

* Add IP Location APIs * Rename these files * Reword this description * Handle ipinfo in addition to maxmind * A slightly more faithful alternative approach * Remove accidentally-added line * A more-correct implementation of the second approach * Another approach using @Variants container * Fix IP location APIs * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Laura Trotta <[email protected]> * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]> * Fix an import and then make generate, etc * Remove unused import --------- Co-authored-by: Joe Gallo <[email protected]> Co-authored-by: Laura Trotta <[email protected]> (cherry picked from commit dae081f) Co-authored-by: Quentin Pradet <[email protected]>

@Variants

* Add IP Location APIs * Rename these files * Reword this description * Handle ipinfo in addition to maxmind * A slightly more faithful alternative approach * Remove accidentally-added line * A more-correct implementation of the second approach * Another approach using @Variants container * Fix IP location APIs * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Laura Trotta <[email protected]> * Update specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]> * Fix an import and then make generate, etc * Remove unused import --------- Co-authored-by: Joe Gallo <[email protected]> Co-authored-by: Laura Trotta <[email protected]> (cherry picked from commit dae081f) Co-authored-by: Quentin Pradet <[email protected]>

Add IP Location APIs

0b49cde

pquentin requested a review from joegallo November 8, 2024 09:49

pquentin requested a review from a team as a code owner November 8, 2024 09:49

github-actions bot added the specification label Nov 8, 2024

This comment was marked as outdated.

Sign in to view

pquentin added backport 8.x backport 8.16 labels Nov 8, 2024

This comment was marked as resolved.

Sign in to view

joegallo added 3 commits November 8, 2024 17:02

Rename these files

d83e249

Reword this description

01e941c

Merge branch 'main' into ip-location-apis

1a086d8

This comment was marked as outdated.

Sign in to view

pquentin mentioned this pull request Nov 12, 2024

Update rest-api-spec 8.16 #3121

Merged

This was referenced Nov 15, 2024

Update rest-api-spec main #3019

Merged

Update rest-api-spec 8.x #3013

Merged

Merge branch 'main' into ip-location-apis

a984d15

This comment was marked as outdated.

Sign in to view

Handle ipinfo in addition to maxmind

9f1d783

joegallo force-pushed the ip-location-apis branch from 7fe4822 to 9f1d783 Compare November 18, 2024 18:30

This comment was marked as outdated.

Sign in to view

A slightly more faithful alternative approach

74230fb

This comment was marked as outdated.

Sign in to view

pquentin commented Nov 19, 2024

View reviewed changes

joegallo added 3 commits November 19, 2024 13:23

Remove accidentally-added line

91e9c64

A more-correct implementation of the second approach

31fd86a

Merge branch 'main' into ip-location-apis

ec6f46b

This comment was marked as outdated.

Sign in to view

Fix IP location APIs

242310b

This comment was marked as outdated.

Sign in to view

JoshMock mentioned this pull request Dec 11, 2024

putGeoipDatabase without body parameter returns an error elastic/elasticsearch-js#2482

Closed

l-trotta reviewed Dec 16, 2024

View reviewed changes

specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Outdated Show resolved Hide resolved

l-trotta reviewed Dec 16, 2024

View reviewed changes

specification/ingest/get_ip_location_database/GetIpLocationDatabaseResponse.ts Outdated Show resolved Hide resolved

pquentin and others added 2 commits December 17, 2024 10:43

Update specification/ingest/get_ip_location_database/GetIpLocationDat…

944f4aa

…abaseResponse.ts Co-authored-by: Laura Trotta <[email protected]>

Update specification/ingest/get_ip_location_database/GetIpLocationDat…

f3a5f78

…abaseResponse.ts Co-authored-by: Quentin Pradet <[email protected]>

l-trotta approved these changes Dec 17, 2024

View reviewed changes

joegallo added 3 commits December 17, 2024 23:56

Merge branch 'main' into ip-location-apis

680b548

Fix an import

c2c5f85

and then make generate, etc

Remove unused import

be050a8

This comment was marked as outdated.

Sign in to view

joegallo approved these changes Dec 18, 2024

View reviewed changes

pquentin added the backport 8.17 label Dec 18, 2024

pquentin merged commit dae081f into main Dec 18, 2024
8 checks passed

pquentin deleted the ip-location-apis branch December 18, 2024 05:08

github-actions bot mentioned this pull request Dec 18, 2024

[Backport 8.x] Add IP Location APIs #3355

Merged

github-actions bot mentioned this pull request Dec 18, 2024

[Backport 8.16] Add IP Location APIs #3356

Merged

github-actions bot mentioned this pull request Dec 18, 2024

[Backport 8.17] Add IP Location APIs #3357

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add IP Location APIs #3107

Add IP Location APIs #3107

pquentin commented Nov 8, 2024

This comment was marked as outdated.

This comment was marked as resolved.

This comment was marked as outdated.

joegallo commented Nov 8, 2024

lcawl commented Nov 13, 2024

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

pquentin Nov 19, 2024

joegallo Nov 19, 2024

joegallo Nov 19, 2024

pquentin Nov 20, 2024

joegallo Dec 5, 2024

This comment was marked as outdated.

This comment was marked as outdated.

joegallo commented Dec 5, 2024

This comment was marked as outdated.

pquentin commented Dec 10, 2024

l-trotta left a comment

This comment was marked as outdated.

github-actions bot commented Dec 18, 2024

joegallo left a comment

Add IP Location APIs #3107

Add IP Location APIs #3107

Conversation

pquentin commented Nov 8, 2024

This comment was marked as outdated.

This comment was marked as resolved.

This comment was marked as outdated.

joegallo commented Nov 8, 2024

lcawl commented Nov 13, 2024

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

pquentin Nov 19, 2024

Choose a reason for hiding this comment

joegallo Nov 19, 2024

Choose a reason for hiding this comment

joegallo Nov 19, 2024

Choose a reason for hiding this comment

pquentin Nov 20, 2024

Choose a reason for hiding this comment

joegallo Dec 5, 2024

Choose a reason for hiding this comment

This comment was marked as outdated.

This comment was marked as outdated.

joegallo commented Dec 5, 2024

This comment was marked as outdated.

pquentin commented Dec 10, 2024

l-trotta left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

github-actions bot commented Dec 18, 2024

joegallo left a comment

Choose a reason for hiding this comment