[Connector API] Add missing 8.14 changes

[jedrazb]

Changes

Adding missing Connector API features that were shipped in 8.14 after #2367 was merged.

2 new endpoints:

• validate draft filtering
• activate draft filtering

2 endpoint changes

• update filtering endpoint takes now rules and advanced_snippet as optional params, filtering is now optional (still present for backward compatibility)
• delete connector endpoint accepts delete_sync_jobs url param

filtering-related timestamps (rules and advanced_snippet) were made optional in ES backend in 8.14

[l-trotta]

From the other PR: FilteringConfig -> active and draft should be optional, domain should be removed, it seems like an internal parameter

[jedrazb]

From the other PR: FilteringConfig -> active and draft should be optional

@l-trotta I'm not sure if they should become optional, they are always required in the Filtering config representation in the server implementation

domain should be removed, it seems like an internal parameter

Yes, domain is not part of the request payload for any endpoint, but we do populate it in the index and return in the response when getting/listing connectors (the response is Connector or Connectors[]). We still need to populate it (even with the default value) as this is required by the Connector Protocol.

If I remove this from the FilteringConfig could it impact e.g. the get connector response from clients? (as Connector type references filtering: FilteringConfig[])

I'm unsure how to handle that case correctly in clients

[l-trotta]

Sorry for the confusion, I didn't understand how the filtering field was outputted differently if used in the context of SyncJobs. The correct fix is to replace FilteringConfig with FilteringRules in SyncJobConnectorReference. Server code

[l-trotta]

`features' in Connector should be nullable

[jedrazb]

The correct fix is to replace FilteringConfig with FilteringRules in SyncJobConnectorReference

Thank you for catching this! 👍 I addressed your feedback

[l-trotta]

LGTM!

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
connector.check_in	⚪	Missing test	Missing test
connector.delete	⚪	Missing test	Missing test
connector.get	⚪	Missing test	Missing test
connector.last_sync	⚪	Missing test	Missing test
connector.list	⚪	Missing test	Missing test
connector.post	⚪	Missing test	Missing test
connector.put	⚪	Missing test	Missing test
connector.sync_job_cancel	⚪	Missing test	Missing test
connector.sync_job_delete	⚪	Missing test	Missing test
connector.sync_job_get	⚪	Missing test	Missing test
connector.sync_job_list	⚪	Missing test	Missing test
connector.sync_job_post	⚪	Missing test	Missing test
connector.update_active_filtering	⚪	Missing test	Missing test
connector.update_api_key_id	⚪	Missing test	Missing test
connector.update_configuration	⚪	Missing test	Missing test
connector.update_error	⚪	Missing test	Missing test
connector.update_filtering_validation	⚪	Missing test	Missing test
connector.update_filtering	⚪	Missing test	Missing test
connector.update_index_name	⚪	Missing test	Missing test
connector.update_name	⚪	Missing test	Missing test
connector.update_native	⚪	Missing test	Missing test
connector.update_pipeline	⚪	Missing test	Missing test
connector.update_scheduling	⚪	Missing test	Missing test
connector.update_service_type	⚪	Missing test	Missing test
connector.update_status	⚪	Missing test	Missing test

You can validate these APIs yourself by using the make validate target.

[github-actions]

The backport to 8.14 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.14 8.14
# Navigate to the new working tree
cd .worktrees/backport-8.14
# Create a new branch
git switch --create backport-2527-to-8.14
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 6420945e1f3b0cdbd13d6031ee50fea046f6a908
# Push it to GitHub
git push --set-upstream origin backport-2527-to-8.14
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.14

Then, create a pull request where the base branch is 8.14 and the compare/head branch is backport-2527-to-8.14.

[jedrazb]

💚 All backports created successfully

Status	Branch	Result
✅	8.14

Questions ?

Please refer to the Backport tool documentation