Fix most ML APIs

[l-trotta]

References for the fixes:

• DiscoveryNode: two possible representation, one is the standard one, the other is the "compact one" (can discuss naming of course) used by ml.get_job_stats and ml.get_datafeed_stats
• IndicesOptions - adding missing parameters
• AnalysisLimit - model_memory_limit can accept both string and long
• Bucket - timestamp_string is nullable (depends on "humanReadable")
• DatafeedTimingStats - added missing field ExponentialAverageCalculationContext
• Datafeed (response) - timing_stats is optional
• DataframeAnalyticsSummary (or "config" in the server) - added missing metadata
• UpdateJob (request) - added missing field detectors - DetectorUpdate
• Model - added missing field output_memory_allocator_bytes
• TrainedModelDeploymentStats (AssignmentStatsin the server code) - several missing properties and optional ones
• TrainedModelDeploymentNodesStats (inner class of AssignmentStats in the server code) - many missing/optional properties
• TrainedModelConfig - missing model_package field, implemented as ModelPackageConfig
• TrainedModelAssignmentTaskParameters (inner class in the server code) - missing 2 parameters, another one optional
• TrainedModelAssignmentRoutingTable (RoutingInfo in server code) - some optional parameters
• TrainedModelAssignment - missing fields
• TokenizationConfigContainer - added bert_ja option
• NlpRobertaTokenizationConfig - missing do_lower_case
• TextEmbeddingInferenceOptions - missing vocabulary
• TextExpansionInferenceOptions - missing vocabulary
• FillMaskInferenceOptions - missing vocabulary
• MlGetTrainedModelRequest - missing include_model_definition
• Limits (inner structure in server code) - missing properties
• MlPostJobDataResponse (DataCounts in server code) - multiple optional properties
• MlPutDataFrameAnalyticsRequest - added missing metadata
• MlPutDataFrameAnalyticsResponse - added missing metadata (same object as the request)
• MlPutDatafeedRequest - added aggs as alias for aggregations, both accepted
• MlPutJobRequest - added query parameter ignore_throttled (could not find server proof), body parameter job_id
• MlUpdateJobRequest - detectors uses different class, DetectorUpdate

The only thins missing server-side proof is ignore_throttled as query parameter in MlPutJobRequest, I couldn't find it in parent classes either, so I'm trusting the validation on that one.

[pquentin]

The failure is:

Error: A type cannot end with "Request" or "Response"
At: /home/runner/work/elasticsearch-specification/elasticsearch-specification/elasticsearch-specification/output/typescript/types.ts:46:73

[l-trotta]

my bad, messed up an import, fixing it now

[l-trotta]

@pquentin I just understood the source of my confusion: it seems like DiscoveryNode can be written in 2 ways, one is implemented in toXContent and it builds it like this:

          "node": {
              "UdmgtQ3gRly7A3Z6PmLrPQ": {
                  "name": "instance-0000000004",
                  "ephemeral_id": "RjniA_UvRLq0bqo2aYL-jg",
                  "transport_address": "10.45.0.50:19529",
                ...
              }
           }

with the id as an upper level json object, while the method writeTo build it like:

          "node": {
                "name": "instance-0000000004",
                "id": "UdmgtQ3gRly7A3Z6PmLrPQ",
                "ephemeral_id": "RjniA_UvRLq0bqo2aYL-jg",
                "transport_address": "10.45.0.50:19529",
                ...
              }
           }

with the id inside the node object. Both ways seem to be accepted and present in the recordings.
I'm thinking this could be a shortcut, but I'd like to confirm this in the next spec meet

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
ml.clear_trained_model_deployment_cache	🟢	1/1	1/1
ml.close_job	🟢	64/64	63/63
ml.delete_calendar_event	🟢	4/4	4/4
ml.delete_calendar_job	🟢	3/3	3/3
ml.delete_calendar	🟢	5/5	5/5
ml.delete_data_frame_analytics	🟢	2/2	2/2
ml.delete_datafeed	🟢	3/3	3/3
ml.delete_expired_data	🟢	5/5	5/5
ml.delete_filter	🟢	27/27	27/27
ml.delete_forecast	🟢	3/3	3/3
ml.delete_job	🟢	47/47	47/47
ml.delete_model_snapshot	🟢	2/2	2/2
ml.delete_trained_model_alias	🟢	3/3	3/3
ml.delete_trained_model	🟢	5/5	5/5
ml.estimate_model_memory	🟢	16/16	16/16
ml.evaluate_data_frame	🟢	15/15	15/15
ml.explain_data_frame_analytics	🟢	7/7	7/7
ml.flush_job	🟢	15/15	15/15
ml.forecast	🟢	1/1	1/1
ml.get_buckets	🟢	14/14	14/14
ml.get_calendar_events	🟢	15/15	15/15
ml.get_calendars	🟢	17/17	17/17
ml.get_categories	🟢	12/12	12/12
ml.get_data_frame_analytics_stats	🟢	12/12	12/12
ml.get_data_frame_analytics	🟢	17/17	17/17
ml.get_datafeed_stats	🟢	27/27	27/27
ml.get_datafeeds	🟢	20/20	20/20
ml.get_filters	🟢	13/13	13/13
ml.get_influencers	🟢	11/11	11/11
ml.get_job_stats	🟢	32/32	32/32
ml.get_jobs	🟢	31/31	31/31
ml.get_memory_stats	🟢	6/6	6/6
ml.get_model_snapshot_upgrade_stats	🟢	3/3	3/3
ml.get_model_snapshots	🟢	18/18	18/18
ml.get_overall_buckets	🟢	16/16	15/15
ml.get_records	🟢	8/8	8/8
ml.get_trained_models_stats	🟢	17/17	17/17
ml.get_trained_models	🟢	37/37	37/37
ml.infer_trained_model	🟢	10/10	10/10
ml.info	🟢	10/10	10/10
ml.open_job	🟢	83/83	83/83
ml.post_calendar_events	🟢	12/12	12/12
ml.post_data	🔴	9/11	18/18
ml.preview_data_frame_analytics	🟢	3/3	3/3
ml.preview_datafeed	🟢	17/17	17/17
ml.put_calendar_job	🟢	12/12	12/12
ml.put_calendar	🟢	135/135	135/135
ml.put_data_frame_analytics	🟢	33/33	33/33
ml.put_datafeed	🟢	71/71	71/71
ml.put_filter	🟢	27/27	27/27
ml.put_job	🟢	227/227	225/225
ml.put_trained_model_alias	🟢	13/13	13/13
ml.put_trained_model_definition_part	🟢	1/1	1/1
ml.put_trained_model_vocabulary	🟢	1/1	1/1
ml.put_trained_model	🟢	16/16	16/16
ml.reset_job	🟢	2/2	2/2
ml.revert_model_snapshot	🟢	2/2	2/2
ml.set_upgrade_mode	🟢	6/6	6/6
ml.start_data_frame_analytics	🟢	1/1	1/1
ml.start_datafeed	🟢	24/24	24/24
ml.start_trained_model_deployment	🟢	14/14	14/14
ml.stop_data_frame_analytics	🟢	5/5	5/5
ml.stop_datafeed	🟢	17/17	17/17
ml.stop_trained_model_deployment	🟢	10/10	10/10
ml.update_data_frame_analytics	🟢	2/2	2/2
ml.update_datafeed	🟢	7/7	7/7
ml.update_filter	🟢	3/3	3/3
ml.update_job	🟢	5/5	5/5
ml.update_model_snapshot	🟢	3/3	3/3
ml.update_trained_model_deployment	🟢	4/4	4/4
ml.upgrade_job_snapshot	🟢	3/3	3/3
ml.validate_detector	🟢	2/2	2/2
ml.validate	🟢	3/3	3/3

You can validate these APIs yourself by using the make validate target.

[pquentin]

OK, I've covered every API now! Thanks a lot for doing this.

[pquentin]

Would you mind creating a base class as done in the Elasticsearch server code to avoid missing fields shared by all tokenization configs like this in the future?

[l-trotta]

finished addressing the reviews, thank you so much @pquentin for double checking everything!

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
ml.clear_trained_model_deployment_cache	🟢	1/1	1/1
ml.close_job	🟢	64/64	63/63
ml.delete_calendar_event	🟢	4/4	4/4
ml.delete_calendar_job	🟢	3/3	3/3
ml.delete_calendar	🟢	5/5	5/5
ml.delete_data_frame_analytics	🟢	2/2	2/2
ml.delete_datafeed	🟢	3/3	3/3
ml.delete_expired_data	🟢	5/5	5/5
ml.delete_filter	🟢	27/27	27/27
ml.delete_forecast	🟢	3/3	3/3
ml.delete_job	🟢	47/47	47/47
ml.delete_model_snapshot	🟢	2/2	2/2
ml.delete_trained_model_alias	🟢	3/3	3/3
ml.delete_trained_model	🟢	5/5	5/5
ml.estimate_model_memory	🟢	16/16	16/16
ml.evaluate_data_frame	🟢	15/15	15/15
ml.explain_data_frame_analytics	🟢	7/7	7/7
ml.flush_job	🟢	15/15	15/15
ml.forecast	🟢	1/1	1/1
ml.get_buckets	🟢	14/14	14/14
ml.get_calendar_events	🟢	15/15	15/15
ml.get_calendars	🟢	17/17	17/17
ml.get_categories	🟢	12/12	12/12
ml.get_data_frame_analytics_stats	🟢	12/12	12/12
ml.get_data_frame_analytics	🟢	17/17	17/17
ml.get_datafeed_stats	🟢	27/27	27/27
ml.get_datafeeds	🔴	20/20	9/20
ml.get_filters	🟢	13/13	13/13
ml.get_influencers	🟢	11/11	11/11
ml.get_job_stats	🟢	32/32	32/32
ml.get_jobs	🔴	31/31	22/31
ml.get_memory_stats	🟢	6/6	6/6
ml.get_model_snapshot_upgrade_stats	🟢	3/3	3/3
ml.get_model_snapshots	🟢	18/18	18/18
ml.get_overall_buckets	🟢	16/16	15/15
ml.get_records	🟢	8/8	8/8
ml.get_trained_models_stats	🔴	17/17	10/17
ml.get_trained_models	🟢	37/37	37/37
ml.infer_trained_model	🟢	10/10	10/10
ml.info	🟢	10/10	10/10
ml.open_job	🟢	83/83	83/83
ml.post_calendar_events	🟢	12/12	12/12
ml.post_data	🔴	9/11	18/18
ml.preview_data_frame_analytics	🟢	3/3	3/3
ml.preview_datafeed	🟢	17/17	17/17
ml.put_calendar_job	🟢	12/12	12/12
ml.put_calendar	🟢	135/135	135/135
ml.put_data_frame_analytics	🟢	33/33	33/33
ml.put_datafeed	🔴	71/71	53/71
ml.put_filter	🟢	27/27	27/27
ml.put_job	🔴	227/227	223/225
ml.put_trained_model_alias	🟢	13/13	13/13
ml.put_trained_model_definition_part	🟢	1/1	1/1
ml.put_trained_model_vocabulary	🟢	1/1	1/1
ml.put_trained_model	🔴	8/16	16/16
ml.reset_job	🟢	2/2	2/2
ml.revert_model_snapshot	🟢	2/2	2/2
ml.set_upgrade_mode	🟢	6/6	6/6
ml.start_data_frame_analytics	🟢	1/1	1/1
ml.start_datafeed	🟢	24/24	24/24
ml.start_trained_model_deployment	🟢	14/14	14/14
ml.stop_data_frame_analytics	🟢	5/5	5/5
ml.stop_datafeed	🟢	17/17	17/17
ml.stop_trained_model_deployment	🟢	10/10	10/10
ml.update_data_frame_analytics	🟢	2/2	2/2
ml.update_datafeed	🔴	7/7	2/7
ml.update_filter	🟢	3/3	3/3
ml.update_job	🟢	5/5	5/5
ml.update_model_snapshot	🟢	3/3	3/3
ml.update_trained_model_deployment	🟢	4/4	4/4
ml.upgrade_job_snapshot	🟢	3/3	3/3
ml.validate_detector	🟢	2/2	2/2
ml.validate	🟢	3/3	3/3

You can validate these APIs yourself by using the make validate target.

[pquentin]

Thanks! LGTM. Thanks for iterating. Let's merge this! 🎉

(And next time, let's try to split this into more PRs 😅)