No way to query product ingestion API since CustomRequestBuilder removed in V6

[Tomanow]

Is your feature request related to a problem? Please describe the problem.

We are unable to upgrade to v6 since we can no longer construct custom URLs for graph requests, and the SDK we use (the partner marketplace product ingestion API) is not supported by the client.

The BaseClient#customRequest() method and CustomRequestBuilder support was removed in v6 with no alternative provided.

Describe the solution you'd like.

Support for building custom graph request URIs and/or SDK support for the partner marketplace product ingestion APIs.

Additional context?

We (MongoDB) are a significant Microsoft Azure Partner with high profile integrations relying on the Java SDK.

[Ndiritu]

@baywet @andrueastman what do you think about us adding a customRequest() method to the service lib's GraphServiceClient to cover this scenario?

public T customRequest(RequestInformation requestInformation, Class<T extends Parsable> responseType)

[andrueastman]

Is this something that can be resolved with the generated withUrl methods?

public T customRequest(RequestInformation requestInformation, Class<T extends Parsable> responseType)

I believe this is functionally equivalent to the send methods in the request adapter.
https://github.com/microsoft/kiota-java/blob/6291fa271db0f3e5844832b6a2f42368d22006de/components/http/okHttp/src/main/java/com/microsoft/kiota/http/OkHttpRequestAdapter.java#L267

[baywet]

This API is "special" and doesn't follow the typical Graph conventions/infrastructure. They just happen to be on the same host. The base URI is https://graph.microsoft.com/rp/ (note the absence of version).
This is why nothing is generated for it, it's not part of the metadata, etc...
Which means the "with URL" pattern won't solve the partner issue here.
What would solve it is to be able to get an OkHttpClient with an auth middleware.
For that we'd need:

1. a middleware that accepts an authentication provider, and gets the access token provide from it.
2. a client factory that adds that middleware to the stack to make things easier
3. a bit of documentation to demonstrate the usage for arbitrary requests. Where the guidance is to use the generated APIs first, then try with URL, if all else fails, this OkHttpClient. And also NOT to insert this middleware in the stack for the generated GraphClient since we already have the authentication provider here.

Assuming this is clear, does one of you want to work on a design here? We'd then generalize to other languages.

[Ndiritu]

Sounds good @baywet. Was targeting re-using our (de)serialization but it would be added effort to make the customer's custom models into Parsable's yet v5 used GSON to deserialize to models.

Using an OkHttpClient with Graph auth middleware works better.

Could the Graph auth middleware still be present by default & be initialized with the same auth provider the RequestAdapter uses to retain allowed hosts & other config + check for an Authorization header before considering adding one?

I can take this on probably next week/sprint worst case.

[baywet]

Could the Graph auth middleware still be present by default & be initialized with the same auth provider the RequestAdapter uses to retain allowed hosts & other config + check for an Authorization header before considering adding one?

We could explore that approach but we need to be careful not to:

• do auth twice
• break CAE challenges / resolution

This would have the benefit of not asking customers to juggle between multiple okhttp clients.

[andrueastman]

Just to be clear here, do we need the auth middleware/custom client as the token needed is different than the one that would be used for normal graph calls?

In my head, this should ideally work as the sendXXX methods from the adapter should handle the auth as well CAE challenges together with deserialization assuming the right method is called. (what we may need here is method that makes this all easier).

        // Create a request info object
        var requestInformation = new RequestInformation();
        requestInformation.httpMethod = HttpMethod.GET;
        requestInformation.urlTemplate = "https://graph.microsoft.com/rp/xxxxx";

        // Setup an error mapping
        HashMap<String, ParsableFactory<? extends Parsable>> errorMapping = new HashMap();
        errorMapping.put("XXX", ODataError::createFromDiscriminatorValue);

        // get the response as a stream
        var responseStream = graphClient.getRequestAdapter().sendPrimitiveCollection(requestInformation, errorMapping, Stream.class);
        // TODO deserialize the stream as desired.

        // OR get the parsable using a factory as User object
        var deserializedResponse = graphClient.getRequestAdapter().send(requestInformation, errorMapping, User::createFromDiscriminatorValue);

a middleware that accepts an authentication provider, and gets the access token provide from it.

A small challenge we may find with this is that the authentication providers act on RequestInformation instance from kiota abstractions while the middleware is handling transport specific objects(Request from okhttp) objects. Unless you mean hooking up on the AccessTokenProvider?

[Ndiritu]

The authentication request is a standard Azure Identity request with the default Graph scopes.

My concerns with using the sendXXX methods is our SDK doesn't generate models for this API so customers have probably implemented their own. This would mean asking devs to implement Parsable & ParsableFactory.

With the okHttpClient the methods to call are more direct & (de)serialization in the control of the developer.

A small challenge we may find with this is that the authentication providers act on RequestInformation instance from kiota abstractions while the middleware is handling transport specific objects(Request from okhttp) objects. Unless you mean hooking up on the AccessTokenProvider?

Yeah, good point. We may have to add a method that converts from the native okHttp Request to RequestInformation

[baywet]

@andrueastman the request adapter interface (and the associated request information) was never designed to be used by humans, but rather to facilitate the work of the generated clients/reduce the amount of generated code that's required. (on top of providing a level of indirection to avoid tight coupling with any given http client).

The reason why we used this interface to implement tasks (batching, etc...) is mostly because we didn't want to have customers juggle with multiple clients, auth setup etc...

For arbitrary requests, customers will get a much better experience using the native client augmented with the middleware handlers. In fact we used to support that in the previous generation of SDKs. The reason why we didn't feel the need to port it to this newer generation was because the generated portion supports so much more of the API surface, so there was a much smaller need to implement it. That's until we ran in this "odd API".

[andrueastman]

@andrueastman the request adapter interface (and the associated request information) was never designed to be used by humans, but rather to facilitate the work of the generated clients/reduce the amount of generated code that's required. (on top of providing a level of indirection to avoid tight coupling with any given http client).

Thanks for the insight here. This is the bit I was missing.

My concerns with using the sendXXX methods is our SDK doesn't generate models for this API so customers have probably implemented their own. This would mean asking devs to implement Parsable & ParsableFactory.

Not necessarily as they could use the methods returning a stream/string and then deserialize the values as they wish. However this is now moot. Let's go ahead with the earlier suggestion here.

[baywet]

related microsoft/kiota-dotnet#270

[baywet]

authored microsoftgraph/msgraph-sdk-design#111 to help move this along

[Tomanow]

Thank you all for your help patching this! 🎉

[mattchannn]

Hi all, so how can the latest change fulfill the need from @Tomanow .

I am a bit lost now but I have similar requirement.

I have tried @andrueastman 's solution and works but looks like @baywet doesn't find this way "desired"

the request adapter interface (and the associated request information) was never designed to be used by humans

[mattchannn]

My requirement is to call the below Graph API to access Sharepoint (aka SITE) and retrieve site id, but it doesn't seems the existing fluent-style API has such method

https://graph.microsoft.com/v1.0/sites/{domain}:/sites/{path}
e.g.: https://graph.microsoft.com/v1.0/sites/abccom.sharepoint.com:/sites/Folder1

[Ndiritu]

Hi @mattchannn,
Thanks for reaching out.
This solution applies to calling Microsoft Graph API endpoints that don't follow the "traditional" Graph API pattern e.g. Product Ingestion API or endpoints not on the v1 reference In these cases we don't generate models for now.

For a "traditional" Graph API endpoint whose models we generate etc, we provide the withUrl() method as a work-around.
For your case, you can replace the siteId with the path or use withUrl as follows:

Site site = client.sites().bySiteId("{hostname}:/{server-relative-path}").get();
// OR
Site site = client.sites().bySiteId("").withUrl(
    "https://graph.microsoft.com/v1.0/sites/{hostname}:/{server-relative-path}"
).get();

[mattchannn]

@Ndiritu

Thank you for your advice, both options work!
Clearly I am not aware we can put it like bySiteId("{hostname}:/{server-relative-path}")
but I do tried below without siteId, may I know why it doesn't work? Any documentation for reference?

SiteCollectionResponse site = graphServiceClient
                .sites()
                .withUrl("https://graph.microsoft.com/v1.0/sites/{hostname}:/{server-relative-path}")
                .get();

[mattchannn]

Actually
https://stackoverflow.com/questions/57009982/microsoft-graph-api-java-sdk-get-a-site-with-its-server-relative-url
The solution inside above link matches the first solution provided by @Ndiritu.