Making capabilities() API a factory

[lozy219]

We are expecting the developers to call capabilities() before creating the AILanguageModel but currently this dependency is not enforced, when actually creating the model, it's still necessary to run the availability check.

Would it be better if the capabilities() API returns a factory, which will only be set when the availability is readily or after-download, and we move the create() method to the factory?

The usage becomes:

const capabilities = await ai.languageModel.capabilities();
if (capabilities.available !== "no") {
  // otherwise the factory will be null
  const session = await capabilities.factory.create({
    monitor(m) {
      m.addEventListener("downloadprogress", e => {
        console.log(`Downloaded ${e.loaded} of ${e.total} bytes.`);
      });
    }
  });
}

[domenic]

To be clear, it's not the case that we expect web developers to always call capabilities(). They should only do this if they want to give a different user experience depending on download status, e.g., "summarization will require your browser to download an AI model. Do you want to continue?" In most cases, just calling create() and handling any failures is reasonable.

Given this, it sounds like you're suggesting that instead of the implementation caching, we should make the developer do this work? I feel like it would be better for the implementation to transparently cache, to keep the current API for developers so that they don't have to do this two-step process.

[lozy219]

Thanks, I got the point that the capabilities API is used to provide extra info but not supposed to block the creation. It's perfectly fine if the developer doesn't care about the capabilities and just want to create a session straightaway.

There is another case that was raised during the discussion: if the model/metadata has changed between calling capabilities() and create(), it could be detected if the create() is called from the factory that is returned from the capabilities(). This will be helpful to the developers who want to create the model based on the criteria of the capability check.

For example, they only want to download the model when Japanese is supported, this is what they might do

const capabilities = await ai.languageModel.capabilities();
if (capabilities.available !== "no" && capabilities.languageAvailable("ja") !== "no") {
  // at this point the model is changed, and ja support is removed
  const session = await ai.languageModel.create();
  // this session doesn't support Japanese!
}

In this case if the create method is called from the capabilities then the version mismatch can be detected and we could throw a clear error indicating that the mode file has been updated, and the developer need to be retry the capabilities().

I agree that this is a extreme corner case though.

[domenic]

For example, they only want to download the model when Japanese is supported, this is what they might do

In that case create() must throw, per spec. This should be possible with the current API. I'm not sure why changing the API shape would make a difference here. The browser can cache any appropriate information on ai.languageModel; a new object doesn't help.