Parametrized OpenAI Models

[john-thomas-dotcms]

Parent Issue

No response

User Story

Currently, the OpenAI models we support, and the parameters for those models (such as maximum tokens) are hard-coded in a Java file: https://github.com/dotCMS/core/blob/master/dotCMS/src/main/java/com/dotcms/ai/util/OpenAIModel.java

However, these models and parameters are not static. For example, OpenAI has added new models which are not in our list, and has already deprecated the gpt-3.5-turbo-16k model (and will decommission it completely in 3 months). But the only way we can change those for dotAI is with a Java code change (which will then also require backporting to 24.04 LTS).

We will have a more future-proof solution for this in dotAI phase 2. However, for now we need to:

1. Not restrict the model name used in our code.
   • Users should be able to type in any model name they want, and we will pass it through to OpenAI.
   • But, we need to be able to detect in code if the OpenAI API returns an error indicating either that the model name is invalid, or that the model is no longer supported.
2. Move all of the parameters specified in the Java code to configuration parameters in the dotAI App.
   • The code should then pull those parameters from the App.
   • Note that we will need 2 sets of parameters in the dotAI App - one for the text model and one for the image model.

Because of both our inability to support new OpenAI models, and the impending decommissioning of a model we support, this change is a high priority.

Acceptance Criteria

1. Enable configuration of each of the following models and parameter sets via the dotAI App:
2. When an error is returned from OpenAI indicating an invalid model name was entered, we capture it.
   • Although it would be ideal to differentiate between a model which doesn't exist vs. a model that's no longer supported, I'm not sure the OpenAI API will make that distinction, so this isn't required.
3. When an error is returned from OpenAI indicating that the parameters are wrong (e.g. maximum tokens exceeded), we capture it.
4. Any time we capture one of the above types of errors, we do all of the following:
   • Display a popup notification to the user attempting to use dotAI.
     • For the AI blocks, this notification can be displayed in the existing popups or in a separate error popup.
     • For other places where dotAI is used, we display the error in the standard way for that modality.
       • For example, if an error is generated with an API method, we return an error code but do not notify any users.
   • Display a Notification to the admin (in the Notifications pulldown; same permissions as reindex messages).
   • Write a message to the log (ERROR level).

Proposed Objective

Reliability

Proposed Priority

Priority 2 - Important

External Links... Slack Conversations, Support Tickets, Figma Designs, etc.

No response

Assumptions & Initiation Needs

No response

Quality Assurance Notes & Workarounds

No response

Sub-Tasks & Estimates

No response

Tasks

Preview Give feedback

1. Relocate OpenAI model parameters to dotAI App params #29281
   Priority : 2 High QA : Approved QA : Passed Internal Release : 24.08.20 Team : Falcon Type : New Functionality dotCMS : AI +7
   
2. Fail gracefully when OpenAI model stops working #29284
   Priority : 2 High QA : Approved QA : Passed Internal Release : 24.09.21 Team : Falcon Type : New Functionality dotCMS : AI +7
3. Enhance dotAI Functionality: Model Listing, Exception Handling, Improved Logging, and Updated Tests #29587
   Priority : 2 High QA : Not Needed Release : 24.08.20 Team : Falcon Type : New Functionality dotCMS : AI +6