Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes #4182: The huggingface examples return strange results #4190

Merged
merged 2 commits into from
Oct 7, 2024
Merged

Fixes #4182: The huggingface examples return strange results #4190

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7b87282
Fixes #4182: The huggingface examples return strange results
vga91 Sep 6, 2024
710e93d
changes review and removed unused imports
vga91 Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/asciidoc/modules/ROOT/pages/ml/openai.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,10 +161,12 @@ For the https://huggingface.co/[HuggingFace API], we have to define the config `
For example:
[source,cypher]
----
CALL apoc.ml.openai.completion('What color is the sky? Answer in one word: ', $huggingFaceApiKey,
{endpoint: 'https://api-inference.huggingface.co/models/gpt2', apiType: 'HUGGINGFACE', model: 'gpt2', path: ''})
CALL apoc.ml.openai.completion('[MASK] is the color of the sky', $huggingFaceApiKey,
{endpoint: 'https://api-inference.huggingface.co/models/google-bert/bert-base-uncased', apiType: 'HUGGINGFACE'})
----

With gpt2 or other text completion models the answers are not valid.

Or also, by using the https://docs.cohere.com/docs[Cohere API], where we have to define `path: '''` not to add the `/completions` suffix to the URL:
[source,cypher]
----
Expand Down
3 changes: 3 additions & 0 deletions extended/src/main/java/apoc/ml/OpenAI.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import static apoc.ExtendedApocConfig.APOC_ML_OPENAI_TYPE;
import static apoc.ExtendedApocConfig.APOC_OPENAI_KEY;
import static apoc.ml.MLUtil.*;
import static apoc.ml.RestAPIConfig.METHOD_KEY;


@Extended
Expand Down Expand Up @@ -103,6 +104,8 @@ private static void handleAPIProvider(OpenAIRequestHandler.Type type,
}
case HUGGINGFACE -> {
configForPayload.putIfAbsent("inputs", inputs);
configuration.putIfAbsent(PATH_CONF_KEY, "");
headers.putIfAbsent(METHOD_KEY, "POST");
configuration.putIfAbsent(JSON_PATH_CONF_KEY, "$[0]");
}
case ANTHROPIC -> {
Expand Down
13 changes: 6 additions & 7 deletions extended/src/test/java/apoc/ml/OpenAIOpenLMIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,17 @@ public void setUp() throws Exception {
public void completionWithHuggingFace() {
String huggingFaceApiKey = System.getenv("HF_API_TOKEN");
Assume.assumeNotNull("No HF_API_TOKEN environment configured", huggingFaceApiKey);
String modelId = "gpt2";

String modelId = "google-bert/bert-base-uncased";
Map<String, String> conf = Map.of(ENDPOINT_CONF_KEY, "https://api-inference.huggingface.co/models/" + modelId,
API_TYPE_CONF_KEY, OpenAIRequestHandler.Type.HUGGINGFACE.name(),
PATH_CONF_KEY, "",
MODEL_CONF_KEY, modelId
API_TYPE_CONF_KEY, OpenAIRequestHandler.Type.HUGGINGFACE.name()
);
testCall(db, COMPLETION_QUERY,

testCall(db, "CALL apoc.ml.openai.completion('[MASK] is the color of the sky', $apiKey, $conf)",
Map.of("conf", conf, "apiKey", huggingFaceApiKey),
(row) -> {
var result = (Map<String,Object>) row.get("value");
String generatedText = (String) result.get("generated_text");
String generatedText = (String) result.get("sequence");
assertTrue(generatedText.toLowerCase().contains("blue"),
"Actual generatedText is " + generatedText);
});
Expand Down
Loading