How do you disable sending system_instruction , while sending query to gemini pro 1.0 ?

[arjandhakal]

I am getting the error Developer instruction is not enabled for models/gemini-pro . It seems to be because of system_instruction that is being sent along.

Log in gptel-log is the following :

{
  "system_instruction": {
    "parts": {
      "text": "You are a large language model living in Emacs and a helpful assistant. Respond concisely."
    }
  },
  "contents": [
    {
      "role": "user",
      "parts": {
        "text": "Hello"
      }
    }
  ],
  "safetySettings": [
    {
      "category": "HARM_CATEGORY_HARASSMENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_HATE_SPEECH",
      "threshold": "BLOCK_NONE"
    }
  ],
  "generationConfig": {
    "temperature": 1.0
  }
}
{
  "gptel": "response body",
  "timestamp": "2024-09-27 18:36:42"
}
[
  {
    "error": {
      "code": 400,
      "message": "Developer instruction is not enabled for models/gemini-pro",
      "status": "INVALID_ARGUMENT"
    }
  }
]

I am not sure but it seems to be because of system_instruction? Any way to disable that or is the problem something else?

[karthink]

@arjandhakal Thanks, this is because of a recent change in both gptel and the Gemini API. I'll fix it, but could you run a little test first?

1. Check if requests to Gemini using the gemini-1.5-pro-latest model work correctly when a system instruction is set
2. Check if requests to Gemini using the gemini-1.5-flash model work correctly when a system instruction is set

You can change the model by setting gptel-model or from gptel's transient menu.

[arjandhakal]

@karthink Ran the test for both gemini-1.5-pro-latest and gemini-1.5-flash. Both are working.

Log for gemini1.5-pro-latest

{
  "system_instruction": {
    "parts": {
      "text": "You are a large language model living in Emacs and a helpful assistant. Respond concisely."
    }
  },
  "contents": [
    {
      "role": "user",
      "parts": {
        "text": "Hello"
      }
    },
    {
      "role": "model",
      "parts": {
        "text": "Greetings. \\360\\237\\221\\213"
      }
    },
    {
      "role": "user",
      "parts": {
        "text": "What is the result of multiplying 2 by 4?"
      }
    }
  ],
  "safetySettings": [
    {
      "category": "HARM_CATEGORY_HARASSMENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_HATE_SPEECH",
      "threshold": "BLOCK_NONE"
    }
  ],
  "generationConfig": {
    "temperature": 1.0
  }
}
{
  "gptel": "response body",
  "timestamp": "2024-09-27 20:44:22"
}
[
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": "8"
            }
          ],
          "role": "model"
        },
        "finishReason": "STOP",
        "index": 0
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 38,
      "candidatesTokenCount": 1,
      "totalTokenCount": 39
    }
  },
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": " \n"
            }
          ],
          "role": "model"
        },
        "finishReason": "STOP",
        "index": 0,
        "safetyRatings": [
          {
            "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_HATE_SPEECH",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_HARASSMENT",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
            "probability": "NEGLIGIBLE"
          }
        ]
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 38,
      "candidatesTokenCount": 1,
      "totalTokenCount": 39
    }
  }
]

Log for gemini-1.5-flash

{
  "system_instruction": {
    "parts": {
      "text": "You are a large language model living in Emacs and a helpful assistant. Respond concisely."
    }
  },
  "contents": [
    {
      "role": "user",
      "parts": {
        "text": "Hello"
      }
    },
    {
      "role": "model",
      "parts": {
        "text": "Greetings. \\360\\237\\221\\213"
      }
    },
    {
      "role": "user",
      "parts": {
        "text": "What is the result of multiplying 2 by 4?"
      }
    },
    {
      "role": "model",
      "parts": {
        "text": "8"
      }
    },
    {
      "role": "user",
      "parts": {
        "text": "What is 2 * 2?"
      }
    }
  ],
  "safetySettings": [
    {
      "category": "HARM_CATEGORY_HARASSMENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_HATE_SPEECH",
      "threshold": "BLOCK_NONE"
    }
  ],
  "generationConfig": {
    "temperature": 1.0
  }
}
{
  "gptel": "response body",
  "timestamp": "2024-09-27 20:47:06"
}
[
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": "4"
            }
          ],
          "role": "model"
        },
        "finishReason": "STOP",
        "index": 0
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 49,
      "candidatesTokenCount": 1,
      "totalTokenCount": 50
    }
  },
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": "\n\n"
            }
          ],
          "role": "model"
        },
        "finishReason": "STOP",
        "index": 0,
        "safetyRatings": [
          {
            "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_HATE_SPEECH",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_HARASSMENT",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
            "probability": "NEGLIGIBLE"
          }
        ]
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 49,
      "candidatesTokenCount": 1,
      "totalTokenCount": 50
    }
  }
]

[karthink]

I pushed a fix, please test both gemini-pro and one of the other models that support a system instruction.

[arjandhakal]

@karthink Thanks for working on this issue so quickly.

So, the recent fix did fix it with a little issue remaining .gemini-pro , gemini-1.5-pro-latest and gemini-1.5-flash, all work.

But, if i switch to gemini-1.5-pro-latest or gemini-1.5-flash, interact with the LLM and switch back to gemini-pro, then its getting the same error with system_instruction being attached in the request

After that I need to restart emacs, to be able to use gemini-pro again.

[karthink]

Hmm that's odd, I'm not seeing why that happens. The choice to use/not-use the system instruction should be stateless.