支持LongCite-glm4-9b模型

[leoterry-ulrica]

Feature request / 功能建议

basic demo中支持Longite-glm4-9b的加载和推理。

Motivation / 动机

现在能跑起来，但chat的时候不正常：没法返回结果。
request：

{
        "model": "glm-4",
        "stream": true,
        "messages": [
            {
                "role": "system",
                "content": "You are a helpful assistant."
            },
            {
                "role": "user",
                "content": "W. Russell Todd, 94, United States Army general (b. 1928). February 13. Tim Aymar, 59, heavy metal singer (Pharaoh) (b. 1963). Marshall \"Eddie\" Conway, 76, Black Panther Party leader (b. 1946). Roger Bonk, 78, football player (North Dakota Fighting Sioux, Winnipeg Blue Bombers) (b. 1944). Conrad Dobler, 72, football player (St. Louis Cardinals, New Orleans Saints, Buffalo Bills) (b. 1950). Brian DuBois, 55, baseball player (Detroit Tigers) (b. 1967). Robert Geddes, 99, architect, dean of the Princeton University School of Architecture (1965–1982) (b. 1923). Tom Luddy, 79, film producer (Barfly, The Secret Garden), co-founder of the Telluride Film Festival (b. 1943). David Singmaster, 84, mathematician (b. 1938)."
            },
            {
                "role": "user",
                "content": "how old is W. Russell Todd?"
            }
        ]
    }

response：无限返回叹号！

Your contribution / 您的贡献

无

[ghost]

你使用的是哪个推理脚本

[leoterry-ulrica]

你使用的是哪个推理脚本

https://github.com/THUDM/GLM-4/blob/main/basic_demo/openai_api_server.py

[zRzRzRzRzRzRzR]

这个问题已经被记录，我们复现了，我们看一下这个实现文件要怎么改

[sixsixcoder]

修改https://github.com/THUDM/GLM-4/blob/main/basic_demo/glm_server.py文件第670行为dtype=torch.bfloat16，
或者使用huggingface上的推理代码进行推理https://huggingface.co/THUDM/LongCite-glm4-9b

[leoterry-ulrica]

修改https://github.com/THUDM/GLM-4/blob/main/basic_demo/glm_server.py文件第670行为dtype=torch.bfloat16， 或者使用huggingface上的推理代码进行推理https://huggingface.co/THUDM/LongCite-glm4-9b

虽然能正常输出了，但返回的结果没有引用源标注，是不是只能自己去适配：

model.query_longcite(context, query, tokenizer=tokenizer, max_new_tokens=1024)

[sixsixcoder]

请按照如下方式解析result，参考https://huggingface.co/THUDM/LongCite-glm4-9b

print("Answer:\n{}\n".format(result['answer']))
print("Statement with citations:\n{}\n".format(
  json.dumps(result['statements_with_citations'], indent=2, ensure_ascii=False)))
print("Context (divided into sentences):\n{}\n".format(result['splited_context']))