C Transformers
C Transformers库提供了GGML模型的Python绑定。
本示例介绍如何使用LangChain与C Transformers 模型进行交互。
安装
%pip install --upgrade --quiet ctransformers
加载模型
from langchain_community.llms import CTransformers
llm = CTransformers(model="marella/gpt-2-ggml")
生成文本
print(llm.invoke("AI is going to"))
流式处理
from langchain_core.callbacks import StreamingStdOutCallbackHandler
llm = CTransformers(
model="marella/gpt-2-ggml", callbacks=[StreamingStdOutCallbackHandler()]
)
response = llm.invoke("AI is going to")
LLMChain
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
template = """Question: {question}
Answer:"""
prompt = PromptTemplate.from_template(template)
llm_chain = LLMChain(prompt=prompt, llm=llm)
response = llm_chain.run("What is AI?")