Skip to main content

ChatHuggingFace

这将帮助您开始使用 langchain_huggingface 聊天模型。有关所有 ChatHuggingFace 功能和配置的详细文档，请访问 API 参考。要查看 Hugging Face 支持的模型列表，请查看 此页面。

概述​

集成细节​

集成详情​

类包本地可序列化JS 支持包下载量包最新版本
ChatHuggingFacelangchain-huggingface✅beta❌PyPI - DownloadsPyPI - Version

模型特性​

工具调用结构化输出JSON 模式图像输入音频输入视频输入令牌级流式传输原生异步令牌使用Logprobs
✅✅❌✅✅✅❌✅✅❌

设置​

要访问 Hugging Face 模型，您需要创建一个 Hugging Face 账户，获取一个 API 密钥，并安装 langchain-huggingface 集成包。

凭证​

生成一个 Hugging Face 访问令牌 并将其存储为环境变量： HUGGINGFACEHUB_API_TOKEN。

import getpass
import os

if not os.getenv("HUGGINGFACEHUB_API_TOKEN"):
    os.environ["HUGGINGFACEHUB_API_TOKEN"] = getpass.getpass("Enter your token: ")

安装​

类别包本地可序列化JS 支持包下载量包最新版本
ChatHuggingFacelangchain_huggingface✅❌❌PyPI - DownloadsPyPI - Version

模型特性​

工具调用结构化输出JSON模式图像输入音频输入视频输入令牌级流式传输原生异步令牌使用对数概率
✅✅❌❌❌❌❌❌❌❌

设置​

要访问 langchain_huggingface 模型，您需要创建一个 Hugging Face 账户，获取一个 API 密钥，并安装 langchain_huggingface 集成包。

证书​

您需要将 Hugging Face 访问令牌 保存为环境变量： HUGGINGFACEHUB_API_TOKEN。

import getpass
import os

os.environ["HUGGINGFACEHUB_API_TOKEN"] = getpass.getpass(
    "Enter your Hugging Face API key: "
)
%pip install --upgrade --quiet  langchain-huggingface text-generation transformers google-search-results numexpr langchainhub sentencepiece jinja2 bitsandbytes accelerate

[notice] A new release of pip is available: 24.0 -> 24.1.2
[notice] To update, run: pip install --upgrade pip
Note: you may need to restart the kernel to use updated packages.

实例化​

您可以通过 HuggingFaceEndpoint 或 HuggingFacePipeline 以两种不同方式实例化 ChatHuggingFace 模型。

HuggingFaceEndpoint​

from langchain_huggingface import ChatHuggingFace, HuggingFaceEndpoint

llm = HuggingFaceEndpoint(
    repo_id="HuggingFaceH4/zephyr-7b-beta",
    task="text-generation",
    max_new_tokens=512,
    do_sample=False,
    repetition_penalty=1.03,
)

chat_model = ChatHuggingFace(llm=llm)
该令牌尚未保存到 git 凭据助手中。如果您希望同时设置 git 凭据，请直接在此函数中传递 `add_to_git_credential=True`，或者在通过 `huggingface-cli` 使用时使用 `--add-to-git-credential`。
令牌有效（权限：fineGrained）。
您的令牌已保存至 /Users/isaachershenson/.cache/huggingface/token
登录成功

HuggingFacePipeline​

from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline

llm = HuggingFacePipeline.from_model_id(
    model_id="HuggingFaceH4/zephyr-7b-beta",
    task="text-generation",
    pipeline_kwargs=dict(
        max_new_tokens=512,
        do_sample=False,
        repetition_penalty=1.03,
    ),
)

chat_model = ChatHuggingFace(llm=llm)
config.json:   0%|          | 0.00/638 [00:00<?, ?B/s]
model.safetensors.index.json:   0%|          | 0.00/23.9k [00:00<?, ?B/s]
Downloading shards:   0%|          | 0/8 [00:00<?, ?it/s]
model-00001-of-00008.safetensors:   0%|          | 0.00/1.89G [00:00<?, ?B/s]
model-00002-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
model-00003-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
model-00004-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
model-00005-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
model-00006-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
model-00007-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
model-00008-of-00008.safetensors:   0%|          | 0.00/816M [00:00<?, ?B/s]
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
generation_config.json:   0%|          | 0.00/111 [00:00<?, ?B/s]

使用量化进行实例化​

要运行模型的量化版本，您可以按如下方式指定 bitsandbytes 量化配置：

from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="float16",
    bnb_4bit_use_double_quant=True,
)

并将其作为 model_kwargs 的一部分传递给 HuggingFacePipeline：

llm = HuggingFacePipeline.from_model_id(
    model_id="HuggingFaceH4/zephyr-7b-beta",
    task="text-generation",
    pipeline_kwargs=dict(
        max_new_tokens=512,
        do_sample=False,
        repetition_penalty=1.03,
    ),
    model_kwargs={"quantization_config": quantization_config},
)

chat_model = ChatHuggingFace(llm=llm)

调用​

from langchain_core.messages import (
    HumanMessage,
    SystemMessage,
)

messages = [
    SystemMessage(content="You're a helpful assistant"),
    HumanMessage(
        content="What happens when an unstoppable force meets an immovable object?"
    ),
]

ai_msg = chat_model.invoke(messages)
print(ai_msg.content)
According to the popular phrase and hypothetical scenario, when an unstoppable force meets an immovable object, a paradoxical situation arises as both forces are seemingly contradictory. On one hand, an unstoppable force is an entity that cannot be stopped or prevented from moving forward, while on the other hand, an immovable object is something that cannot be moved or displaced from its position. 

In this scenario, it is un

API 参考​

有关所有 ChatHuggingFace 功能和配置的详细文档，请访问 API 参考： https://api.python.langchain.com/en/latest/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html

API 参考​

有关所有 ChatHuggingFace 功能和配置的详细文档，请访问 API 参考： https://api.python.langchain.com/en/latest/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html

相关​

此页面是否有帮助？


您还可以留下详细的反馈 在 GitHub 上