Photo by Editor | Chat GPT
. Introduction
Hugs the facial transformer The library has become a toll cut for the works of natural language processing (NLP) and (large) language model (LLM) in the ecosystem. Her pipeline() The function is an important summary, which enables data scientists and developers to minimize complex tasks to minimize text classification, summary, and designated entity identification.
Although the default settings are excellent to start the settings, some small opportunities can significantly increase the performance, improve the use of memory, and strengthen your code. In this article, we offer 10 powerful one -liner that will help you improve your hugs face pipeline() Floose
. 1. Fast estimation with GPU Exception
One of the easiest but most effective improvements is to transfer your model and its count to GPU. If you have a CUDA-capable GPU available, explaining the device is a parameter change that can accelerate the reduction by the order of intensity.
classifier = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english", device=0)This One Liner tells the pipeline to load the model on the first available GPU (device=0) Only CPU estimates LOY, you can set device=-1.
. 2. To take action on multiple inputs with batching
Instead of repeating and feeding single inputs in the pipeline, you can act on the list of texts at the same time, and pass them completely. The use of batching significantly improves by allowing the model to perform parallel computation on GPU.
results = text_generator(list_of_texts, batch_size=8)Here, list_of_texts There is a list of a standard wire. You can adjust batch_size Based on your GPU memory capacity for maximum performance.
. 3. Activate half -precision high -speed estimates
With modern nvidia gpus Tensor cover Support, half -precision floating point number (using (float16) Can speed up its speed with a minimum effect on accuracy. It also reduces the model of the model’s memory. You will need to import torch The library for her
transcriber = pipeline("automatic-speech-recognition", model="openai/whisper-base", torch_dtype=torch.float16, device="cuda:0")Make sure you have Piturich Installed and imported (import torch) This one liner is especially effective for large models such as whispers or GPT variations.
. 4. Grouping sub -words with collective strategies
When performing the name of the designated entity (NER), models often break the words into a sub -word token (such as “New York” “New” and “## York”). aggregation_strategy The parameter connects it as a single, integrated entity to a token group.
ner_pipeline = pipeline("ner", model="dslim/bert-base-NER", aggregation_strategy="simple") simple The strategy automatically groups the entities, which gives you a clean output {'entity_group': 'LOC', 'score': 0.999, 'word': 'New York'}.
. 5. Beautifully with handling long text
Transformer models have maximum input layout length. Feeding them, which exceeds that limit, will result in a mistake. Activation of the trim ensures that any major input is automatically cut into maximum length of the model.
summarizer = pipeline("summarization", model="sshleifer/distilbart-cnn-12-6", truncation=True)It is a simple one liner to build applications that can handle the real world, unexpected text inputs.
. 6. Activation of high -speed tufts
The Transformers Library contains two sets of tokenis: a slow, pure implementation and sharp, rust -based version. You can ensure that you are using the fast version to promote performance, especially on the CPU. This needs to be loaded first separately.
fast_tokenizer_pipe = pipeline("text-classification", tokenizer=AutoTokenizer.from_pretrained("bert-base-uncased", use_fast=True))Remember to import required class: from transformers import AutoTokenizer. This easy change can make a significant difference in the heavy pre -processing stages from the data.
. 7. Returning of raw tensors for further processing
As default, pipelines return the list of human readings and dictionary. However, if you are integrating the pipeline into a major machine learning workflow, such as embedded in another model, you can directly access raw output tensors.
feature_extractor = pipeline("feature-extraction", model="sentence-transformers/all-MiniLM-L6-v2", return_tensors=True)Sequence return_tensors=True Depending on your installed backdrop, you will get piturich or tensor flu tensors, eliminating unnecessary data conversion.
. 8. Disabling growth rods for cleaner logs
When you use pipelines in an automatic script or production environment, the default progress bars can disrupt your logs. You can disable them globally with the same function call.
You can add from transformers.utils.logging import disable_progress_bar More clean, production friendly output in the upper part of your own script.
Alternatively, and not in individuals related, you can meet the same results by setting the environmental variable (for those who are interested).
export HF_HUB_DISABLE_PROGRESS_BARS=1. 9. Loading a specified model for reproductive capacity
On the model Hug the face hub Can be updated by their owners. To ensure that your application behavior does not change unexpectedly, you can pin your pipeline into a particular model commort hash or branch. It has been fulfilled using this one -liner:
stable_pipe = pipeline("fill-mask", model="bert-base-uncased", revision="e0b3293T")Using a particular revision This guarantees that you always use the exact version of the model, which makes your results quite reproductive. You can find the hash on the model page on the hub.
. 10. to accelerate the pipeline with a pre -loaded model
It may take time to load a large model. If you need to use the same model in different pipeline configurations, you can load it once and reach the Object to the Lord pipeline() Function, time and memory savings.
qa_pipe = pipeline("question-answering", model=my_model, tokenizer=my_tokenizer, device=0)It assumes that you are already loaded my_model And my_tokenizer Items, for example AutoModel.from_pretrained(...). This technique gives you the most potential control and performance when reusing the model assets.
. Wrap
The hugs to the throat pipeline() The function is the gateway to powerful NLP models, and with these 10 one -liners, you can make it faster, more efficient and the use of production of Better better. By moving to the GPU, activating the batch, and using faster the tokeners, you can dramatically improve the performance. By carving, collecting and organizing specific reviews, you can create a stronger and reproductive work flow.
Experience in your own plans with these one -liners and see how small code changes can lead to major correction.
Matthew Mayo For,,,,,,,,,, for,, for,,,, for,,,, for,,, for,,,, for,,,, for,,,, for,,, for,,, for,,, for,,, for,,,, for,,, for,,, for,,,, for,,, for,,,, for,,, for,,, for,,,, for,,, for,,, for,,,, for,,, for,,,, for,,, for,,,, for,,, for,,,, for,,, for,,,, for,,,, for,,,, for,,,, for,,,, for,,,, for,,,, for,,, for,,, for,,, for,,, for,,,,, for,,,, for,,,, for,,,, for,, for,.@MattMayo13) Computer science is a graduate diploma in master’s degree and data mining. As the Managing Editor of Kdnuggets & StatologyAnd supporters in the editor Machine specializes in learningMatthew aims to make complex data science concepts accessible. Its professional interests include natural language processing, language models, machine learning algorithms, and the search for emerging AI. He is driven by a mission to democratic knowledge in the data science community. Matthew has been coding since the age of 6.