The pipeline object can do that for you when you set the parameter:

  • transformers < 4.7.0: grouped_entities to True.
  • transformers >= 4.7.0: aggregation_strategy to simple
from transformers import pipeline

#transformers < 4.7.0
#ner = pipeline("ner", grouped_entities=True)

ner = pipeline("ner", aggregation_strategy='simple')

sequence = "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very close to the Manhattan Bridge which is visible from the window."

output = ner(sequence)

print(output)

Output:

[{'entity_group': 'I-ORG', 'score': 0.9970663785934448, 'word': 'Hugging Face Inc'}
, {'entity_group': 'I-LOC', 'score': 0.9993778467178345, 'word': 'New York City'}
, {'entity_group': 'I-LOC', 'score': 0.9571147759755453, 'word': 'DUMBO'}
, {'entity_group': 'I-LOC', 'score': 0.9838141202926636, 'word': 'Manhattan Bridge'}
, {'entity_group': 'I-LOC', 'score': 0.9838141202926636, 'word': 'Manhattan Bridge'}]
Answer from cronoik on Stack Overflow
🌐
Hugging Face
huggingface.co › dslim › bert-base-NER
dslim/bert-base-NER · Hugging Face
bert-base-NER is a fine-tuned BERT model that is ready to use for Named Entity Recognition and achieves state-of-the-art performance for the NER task.
🌐
GitHub
microsoft.github.io › PyMarlin › docs › plugins › hf_ner
Named Entity Recognition with HuggingFace models | PyMarlin
We designed this plugin to allow for out-of-the-box training and evaluation of HuggingFace models for NER tasks. We provide a golden config file (config.yaml) which you can adapt to your task. This config will make experimentations easier to schedule and track. All the source code and notebooks to submit jobs can be found here · We will go through how to adapt any dataset/task for PyMarlin and how to setup the plugin. For this purpose we will use the GermEval dataset - this is a dataset with German Named Entity annotation , with data sampled from German Wikipedia and News Corpora.
🌐
Hugging Face
huggingface.co › models
Models – Hugging Face
named-entity-recognition · Inference Endpoints · text-generation-inference · Eval Results · Merge · 4-bit precision · custom_code · 8-bit precision · text-embeddings-inference · Mixture of Experts · Carbon Emissions · Apply filters · 741 · Full-text search Inference Available ·
🌐
Hugging Face
huggingface.co › docs › transformers › main › tasks › token_classification
Token classification
Token classification assigns a label to individual tokens in a sentence. One of the most common token classification tasks is Named Entity Recognition (NER).
🌐
GitHub
github.com › Arshad221b › Named-Entity-Recognition
GitHub - Arshad221b/Named-Entity-Recognition: NER using Huggingface model. Implementation of HF Tokeniser, Trainer and Pipeline.
Huggingface RoBERTa model implementation for NER on conll2003 dataset. ... from transformers import pipeline nlp = pipeline("ner", model=model_, tokenizer=tokenizer) example = "My name is Wolfgang and I live in Berlin" ner_results = nlp(example) print(ner_results) ... [{'entity': 'B-PER', 'score': 0.84195936, 'index': 4, 'word': 'wolfgang', 'start': 11, 'end': 19}, {'entity': 'B-LOC', 'score': 0.9583987, 'index': 9, 'word': 'berlin', 'start': 34, 'end': 40}]
Author   Arshad221b
🌐
KDnuggets
kdnuggets.com › implement-named-entity-recognition-with-hugging-face-transformers
How to Implement Named Entity Recognition with Hugging Face Transformers - KDnuggets
Let's take a look at how we can perform NER using that Swiss army knife of NLP and LLM libraries, Hugging Face's Transformers.
Find elsewhere
🌐
Reddit
reddit.com › r/languagetechnology › named-entity recognition of long texts using huggingface's "ner" pipeline
r/LanguageTechnology on Reddit: Named-Entity Recognition of Long Texts Using HuggingFace's "ner" Pipeline
September 8, 2021 -

I'm trying to fine-tune BERT to do named-entity recognition (i.e. token classification with some extra steps). Most of my documents are longer than BERT's 512-token max length, so I can't evaluate the whole doc in one go.

In theory, I think what I want to do is have a sliding window that averages the logits for the overlapping sections. I am not sure how to accomplish this using TokenClassificationPipeline (source), which seems to automatically truncate the input text to the model's max length.

Anyone know an easy way to accomplish this? Or should I make a feature request to HuggingFace? 3rd option?

🌐
Hugging Face
huggingface.co › mdarhri00 › named-entity-recognition
mdarhri00/named-entity-recognition · Hugging Face
Named Entity Recognition (NER): The primary purpose of this model is to perform Named Entity Recognition (NER) in text data. It identifies and categorizes entities such as names of people, organizations, locations, dates, and more.
🌐
YouTube
youtube.com › watch
Named Entity Recognition with Hugging Face 🤗 NLP Tutorial For Beginners - YouTube
Hi, in this video, we talk about how to perform NER with HuggingFace, and Transformers using BERT00:01 What is NER?00:40 Loading the dataset 01:17 Understand...
Published   November 2, 2023
🌐
SageMaker
sagemaker.readthedocs.io › en › v2.113.0 › algorithms › text › named_entity_recognition_hugging_face.html
Named Entity Recognition - HuggingFace — sagemaker 2.113.0 documentation
Named Entity Recognition - HuggingFace · Edit on GitHub · This is a supervised named entity recognition algorithm which supports fine-tuning of many pre-trained models available in Hugging Face. The following sample notebook demonstrates how to use the Sagemaker Python SDK for Named Entity ...
🌐
Hugging Face
huggingface.co › mansoorhamidzadeh › Named-entity-recognition
mansoorhamidzadeh/Named-entity-recognition · Hugging Face
Model Description: This Named-Entity-Recognition (NER) model is designed to identify and classify named entities in Persian (Farsi) text into predefined categories such as person names, organizations, locations, dates, and more.
🌐
Hugging Face
huggingface.co › blog › minibase-ai › named-entity-recognition
A lightweight model for Named Entity Recognition (NER)
TL;DR: We’re releasing compact models for Named Entity Recognition (NER). These model can run locally on a CPU and quickly identifies people, organizations, and locations with near-perfect recall. There is a Standard and Small version. Both models are available on HuggingFace (Standard & ...
🌐
Hugging Face
huggingface.co › papers › 2311.08526
Paper page - GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer
However, their size and cost, particularly for those accessed via APIs like ChatGPT, make them impractical in resource-limited scenarios. In this paper, we introduce a compact NER model trained to identify any type of entity.
🌐
freeCodeCamp
freecodecamp.org › news › getting-started-with-ner-models-using-huggingface
How to Fine-Tune BERT for NER Using HuggingFace
January 31, 2022 - For example, in the sentence "Last week Gandalf visited the Shire", we can consider entities to be "Gandalf" with label "Person" and "Shire" with label "Location". To build a model that'll perform this task, first of all we need a dataset. We'll be using the WikiANN dataset for the Bengali language, which is easily available via the datasets module of HuggingFace.
🌐
Hugging Face
huggingface.co › course › chapter7 › 2
Token classification - Hugging Face Course
Named entity recognition (NER): Find the entities (such as persons, locations, or organizations) in a sentence.
🌐
Hugging Face
huggingface.co › dslim › distilbert-NER
dslim/distilbert-NER · Hugging Face
distilbert-NER is the fine-tuned version of DistilBERT, which is a distilled variant of the BERT model. DistilBERT has fewer parameters than BERT, making it smaller, faster, and more efficient. distilbert-NER is specifically fine-tuned for the task of Named Entity Recognition (NER).
🌐
Medium
medium.com › @lokaregns › named-entity-recognition-with-hugging-face-transformers-a-beginners-guide-e1ac6085fb3c
Named Entity Recognition with Hugging Face Transformers: A Beginner’s Guide | by Ganesh Lokare | Medium
February 7, 2023 - Named entity recognition: Use the information captured in the token embeddings to identify named entities and categorize them into predefined categories.