Named-entity recognition

If spaCy’s NER isn’t picking up what you need, you’ll probably need to look into creating your own annotations and fine tuning a model or training a custom model. It isn’t too hard using BIO/BILOU tags. Things like “raw materials” and particularly niche models and brands are unlikely to be picked up by off the shelf solutions. More on reddit.com

I did exactly this at my last job. We wanted to do NER on documents thats were sometimes 10,000 words long. We wrote a lot of custom code to split the documents into overlapping chunks, perform NER on each chunk, extract terms and labels from each chunk and then stitch the output back together. I tried to push management to let us open source the code but unfortunately to no avail 🥲 More on reddit.com

Hi, NER is basically a token level text classification problem, which can be considered to be similar to semantic segmentation in vision tasks, which is pixel level classification. To prepare the dataset, first you need to have a fixed number of labels, like any other classification problem, and each word should be labelled (an label for all words doesn't have an entity). Please ensure no words are left unlabelled. Once you have this dataset, you can try these, based on your dataset aswell: as mentioned in other comments, few-shot learning with LLMs using spacy custom NER model ( Ref: https://medium.com/@mjghadge9007/building-your-own-custom-named-entity-recognition-ner-model-with-spacy-v3-a-step-by-step-guide-15c7dcb1c416 ) BERT token level classifier (Ref: https://huggingface.co/docs/transformers/en/tasks/token_classification ) An RNN or LSTM classifier with some dense embedded features (glove, word2vec etc), and a prediction layer at each time step after the stack of (if multi-layer) RNNs I would suggest you try the 4th one only if you have enough time, otherwise invest more on preparing a good enough custom dataset and work on any of the first 3. More on reddit.com

I don't know about tutorials, but you should check the seqeval library. I also recommend Lignos and Kamyab (2020) about results reproductibility in NER More on reddit.com