Named-entity recognition

Hi, NER is basically a token level text classification problem, which can be considered to be similar to semantic segmentation in vision tasks, which is pixel level classification. To prepare the dataset, first you need to have a fixed number of labels, like any other classification problem, and each word should be labelled (an label for all words doesn't have an entity). Please ensure no words are left unlabelled. Once you have this dataset, you can try these, based on your dataset aswell: as mentioned in other comments, few-shot learning with LLMs using spacy custom NER model ( Ref: https://medium.com/@mjghadge9007/building-your-own-custom-named-entity-recognition-ner-model-with-spacy-v3-a-step-by-step-guide-15c7dcb1c416 ) BERT token level classifier (Ref: https://huggingface.co/docs/transformers/en/tasks/token_classification ) An RNN or LSTM classifier with some dense embedded features (glove, word2vec etc), and a prediction layer at each time step after the stack of (if multi-layer) RNNs I would suggest you try the 4th one only if you have enough time, otherwise invest more on preparing a good enough custom dataset and work on any of the first 3. More on reddit.com

If spaCy’s NER isn’t picking up what you need, you’ll probably need to look into creating your own annotations and fine tuning a model or training a custom model. It isn’t too hard using BIO/BILOU tags. Things like “raw materials” and particularly niche models and brands are unlikely to be picked up by off the shelf solutions. More on reddit.com

spacy does a great job of supporting domain specific and custom named entities More on reddit.com

Can you kill a mosquito with a bazooka? Yes. Is it the most efficient tool to do so? No. There will always be room for specialized, effective and efficient tools to do specific tasks. More on reddit.com