If spaCy’s NER isn’t picking up what you need, you’ll probably need to look into creating your own annotations and fine tuning a model or training a custom model. It isn’t too hard using BIO/BILOU tags. Things like “raw materials” and particularly niche models and brands are unlikely to be picked up by off the shelf solutions. More on reddit.com

Can you make more examples? Annotating NER isn't that hard, there are user-friendly annotation tools and you (or an exploited student) can get much more than 400 annotated lines within a single day. Alternatively, is your problem really that unique so that your 400 lines is all there is, is there really no way to find a larger existing dataset somewhere in the world?

Also, NER tends to rely on two types of training data - contextual examples (annotated sentences i.e. BIO format) and gazetteers (databases of known entity names). If you have few examples, perhaps can you get a good/large gazetteer for your problem domain?

More on reddit.com

I am not sure, if there is huge differences from one model to another. This is heavily depending on the training data that you can get. I would suggest using some existing NER nodels and possibly fine tune them on your own data. Have a look at GENRE https://github.com/facebookresearch/GENRE More on reddit.com

Hi, NER is basically a token level text classification problem, which can be considered to be similar to semantic segmentation in vision tasks, which is pixel level classification. To prepare the dataset, first you need to have a fixed number of labels, like any other classification problem, and each word should be labelled (an label for all words doesn't have an entity). Please ensure no words are left unlabelled. Once you have this dataset, you can try these, based on your dataset aswell: as mentioned in other comments, few-shot learning with LLMs using spacy custom NER model ( Ref: https://medium.com/@mjghadge9007/building-your-own-custom-named-entity-recognition-ner-model-with-spacy-v3-a-step-by-step-guide-15c7dcb1c416 ) BERT token level classifier (Ref: https://huggingface.co/docs/transformers/en/tasks/token_classification ) An RNN or LSTM classifier with some dense embedded features (glove, word2vec etc), and a prediction layer at each time step after the stack of (if multi-layer) RNNs I would suggest you try the 4th one only if you have enough time, otherwise invest more on preparing a good enough custom dataset and work on any of the first 3. More on reddit.com