SpaCy

I think what they're trying to say is the package is not academic in the way that some of the other ones might be. Instead that spaCy is meant to commonly work means that it is more about getting the job done than being faithful to the paper(s). To me it represents the divide between academia and engineering. If I'm reading it right they're suggesting they are an engineering package, not an academic one. Depending on your goals this could be a boon or bane. I'd suggest if you just want things to work and already know spaCy then just stick with that and don't overthink it. More on reddit.com

I agree with both of the previous commenters in that: I use {spacyr} + {reticulate} for NLP work that involves using spaCy. This is because I like support for Python from RStudio IDE and rmarkdown. It's miles better for R + Python work than its closest neighbor, Jupyter Notebook. But at the same time, even as someone who also mostly uses R, I agree with Python's approach to NLP. Actually, if I'm being more precise, it's not really R vs. Python but more functional vs. object-oriented approaches to NLP. The difference is between working with words as observations representated as rows in dataframes versus working with words as objects that have special attributes. Let me extrapolate a bit more on my point #2. Consider a case where you want to know whether the head of a word is a Verb. In {spacyr}, this means finding the value of the head column at the row of the word, then finding the row of the head word using that value, then finding the value of the POS column at that row. This approach involves a lot of slice() and filter() and subsetting. In Python's spaCy, on the other hand, this just involves getting the attribute head of the word and then getting the attribute of that word. If that word is stored in a variable called my_word, checking whether the head of that word is a Verb is simply my_word.head.pos_ == "VERB" So at least in the case of analyzing depedency relations, the difference is between working with indices of a data frame of words vs. working with pointers to words that are accessible as an attribute of word objects. I favor the latter object-oriented approach in Python because it's faster, less code, more readable, and more intuitive. But that's probably partly because my work involves a lot of analyzing dependency relatios. tl;dr #1: Depends on the task, but object-oriented approach of Python is overall better IMO. But it's not an either-or: you can do all that work within the comfort of the RStudio IDE and your R-centered workflow with {reticulate} and R markdown. Just to be clear though, functional vs. object-oriented approach to NLP is also not really a black-white issue, much like "R vs. Python". In fact, the part that I love the most about the object-oriented NLP workflow in Python is list comprehension and that's about as functional programming as you can get. Working with a list of word objects is IMO the perfect marriage between functional and object-oriented programming as it vastly improves the quality of my work. tl;dr #2: If you're going to learn spaCy (the Python way), learn Python's list comprehension too. More on reddit.com

Hi,

First — I'm sorry to hear that this has been a bad experience. Thanks for taking the time to write this up. There's often a shortage of this sort of challenging feedback, and it's necessary for improvement.

I will say that the versions over the last couple of months have been a bit less stable than we want. This is an awkward transitional period: we're getting spaCy ready for neural networks in 2.0, and fixing a lot of long-standing issues. We're also getting ready to push v1.7.0.

In response to your specific comments:

• spaCy tries to use as much of the same code for training and runtime as possible. There's therefore no plans to have a separate "runtime only" mode.

• What isn't compiling out of the box?

• I've never built a Debian package myself. I think this makes sense, and I should add it to our test service.

• Here's how we're fixing the model download problem: We're introducing thin wrappers around the data assets so that you'll be able to install them as pip packages. You'll therefore be able to serve these however else you're serving pip dependencies. You can also point pip to a location on your file system, run a pip service, etc. But this isn't the only way to get the data installed.

Ultimately spaCy just needs to find the files on the filesystem somewhere. I had imagined that production users would either copy the data inside the spaCy package themselves, or create a package of spaCy that included the data they needed by default. I realise this wasn't clear --- but it's hard to know what a different production environment might need. You can also point spaCy to a location on your file system with the util.set_data_path() command.

• I've been installing v1.6.0 from PyPi regularly, without issue --- so I'm not sure what's different in your setup. I wonder whether one of us could be pulling from a cached version? We do have a CI process which builds an sdist on a server, and then the test installs from there. We plan to keep improving the test infrastructure.

• It's tricky to interpret what semver should mean when the data changes. I think if we bumped major version for something like a change to the stop words, we'd have no way to communicate deeper breaking changes. This is especially true as more languages are added. If we change an Estonian lemmatizer rule, should we increment the version?

Thanks again, Matthew Honnibal

More on reddit.com

From what I understand it was a small cannon piece where the spacey got consolidated into the federation to become the eff but I think it's somewhat been retconned by it's disappearance, I'm slightly speculating though More on reddit.com