Videos
Hey all,
Anyone who knows where to find a really big list of all English words which can be downloaded?
Need for NLP (natural language processing)
Thanks in advance
I have an assignment due in a week where we have to create a password file of all known dictionary words with their hash values. Now i know how to hash the words and store it. But i am having trouble in getting a list of all dictionary words.
I was thinking to implement this by having 26 files for words starting with each letter. For eg, A.txt will have all words starting with A, B.txt will have words starting with B. with one word in one line
I want to read a file with these words, run a loop through it and store its hash in a different output file. For password: <hash value for password>
Please help me get this list of words lol.
Is there a way to make a list of all the valid words in the english language, like in the game scrabble or wordle?
The gold standard for natural language processing is Wordnet at http://wordnet.princeton.edu/. This has an active user group, has semantics and syntax associated with words, and interfaces with other NLP tools. If you are thinking of doing computation with the words you should definitely have a look.
However selecting words at random does not generate a useful sentence and I suspect you will be disappointed with the results. Have a look at toolkits such as OpenNLP where there are many tools including part-of-speech (POS) which you will certainly need.
Even when you have sentences that may have valid syntax, you will need to read the work of Chomsky and others. His "Colorless green ideas sleep furiously" http://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_furiously illustrates the problem.
Check for Lorem Ipsum on site http://www.lipsum.com/ for generating "Void text"
There are lot of generators on net http://loremipsum.sourceforge.net/
Reference text: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed consectetur viverra fringilla. Donec at lectus at turpis bibendum placerat. Vivamus non nibh mauris. Nulla metus metus, sollicitudin nec egestas id, fermentum at nisl. Pellentesque at nisl est. In nec sem tellus, ac imperdiet lectus. Pellentesque tortor turpis, sagittis vel facilisis tristique, cursus in tortor. Mauris non neque magna, vel dignissim sem. Suspendisse interdum diam tempus dui mattis molestie. Donec in mauris urna, at vulputate ipsum. Sed sodales venenatis quam non tincidunt.