pdfplumber has yielded the best results in my testing. You can modify the default table extraction settings to extract "columns" (assuming the default doesn't already detect them). https://github.com/jsvine/pdfplumber#table-extraction-methods You can also check of they are detected by .rects The Visual Debugging can be helpful in actually seeing what the settings currently matching. https://github.com/jsvine/pdfplumber#visual-debugging There are also several threads in the github discussions showing examples of customized settings which may be useful to read through. https://github.com/jsvine/pdfplumber/discussions More on reddit.com

the side panel is definitely not a DeX feature More on reddit.com

The platform I work on most of the time has to generate PDFs with varying degrees of accessibility. This does a good job of starting to scratch the surface of the pains of reading the PDF format for extraction or anything. We try super hard (last year almost 1/3rd of all our dev effort went to this, maybe more) on the PDF authoring side, please do believe me. At least for any PDFs generated "at scale". I am going to ignore more-or-less "one offs" by office workers using "word to PDF" or such things and touching up from there. The main problem is that PDF was never really designed as a data input format, but rather, it was designed as an output format giving fine grained control over the resulting document. This is exactly the root of the problem, compound it with being a format that grew out of PostScript and other 80s tech, growing crazy (now having embedded animations, scripting and 3d models and more!) things along the way. In particular, text data isn’t stored as paragraphs - or even words - but as characters which are painted at certain locations on the page. For added insanity beyond the depth of this article, note this is applying some hand-wavyium. It is exceedingly common to also see text rendered instead of with fonts but raw "draw/stroke" paths (think SVG stuff). This path-map-per letter might fit a definition of a "font" (or font subset) but is at this point lacking actual labeling on "this is letter a" and instead is just a random pointer into the path-map. Basically think/duplicate the whole section on "PDF Fonts" with "but what if it was bare stroke macros/instructions/scripts with zero metadata left over". This is mostly for when exceedingly pickyness on styling/kerning is being asked for, or when the PDF writing/generating library struggles with multi-language stuff. For example our PDF writing basically can't handle mixed right-to-left/left-to-right text in the same tags. Instead we just distill down to raw paths for most non-english text. We still tag with the misc accessibility PDF tag standards the original unicode so we aren't pure evil, just stuck in impossible situation of PDFs are complex monsters. TL;DR: asking a computer to "just read a PDF and extract the words/etc" is even harder than the article says, its just the tip. Falling back on OCR with human oversight/checking is generally far easier. More on reddit.com

Hello sirius_c, first of all welcome to python! I hope I understood your problem correctly, so I googled around a bit. I dont quite understand the need to convert the pdf to xml, can you explain abit further? If you only need plaintext, you can use this: http://stackoverflow.com/a/26495057 The amazing thing with Python is, you can hack together what you need and you dont need to learn a whole package to do what you want. Thats what I love in python. So, to help abit, here a code that transforms 1 PDF to text and checks the sentences in it for a certain word, hope you can customize it to your needs (it also extracts the metadata): from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter from pdfminer.converter import TextConverter from pdfminer.layout import LAParams from pdfminer.pdfpage import PDFPage from io import StringIO from pdfminer.pdfparser import PDFParser from pdfminer.pdfdocument import PDFDocument from nltk import ngrams def convert_pdf_to_txt_and_metadata(path): rsrcmgr = PDFResourceManager() retstr = StringIO() codec = 'utf-8' laparams = LAParams() device = TextConverter(rsrcmgr, retstr, codec=codec, laparams=laparams) fp = open(path, 'rb') metadata=PDFDocument(PDFParser(fp)).info interpreter = PDFPageInterpreter(rsrcmgr, device) password = "" maxpages = 0 caching = True pagenos=set() for page in PDFPage.get_pages(fp, pagenos, maxpages=maxpages, password=password,caching=caching, check_extractable=True): interpreter.process_page(page) text = retstr.getvalue() fp.close() device.close() retstr.close() return text, metadata pdf, metadata=convert_pdf_to_txt_and_metadata("NAACL2013Regularities.pdf" ) def findWord(word, pdftext): for sentence in pdftext.split("."): if word in sentence: print(sentence.replace("\n", " ")) findWord("reasoning", pdf) print(metadata) Output is then: This allows vector-oriented reasoning based on the offsets between words [{'PTEX.Fullbanner': b'This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4) kpathsea version 3.5.4', 'Producer': b'pdfeTeX-1.21a', 'Creator': b'TeX', 'CreationDate': b"D:20130404113049-07'00'"}] More on reddit.com