The other answers are great. But I thought I (the OP) ought to share what I do these days (a year or two after the question).
I use Sphinx and its Markdown extension. Do the following:
TL;DR: See Gist snippet.
Sphinx-markdown-builder
You need sphinx-markdown-builder python module.
pip install sphinx sphinx-markdown-builder;
Run Sphinx
Not the autodoc, the apidoc!
sphinx-apidoc -o Sphinx-docs . sphinx-apidoc --full -A 'Matteo Ferla'; cd Sphinx-docs;
Configuration
Fix the conf.py file, by following the following or just lazily copy paste the echo command below.
Manual
First uncomment the lines. These are otherwise commented out.
import os
import sys
sys.path.insert(0, os.path.abspath('../'))
Note the change to ../
One weirdness is that the magic methods get ignored. To override this, add this anywhere:
def skip(app, what, name, obj, would_skip, options):
if name in ( '__init__',):
return False
return would_skip
def setup(app):
app.connect('autodoc-skip-member', skip)
A thing to note: The docstrings ought to be written in restructuredtext (RST). If they are in Markdown, you need to add a mod - see this. The two are similar, but different. For example, a single backquote is required for <code> in Markdown, while two are for RST. If in doubt, several blog posts discuss the merits of RST documentation over Markdown.
Typehinting
RST typehints (:type variable: List) are obsolete as proper typehinting def foo(variable: Optional[List[int]]=None) -> Dict[str,int]: has been introduced since 3.6. To make these work:
pip install sphinx-autodoc-typehints
And add 'sphinx_autodoc_typehints' at the end of the extensions list. Note the package has hyphens while the module has underscores.
TL;DR
Copy paste this:
echo " import os
import sys
sys.path.insert(0,os.path.abspath('../'))
def skip(app, what, name, obj,would_skip, options):
if name in ( '__init__',):
return False
return would_skip
def setup(app):
app.connect('autodoc-skip-member', skip)
extensions.append('sphinx_autodoc_typehints')
" >> conf.py;
Showtime
Then it is showtime.
make markdown;
Copy the files and clean however you fancy.
mv _build/markdown/* ../; rm -r Sphinx-docs;
Repeat Apidoc for new files
It should be noted that when new files are added, the apidoc command needs to be repeated. Nevertheless, I highly recommend generating documentation midway as I often realise I am doing something wrong when I see the docs.
But briefly, apidoc will add for each file a automodule command, so this could be added manually or even expanded:
.. automodule:: my_module
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
There's also the commands autoclass, autofunction, autoexception, for specific cases. In the case of autoclass if the class inherits many base classes in separate files to rightfully keep filesizes under 250 lines, the property :inherited-members: is a nice addition to this —thus avoiding having to describe the private base classes.
Read the docs: the common way
It should be said that there's a trend to not have documentation in GitHub but in Read the docs. My guess is because:
- avoids this docstrings-to-markdown business
- some users get confused by GitHub
- looks nicer
- other do it
Despite this, it requires some set up due to the module requirements. In another SO post is a long list of pitfalls and tricks —briefly IMO users, such as myself, make three mistakes:
- missing modules or the target module
- forget to hard refresh the browser
- enabling the
sphinx.ext.autodocextension
However, if one has written markdown documentation in GitHub these can be imported too. Formerly, the m2r2 (a fix of the deprecated m2r) was a good solution, but the divergence of its dependency mistune, which would require it to be frozen at version 0.8.4 as opposed to being at 2.0.0, which breaks other sphinx modules, therefore a new split works best and better: sphinx-mdinclude.
This is pip installed as sphinx-mdinclude but included as sphinx_mdinclude and allows md files to be read alongside rst files.
So a simple workaround in the docs/source/config.py file is to copy the files from the project root to the folder of config.py One issue is that links may need to be checked, especially if files moved around or are relative to the base URL (slash prefixed), eg. Foo.
The other answers are great. But I thought I (the OP) ought to share what I do these days (a year or two after the question).
I use Sphinx and its Markdown extension. Do the following:
TL;DR: See Gist snippet.
Sphinx-markdown-builder
You need sphinx-markdown-builder python module.
pip install sphinx sphinx-markdown-builder;
Run Sphinx
Not the autodoc, the apidoc!
sphinx-apidoc -o Sphinx-docs . sphinx-apidoc --full -A 'Matteo Ferla'; cd Sphinx-docs;
Configuration
Fix the conf.py file, by following the following or just lazily copy paste the echo command below.
Manual
First uncomment the lines. These are otherwise commented out.
import os
import sys
sys.path.insert(0, os.path.abspath('../'))
Note the change to ../
One weirdness is that the magic methods get ignored. To override this, add this anywhere:
def skip(app, what, name, obj, would_skip, options):
if name in ( '__init__',):
return False
return would_skip
def setup(app):
app.connect('autodoc-skip-member', skip)
A thing to note: The docstrings ought to be written in restructuredtext (RST). If they are in Markdown, you need to add a mod - see this. The two are similar, but different. For example, a single backquote is required for <code> in Markdown, while two are for RST. If in doubt, several blog posts discuss the merits of RST documentation over Markdown.
Typehinting
RST typehints (:type variable: List) are obsolete as proper typehinting def foo(variable: Optional[List[int]]=None) -> Dict[str,int]: has been introduced since 3.6. To make these work:
pip install sphinx-autodoc-typehints
And add 'sphinx_autodoc_typehints' at the end of the extensions list. Note the package has hyphens while the module has underscores.
TL;DR
Copy paste this:
echo " import os
import sys
sys.path.insert(0,os.path.abspath('../'))
def skip(app, what, name, obj,would_skip, options):
if name in ( '__init__',):
return False
return would_skip
def setup(app):
app.connect('autodoc-skip-member', skip)
extensions.append('sphinx_autodoc_typehints')
" >> conf.py;
Showtime
Then it is showtime.
make markdown;
Copy the files and clean however you fancy.
mv _build/markdown/* ../; rm -r Sphinx-docs;
Repeat Apidoc for new files
It should be noted that when new files are added, the apidoc command needs to be repeated. Nevertheless, I highly recommend generating documentation midway as I often realise I am doing something wrong when I see the docs.
But briefly, apidoc will add for each file a automodule command, so this could be added manually or even expanded:
.. automodule:: my_module
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
There's also the commands autoclass, autofunction, autoexception, for specific cases. In the case of autoclass if the class inherits many base classes in separate files to rightfully keep filesizes under 250 lines, the property :inherited-members: is a nice addition to this —thus avoiding having to describe the private base classes.
Read the docs: the common way
It should be said that there's a trend to not have documentation in GitHub but in Read the docs. My guess is because:
- avoids this docstrings-to-markdown business
- some users get confused by GitHub
- looks nicer
- other do it
Despite this, it requires some set up due to the module requirements. In another SO post is a long list of pitfalls and tricks —briefly IMO users, such as myself, make three mistakes:
- missing modules or the target module
- forget to hard refresh the browser
- enabling the
sphinx.ext.autodocextension
However, if one has written markdown documentation in GitHub these can be imported too. Formerly, the m2r2 (a fix of the deprecated m2r) was a good solution, but the divergence of its dependency mistune, which would require it to be frozen at version 0.8.4 as opposed to being at 2.0.0, which breaks other sphinx modules, therefore a new split works best and better: sphinx-mdinclude.
This is pip installed as sphinx-mdinclude but included as sphinx_mdinclude and allows md files to be read alongside rst files.
So a simple workaround in the docs/source/config.py file is to copy the files from the project root to the folder of config.py One issue is that links may need to be checked, especially if files moved around or are relative to the base URL (slash prefixed), eg. Foo.
I've found pydoc-markdown quite easy to use. The first command will install the library and the second one will create a README from your module named MY_MODULE:
pip install pydoc-markdown
pydoc-markdown -m MY_MODULE -I $(pwd) > README.md
» pip install python-docstring-markdown
» pip install docstring-to-markdown
Sphinx's Autodoc extension emits an event named autodoc-process-docstring every time it processes a doc-string. We can hook into that mechanism to convert the syntax from Markdown to reStructuredText.
Unfortunately, Recommonmark does not expose a Markdown-to-reST converter. It maps the parsed Markdown directly to a Docutils object, i.e., the same representation that Sphinx itself creates internally from reStructuredText.
Instead, I use Commonmark for the conversion in my projects. Because it's fast — much faster than Pandoc, for example. Speed is important as the conversion happens on the fly and handles each doc-string individually. Other than that, any Markdown-to-reST converter would do. M2R2 would be a third example. The downside of any of these is that they do not support Recommonmark's syntax extensions, such as cross-references to other parts of the documentation. Just the basic Markdown.
To plug in the Commonmark doc-string converter, make sure that package is installed (pip install commonmark) and add the following to Sphinx's configuration file conf.py:
import commonmark
def docstring(app, what, name, obj, options, lines):
md = '\n'.join(lines)
ast = commonmark.Parser().parse(md)
rst = commonmark.ReStructuredTextRenderer().render(ast)
lines.clear()
lines += rst.splitlines()
def setup(app):
app.connect('autodoc-process-docstring', docstring)
Meanwhile, Recommonmark was deprecated in May 2021. The Sphinx extension MyST, a more feature-rich Markdown parser, is the replacement recommended by Sphinx and by Read-the-Docs. With MyST, one could use the same "hack" as above to get limited Markdown support. Though in February 2023, the extension Sphinx-Autodoc2 was published, which promises full (MyST-flavored) Markdown support in doc-strings, including cross-references.
A possible alternative to the approach outlined here is using MkDocs with the MkDocStrings plug-in, which would eliminate Sphinx and reStructuredText entirely from the process.
I had to extend the accepted answer by john-hen to allow multi-line descriptions of Args: entries to be considered a single parameter:
def docstring(app, what, name, obj, options, lines):
wrapped = []
literal = False
for line in lines:
if line.strip().startswith(r'```'):
literal = not literal
if not literal:
line = ' '.join(x.rstrip() for x in line.split('\n'))
indent = len(line) - len(line.lstrip())
if indent and not literal:
wrapped.append(' ' + line.lstrip())
else:
wrapped.append('\n' + line.strip())
ast = commonmark.Parser().parse(''.join(wrapped))
rst = commonmark.ReStructuredTextRenderer().render(ast)
lines.clear()
lines += rst.splitlines()
def setup(app):
app.connect('autodoc-process-docstring', docstring)