misc documentation updates

[mr-c]

This list is a bit old and not all entries may make sense. Moved over from #321

• add a subtitle
• document our documentation conventions
• building on the above: use the Sphinx unix manual page builder to build and ship manual pages for each script
• for each part: list intended audiences and expectations
• merge the following chapters elsewhere or drop them completely: "6. Blog posts and additional documentation", "8. Partitioning large data sets", "9. ktable, simple k-mer counting", "11. Miscellaneous implementation details", "16. Sequence access patterns in khmer"
• remove the incomplete '22. Indices and tables" and "Python Module Index" (for now)
• In "1.1 Introduction" explain what hashing & hash tables for the non-C.S. folk.
• in "1.2 Using khmer" make sure we mention the appropriate script name for every use case listed. They should be auto linked using sphinx magic
• split the install instructions up for user & dev use
• document how to install a specific version and do a system-wide install
• add a section under "Chapter 3. A few examples" describing and linking to the khmer protocols
• review the usefulness of examples/
• determine which scripts can make use of stdin or stdout as input or outputs. Document this and file issues if it doesn't work and it would be useful if it did.
• ensure that every script includes example usage (about half do already)
• document exactly how filter-stoptags trims
• elevate the following sandbox scripts to scripts/ or remove references to them: fasta-to-abundance-hist.py, abundance-hist-by-position.py, hi-lo-abundance-by-position.py, velvet-assemble.sh
• work with @qingpeng to update "Chapter 7: Choosing hash sizes for khmer"
• Possibly remove the "Architecture and Design" chapter as it is out of date. If we keep it then many sections need removing and updating. All TODOs within should be removed and become GitHub issues if they aren't already.
• Acknowledge the in-flux nature of the codebase; if a contributor modifies a file then they are responsible for making the entire file well formatted (using autopep8 / astyle / their hands)
• link to https://github.com/pjotrp/bioinformatics & file issue for outstanding items
• write up motivation for the coding standards, use Software Sustainability Institute articles as they are useful
• move the git/github strategies before the code review section
• talk about the two types of pull requests & workflow including requesting code review and who does the merge
• mention specific git techniques including rebasing to clean up commits
• configure pylint to respect our script naming convention + doc that if needed
• specific exit codes can be 2-255 but their meaning needs to be documented. I don't know how much demand for fine grained exit codes there are, though
• "Chapter 14: Deploying the khmer project tools on Galaxy": we also support filter-abund.py
• fix numbered list in 14.2
• "Chapter 17: How to make a khmer release": add step to scan for TODOs BUGs and FIXMEs that need to be moved to Github
• add version check to each stage of the release check; replace interrogative comments with statements
• document that the setuptools version requirement is enforced by setup.py
• document .gitignore usage http://stackoverflow.com/questions/1753070/git-ignore-files-only-locally

[ACharbonneau]

Can I just add to this list?

• scripts folder README refers me to non-existant file "See ../doc/scripts.txt for basic documentation."

[mr-c]

Yes, thanks! Also happy to hear if any items on the above list no longer make any sense

[standage]

We should review this and see what's still relevant for 2.1.

[betatim]

I will tick things off if they have been resolved or are not relevant anymore.

[standage]

I'm closing this issue for now. The documentation seems to have changed substantially in two years, and I'm having a hard time determining which of these items are still relevant. For those that are, I'm mostly inclined to punt with respect to the 2.1 release.

If anyone has objections, feel free to comment, but I'd suggest starting a dedicated thread for each issue. This thread is essentially unactionable at this point.