Add automatic linking to markdown

[lorenzo93]

Hi,

I'm starting to work with your tool to make man-pages for my app.
I've managed to get the automatic linking feature working in the HTML files with index.txt.

I want to publish also the markdown version to GitHub Wiki, is there a way to make the automatic linking working also in markdown output? (--markdown option)

Thanks in advance

[mttbernardini]

Hi @lorenzo93,

I think that, apparently, markdown linking is supposed to work but is not actually working, or at least it's almost there.

I made a simple test using this source:

sample(7) - a sample test
=========================

## DESCRIPTION

This is a sample test with a link to `some-command`(1)

and the following index.txt:

some-command(1)  some-command.1

which produces the following output when compiled using ronn --markdown:

sample(7) - a sample test
=========================

## DESCRIPTION

This is a sample test with a link to `some-command`(1)


[DESCRIPTION]: #DESCRIPTION "DESCRIPTION"


[some-command(1)]: some-command.1.html
[sample(7)]: test.html

As you can see the references are correctly added, however I can see various issues:

1. The link [`some-command`(1)][some-command(1)] is not actually created, contrary to the html version where tokens are automatically detected (even when formatted like in this case).
2. I don't think we are supposed to create the link manually, as it messes with the roff format (it prints the url next to the link, which is not what we want here), and, again, the html version already does this job automatically, so this is most probably a bug.
3. The reference has the wrong extension, it should be some-command.1.md. This is especially true when using markdown pages with GitHub (as you plan to do), because GitHub expects you to link to other markdown pages as it already takes care of the html rendering and there's no actual html url to point to. I would say that adding links to other html pages is actually the job of the html version.

Unfortunately I don't speak Ruby, so I don't know how to help solving these issues. I hope, however, someone among the maintainers will take a look at this and could use my comment as a starting point.

Thanks for the consideration.

[lorenzo93]

Thanks @mttbernardini,
I didn't notice that the work was half done yet.

Unfortunately I don't speak Ruby too, so correcting the behavior could be impossibile for me.

Hoping someone will fix this bug

[apjanke]

I'll take care of it once I have some time to spare!

[apjanke]

Some responses to @mttbernardini's questions:

1. / 2. Nope, you're not supposed to create the link manually. I think this is just a bug in the markdown output module.
2. I think .html actually is the correct extension: even though the target file will be defined as an .md file and automatically converted to HTML by GitHub pages (or whatever other site builder you're using), the final pages are accessed through ...*.html URLs, and GitHub Pages doesn't auto-translate .md link targets into the equivalent .html targets. So the links should reference the .html files that will appear after processing. This is how it works in other Jekyll-based sites I've worked with, anyway.

[mttbernardini]

I think .html actually is the correct extension: even though the target file will be defined as an .md file and automatically converted to HTML by GitHub pages (or whatever other site builder you're using), the final pages are accessed through ...*.html URLs, and GitHub Pages doesn't auto-translate .md link targets into the equivalent .html targets. So the links should reference the .html files that will appear after processing. This is how it works in other Jekyll-based sites I've worked with, anyway.

Actually, I'm referring to standard GitHub repository navigation, not GitHub Pages. I would agree if that were the case, but in a regular GitHub repository, GitHub expects links to point directly to the .md files. In the wikis it doesn't even expect an extension, so I don't know how this should be handled in that case 🤔. For GitHub pages I don't see the point in using the md files instead of using directly the html files generated by ronn.

But anyway, since use cases can really vary, maybe some kind of option/configuration for this is needed? Like an option to specify the extension, or using different types of index.txt which then get indicated by a ronn option?

[apjanke]

Ah, I see. GitHub repo navigation is a special case. Yeah, I think an option will be needed.

Let's think a little bit about what all targets people might be using with the --markdown output format. I can see:

• Wikis that support Markdown. (But there's so many wikis out there; we can't really assume what their link structure is, and I don't want to add support for specific wikis. Maybe this is something we just don't have special support for. Or maybe we add an option that specifies an arbitrary link transformation rule?)
• GitHub repos on github.com or things that act like it, supporting direct linking to the source .md files.
• Static site generators like Jekyll, that expect .html links. But I think you're right: why wouldn't one just use the HTML output?
• Arbitrary Markdown environments we know nothing about.

The GitHub repos could be handled with a --dot-md-links option that tells ronn to use .md link targets instead of .html ones.

An arbitrary transformation rule: that sounds pretty complex, and seems like an obscure use case to me. I'm not even sure how that would work: maybe the user would supply a Ruby function that took a man page specification and returned the relative link path? Maybe we should just not support that until someone actually asks for it with a concrete use case?

[mttbernardini]

I kinda have two possible ideas:

1. The fact that the file must be called index.txt is not written on stone, rather it can be passed to ronn with an option (let's say --index index.txt). This thing, coupled with not doing any processing on the index file (i.e. not adding an extension to links, maybe this could be set by an option too, like --no-index-processing), makes the user free to specify whichever link structure they're using and create different index files for different build types (for example one could have an index for markdown builds and another for html builds). This option gives the user the most freedom, but has a drawback of potential code duplication for simpler uses by having to manually create different indexes for different build types where almost nothing is changed.

2. So that's where the second option comes in: post-process the index file but letting the user choose the extension (which could default to .html for html builds and .markdown for md ones) with an option, like --index-extensions .md. This should handle simple cases like github/gitlab/bitbucket repos (they all behave the same way) and github wikis (by specifying an empty extension). So anytime the transformation rule only consist in having a particular file extension this would apply, otherwise for more complex transformation the job could be left to the user by defining their own indexes as described in (1).

This should cover most use cases without getting too much into the details of each of them. What do you think?

PS:

An arbitrary transformation rule: that sounds pretty complex, and seems like an obscure use case to me. I'm not even sure how that would work: maybe the user would supply a Ruby function that took a man page specification and returned the relative link path? Maybe we should just not support that until someone actually asks for it with a concrete use case?

I agree with this point, I think that (1) could be enough until someone actually asks for a different (possibly better) way of doing this.