Support YAML metadata block in gfm

[abitrolly]

As explained in #2827 (comment) the gfm format specifier doesn't support YAML metadata blocks. This doesn't work currently.

pandoc mwe.md -f gfm+yaml_metadata_block -o mwe.pdf

The workaround for now is to use deprecated markdown_github format specifier.

pandoc mwe.md -f markdown_github+yaml_metadata_block -o mwe.pdf

✗ pandoc -v

pandoc 2.7.3
Compiled with pandoc-types 1.17.6.1, texmath 0.11.3, skylighting 0.8.2.1
Default user data directory: /home/anatoli/.local/share/pandoc or /home/anatoli/.pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

[craigbarnes]

GitHub has had special rendering for YAML metadata in Markdown for a few years now, so it might make sense to support YAML blocks even when just using -f gfm.

[caressofsteel]

I can't think of any good reason not to support this. I wasted an hour or two trying to figure out why the YAML was being spit as part of the PDF before I stumbled upon the workaround.

[tarleb]

I think the merits of this request are undisputed. What's missing is the contributor power to make it happen.

[abitrolly]

@taleb so the task is to find the code for yaml_metadata_block and merge it into gfm to make it the default behavior? Or the task is to fix behavior of compound format specifier gfm+yaml_metadata_block?

[tarleb]

GitHub Flavored Markdown is a variant of CommonMark and pandoc uses the commonmark-hs library for parsing it. The challenge is to add YAML support there. The repo is https://github.com/jgm/commonmark-hs.

[tarleb]

See also jgm/commonmark-hs#17

[jgm]

This is a bit more complicated than the average commonmark extension, because here we need to parse the YAML into a YAML object, and then parse the terminal nodes as commonmark. I have to think about the best way to handle this.

[shiftybit]

In the mean time, is there any way to hide the metadata block from the output? For my purposes, just discarding it is fine.

[jgm]

Your best bet is probably to preprocess the input and strip it off before passing to pandoc.
An alternative approach would be to use pandoc's markdown instead of gfm as your input format.

[shiftybit]

Ended up doing this. --- worked out to return as horizontal rule, so for my needs, I just sort of modified one of the example scripts.

from pandocfilters import toJSONFilter
incomment = False
firstLine = True

def comment(k, v, fmt, meta):
	global incomment
	global firstLine
	if(firstLine and k == 'HorizontalRule'):
		incomment = True
		firstLine = False
		return []
	elif(incomment and k == 'HorizontalRule'):
		incomment = False
		return []

	if incomment:
		return []  # suppress anything in a header

if __name__ == "__main__":
	toJSONFilter(comment)