Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Citations in tables do not render correctly outside of raw markdown and kable #3340

Closed
4 tasks done
andrewheiss opened this issue Nov 16, 2022 · 27 comments
Closed
4 tasks done
Assignees
Labels
bug Something isn't working crossref tables Issues with Tables including the gt integration
Milestone

Comments

@andrewheiss
Copy link

Bug description

There is inconsistent behavior when including citation keys inside tables, but it's a really tricky issue and I'm not sure if it's a Quarto issue or an issue with other table-making packages.

When making a table with regular Markdown or with knitr::kable(), citation keys that are included in table rows are rendered correctly by pandoc.

However, when making a table with kableExtra::kbl() or gt::gt(), pandoc doesn't see the citation keys.

Here's a reproducible example:

---
title: "Citations in tables"
format: html

references:
- type: article-journal
  id: Lovelace1842
  author:
  - family: Lovelace
    given: Augusta Ada
  issued:
    date-parts:
    - - 1842
  title: >-
    Sketch of the analytical engine invented by Charles Babbage, by LF Menabrea, 
    officer of the military engineers, with notes upon the memoir by the translator
  title-short: Molecular structure of nucleic acids
  container-title: Taylor’s Scientific Memoirs
  volume: 3
  page: 666-731
  language: en-GB
---

## Raw Markdown

This works.

| Thing | Citation      |
|-------|---------------|
| 1234  | @Lovelace1842 |


## `knitr::kable()`

This works.

```{r}
library(knitr)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kable()
```


## `kableExtra::kbl()`

This doesn't work.

```{r}
library(kableExtra)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kbl() |>
  kable_styling()
```


## `gt::gt()`

This doesn't work.

```{r}
library(gt)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt()
```

Raw Markdown and knitr::kable() work correctly:

image

kableExtra::kbl() and gt::gt() do not render the citation:

image


This is a known issue in both kableExtra (haozhu233/kableExtra#214 (comment)) and gt (rstudio/gt#112 (comment)), and the existing solution for both packages is to use bookdown's text reference feature to render the citation outside of the table and then automatically include the rendered citation inside the table when knitting, like this:

(ref:lovelace) @Lovelace1842

```{r}
library(kableExtra)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "(ref:lovelace)"
) |>
  kbl() |>
  kable_styling()
```

This doesn't work in Quarto, though. #1959 and #1785 are about adding bookdown-esque text references to Quarto, which would potentially solve this issue.


This is with RStudio 2022.07.2+576 on macOS Monterey 12.6, but it happens across platforms

Checklist

  • Please include a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
  • Please format your issue so it is easier for us to read the bug report.
  • Please document the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
  • Please document the operating system you're running. If on Linux, please provide the specific distribution.
@andrewheiss andrewheiss added the bug Something isn't working label Nov 16, 2022
@andrewheiss
Copy link
Author

Ooh, @debruine's issue/solution in #1710 also works here! This can be fixed by removing the class with unclass(), printing the result with cat(), and including results="asis" in the chunk options

## `kableExtra::kbl()` with `unclass()`

This works!

```{r results="asis"}
library(kableExtra)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kbl() |>
  kable_styling() |>
  unclass() |> cat()
```

image

@cscheid cscheid added crossref tables Issues with Tables including the gt integration labels Nov 16, 2022
@cscheid cscheid self-assigned this Nov 16, 2022
@cscheid cscheid added this to the v1.3 milestone Nov 16, 2022
@MohdAzmiSuliman
Copy link

i was looking for this too. however if i use result = "asis", the rendered html files unable to recognise it as table for cross reference.

---
title: "kable extra citation"
format: html
---

refer @tbl-table

```{r}
#| label: tbl-table
#| tab-cap: "table with citation"
#| results: asis

library(kableExtra)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kbl() |>
  kable_styling() |>
  unclass() |> cat()
```

image

is there any way to caption the table and crossref it?

@cscheid
Copy link
Collaborator

cscheid commented Nov 17, 2022

I think you need tbl-cap instead of tab-cap?

@andrewheiss
Copy link
Author

Using tbl-cap adds the caption to the table itself, but it is still not cross-referencable:

See @tbl-table

```{r}
#| echo: fenced
#| label: tbl-table
#| tbl-cap: Table with citation
#| results: asis

library(kableExtra)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kbl() |>
  kable_styling() |>
  unclass() |> cat()
```

image

@cscheid
Copy link
Collaborator

cscheid commented Feb 8, 2023

This is now fixed, although kableExtra will need to use a slightly different method (it's currently emitting plain HTML without enclosing it in a RawBlock, which is not HTML syntax that we officially support).

For a preview of how this is going to work in the future, consider this snippet using an in-development branch of gt:

```{r}
library(gt)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt() |> fmt_markdown()
```

If you tell gt that you're providing it with markdown input (through fmt_markdown, which can also be scoped to cells, columns, rows, etc), then you get this:

image

If you don't call fmt_markdown(), you get this:

image

@cderv
Copy link
Collaborator

cderv commented Feb 9, 2023

This is now fixed, although kableExtra will need to use a slightly different method (it's currently emitting plain HTML without enclosing it in a RawBlock, which is not HTML syntax that we officially support).

@cscheid about this, Aren't we enclosing ourself the result in Raw Block ? I believe we do it here

What more would be needed ? I think we need to make sure what is planned for gt will work for other framework.

Also, some comment on why in R Markdown ecosystem the raw block is not used. This is because that way it can benefit with Pandoc from the markdown_in_html_blocks extension.
With Quarto we loose this feature.

This means that with rmarkdown,

Rmd file
---
title: "Citations in tables"
output: 
  bookdown::html_document2:
    keep_md: true
references:
- type: article-journal
  id: Lovelace1842
  author:
  - family: Lovelace
    given: Augusta Ada
  issued:
    date-parts:
    - - 1842
  title: >-
    Sketch of the analytical engine invented by Charles Babbage, by LF Menabrea, 
    officer of the military engineers, with notes upon the memoir by the translator
  title-short: Molecular structure of nucleic acids
  container-title: Taylor’s Scientific Memoirs
  volume: 3
  page: 666-731
  language: en-GB
---

## Raw Markdown

This works.

| Thing | Citation      |
|-------|---------------|
| 1234  | @Lovelace1842 |


## `knitr::kable()`

This works.

```{r}
library(knitr)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kable()
```


## `kableExtra::kbl()`

This doesn't work.

```{r}
library(kableExtra)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kbl() |>
  kable_styling()
```


## `gt::gt()`

This doesn't work.

```{r}
library(gt)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt()
```
  • We get the reference in kable() because it outputs markdown
  • We get the reference in kableExtra() because the inside markdown is process thanks to markdown_in_html_blocks
  • We don't get it working in gt because it outputs its code in inside raw block

Putting that here in case R Markdown user are wondering why the difference.

@cscheid cscheid modified the milestones: v1.3, v1.4 Feb 24, 2023
@andrewheiss
Copy link
Author

@cscheid Do you know which branch of gt has support for the preview version here (#3340 (comment))? Using the gt() > fmt_markdown() approach with today's GitHub version still leads to non-working output:

image

@cscheid
Copy link
Collaborator

cscheid commented Mar 6, 2023

@rich-iannone ?

@cscheid cscheid mentioned this issue May 23, 2023
62 tasks
@jameshowison
Copy link

jameshowison commented Aug 29, 2023

Anyone know if this should be working now? I don't think it is for me.

---
title: "trial"
format: html
---

## Quarto {#sec-testing}

Quarto enables you to weave together content and executable code into a finished document. To learn more about Quarto see <https://quarto.org>.

Inside a table cross-refs don't render:

```{r}
library(gt)
tibble::tribble(
  ~Thing, ~Link,
  1234, "@sec-testing"
) |>
  gt() |> fmt_markdown()
```

But outside a table it works fine @sec-testing

Produces this output:

Screen Shot 2023-08-29 at 1 43 11 PM

I'm expecting a Linked "Section 1" to appear inside the table? This also doesn't work with kable for me either.

Current versions:
gt (0.9.0)
quarto (cmdline) 1.3.242

@cscheid
Copy link
Collaborator

cscheid commented Aug 29, 2023

Actually, now that I say that, I wonder. @rich-iannone, shouldn't that fmt_markdown be wrapping the markdown content in a <span data-qmd="..."></span> element? It doesn't seem to be right now. This is the HTML being emitted:

<table class="gt_table" data-quarto-disable-processing="false" data-quarto-bootstrap="false">
  <thead>
    
    <tr class="gt_col_headings">
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="Thing">Thing</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" scope="col" id="Link">Link</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td headers="Thing" class="gt_row gt_right"><div class='gt_from_md'><p>1234</p>
</div></td>
<td headers="Link" class="gt_row gt_left"><div class='gt_from_md'><p>@sec-testing</p>
</div></td></tr>
  </tbody>
  
  
</table>

@cscheid
Copy link
Collaborator

cscheid commented Aug 29, 2023

We do have a second issue, though, which is that gt emits data-quarto-disable-processing="false", and quarto checks for

              if table.attributes[constants.kDisableProcessing] ~= nil then

This one is a quarto bug and I'll fix that one. In my testing, if <div class='gt_from_md'><p>...</p></div> is replaced with <span data-qmd="..."></span>, then quarto (after this bugfix) actually renders this crossref correctly.

@jameshowison : you'll need to install the latest 1.4 prerelease (and wait for the confirmation and fix of the gt issue) but this should all be working very soon. Thanks for the patience!

@rich-iannone
Copy link
Collaborator

Okay, there was a change required in gt to ensure that Quarto is detected during the render. I've made that change just now as rstudio/gt@e1feb6b and the interface mentioned above by Carlos now works. You'll need to get the latest dev version of gt with devtools::install_github("rstudio/gt").

I tested with this .qmd:

---
format: html
references:
- type: article-journal
  id: Lovelace1842
  author:
  - family: Lovelace
    given: Augusta Ada
  issued:
    date-parts:
    - - 1842
  title: >-
    Sketch of the analytical engine invented by Charles Babbage, by LF Menabrea, 
    officer of the military engineers, with notes upon the memoir by the translator
  title-short: Molecular structure of nucleic acids
  container-title: Taylor’s Scientific Memoirs
  volume: 3
  page: 666-731
  language: en-GB
---

```{r}
library(gt)

tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt() |> fmt_markdown()
```

and got this document, where hovering on the formatted citation yields the full reference.

quarto-gt-citation-in-table

Really sorry it took so long to discover the underlying gt issue. Try it out and let me know if it doesn't work on your system.

@cscheid
Copy link
Collaborator

cscheid commented Sep 20, 2023

I can confirm this works on main(and the version of gt on github) as well. kableExtra will need to emit its output slightly differently, so I'm going to go ahead and close this one.

@cderv
Copy link
Collaborator

cderv commented Sep 20, 2023

kableExtra will need to emit its output slightly differently,

I believe this could be in knitr directly that this should be done. So we could try sync an update with 1.4 release for this. I opened an issue there.

@andrewheiss
Copy link
Author

This issue persists when rendering to PDF (using gt 0.10.0 and Quarto 1.4.424):

---
title: "Citations in tables"
format: pdf

references:
- type: article-journal
  id: Lovelace1842
  author:
  - family: Lovelace
    given: Augusta Ada
  issued:
    date-parts:
    - - 1842
  title: >-
    Sketch of the analytical engine invented by Charles Babbage, by LF Menabrea, 
    officer of the military engineers, with notes upon the memoir by the translator
  title-short: Sketch of the analytical engine
  container-title: Taylor’s Scientific Memoirs
  volume: 3
  page: 666-731
  language: en-GB
---

```{r}
library(gt)

tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt() |> fmt_markdown()
```
image

@andrewheiss
Copy link
Author

andrewheiss commented Oct 31, 2023

knitr::kable() works with PDF:

```{r}
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  knitr::kable()
```
image

 

kableExtra::kbl() doesn't work with PDF:

```{r}
library(kableExtra)

tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  kbl() |> kable_styling()
```
image

 

That might be related to yihui/knitr#2289, though that issue is specifically about HTML output

@cscheid
Copy link
Collaborator

cscheid commented Oct 31, 2023

```{r}
library(gt)

tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt() |> fmt_markdown()
```

Somehow, gt is not formatting this with markdown, but is emitting LaTeX, cf keep-md: true

::: {.cell}

```{.r .cell-code}
library(gt)

tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt() |> fmt_markdown()
```

::: {.cell-output-display}
\begin{longtable}{rl}
\toprule
Thing & Citation \\ 
\midrule\addlinespace[2.5pt]
1234 & @Lovelace1842 \\ 
\bottomrule
\end{longtable}

:::
:::

@rich-iannone, do you know what's going on here?

@cderv
Copy link
Collaborator

cderv commented Oct 31, 2023

The document is using format: pdf here, this means that PDF output is expected. gt will then output as_latex() because this is used in a format: pdf rendering.

I don't think we have yet an equivalent of data-qmd trick on Span that works with LaTeX tables. (probably because we don't parse tables).

If you want HTML tables, even in format: pdf document, because you known Quarto will parse the HTML table anyway, you can ask for it

```{r}
library(gt)

tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842"
) |>
  gt() |> fmt_markdown() |> as_raw_html()
```

This will correctly process the citation in the table, even for PDF outputs.

So at the end, this is a gt choice: when inside quarto document rendering, if output format is LaTeX, should it generate a HTML table anyway ? At least when fmt_markdown() is used maybe ?

Otherwise, it will be to the user to explicitly ask for HTML version of the table.

@cscheid
Copy link
Collaborator

cscheid commented Oct 31, 2023

I don't think we have yet an equivalent of data-qmd trick on Span that works with LaTeX tables. (probably because we don't parse tables).

But if you provide an HTML quarto-style table in LaTeX, we will do the right thing. That was the whole point of doing a uniform parser for tables in quarto. See:

---
format: pdf
---

```{r}
#| label: fig-1
#| fig-cap: A caption
plot(1:100)
```

```{r}
#| label: fig-2
#| fig-cap: Another caption
plot(20:30)
```


```{=html}
<table>
<thead>
<tr><th>What</th><th>Where</th></tr>
</thead>
<tbody><tr>
<td><span data-qmd="@fig-1"></span></td>
<td><span data-qmd="@fig-2"></span></td>
</tr></tbody>
</table>
```
image
image

@cscheid
Copy link
Collaborator

cscheid commented Oct 31, 2023

And, more specifically, I thought that gt had been changed to have a way to send HTML tables to quarto, even in non-HTML formats.

@andrewheiss
Copy link
Author

Ok cool, yeah, I also thought that {gt} had changed for non-HTML formats and emitted HTML. Adding as_raw_html() works for PDF as expected, but leaving it on messes with regular HTML output, so some conditional work has to be done to not use it with HTML

Normal HTML output:

image

 

With as_raw_html() so that LaTeX is happy, but now the HTML output is messed up because something changed with the gt CSS or something:

image

But that's definitely a {gt} issue, not a Quarto issue

@cderv
Copy link
Collaborator

cderv commented Oct 31, 2023

But if you provide an HTML quarto-style table in LaTeX, we will do the right thing. That was the whole point of doing a uniform parser for tables in quarto.

yeah I know that. We all agree here ! The code I shared was about specifically outputting HTML in the PDF format for it to work.

And, more specifically, I thought that gt had been changed to have a way to send HTML tables to quarto, even in non-HTML formats.

I don't think it has. @rich-iannone is this a plan to output HTML table for LaTeX document when rendered in Quarto ?

Only drawback of this approach is possibly loosing all the custom LaTeX styling for gt table if any, that can't be set through Quarto/Pandoc writing the raw LaTeX instead of gt directly.

The example share by @andrewheiss is a hint about this I believe. Possibly some quarto improvement on this not gt as this is Quarto/Pandoc that is generating the raw LaTeX inserted, and no more gt.

@rich-iannone I let you open a ticket on your repo to track potential improvement and more testing.

@andrewheiss
Copy link
Author

Yeah, using as_raw_html() strips away all custom LaTeX stuff like column widths when outputting to HTML

With HTML:

image

With LaTeX:

image

@rich-iannone
Copy link
Collaborator

rich-iannone commented Nov 2, 2023

@andrewheiss me and Carlos discussed this and we have a good plan in place that will preserve the LaTeX table code that gt generates while also allowing Quarto to properly handle citations as well as performing the Markdown conversion. More to come.

@cscheid
Copy link
Collaborator

cscheid commented Nov 2, 2023

@rich-iannone the PR on the quarto side is here #7451

@andrewheiss
Copy link
Author

Following up on this (and related to #9342), @rich-iannone is there a way to use \QuartoMarkdownBase64{} from #7451 in {gt}?

For example, this qmd has an equation and a citation:

---
title: "Citations in tables"
format: 
  html: default
  pdf: 
    keep-tex: true

references:
- type: article-journal
  id: Lovelace1842
  author:
  - family: Lovelace
    given: Augusta Ada
  issued:
    date-parts:
    - - 1842
  title: >-
    Sketch of the analytical engine invented by Charles Babbage, by LF Menabrea, 
    officer of the military engineers, with notes upon the memoir by the translator
  title-short: Molecular structure of nucleic acids
  container-title: Taylor’s Scientific Memoirs
  volume: 3
  page: 666-731
  language: en-GB
---

$$
a^2 + b^2 = c^2
$${#eq-math}

```{r}
library(gt)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "@Lovelace1842",
  5678, "@eq-math"
) |>
  gt() |> 
  fmt_markdown(Citation)
```

When rendering to PDF, those @ things don't get processed, but doing it manually with LaTex and \QuartoMarkdownBase64{} works (kind of; in #9342 this doesn't work with citation keys yet)

```{=latex}
% QExvdmVsYWNlMTg0Mg==} is "@Lovelace1842" in base-64 encoding
% QGVxLW1hdGg= is "@eq-math" in base-64 encoding
\begin{tabular}{cc}
Thing & Citation \\
1234 & \QuartoMarkdownBase64{QExvdmVsYWNlMTg0Mg==} \\
5678 & \QuartoMarkdownBase64{QGVxLW1hdGg=} \\
\end{tabular}
```

I tried adding the base64 content myself in gt():

tibble::tribble(
  ~Thing, ~Citation,
  1234, "\\QuartoMarkdownBase64{QGVxLW1hdGg=}",
  5678, "\\QuartoMarkdownBase64{QExvdmVsYWNlMTg0Mg==}"
) |>
  gt()

…but {gt} automatically converts \\ to \textbackslash{}, so we end up with this LaTeX:

\begin{longtable*}{rl}
\toprule
Thing & Citation \\ 
\midrule\addlinespace[2.5pt]
1234 & \textbackslash{}QuartoMarkdownBase64\{QExvdmVsYWNlMTg0Mg==\} \\ 
5678 & \textbackslash{}QuartoMarkdownBase64\{QGVxLW1hdGg=\} \\ 
\bottomrule
\end{longtable*}

and this PDF:

image

Is there a way to get {gt} to emit \QuartoMarkdownBase64{} without automatically escaping the LaTeX?

@cderv
Copy link
Collaborator

cderv commented Apr 11, 2024

@andrewheiss I believe this is somewhat related to this issue I opened in GT for the case for HTML

But same workaround applies

library(gt)
tibble::tribble(
  ~Thing, ~Citation,
  1234, "\\QuartoMarkdownBase64{QGVxLW1hdGg=}",
  5678, "\\QuartoMarkdownBase64{QExvdmVsYWNlMTg0Mg==}"
) |>
  gt() |> 
  fmt_passthrough(
    columns = Citation,
    escape = FALSE
  )

Using fmt_passthrough with escape = FALSE should help here.

It should write this table

\begin{longtable}{rl}
\toprule
Thing & Citation \\ 
\midrule\addlinespace[2.5pt]
1234 & \QuartoMarkdownBase64{QGVxLW1hdGg=} \\ 
5678 & \QuartoMarkdownBase64{QExvdmVsYWNlMTg0Mg==} \\ 
\bottomrule
\end{longtable}

Hope it helps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working crossref tables Issues with Tables including the gt integration
Projects
None yet
Development

No branches or pull requests

6 participants