Error `Package soul Error: Reconstruction failed.` with PDF vignette

[grimbough]

Hi,

In my case, I ran into another error using the files from https://github.com/LieberInstitute/spatialLIBD/tree/RELEASE_3_14.

Rscript -e 'rmarkdown::render("spatialLIBD/vignettes/TenX_data_download.Rmd", output_format = "BiocStyle::pdf_document", output_file = "AdditionalFile1.pdf")'

output file: TenX_data_download.knit.md

/usr/local/bin/pandoc +RTS -K512m -RTS TenX_data_download.knit.md --to latex --from markdown+autolink_bare_uris+tex_math_single_backslash --output AdditionalFile1.tex --lua-filter /Library/Frameworks/R.framework/Versions/4.1/Resources/library/bookdown/rmarkdown/lua/custom-environment.lua --lua-filter /Library/Frameworks/R.framework/Versions/4.1/Resources/library/rmarkdown/rmarkdown/lua/pagebreak.lua --lua-filter /Library/Frameworks/R.framework/Versions/4.1/Resources/library/rmarkdown/rmarkdown/lua/latex-div.lua --self-contained --table-of-contents --toc-depth 2 --number-sections --highlight-style tango --pdf-engine pdflatex --include-in-header /var/folders/cx/n9s558kx6fb7jf5z_pgszgb80000gn/T//RtmpQsGD98/1954357f612e.tex --variable graphics --wrap preserve --variable subparagraph --template /var/folders/cx/n9s558kx6fb7jf5z_pgszgb80000gn/T//RtmpQsGD98/BiocStyle/template.tex --variable tables=yes --standalone --variable 'compact-title:yes'
! Package soul Error: Reconstruction failed.

Error: LaTeX failed to compile AdditionalFile1.tex. See https://yihui.org/tinytex/r/#debugging for debugging tips. See AdditionalFile1.log for more info.
Execution halted

Failed outputs are at: Archive.zip

I don't get the error with:

rmarkdown::render("spatialLIBD/vignettes/TenX_data_download.Rmd", output_format = "rmarkdown::pdf_document", output_file = "AdditionalFile1.pdf")'

AdditionalFile1.pdf

R session info

> library("BiocStyle")
> library("spatialLIBD")
> options(width = 120)
> sessioninfo::session_info()
─ Session info ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.1.2 (2021-11-01)
 os       macOS Big Sur 10.16
 system   x86_64, darwin17.0
 ui       X11
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       America/New_York
 date     2021-12-17
 pandoc   2.16.2 @ /usr/local/bin/ (via rmarkdown)

─ Packages ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 package                * version    date (UTC) lib source
 AnnotationDbi            1.56.2     2021-11-09 [1] Bioconductor
 AnnotationHub            3.2.0      2021-10-26 [1] Bioconductor
 assertthat               0.2.1      2019-03-21 [1] CRAN (R 4.1.0)
 attempt                  0.3.1      2020-05-03 [1] CRAN (R 4.1.0)
 beachmat                 2.10.0     2021-10-26 [1] Bioconductor
 beeswarm                 0.4.0      2021-06-01 [1] CRAN (R 4.1.0)
 benchmarkme              1.0.7      2021-03-21 [1] CRAN (R 4.1.0)
 benchmarkmeData          1.0.4      2020-04-23 [1] CRAN (R 4.1.0)
 Biobase                * 2.54.0     2021-10-26 [1] Bioconductor
 BiocFileCache            2.2.0      2021-10-26 [1] Bioconductor
 BiocGenerics           * 0.40.0     2021-10-26 [1] Bioconductor
 BiocIO                   1.4.0      2021-10-26 [1] Bioconductor
 BiocManager              1.30.16    2021-06-15 [1] CRAN (R 4.1.0)
 BiocNeighbors            1.12.0     2021-10-26 [1] Bioconductor
 BiocParallel             1.28.3     2021-12-09 [1] Bioconductor
 BiocSingular             1.10.0     2021-10-26 [1] Bioconductor
 BiocStyle              * 2.23.1     2021-12-17 [1] Github (Bioconductor/BiocStyle@932ae2e)
 BiocVersion              3.14.0     2021-05-19 [1] Bioconductor
 Biostrings               2.62.0     2021-10-26 [1] Bioconductor
 bit                      4.0.4      2020-08-04 [1] CRAN (R 4.1.0)
 bit64                    4.0.5      2020-08-30 [1] CRAN (R 4.1.0)
 bitops                   1.0-7      2021-04-24 [1] CRAN (R 4.1.0)
 blob                     1.2.2      2021-07-23 [1] CRAN (R 4.1.0)
 bslib                    0.3.1      2021-10-06 [1] CRAN (R 4.1.0)
 cachem                   1.0.6      2021-08-19 [1] CRAN (R 4.1.0)
 callr                    3.7.0      2021-04-20 [1] CRAN (R 4.1.0)
 cli                      3.1.0      2021-10-27 [1] CRAN (R 4.1.0)
 codetools                0.2-18     2020-11-04 [1] CRAN (R 4.1.2)
 colorout                 1.2-2      2021-11-23 [1] Github (jalvesaq/colorout@79931fd)
 colorspace               2.0-2      2021-06-24 [1] CRAN (R 4.1.0)
 config                   0.3.1      2020-12-17 [1] CRAN (R 4.1.0)
 cowplot                  1.1.1      2020-12-30 [1] CRAN (R 4.1.0)
 crayon                   1.4.2      2021-10-29 [1] CRAN (R 4.1.0)
 curl                     4.3.2      2021-06-23 [1] CRAN (R 4.1.0)
 data.table               1.14.2     2021-09-27 [1] CRAN (R 4.1.0)
 DBI                      1.1.1      2021-01-15 [1] CRAN (R 4.1.0)
 dbplyr                   2.1.1      2021-04-06 [1] CRAN (R 4.1.0)
 DelayedArray             0.20.0     2021-10-26 [1] Bioconductor
 DelayedMatrixStats       1.16.0     2021-10-26 [1] Bioconductor
 desc                     1.4.0      2021-09-28 [1] CRAN (R 4.1.0)
 devtools               * 2.4.3      2021-11-30 [1] CRAN (R 4.1.2)
 digest                   0.6.29     2021-12-01 [1] CRAN (R 4.1.2)
 dockerfiler              0.1.4      2021-09-03 [1] CRAN (R 4.1.0)
 doParallel               1.0.16     2020-10-16 [1] CRAN (R 4.1.0)
 dotCall64                1.0-1      2021-02-11 [1] CRAN (R 4.1.0)
 dplyr                    1.0.7      2021-06-18 [1] CRAN (R 4.1.0)
 dqrng                    0.3.0      2021-05-01 [1] CRAN (R 4.1.0)
 DropletUtils             1.14.1     2021-11-08 [1] Bioconductor
 DT                       0.20       2021-11-15 [1] CRAN (R 4.1.0)
 edgeR                    3.36.0     2021-10-26 [1] Bioconductor
 ellipsis                 0.3.2      2021-04-29 [1] CRAN (R 4.1.0)
 evaluate                 0.14       2019-05-28 [1] CRAN (R 4.1.0)
 ExperimentHub            2.2.0      2021-10-26 [1] Bioconductor
 fansi                    0.5.0      2021-05-25 [1] CRAN (R 4.1.0)
 fastmap                  1.1.0      2021-01-25 [1] CRAN (R 4.1.0)
 fields                   13.3       2021-10-30 [1] CRAN (R 4.1.0)
 filelock                 1.0.2      2018-10-05 [1] CRAN (R 4.1.0)
 foreach                  1.5.1      2020-10-15 [1] CRAN (R 4.1.0)
 fs                       1.5.2      2021-12-08 [1] CRAN (R 4.1.0)
 generics                 0.1.1      2021-10-25 [1] CRAN (R 4.1.0)
 GenomeInfoDb           * 1.30.0     2021-10-26 [1] Bioconductor
 GenomeInfoDbData         1.2.7      2021-11-23 [1] Bioconductor
 GenomicAlignments        1.30.0     2021-10-26 [1] Bioconductor
 GenomicRanges          * 1.46.1     2021-11-18 [1] Bioconductor
 ggbeeswarm               0.6.0      2017-08-07 [1] CRAN (R 4.1.0)
 ggplot2                  3.3.5      2021-06-25 [1] CRAN (R 4.1.0)
 ggrepel                  0.9.1      2021-01-15 [1] CRAN (R 4.1.0)
 glue                     1.6.0      2021-12-17 [1] CRAN (R 4.1.2)
 golem                    0.3.1      2021-04-17 [1] CRAN (R 4.1.0)
 gridExtra                2.3        2017-09-09 [1] CRAN (R 4.1.0)
 gtable                   0.3.0      2019-03-25 [1] CRAN (R 4.1.0)
 HDF5Array                1.22.1     2021-11-14 [1] Bioconductor
 htmltools                0.5.2      2021-08-25 [1] CRAN (R 4.1.0)
 htmlwidgets              1.5.4      2021-09-08 [1] CRAN (R 4.1.0)
 httpuv                   1.6.4      2021-12-14 [1] CRAN (R 4.1.0)
 httr                     1.4.2      2020-07-20 [1] CRAN (R 4.1.0)
 interactiveDisplayBase   1.32.0     2021-10-26 [1] Bioconductor
 IRanges                * 2.28.0     2021-10-26 [1] Bioconductor
 irlba                    2.3.5      2021-12-06 [1] CRAN (R 4.1.0)
 iterators                1.0.13     2020-10-15 [1] CRAN (R 4.1.0)
 jquerylib                0.1.4      2021-04-26 [1] CRAN (R 4.1.0)
 jsonlite                 1.7.2      2020-12-09 [1] CRAN (R 4.1.0)
 KEGGREST                 1.34.0     2021-10-26 [1] Bioconductor
 knitr                    1.37       2021-12-16 [1] CRAN (R 4.1.2)
 later                    1.3.0      2021-08-18 [1] CRAN (R 4.1.0)
 lattice                  0.20-45    2021-09-22 [1] CRAN (R 4.1.2)
 lazyeval                 0.2.2      2019-03-15 [1] CRAN (R 4.1.0)
 lifecycle                1.0.1      2021-09-24 [1] CRAN (R 4.1.0)
 limma                    3.50.0     2021-10-26 [1] Bioconductor
 locfit                   1.5-9.4    2020-03-25 [1] CRAN (R 4.1.0)
 magick                   2.7.3      2021-08-18 [1] CRAN (R 4.1.0)
 magrittr                 2.0.1      2020-11-17 [1] CRAN (R 4.1.0)
 maps                     3.4.0      2021-09-25 [1] CRAN (R 4.1.0)
 Matrix                   1.4-0      2021-12-08 [1] CRAN (R 4.1.2)
 MatrixGenerics         * 1.6.0      2021-10-26 [1] Bioconductor
 matrixStats            * 0.61.0     2021-09-17 [1] CRAN (R 4.1.0)
 memoise                  2.0.1      2021-11-26 [1] CRAN (R 4.1.0)
 mime                     0.12       2021-09-28 [1] CRAN (R 4.1.0)
 munsell                  0.5.0      2018-06-12 [1] CRAN (R 4.1.0)
 pillar                   1.6.4      2021-10-18 [1] CRAN (R 4.1.0)
 pkgbuild                 1.3.0      2021-12-09 [1] CRAN (R 4.1.0)
 pkgconfig                2.0.3      2019-09-22 [1] CRAN (R 4.1.0)
 pkgload                  1.2.4      2021-11-30 [1] CRAN (R 4.1.2)
 plotly                   4.10.0     2021-10-09 [1] CRAN (R 4.1.0)
 png                      0.1-7      2013-12-03 [1] CRAN (R 4.1.0)
 Polychrome               1.3.1      2021-07-16 [1] CRAN (R 4.1.0)
 prettyunits              1.1.1      2020-01-24 [1] CRAN (R 4.1.0)
 processx                 3.5.2      2021-04-30 [1] CRAN (R 4.1.0)
 promises                 1.2.0.1    2021-02-11 [1] CRAN (R 4.1.0)
 prompt                   1.0.1      2021-11-24 [1] Github (gaborcsardi/prompt@7ef0f2e)
 ps                       1.6.0      2021-02-28 [1] CRAN (R 4.1.0)
 purrr                    0.3.4      2020-04-17 [1] CRAN (R 4.1.0)
 R.methodsS3              1.8.1      2020-08-26 [1] CRAN (R 4.1.0)
 R.oo                     1.24.0     2020-08-26 [1] CRAN (R 4.1.0)
 R.utils                  2.11.0     2021-09-26 [1] CRAN (R 4.1.0)
 R6                       2.5.1      2021-08-19 [1] CRAN (R 4.1.0)
 rappdirs                 0.3.3      2021-01-31 [1] CRAN (R 4.1.0)
 RColorBrewer             1.1-2      2014-12-07 [1] CRAN (R 4.1.0)
 Rcpp                     1.0.7      2021-07-07 [1] CRAN (R 4.1.0)
 RCurl                    1.98-1.5   2021-09-17 [1] CRAN (R 4.1.0)
 remotes                  2.4.2      2021-11-30 [1] CRAN (R 4.1.2)
 restfulr                 0.0.13     2017-08-06 [1] CRAN (R 4.1.0)
 rhdf5                    2.38.0     2021-10-26 [1] Bioconductor
 rhdf5filters             1.6.0      2021-10-26 [1] Bioconductor
 Rhdf5lib                 1.16.0     2021-10-26 [1] Bioconductor
 rjson                    0.2.20     2018-06-08 [1] CRAN (R 4.1.0)
 rlang                    0.4.12     2021-10-18 [1] CRAN (R 4.1.0)
 rmarkdown                2.11       2021-09-14 [1] CRAN (R 4.1.0)
 roxygen2                 7.1.2      2021-09-08 [1] CRAN (R 4.1.0)
 rprojroot                2.0.2      2020-11-15 [1] CRAN (R 4.1.0)
 Rsamtools                2.10.0     2021-10-26 [1] Bioconductor
 RSQLite                  2.2.9      2021-12-06 [1] CRAN (R 4.1.0)
 rsthemes                 0.3.1      2021-11-23 [1] Github (gadenbuie/rsthemes@bbe73ca)
 rstudioapi               0.13       2020-11-12 [1] CRAN (R 4.1.0)
 rsvd                     1.0.5      2021-04-16 [1] CRAN (R 4.1.0)
 rtracklayer              1.54.0     2021-10-26 [1] Bioconductor
 S4Vectors              * 0.32.3     2021-11-21 [1] Bioconductor
 sass                     0.4.0.9000 2021-11-23 [1] Github (rstudio/sass@f7a9540)
 ScaledMatrix             1.2.0      2021-10-26 [1] Bioconductor
 scales                   1.1.1      2020-05-11 [1] CRAN (R 4.1.0)
 scater                   1.22.0     2021-10-26 [1] Bioconductor
 scatterplot3d            0.3-41     2018-03-14 [1] CRAN (R 4.1.0)
 scuttle                  1.4.0      2021-10-26 [1] Bioconductor
 sessioninfo              1.2.2      2021-12-06 [1] CRAN (R 4.1.0)
 shiny                    1.7.1      2021-10-02 [1] CRAN (R 4.1.0)
 shinyWidgets             0.6.2      2021-09-17 [1] CRAN (R 4.1.0)
 SingleCellExperiment   * 1.16.0     2021-10-26 [1] Bioconductor
 spam                     2.7-0      2021-06-25 [1] CRAN (R 4.1.0)
 sparseMatrixStats        1.6.0      2021-10-26 [1] Bioconductor
 SpatialExperiment      * 1.4.0      2021-10-26 [1] Bioconductor
 spatialLIBD            * 1.6.4      2021-12-08 [1] Github (LieberInstitute/spatialLIBD@ea2c037)
 stringi                  1.7.6      2021-11-29 [1] CRAN (R 4.1.2)
 stringr                  1.4.0      2019-02-10 [1] CRAN (R 4.1.0)
 SummarizedExperiment   * 1.24.0     2021-10-26 [1] Bioconductor
 testthat               * 3.1.1      2021-12-03 [1] CRAN (R 4.1.0)
 tibble                   3.1.6      2021-11-07 [1] CRAN (R 4.1.0)
 tidyr                    1.1.4      2021-09-27 [1] CRAN (R 4.1.0)
 tidyselect               1.1.1      2021-04-30 [1] CRAN (R 4.1.0)
 usethis                * 2.1.5      2021-12-09 [1] CRAN (R 4.1.0)
 utf8                     1.2.2      2021-07-24 [1] CRAN (R 4.1.0)
 vctrs                    0.3.8      2021-04-29 [1] CRAN (R 4.1.0)
 vipor                    0.4.5      2017-03-22 [1] CRAN (R 4.1.0)
 viridis                  0.6.2      2021-10-13 [1] CRAN (R 4.1.0)
 viridisLite              0.4.0      2021-04-13 [1] CRAN (R 4.1.0)
 withr                    2.4.3      2021-11-30 [1] CRAN (R 4.1.2)
 xfun                     0.29       2021-12-14 [1] CRAN (R 4.1.0)
 XML                      3.99-0.8   2021-09-17 [1] CRAN (R 4.1.0)
 xml2                     1.3.3      2021-11-30 [1] CRAN (R 4.1.2)
 xtable                   1.8-4      2019-04-21 [1] CRAN (R 4.1.0)
 XVector                  0.34.0     2021-10-26 [1] Bioconductor
 yaml                     2.2.1      2020-02-01 [1] CRAN (R 4.1.0)
 zlibbioc                 1.40.0     2021-10-26 [1] Bioconductor

 [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library

With small example it does work

However, with the small test case it does work. I edited the small example above to use sessioninfo::session_info() instead of sessionInfo().

test.Rmd contents:

```# ---
# title: "Test"
# output: BiocStyle::pdf_document
# ---
# 
# ```{r setup, include=FALSE}
# knitr::opts_chunk$set(echo = TRUE)
# ```
# 
# ```{r}
# options(width = 120)
# sessioninfo::session_info()
#

> rmarkdown::render("~/Desktop/test.Rmd")


processing file: test.Rmd
  |............................                                                                                  |  25%
  ordinary text without R code

  |.......................................................                                                       |  50%
label: setup (with options)
List of 1
 $ include: logi FALSE

  |..................................................................................                            |  75%
  ordinary text without R code

  |..............................................................................................................| 100%
label: unnamed-chunk-1

output file: test.knit.md

/usr/local/bin/pandoc +RTS -K512m -RTS test.knit.md --to latex --from markdown+autolink_bare_uris+tex_math_single_backslash --output test.tex --lua-filter /Library/Frameworks/R.framework/Versions/4.1/Resources/library/bookdown/rmarkdown/lua/custom-environment.lua --lua-filter /Library/Frameworks/R.framework/Versions/4.1/Resources/library/rmarkdown/rmarkdown/lua/pagebreak.lua --lua-filter /Library/Frameworks/R.framework/Versions/4.1/Resources/library/rmarkdown/rmarkdown/lua/latex-div.lua --self-contained --table-of-contents --toc-depth 2 --number-sections --highlight-style tango --pdf-engine pdflatex --include-in-header /var/folders/cx/n9s558kx6fb7jf5z_pgszgb80000gn/T//RtmpAcAKad/1b082ec3f17b.tex --variable graphics --wrap preserve --variable subparagraph --template /var/folders/cx/n9s558kx6fb7jf5z_pgszgb80000gn/T//RtmpAcAKad/BiocStyle/template.tex --variable tables=yes --standalone --variable 'compact-title:yes'
Warning: LaTeX Warning: You have requested package `/Library/Frameworks/R.framework/Vers
Warning: ions/4.1/Resources/library/BiocStyle/resources/tex/Bioconductor',
Warning:                but the package provides `Bioconductor'.
Warning: Package geometry Warning: Over-specification in `h'-direction.
Warning:     `width' (384.1122pt) is ignored.
Warning: Package fancyhdr Warning: \fancyhead's `E' option without twoside option is use
Warning: less on input line 162.
Warning: Package fancyhdr Warning: \headheight is too small (42.67912pt):
Warning: (fancyhdr)                Make it at least 46.27916pt, for example:
Warning: (fancyhdr)                \setlength{\headheight}{46.27916pt}.
Warning: (fancyhdr)                You might also make \topmargin smaller to compensate:
Warning: Package fancyhdr Warning: \headheight is too small (42.67912pt):
Warning: (fancyhdr)                Make it at least 46.27916pt, for example:
Warning: (fancyhdr)                \setlength{\headheight}{46.27916pt}.
Warning: (fancyhdr)                You might also make \topmargin smaller to compensate:
Warning: Package fancyhdr Warning: \headheight is too small (42.67912pt):
Warning: (fancyhdr)                Make it at least 46.27916pt, for example:
Warning: (fancyhdr)                \setlength{\headheight}{46.27916pt}.
Warning: (fancyhdr)                You might also make \topmargin smaller to compensate:
Warning: Package fancyhdr Warning: \headheight is too small (42.67912pt):
Warning: (fancyhdr)                Make it at least 46.27916pt, for example:
Warning: (fancyhdr)                \setlength{\headheight}{46.27916pt}.
Warning: (fancyhdr)                You might also make \topmargin smaller to compensate:

Output created: test.pdf

PDF result:

test.pdf

Originally posted by @lcolladotor in #89 (comment)

[grimbough]

This is definitely a bit odd. I can reproduce the issue if if compile the document using Rscript -e 'rmarkdown::render()' but it builds fine if I press the 'knit' button in Rstudio.

This is because I'm dumb and didn't notice that the default is BiocStyle::html_document so pressing the knit button is doing something else entirely.

[grimbough]

I'm beginning to think this is an oddity (bug?) in rmarkdown. Consider the following Rmd:

---
title: "Vignette Title"
output:
  BiocStyle::pdf_document
---

Using the `images = "lowres"` is fine.

This will render fine with

rmarkdown::render("test.Rmd")
#> ...
#> Output created: test.pdf

However it fails when supply an output_file argument, even if that is the same as the default e.g.

rmarkdown::render("test.Rmd", output_file = "test.pdf")
#> ....
#> ! Package soul Error: Reconstruction failed.

[lcolladotor]

That is a really weird oddity / bug in rmarkdown! Thanks for figuring out a tiny reprex! Should I report this at rmarkdown? Or do you want to? Aka, you might have other details / hints that could help them resolve this.

[grimbough]

I'm not sure it's really a bug in rmarkdown. The patch for the soul issues is executed a post-processing step to the intermediate markdown file, before it is converted to PDF. However when you specify a PDF output file name it does the conversion first and fails because the patch hasn't been applied. I'm currently not sure if there's some quirky behaviour in rmarkdown or if BiocStyle is using some hack which this particular usecase exposes.

If you want more unexpected behaviour, try this:

rmarkdown::render("test.Rmd", output_file = "AdditionalFile.tex")

[lcolladotor]

Hmm I see. I have no idea either and well, my natural inclination would be to ask the rmarkdown authors for help.

I searched for output_file at https://github.com/rstudio/rmarkdown/blob/main/R/render.R and well nothing really pops to me. Like https://github.com/rstudio/rmarkdown/blob/main/R/render.R#L1000-L1006 calls https://github.com/Bioconductor/BiocStyle/blob/master/R/pdf_document.R#L146-L163, right? But I can't spot anything that would indicate an issue with the output_file isn't the default NULL and thus triggers by default https://github.com/rstudio/rmarkdown/blob/main/R/render.R#L487-L491.

[grimbough]

I've drafted an issue on the rmakdown repo, but I haven't posted as I want to check it isn't us doing a dirty hack & then blaming someone else. I'll do some more digging and then post it.

[grimbough]

I've posted an issue with the current state of my investigations. In the interim, you can actually get the desired behaviour by providing an output file name without an extension e.g.

> rmarkdown::render("test.Rmd", output_file = "test1")
Warning: LaTeX Warning: You have requested package `/mnt/data/R-lib/4.1.1-bioc_3.14/Bioc
Warning: Style/resources/tex/Bioconductor',
Warning:                but the package provides `Bioconductor'.
Warning: Package geometry Warning: Over-specification in `h'-direction.
Warning:     `width' (384.1122pt) is ignored.
Warning: Package Fancyhdr Warning: \fancyhead's `E' option without twoside option is use
Warning: less on input line 162.

Output created: test1.pdf