Compare commits

...

7 commits

Author SHA1 Message Date
Sebastian Höffner 4ff059f561
Merge c91c64a1aa into 68e95327f7 2024-05-03 18:56:15 +02:00
dependabot[bot] 68e95327f7 build(deps): bump github.com/pelletier/go-toml/v2 from 2.2.1 to 2.2.2
Bumps [github.com/pelletier/go-toml/v2](https://github.com/pelletier/go-toml) from 2.2.1 to 2.2.2.
- [Release notes](https://github.com/pelletier/go-toml/releases)
- [Changelog](https://github.com/pelletier/go-toml/blob/v2/.goreleaser.yaml)
- [Commits](https://github.com/pelletier/go-toml/compare/v2.2.1...v2.2.2)

---
updated-dependencies:
- dependency-name: github.com/pelletier/go-toml/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 11:29:07 +02:00
Bjørn Erik Pedersen 9cd7db61d3
Run mage generate 2024-05-02 10:49:09 +02:00
Bjørn Erik Pedersen c892e75fbc
resources/page: Pull internal Page methods into its own interface
So it doesn't get visible when passing it to `jsonify`.
2024-05-02 10:49:02 +02:00
hugoreleaser 4255d13d3e releaser: Prepare repository for 0.126.0-DEV
[ci skip]
2024-05-01 15:36:00 +00:00
hugoreleaser c8b9f9f81c releaser: Bump versions for release of 0.125.5
[ci skip]
2024-05-01 15:22:11 +00:00
Sebastian Höffner c91c64a1aa
markup: add --citeproc to pandoc converter
Adds the citeproc filter to the pandoc converter.

There are several PRs for it this feature already. However, I think
simply adding `--citeproc` is the cleanest way to enable this feature,
with the option to flesh it out later, e.g., in #7529.

Some PRs and issues attempt adding more config options to Hugo which
indirectly configure pandoc, but I think simply configuring Pandoc via
Pandoc itself is simpler, as it is already possible with two YAML
blocks -- one for Hugo, and one for Pandoc:

    ---
    title: This is the Hugo YAML block
    ---
    ---
    bibliography: assets/pandoc-yaml-block-bibliography.bib
    ...
    Document content with @citation!

There are other useful options, e.g., #4800 attempts to use `nocite`,
which works out of the box with this PR:

    ---
    title: This is the Hugo YAML block
    ---
    ---
    bibliography: assets/pandoc-yaml-block-bibliography.bib
    nocite: |
      @*
    ...
    Document content with no citations but a full bibliography:

    ## Bibliography

Other useful options are `csl: ...` and `link-citations: true`, which
set the path to a custom CSL file and create HTML links between the
references and the bibliography.

The following issues and PRs are related:

- Add support for parsing citations and Jupyter notebooks via Pandoc and/or Goldmark extension #6101
  Bundles multiple requests, this PR tackles citation parsing.

- WIP: Bibliography with Pandoc #4800
  Passes the frontmatter to Pandoc and still uses
  `--filter pandoc-citeproc` instead of `--citeproc`.
- Allow configuring Pandoc #7529
  That PR is much more extensive and might eventually supersede this PR,
  but I think --bibliography and --citeproc should be independent
  options (--bibliography should be optional and citeproc can always be
  specified).
- Pandoc - allow citeproc extension to be invoked, with bibliography. #8610
  Similar to #7529, #8610 adds a new config option to Hugo.
  I think passing --citeproc and letting the users decide on the
  metadata they want to pass to pandoc is better, albeit uglier.
2023-08-08 15:45:27 +02:00
11 changed files with 284 additions and 20 deletions

View file

@ -43,7 +43,7 @@ Hugo passes reasonable default arguments to these external helpers by default:
- `asciidoctor`: `--no-header-footer -`
- `rst2html`: `--leave-comments --initial-header-level=2`
- `pandoc`: `--mathjax`
- `pandoc`: `--mathjax` and, for pandoc >= 2.11, `--citeproc`
{{% note %}}
Because additional formats are external commands, generation performance will rely heavily on the performance of the external tool you are using. As this feature is still in its infancy, feedback is welcome.
@ -63,7 +63,59 @@ Some Asciidoctor parameters can be customized in Hugo. See&nbsp;[details].
[details]: /getting-started/configuration-markup/#asciidoc
## Learn markdown
### External Helper Pandoc
[Pandoc](https://pandoc.org) is a universal document converter and can be used to convert markdown files.
In Hugo, Pandoc can be used for LaTeX-style math (the `--mathjax` command line option is provided):
```
---
title: Math document
---
Some inline math: $a^2 + b^2 = c^2$.
```
This will render in your HTML as:
```
<p>Some inline math: <span class="math inline">\(a^2 + b^2 = c^2\)</span></p>
```
You will have to [add MathJax](https://www.mathjax.org/#gettingstarted) to your template to properly render the math.
For **Pandoc >= 2.11**, you can use [citations](https://pandoc.org/MANUAL.html#extension-citations).
One way is to employ [BibTeX files](https://en.wikibooks.org/wiki/LaTeX/Bibliography_Management#BibTeX) to cite:
```
---
title: Citation document
---
---
bibliography: assets/bibliography.bib
...
This is a citation: @Doe2022
```
Note that Hugo will **not** pass its metadata YAML block to Pandoc; however, it will pass the **second** meta data block, denoted with `---` and `...` to Pandoc.
Thus, all Pandoc settings should go there.
You can also add all elements from a bibliography file (without citing them explicitly) using:
```
---
title: My Publications
---
---
bibliography: assets/bibliography.bib
nocite: |
@*
...
```
It is also possible to provide a custom [CSL style](https://citationstyles.org/authors/) by passing `csl: path-to-style.csl` as a Pandoc option.
## Learn Markdown
Markdown syntax is simple enough to learn in a single sitting. The following are excellent resources to get you up and running:

2
go.mod
View file

@ -55,7 +55,7 @@ require (
github.com/niklasfasching/go-org v1.7.0
github.com/olekukonko/tablewriter v0.0.5
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58
github.com/pelletier/go-toml/v2 v2.2.1
github.com/pelletier/go-toml/v2 v2.2.2
github.com/rogpeppe/go-internal v1.12.0
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd
github.com/sanity-io/litter v1.5.5

4
go.sum
View file

@ -376,8 +376,8 @@ github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N
github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0=
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y=
github.com/pelletier/go-toml/v2 v2.2.1 h1:9TA9+T8+8CUCO2+WYnDLCgrYi9+omqKXyjDtosvtEhg=
github.com/pelletier/go-toml/v2 v2.2.1/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
github.com/perimeterx/marshmallow v1.1.5 h1:a2LALqQ1BlHM8PZblsDdidgv1mWi1DgC2UmX50IvK2s=
github.com/perimeterx/marshmallow v1.1.5/go.mod h1:dsXbUu8CRzfYP5a87xpp0xq9S3u0Vchtcl8we9tYaXw=
github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 h1:KoWmjvw+nsYOo29YJK9vDA65RGE3NrOnUtO7a+RF9HU=

View file

@ -67,6 +67,7 @@ type pageCommon struct {
page.InSectionPositioner
page.OutputFormatsProvider
page.PageMetaProvider
page.PageMetaInternalProvider
page.Positioner
page.RawContentProvider
page.RelatedKeywordsProvider

View file

@ -184,6 +184,7 @@ func (h *HugoSites) newPage(m *pageMeta) (*pageState, *paths.Path, error) {
ResourceNameTitleProvider: m,
ResourceParamsProvider: m,
PageMetaProvider: m,
PageMetaInternalProvider: m,
RelatedKeywordsProvider: m,
OutputFormatsProvider: page.NopPage,
ResourceTypeProvider: pageTypesProvider,

View file

@ -1,7 +1,8 @@
# Release env.
# These will be replaced by script before release.
HUGORELEASER_TAG=v0.125.4
HUGORELEASER_COMMITISH=cc3574ef4f41fccbe88d9443ed066eb10867ada2
HUGORELEASER_TAG=v0.125.5
HUGORELEASER_COMMITISH=c8b9f9f81c375f5b391e61bae711ee63fc76c1fd

View file

@ -15,10 +15,14 @@
package pandoc
import (
"bytes"
"strconv"
"strings"
"sync"
"github.com/gohugoio/hugo/common/hexec"
"github.com/gohugoio/hugo/htesting"
"github.com/gohugoio/hugo/identity"
"github.com/gohugoio/hugo/markup/converter"
"github.com/gohugoio/hugo/markup/internal"
)
@ -64,6 +68,9 @@ func (c *pandocConverter) getPandocContent(src []byte, ctx converter.DocumentCon
return src, nil
}
args := []string{"--mathjax"}
if supportsCitations(c.cfg) {
args = append(args[:], "--citeproc")
}
return internal.ExternallyRenderContent(c.cfg, ctx, src, binaryName, args)
}
@ -76,6 +83,69 @@ func getPandocBinaryName() string {
return ""
}
type pandocVersion struct {
major, minor int64
}
func (left pandocVersion) greaterThanOrEqual(right pandocVersion) bool {
return left.major > right.major || (left.major == right.major && left.minor >= right.minor)
}
var versionOnce sync.Once
var foundPandocVersion pandocVersion
// getPandocVersion parses the pandoc version output
func getPandocVersion(cfg converter.ProviderConfig) (pandocVersion, error) {
var err error
versionOnce.Do(func() {
argsv := []any{"--version"}
var out bytes.Buffer
argsv = append(argsv, hexec.WithStdout(&out))
cmd, err := cfg.Exec.New(pandocBinary, argsv...)
if err != nil {
cfg.Logger.Errorf("Could not call pandoc: %v", err)
foundPandocVersion = pandocVersion{0, 0}
return
}
err = cmd.Run()
if err != nil {
cfg.Logger.Errorf("%s --version: %v", pandocBinary, err)
foundPandocVersion = pandocVersion{0, 0}
return
}
outbytes := bytes.Replace(out.Bytes(), []byte("\r"), []byte(""), -1)
output := strings.Split(string(outbytes), "\n")[0]
// Split, e.g., "pandoc 2.5" into 2 and 5 and convert them to integers
versionStrings := strings.Split(strings.Split(output, " ")[1], ".")
majorVersion, err := strconv.ParseInt(versionStrings[0], 10, 64)
if err != nil {
println(err)
}
minorVersion, err := strconv.ParseInt(versionStrings[1], 10, 64)
if err != nil {
println(err)
}
foundPandocVersion = pandocVersion{majorVersion, minorVersion}
})
return foundPandocVersion, err
}
// SupportsCitations returns true for pandoc versions >= 2.11, which include citeproc
func supportsCitations(cfg converter.ProviderConfig) bool {
if Supports() {
foundPandocVersion, err := getPandocVersion(cfg)
supportsCitations := foundPandocVersion.greaterThanOrEqual(pandocVersion{2, 11}) && err == nil
return supportsCitations
}
return false
}
// Supports returns whether Pandoc is installed on this computer.
func Supports() bool {
hasBin := getPandocBinaryName() != ""

View file

@ -25,7 +25,7 @@ import (
qt "github.com/frankban/quicktest"
)
func TestConvert(t *testing.T) {
func setupTestConverter(t *testing.T) (*qt.C, converter.Converter, converter.ProviderConfig) {
if !Supports() {
t.Skip("pandoc not installed")
}
@ -38,7 +38,140 @@ func TestConvert(t *testing.T) {
c.Assert(err, qt.IsNil)
conv, err := p.New(converter.DocumentContext{})
c.Assert(err, qt.IsNil)
b, err := conv.Convert(converter.RenderContext{Src: []byte("testContent")})
c.Assert(err, qt.IsNil)
c.Assert(string(b.Bytes()), qt.Equals, "<p>testContent</p>\n")
return c, conv, cfg
}
func TestConvert(t *testing.T) {
c, conv, _ := setupTestConverter(t)
output, err := conv.Convert(converter.RenderContext{Src: []byte("testContent")})
c.Assert(err, qt.IsNil)
c.Assert(string(output.Bytes()), qt.Equals, "<p>testContent</p>\n")
}
func runCiteprocTest(t *testing.T, content string, expected string) {
c, conv, cfg := setupTestConverter(t)
if !supportsCitations(cfg) {
t.Skip("pandoc does not support citations")
}
output, err := conv.Convert(converter.RenderContext{Src: []byte(content)})
c.Assert(err, qt.IsNil)
c.Assert(string(output.Bytes()), qt.Equals, expected)
}
func TestGetPandocVersionCallTwice(t *testing.T) {
c, _, cfg := setupTestConverter(t)
version1, err1 := getPandocVersion(cfg)
version2, err2 := getPandocVersion(cfg)
c.Assert(version1, qt.Equals, version2)
c.Assert(err1, qt.IsNil)
c.Assert(err2, qt.IsNil)
}
func TestPandocVersionEquality(t *testing.T) {
c := qt.New(t)
v1 := pandocVersion{1, 0}
v2 := pandocVersion{2, 0}
v3 := pandocVersion{2, 2}
v4 := pandocVersion{1, 2}
v5 := pandocVersion{2, 11}
// 1 >= 1 -> true
c.Assert(v1.greaterThanOrEqual(v1), qt.IsTrue)
// 1 >= 2 -> false, 2 >= 1 -> tru
c.Assert(v1.greaterThanOrEqual(v2), qt.IsFalse)
c.Assert(v2.greaterThanOrEqual(v1), qt.IsTrue)
// 2.0 >= 2.2 -> false, 2.2 >= 2.0 -> true
c.Assert(v2.greaterThanOrEqual(v3), qt.IsFalse)
c.Assert(v3.greaterThanOrEqual(v2), qt.IsTrue)
// 2.2 >= 1.2 -> true, 1.2 >= 2.2 -> false
c.Assert(v3.greaterThanOrEqual(v4), qt.IsTrue)
c.Assert(v4.greaterThanOrEqual(v3), qt.IsFalse)
// 2.11 >= 2.2 -> true, 2.2 >= 2.11 -> false
c.Assert(v5.greaterThanOrEqual(v3), qt.IsTrue)
c.Assert(v3.greaterThanOrEqual(v5), qt.IsFalse)
}
func TestCiteprocWithHugoMeta(t *testing.T) {
content := `
---
title: Test
published: 2022-05-30
---
testContent
`
expected := "<p>testContent</p>\n"
runCiteprocTest(t, content, expected)
}
func TestCiteprocWithPandocMeta(t *testing.T) {
content := `
---
---
---
...
testContent
`
expected := "<p>testContent</p>\n"
runCiteprocTest(t, content, expected)
}
func TestCiteprocWithBibliography(t *testing.T) {
content := `
---
---
---
bibliography: testdata/bibliography.bib
...
testContent
`
expected := "<p>testContent</p>\n"
runCiteprocTest(t, content, expected)
}
func TestCiteprocWithExplicitCitation(t *testing.T) {
content := `
---
---
---
bibliography: testdata/bibliography.bib
...
@Doe2022
`
expected := `<p><span class="citation" data-cites="Doe2022">Doe and Mustermann
(2022)</span></p>
<div id="refs" class="references csl-bib-body hanging-indent"
role="doc-bibliography">
<div id="ref-Doe2022" class="csl-entry" role="doc-biblioentry">
Doe, Jane, and Max Mustermann. 2022. <span>A Treatise on Hugo
Tests.</span> <em>Hugo Websites</em>.
</div>
</div>
`
runCiteprocTest(t, content, expected)
}
func TestCiteprocWithNocite(t *testing.T) {
content := `
---
---
---
bibliography: testdata/bibliography.bib
nocite: |
@*
...
`
expected := `<div id="refs" class="references csl-bib-body hanging-indent"
role="doc-bibliography">
<div id="ref-Doe2022" class="csl-entry" role="doc-biblioentry">
Doe, Jane, and Max Mustermann. 2022. <span>A Treatise on Hugo
Tests.</span> <em>Hugo Websites</em>.
</div>
</div>
`
runCiteprocTest(t, content, expected)
}

View file

@ -0,0 +1,6 @@
@article{Doe2022,
author = "Jane Doe and Max Mustermann",
title = "A Treatise on Hugo Tests",
journal = "Hugo Websites",
year = "2022",
}

View file

@ -225,9 +225,6 @@ type PageMetaProvider interface {
// to the source of this Page. It will be relative to any content root.
Path() string
// This is for internal use only.
PathInfo() *paths.Path
// The slug, typically defined in front matter.
Slug() string
@ -253,6 +250,12 @@ type PageMetaProvider interface {
Weight() int
}
// PageMetaInternalProvider provides internal page metadata.
type PageMetaInternalProvider interface {
// This is for internal use only.
PathInfo() *paths.Path
}
// PageRenderProvider provides a way for a Page to render content.
type PageRenderProvider interface {
// Render renders the given layout with this Page as context.
@ -273,6 +276,7 @@ type PageWithoutContent interface {
RenderShortcodesProvider
resource.Resource
PageMetaProvider
PageMetaInternalProvider
resource.LanguageProvider
// For pages backed by a file.

View file

@ -17,9 +17,8 @@ package page
import (
"encoding/json"
"time"
"github.com/gohugoio/hugo/config"
"time"
)
func MarshalPageToJSON(p Page) ([]byte, error) {
@ -39,7 +38,6 @@ func MarshalPageToJSON(p Page) ([]byte, error) {
isNode := p.IsNode()
isPage := p.IsPage()
path := p.Path()
pathc := p.Path()
slug := p.Slug()
lang := p.Lang()
isSection := p.IsSection()
@ -65,7 +63,6 @@ func MarshalPageToJSON(p Page) ([]byte, error) {
IsNode bool
IsPage bool
Path string
Pathc string
Slug string
Lang string
IsSection bool
@ -90,7 +87,6 @@ func MarshalPageToJSON(p Page) ([]byte, error) {
IsNode: isNode,
IsPage: isPage,
Path: path,
Pathc: pathc,
Slug: slug,
Lang: lang,
IsSection: isSection,