Commit graph

8059 commits

Author SHA1 Message Date
Bjørn Erik Pedersen f31a6db797 Add path, kind and lang to content front matter
Note that none of these can be set via cascade (you will get an error)

Fixes #11544
2024-01-30 20:12:03 +01:00
Lars Lehtonen ec22bb31a8
hugofs/glob: Fix dropped test error 2024-01-28 23:31:10 +01:00
Bjørn Erik Pedersen a795acbcd8 all: Run gofumpt -l -w . 2024-01-28 23:14:09 +01:00
Bjørn Erik Pedersen 982d9513e7
testing: Simplify some integration tests 2024-01-28 22:17:22 +01:00
Bjørn Erik Pedersen 6dedb4efc7 Add the [params] concept to front matter
This is deliberately very simple, but should not break anything. We need to introduce this in baby steps, but this should allow us to introduce this in the documentation.

Note that the `params` section's key/values will be added to `.Params` last. This means that you can have different values for "Hugo's summary" and the custom ".Params.summary" if you want to.

Updates #11055
2024-01-28 21:38:40 +01:00
Bjørn Erik Pedersen 292626e679 tpl/data: Deprecate data.GetJSON and data.GetCSV 2024-01-28 16:37:36 +01:00
razonyang 60d954c785 modules: Print required Hugo version for incompatible modules 2024-01-28 15:23:21 +01:00
Bjørn Erik Pedersen 63e0a92894
hugolib: Remove unused test image 2024-01-28 15:14:53 +01:00
Kandula Naveen ce7daa6156
navigation: Improve menu cache 2024-01-28 15:11:57 +01:00
Bjørn Erik Pedersen 2a0329423c
testing: Rename integration_test.go to PACKAGE_integration_test.go
Primary motivation making them easier to find in the code editor.
2024-01-28 11:41:59 +01:00
Bjørn Erik Pedersen 50dc327d1a
Port some integration tests to new test setup
The method I'm currently using (if other want to help) is:

* Add fmt.Println(b.DumpTxtar()) after the Build step
* Add that to a files var and pass that to Test(t, files) or similar
* Then, if possible, try to reduce the files/content down to what's needed in test.

Note that if the test is small, it's probably faster just to manually re-create the test.
2024-01-28 11:29:23 +01:00
Bjørn Erik Pedersen 7285e74090
all: Rework page store, add a dynacache, improve partial rebuilds, and some general spring cleaning
There are some breaking changes in this commit, see #11455.

Closes #11455
Closes #11549

This fixes a set of bugs (see issue list) and it is also paying some technical debt accumulated over the years. We now build with Staticcheck enabled in the CI build.

The performance should be about the same as before for regular sized Hugo sites, but it should perform and scale much better to larger data sets, as objects that uses lots of memory (e.g. rendered Markdown, big JSON files read into maps with transform.Unmarshal etc.) will now get automatically garbage collected if needed. Performance on partial rebuilds when running the server in fast render mode should be the same, but the change detection should be much more accurate.

A list of the notable new features:

* A new dependency tracker that covers (almost) all of Hugo's API and is used to do fine grained partial rebuilds when running the server.
* A new and simpler tree document store which allows fast lookups and prefix-walking in all dimensions (e.g. language) concurrently.
* You can now configure an upper memory limit allowing for much larger data sets and/or running on lower specced PCs.
We have lifted the "no resources in sub folders" restriction for branch bundles (e.g. sections).
Memory Limit
* Hugos will, by default, set aside a quarter of the total system memory, but you can set this via the OS environment variable HUGO_MEMORYLIMIT (in gigabytes). This is backed by a partitioned LRU cache used throughout Hugo. A cache that gets dynamically resized in low memory situations, allowing Go's Garbage Collector to free the memory.

New Dependency Tracker: Hugo has had a rule based coarse grained approach to server rebuilds that has worked mostly pretty well, but there have been some surprises (e.g. stale content). This is now revamped with a new dependency tracker that can quickly calculate the delta given a changed resource (e.g. a content file, template, JS file etc.). This handles transitive relations, e.g. $page -> js.Build -> JS import, or $page1.Content -> render hook -> site.GetPage -> $page2.Title, or $page1.Content -> shortcode -> partial -> site.RegularPages -> $page2.Content -> shortcode ..., and should also handle changes to aggregated values (e.g. site.Lastmod) effectively.

This covers all of Hugo's API with 2 known exceptions (a list that may not be fully exhaustive):

Changes to files loaded with template func os.ReadFile may not be handled correctly. We recommend loading resources with resources.Get
Changes to Hugo objects (e.g. Page) passed in the template context to lang.Translate may not be detected correctly. We recommend having simple i18n templates without too much data context passed in other than simple types such as strings and numbers.
Note that the cachebuster configuration (when A changes then rebuild B) works well with the above, but we recommend that you revise that configuration, as it in most situations should not be needed. One example where it is still needed is with TailwindCSS and using changes to hugo_stats.json to trigger new CSS rebuilds.

Document Store: Previously, a little simplified, we split the document store (where we store pages and resources) in a tree per language. This worked pretty well, but the structure made some operations harder than they needed to be. We have now restructured it into one Radix tree for all languages. Internally the language is considered to be a dimension of that tree, and the tree can be viewed in all dimensions concurrently. This makes some operations re. language simpler (e.g. finding translations is just a slice range), but the idea is that it should also be relatively inexpensive to add more dimensions if needed (e.g. role).

Fixes #10169
Fixes #10364
Fixes #10482
Fixes #10630
Fixes #10656
Fixes #10694
Fixes #10918
Fixes #11262
Fixes #11439
Fixes #11453
Fixes #11457
Fixes #11466
Fixes #11540
Fixes #11551
Fixes #11556
Fixes #11654
Fixes #11661
Fixes #11663
Fixes #11664
Fixes #11669
Fixes #11671
Fixes #11807
Fixes #11808
Fixes #11809
Fixes #11815
Fixes #11840
Fixes #11853
Fixes #11860
Fixes #11883
Fixes #11904
Fixes #7388
Fixes #7425
Fixes #7436
Fixes #7544
Fixes #7882
Fixes #7960
Fixes #8255
Fixes #8307
Fixes #8863
Fixes #8927
Fixes #9192
Fixes #9324
2024-01-27 16:28:14 +01:00
Bjørn Erik Pedersen 5fd1e74903
Merge commit '9b0050e9aabe4be65c78ccf292a348f309d50ccd' as 'docs'
```
git subtree add --prefix=docs/ https://github.com/gohugoio/hugoDocs.git master --squash
```

Closes #11925
2024-01-27 10:48:57 +01:00
Bjørn Erik Pedersen 9b0050e9aa Squashed 'docs/' content from commit 5c085a37b
git-subtree-dir: docs
git-subtree-split: 5c085a37b297bf12f59efeaae591418ec025c10d
2024-01-27 10:48:33 +01:00
Bjørn Erik Pedersen fc7de7136a
docs: Prepare for new sub tree
See #11925
2024-01-27 10:47:28 +01:00
hugoreleaser 1083bf7c08 releaser: Prepare repository for 0.123.0-DEV
[ci skip]
2024-01-26 16:06:12 +00:00
hugoreleaser b9a03bd59d releaser: Bump versions for release of 0.122.0
[ci skip]
2024-01-26 15:54:24 +00:00
dependabot[bot] e0021f4961 build(deps): bump golang.org/x/tools from 0.16.0 to 0.17.0
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.16.0 to 0.17.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-26 13:19:29 +01:00
dependabot[bot] d25902c0d8 build(deps): bump github.com/rogpeppe/go-internal from 1.11.0 to 1.12.0
Bumps [github.com/rogpeppe/go-internal](https://github.com/rogpeppe/go-internal) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/rogpeppe/go-internal/releases)
- [Commits](https://github.com/rogpeppe/go-internal/compare/v1.11.0...v1.12.0)

---
updated-dependencies:
- dependency-name: github.com/rogpeppe/go-internal
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-26 12:55:48 +01:00
Bjørn Erik Pedersen 50042ee1f4
docs: Regen docshelper 2024-01-26 08:23:37 +01:00
Bjørn Erik Pedersen 3758456b31
Merge commit '7125ad401ad043e46262afc7eca8dceb6d54bb9e' 2024-01-26 08:22:42 +01:00
Bjørn Erik Pedersen 7125ad401a Squashed 'docs/' changes from 4dd2d6415..3b1a8579d
3b1a8579d Update theme
b1f04fd47 params: Adjust the twitter handle
2c8ba029f Fix typo
4421e46ec Update Where.md
5f9fec6a5 Update glossary.md
fb16945cf collections.Where - Clarify description of "collection"
4d6d7018b Update Include.md - improve return statement example
037151073 Update IncludeCached - improve return statement example
b053fa09c Improve return types
f7d615267 Describe minimum required Windows version
e97762b2a Update front matter in math.Sub
9e4d03a79 Fix data.GetJSON examples
9029ee123 Update spellcheck GitHub Action
c852fe421 Add version verification step to quick start guide
8fafca49c Correct front matter example
b412fdf65 Fix closing tag in notes docs
9a64d9579 Add hreflang Tag to Translation Link example
2d2a82101 Fix inverted examples in Permalink and RelPermalink
ff3e67e3f Correct spelling
3904b72e3 Fix documentation contribution steps enumeration
348fd3c02 Fix some typos in the content summaries page
a5823459a Fix two links, one to bonsai.io and one to digital.gov
c36978686 Fix typo
0ee3ff44f Vendor theme
6b0a9b75d Fix typo
44e4cb5c6 Fix typos
815071742 Update min Go version to build from source
192d14df5 Fix typo
e4a18946e Miscellaneous corrections
f12b7b74e netlify: Hugo 0.121.2
97dbc1ea1 Merge branch 'tempv0.121.2'
dc1c0aeeb Add cache note to resources.Concat
a8d1a6020 Correct return type
eca22e97c Fix front matter
7de247a03 Correct return type
5058bc520 collections.Seq: Specify max slice size
e24d81a4b Fix typo on FAQ page
ff8cf5a88 Correct 'related' front matter values
88b21fae0 Adjust deprecation handling
f7e19a25b Update terminology on shortcode templates page
01e10942a Adjust formatting
6ed2e19c2 tpl/math: Add math.Rand template function
b494a8d8d Update js.md
6d78035f4 Add interval description to glossary (#2378)
f9b612b64 Templates Lookup Order: Add missing parenthesis
741ab1bb2 Show vendorClosest in module default config
eb8cb878f Fix Babel headings
cdcdfc8e8 Change project name in Codeowners example
f31fe0d63 Update link to image filters
9dee1ee0e Update Solus installation instructions
716636fee Add TOC to collections.Sort
f7c64143c Update collections.Sort
4b8f6b991 Add id attributes to Scratch and Store methods
2a1adea79 Add id attributes to GitInfo, File, and Resources methods
b12d9cb8f Add Alpine Linux installation instructions
dfbebeec7 Change writeStats to buildStats
175b8aac8 resources/images: Create AutoOrient image filter
ba8e05c31 Update img shortcode
7fc15a6cc Fix default value of  pluralizeListTitles
a0be4fd8a Add Comentario to available comment engines
71aa7d8ab Update GitInfo.md
348441491 EXIF: describe applicable image types
743497eda Disable TOC for lang.Translate
b3b1df75d Describe lang.Translate fallback behavior
a6d9c362c Categorize CLI commands to improve search results
08cc2da4b Document canonical config for enabling inline shortcodes
8945be9c6 Update description of the humanize ordinal conversion
b8b4ccb6f Fix grammatical error
f53f9f936 Fix description of strings.HasSuffix
cdbe19b5b Update Params.md
e00c4eea2 Reformat site config descriptions
6f50c0755 Enumerate root configuration keys
1f2e7b9b2 Use canonical key when documenting menu configs
7d5a13d8d Update images.Text filter
315d2f8c3 Add math.Counter example to warnf
8ed9cdaa9 Miscellaneous edits
2bda491eb Fix front matter example
5c5bbbe3a netlify: Bump to Hugo 0.121.1
ed08f5943 Update Gentoo installation instructions
e4eb48457 Miscellaneous edits
8c3700d12 netlify: Bump to Hugo 0.121.0
1bb5a377f Merge branch 'tempv0.121.0'
8cba89504 Improve where function
0dfc00b0f docs: Regen docshelper
028076a66 Fix link to Gentoo package
52d2db4ff Fix typo
120025c19 Improve Gentooo installation instructions
0c31a4b45 Update GitHub hosting workflow example
dee99c36a Defend against missing aliases in quick-reference shortcode
d305278f1 Add Gentoo installation instructions
be98a37e1 docs: Adjust last merge from docs repository
a703dab53 docs: Regen docs helper
2e9c3532f Merge commit '35dec7c96f7ee3eb17dd444f7067f0c776fb56ae'
f57fdb44b deps: Update github.com/tdewolff/minify/v2 v2.20.7 => v2.20.8
50cad5e38 tpl/transform: Add transform.XMLEscape template function

git-subtree-dir: docs
git-subtree-split: 3b1a8579d2e032c1aab8a74d4cf63849ba11e7a8
2024-01-26 08:11:52 +01:00
Jeremy Kun d0d2c6795e markup/goldmark: Support passthrough extension
Fixes #10894
2024-01-26 08:11:35 +01:00
dependabot[bot] 2dd608378d build(deps): bump github.com/pelletier/go-toml/v2 from 2.1.0 to 2.1.1
Bumps [github.com/pelletier/go-toml/v2](https://github.com/pelletier/go-toml) from 2.1.0 to 2.1.1.
- [Release notes](https://github.com/pelletier/go-toml/releases)
- [Changelog](https://github.com/pelletier/go-toml/blob/v2/.goreleaser.yaml)
- [Commits](https://github.com/pelletier/go-toml/compare/v2.1.0...v2.1.1)

---
updated-dependencies:
- dependency-name: github.com/pelletier/go-toml/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-25 19:48:59 +01:00
dependabot[bot] 45f52be7f8 build(deps): bump github.com/evanw/esbuild from 0.19.8 to 0.19.12
Bumps [github.com/evanw/esbuild](https://github.com/evanw/esbuild) from 0.19.8 to 0.19.12.
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.19.8...v0.19.12)

---
updated-dependencies:
- dependency-name: github.com/evanw/esbuild
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-25 19:48:32 +01:00
Daniel Terhorst-North 87bf2b9688 tpl/tplimpl: Fix incorrect lastBuildDate
Fixes #11600

Co-authored-by: Joe Mooring <joe@mooring.com>
2024-01-25 19:46:52 +01:00
Daniel Terhorst-North f281ef8a4a tpl: fix incorrect lastBuildDate
Set the `<lastBuildDate>` field to be the most recent Lastmod date
of all the posts in the current selection.

Fixes #11600
2024-01-25 19:46:52 +01:00
August Feng 46f618756f parser/metadecoders: Accumulate org keywords into arrays
Closes #11743
2024-01-25 19:34:12 +01:00
James Tatum 8915343075 deps: Update github.com/tdewolff/minify/v2 v2.20.9 => v2.20.13
KeepConditionalComments was deprecated in the upstream library and replaced with KeepSpecialComments. This new option reflects that both conditional comments and Server Side Include comments can be optionally stripped by the minifier. As with KeepConditionalComments, the minifier is configured not to strip them by default.
2024-01-25 19:33:14 +01:00
Bjørn Erik Pedersen a541e3b4d4
Upgrade to Go 1.21.6
Closes #11892
2024-01-16 10:49:03 +01:00
Joe Mooring 912c6576bb parser/metadecoders: Add CSV lazyQuotes option to transform.Unmarshal
If true, a quote may appear in an unquoted field and a non-doubled quote
may appear in a quoted field. It defaults to false.

Closes #11884
2024-01-16 09:26:44 +01:00
Joe Mooring 911bc60a7a
README: Update minimum Go version to 1.20 2024-01-06 05:31:49 -08:00
hugoreleaser 5ff6323320 releaser: Prepare repository for 0.122.0-DEV
[ci skip]
2024-01-05 12:34:12 +00:00
hugoreleaser 6d5b44305e releaser: Bump versions for release of 0.121.2
[ci skip]
2024-01-05 12:21:15 +00:00
dependabot[bot] 1ccd3147a0 build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-05 13:17:17 +01:00
Joe Mooring e40b9fbbcf tpl/math: Add math.Rand template function
Closes #11833
2023-12-29 19:37:27 +01:00
Bjørn Erik Pedersen 9cd8fbb332
Adjust site benchmark 2023-12-29 16:42:33 +01:00
Bjørn Erik Pedersen abcc61002a
Simplify baseline benchmark 2023-12-29 15:56:02 +01:00
Joe Mooring 648d00c7d8 resources/images: Create AutoOrient image filter
Closes #11717
2023-12-20 14:18:24 +01:00
Bjørn Erik Pedersen 8adba648cc all: Remove unused code
Using x/tools/cmd/deadcode
2023-12-18 19:51:26 +01:00
hugoreleaser 6f13430d4a releaser: Prepare repository for 0.122.0-DEV
[ci skip]
2023-12-08 09:01:24 +00:00
hugoreleaser 00b46fed8e releaser: Bump versions for release of 0.121.1
[ci skip]
2023-12-08 08:47:45 +00:00
Bjørn Erik Pedersen eb9f1eb657
Upgrade to Go 1.21.5
Closes #11786
2023-12-07 12:40:10 +01:00
hugoreleaser 5186e762ac releaser: Prepare repository for 0.122.0-DEV
[ci skip]
2023-12-05 15:36:07 +00:00
hugoreleaser e321c3502a releaser: Bump versions for release of 0.121.0
[ci skip]
2023-12-05 15:22:31 +00:00
Bjørn Erik Pedersen 255e0a9718
docs: Regen docshelper 2023-12-05 16:19:43 +01:00
dependabot[bot] 558f3258a7 build(deps): bump github.com/alecthomas/chroma/v2 from 2.11.1 to 2.12.0
Bumps [github.com/alecthomas/chroma/v2](https://github.com/alecthomas/chroma) from 2.11.1 to 2.12.0.
- [Release notes](https://github.com/alecthomas/chroma/releases)
- [Changelog](https://github.com/alecthomas/chroma/blob/master/.goreleaser.yml)
- [Commits](https://github.com/alecthomas/chroma/compare/v2.11.1...v2.12.0)

---
updated-dependencies:
- dependency-name: github.com/alecthomas/chroma/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-05 09:42:04 +01:00
Joe Mooring 6580cd30ad docs: Adjust last merge from docs repository 2023-12-04 16:57:15 +01:00
Bjørn Erik Pedersen 7617de86cd
docs: Regen docs helper 2023-12-04 15:24:34 +01:00
Bjørn Erik Pedersen d19ed4d4e6
Merge commit '35dec7c96f7ee3eb17dd444f7067f0c776fb56ae' 2023-12-04 15:24:01 +01:00