Commit graph

583 commits

Author SHA1 Message Date
Bjørn Erik Pedersen fdb0b7fb19
helpers: Remove superflous formatting flag in deprecation warnings 2023-06-19 13:02:47 +02:00
Oleksandr Redko 9009c8cdca all: Fix typos in function names and comments 2023-06-19 09:26:29 +02:00
Bjørn Erik Pedersen 7c9fada778 Replace the old log setup, with structured logging etc.
Fixes #11124
2023-06-18 13:03:04 +02:00
Erica Mays 258884f44f cache: Set default cache path based on $USER
Change the default cache directory to `$TMPDIR/hugo_cache_$USER`, so
that multi-user systems do not have caches that interfere with each
other.  The other cache-choosing logic (e.g. Netlify exceptions,
configuration options) are not affected.

Fixes #7391
2023-06-12 21:33:35 +02:00
Bjørn Erik Pedersen ed7e250068 helpers: Avoid url.Parse in RelURL and AbsURL if we can
```
RelURL-10            159ns ± 5%      18ns ± 4%   -88.89%  (p=0.029 n=4+4)
AbsURL/relurl-10     532ns ± 0%     537ns ± 0%    +1.09%  (p=0.029 n=4+4)
AbsURL/absurl-10     142ns ± 0%       4ns ± 3%   -96.91%  (p=0.029 n=4+4)

name              old alloc/op   new alloc/op   delta
RelURL-10             144B ± 0%        0B       -100.00%  (p=0.029 n=4+4)
AbsURL/relurl-10      544B ± 0%      544B ± 0%      ~     (all equal)
AbsURL/absurl-10      144B ± 0%        0B       -100.00%  (p=0.029 n=4+4)

name              old allocs/op  new allocs/op  delta
RelURL-10             1.00 ± 0%      0.00       -100.00%  (p=0.029 n=4+4)
AbsURL/relurl-10      10.0 ± 0%      10.0 ± 0%      ~     (all equal)
AbsURL/absurl-10      1.00 ± 0%      0.00       -100.00%  (p=0.029 n=4+4)
```
2023-06-12 16:15:59 +02:00
Bjørn Erik Pedersen 6a09e7f28e
Adjust benchmarks 2023-06-12 16:06:11 +02:00
Bjørn Erik Pedersen 29e5cbb699
Adjust benchmark 2023-06-12 15:18:41 +02:00
Bjørn Erik Pedersen ded6866001
Add BenchmarkAbsURL 2023-06-12 15:15:28 +02:00
Joe Mooring e1d43021ca helpers: Improve schema detection when creating relative URLs
Fixes #11080
2023-06-12 15:01:30 +02:00
Bjørn Erik Pedersen 5db215d4d6
helpers: Add a basic benchmark for RelURL 2023-06-12 14:44:07 +02:00
Oleksandr Redko 610cedaa61 all: Fix comments for exported functions and packages 2023-05-18 21:25:27 +02:00
Oleksandr Redko 3d90871e9e helpers: simplify path tests with T.TempDir 2023-05-16 18:35:19 +02:00
Bjørn Erik Pedersen 241b21b0fd Create a struct with all of Hugo's config options
Primary motivation is documentation, but it will also hopefully simplify the code.

Also,

* Lower case the default output format names; this is in line with the custom ones (map keys) and how
it's treated all the places. This avoids doing `stringds.EqualFold` everywhere.

Closes #10896
Closes #10620
2023-05-16 18:01:29 +02:00
Bjørn Erik Pedersen 0fbab7cbc5
commands: Fix data race in test
Note that this is a test fix only.
2023-03-14 12:18:42 +01:00
Oleksandr Redko 36ce3a4a9d Correct typos in Go comments 2023-03-02 16:32:32 +01:00
Oleksandr Redko d453c12742 Replace deprecated ioutil with io and os
https://pkg.go.dev/io/ioutil is deprecated since Go 1.16.
2023-03-01 16:28:43 +01:00
Bjørn Erik Pedersen 6abd15e781 Adjust tests for GO 1.20
Updates #10691
2023-02-22 11:26:52 +01:00
Bjørn Erik Pedersen 4ef9baf5bd Only invoke a given cached partial once
Note that this is backed by a LRU cache (which we soon shall see more usage of), so if you're a heavy user of cached partials it may be evicted and
refreshed if needed. But in most cases every partial is only invoked once.

This commit also adds a timeout (the global `timeout` config option) to make infinite recursion in partials
easier to reason about.

```
name              old time/op    new time/op    delta
IncludeCached-10    8.92ms ± 0%    8.48ms ± 1%   -4.87%  (p=0.016 n=4+5)

name              old alloc/op   new alloc/op   delta
IncludeCached-10    6.65MB ± 0%    5.17MB ± 0%  -22.32%  (p=0.002 n=6+6)

name              old allocs/op  new allocs/op  delta
IncludeCached-10      117k ± 0%       71k ± 0%  -39.44%  (p=0.002 n=6+6)
```

Closes #4086
Updates #9588
2023-01-25 17:35:23 +01:00
Joe Mooring 2d217cba51
helpers: Allow at signs in UnicodeSanitize (note)
Closes #10548
2022-12-19 09:58:56 +01:00
Bjørn Erik Pedersen ad2059878a Also consider wrapped errors when checking for file IsNotExist errors
Fixes #10534
2022-12-14 13:51:06 +01:00
Bjørn Erik Pedersen 35fa192838 deps: Udpate to github.com/alecthomas/chroma/v2
Fixes #9932
Fixes #9931
2022-06-14 14:08:40 +02:00
Bjørn Erik Pedersen a5a4422aae Fix relURL with leading slash when baseURL includes a subdirectory
Fixes #9994
2022-06-13 11:40:25 +02:00
Bjørn Erik Pedersen 3854a6fa6c Fix Plainify edge cases
This commit replaces the main part of `helpers.StripHTML` with Go's implementation in its html/template package.

It's a little slower, but correctness is more important:

```bash
BenchmarkStripHTMLOld-10    	  680316	      1764 ns/op	     728 B/op	       4 allocs/op
BenchmarkStripHTMLNew-10    	  384520	      3099 ns/op	    2089 B/op	      10 allocs/op
```

Fixes #9199
Fixes #9909
Closes #9410
2022-05-25 17:55:23 +02:00
Bjørn Erik Pedersen cd0112a05a Add resources.Copy
Implemented by most Resource objects, but not Page (for now).

Fixes #9313
2022-05-25 10:35:31 +02:00
Bjørn Erik Pedersen f2946da9e8 Improve error messages, esp. when the server is running
* Add file context to minifier errors when publishing
* Misc fixes (see issues)
* Allow custom server error template in layouts/server/error.html

To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go.

Fixes #9852
Fixes #9857
Fixes #9863
2022-05-06 19:43:22 +02:00
Bjørn Erik Pedersen d070bdf10f
Rework the Destination filesystem to make --renderStaticToDisk work
See #9626
2022-04-08 13:26:17 +02:00
Bjørn Erik Pedersen 9539069f5e commands: Improve server startup to make tests less flaky
Do this by announcing/listen on the local address before we start the server.
2022-03-21 09:32:35 +01:00
Bjørn Erik Pedersen b80853de90
all: gofmt -w -r 'interface{} -> any' .
Updates #9687
2022-03-17 22:03:27 +01:00
Bjørn Erik Pedersen 08fdca9d93 Add Markdown diagrams and render hooks for code blocks
You can now create custom hook templates for code blocks, either one for all (`render-codeblock.html`) or for a given code language (e.g. `render-codeblock-go.html`).

We also used this new hook to add support for diagrams in Hugo:

* Goat (Go ASCII Tool) is built-in and enabled by default; just create a fenced code block with the language `goat` and start draw your Ascii diagrams.
* Another popular alternative for diagrams in Markdown, Mermaid (supported by GitHub), can also be implemented with a simple template. See the Hugo documentation for more information.

Updates #7765
Closes #9538
Fixes #9553
Fixes #8520
Fixes #6702
Fixes #9558
2022-02-24 18:59:50 +01:00
Cameron Moore 6407b2cd01 helpers: Allow hyphens in UnicodeSanitize
Improve handling of existing hyphens in input to UnicodeSanitize.
This commit accomplishes three things:

1. Explicitly allow hyphens
2. Avoid appending a hyphen if a preceeding hyphen is found
3. Avoid prepending a hyphen if a trailing hyphen is found

Fixes #7288
2022-02-23 21:21:46 +01:00
Ephex2 22055176d2 general: Fix issue causing log threads to hang indefinitely when print() panics
The function printIfNotPrinted() defined for DistinctLogger unlocked the mutex within
the logger only after the print() function ran. If print panics, the mutex would stay
locked and future attempts to read or write from the logger mutex would cause the goroutine
to hang indefinitely.

Deferred the unlocking of the mutex to prevent this. Also, put l.m[key] before the print()
call since this will prevent another bug where the same warning potentially gets logged
multiple times if the print() call panics.

Fixes #9380
2022-01-27 10:38:50 +01:00
Bjørn Erik Pedersen 55a9bc1e70 helpers: Remove unused code 2022-01-16 17:12:56 +01:00
Bjørn Erik Pedersen c8b5ab75b7 Add --panicOnWarning flag
Fixes #9357
Fixes #9359
2022-01-06 12:27:04 +01:00
Bjørn Erik Pedersen 1651beb2c1 Remove mmark
Closes #9350
2022-01-04 17:10:39 +01:00
Bjørn Erik Pedersen f4389e48ce
Add some basic security policies with sensible defaults
This ommmit contains some security hardening measures for the Hugo build runtime.

There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".

For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.

These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.

You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.

The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.

```toml
[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
```
2021-12-16 09:40:22 +01:00
Bjørn Erik Pedersen 5e0947c5b5 helpers: Make UniqueStringsReuse allocation free
This should be faster for all of our use cases (small string slices), but more important, it makes UniqueStringsReuse allocation free.

```bash
name                                 old time/op    new time/op    delta
UniqueStrings/Safe-16                   776ns ± 0%     305ns ± 0%   -60.68%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice-16            621ns ± 3%     174ns ± 0%   -72.01%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice_sorted-16     319ns ± 4%     313ns ± 0%    -2.01%  (p=0.029 n=4+4)

name                                 old alloc/op   new alloc/op   delta
UniqueStrings/Safe-16                    224B ± 0%      128B ± 0%   -42.86%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice-16            96.0B ± 0%      0.0B       -100.00%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice_sorted-16     24.0B ± 0%     24.0B ± 0%      ~     (all equal)

name                                 old allocs/op  new allocs/op  delta
UniqueStrings/Safe-16                    7.00 ± 0%      1.00 ± 0%   -85.71%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice-16             6.00 ± 0%      0.00       -100.00%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice_sorted-16      1.00 ± 0%      1.00 ± 0%      ~     (all equal)
```
2021-11-17 10:44:03 +01:00
Bjørn Erik Pedersen 91cbb96302 Bump all long-living deprecations to ERRORs 2021-07-25 17:17:21 +02:00
Bjørn Erik Pedersen 022c479551
hugofs: Make FileMeta a struct
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See #8749
2021-07-15 17:14:26 +02:00
Bjørn Erik Pedersen a70da2b74a
Fix the deprecation error/warn log levels 2021-07-15 12:24:20 +02:00
Bjørn Erik Pedersen 4c8552b114
Fix Cloudflare vs Netlify cache dir issue
Re-add the additional environment checks to determine if its Netlify. Seems that Cloudflare also sets `NETLIFY=true`.

This makes it look, basically, like a variant of the conditional we had before we started fixing this, but I have checked this logic on Netlify now and it should work.

Fixes #8714
2021-06-30 12:58:43 +02:00
Bjørn Erik Pedersen 6c8c0c8b6a
config: Fix Netlify default cache dir logic
Fixes #8710
2021-06-29 09:51:09 +02:00
Bjørn Erik Pedersen 93aad3c543
Split out the puthe path/filepath functions into common/paths
So they can be used from the config package without cyclic troubles.

Updates #8654
2021-06-18 10:55:00 +02:00
Bjørn Erik Pedersen d392893cd7
Misc config loading fixes
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster:

```
BenchmarkDefaultConfigProvider/Viper-16         	  252418	      4546 ns/op	    2720 B/op	      30 allocs/op
BenchmarkDefaultConfigProvider/Custom-16        	  450756	      2651 ns/op	    1008 B/op	       6 allocs/op
```

Fixes #8633
Fixes #8618
Fixes #8630
Updates #8591
Closes #6680
Closes #5192
2021-06-14 17:00:32 +02:00
Bjørn Erik Pedersen f55d2f4376
tpl/fmt: Add erroridf template func
Fixes #8613
2021-06-07 19:11:03 +02:00
Cameron Moore 1f7e9f7333 Update emoji import paths and version
Since emoji is v2, we must import it with the `/v2` suffix.  Doing so
allows the Go tools to detect the latest release.
2020-12-30 22:46:19 +01:00
Phil Davis 04b89857e1
all: Fix minor typos 2020-12-16 12:11:32 +01:00
Bjørn Erik Pedersen d90e37e0c6 all: Format code with gofumpt
See https://github.com/mvdan/gofumpt
2020-12-03 13:12:58 +01:00
ivan-meridianbanc-com aebfe156fb
Fix RelURL and AbsURL when path starts with language 2020-12-02 12:47:01 +01:00
Bjørn Erik Pedersen fdfa4a5fe6 Allow getJSON errors to be ignored
This change is mostly motivated to get a more stable CI build (we're building the Hugo site there, with Instagram and Twitter shortcodes sometimes failing).

Fixes #7866
2020-10-22 09:09:29 +02:00
Derk Muenchhausen f0266e2ef3
Rework external asciidoctor integration
This commit solves the relative path problem with asciidoctor tooling. An include will resolve relatively, so you can refer easily to files in the same folder.

Also `asciidoctor-diagram` and PlantUML rendering works now, because the created temporary files will be placed in the correct folder.

This patch covers just the Ruby version of asciidoctor. The old AsciiDoc CLI EOLs in Jan 2020, so this variant is removed from code.

The configuration is completely rewritten and now available in `config.toml` under the key `[markup.asciidocext]`:

```toml
[markup.asciidocext]
    extensions = ["asciidoctor-html5s", "asciidoctor-diagram"]
    workingFolderCurrent = true
    trace = true
    [markup.asciidocext.attributes]
        my-base-url = "https://example.com/"
        my-attribute-name = "my value"
```

- backends, safe-modes, and extensions are now whitelisted to the popular (ruby) extensions and valid values.
- the default for extensions is to not enable any, because they're all external dependencies so the build would break if the user didn't install them beforehand.
- the default backend is html5 because html5s is an external gem dependency.
- the default safe-mode is safe, explanations of the modes: https://asciidoctor.org/man/asciidoctor/
- the config is namespaced under asciidocext_config and the parser looks at asciidocext to allow a future native Go asciidoc.
- `uglyUrls=true` option and `--source` flag are supported
- `--destination` flag is required

Follow the updated documentation under `docs/content/en/content-management/formats.md`. 
  
This patch would be a breaking change, because you need to correct all your absolute include pathes to relative paths, so using relative paths must be configured explicitly by setting `workingFolderCurrent = true`.
2020-06-25 09:51:33 +02:00