Commit graph

1679 commits

Author SHA1 Message Date
Bjørn Erik Pedersen 669ada4367 resource: Move resource processors into sub-packages 2019-01-02 14:25:37 +01:00
Bjørn Erik Pedersen 40ffb0484b hugolib: Restore 0.48 slash handling in taxonomies
Fixes #5571
2018-12-30 18:50:09 +01:00
Bjørn Erik Pedersen 4bae8b04aa Revert " Fix handling of taxonomy terms containing slashes"
See #4090
See #5571

This reverts commit fff132537b.
2018-12-30 18:50:09 +01:00
Bjørn Erik Pedersen 65fa069271
Revert "hugolib: Restore taxonomy term path separation"
See #5571

This reverts commit 9ce0a1fb70.
2018-12-29 10:00:17 +01:00
Bjørn Erik Pedersen ce06bdb16a
Rename CSV option from comma to delimiter
See #5555
2018-12-23 21:09:09 +01:00
Bjørn Erik Pedersen 25ddbb09fe
hugolib: Adjust test
See #5544
2018-12-23 19:42:51 +01:00
Bjørn Erik Pedersen a574469797
Add CSV support to transform.Unmarshal
Fixes #5555
2018-12-23 16:33:21 +01:00
Bjørn Erik Pedersen 822dc627a1
tpl/transform: Add transform.Unmarshal func
Fixes #5428
2018-12-23 10:02:42 +01:00
Bjørn Erik Pedersen 43f9df0194 Prevent resource publishing for transformed inline resources
That is, if only `.Content` is accessed.

This means that, for a transformed resource to be published to `/public`, you need to access either `.RelPermalink` or `Permalink`.

Fixes #4944
2018-12-21 13:41:46 +01:00
Bjørn Erik Pedersen 1021714449
hugolib: Add .Name as a shortcode variable
Fixes #5546
2018-12-21 09:51:15 +01:00
Cameron Moore 0483299bc0 hugolib: Improve logic of output path trimming
Fixes #4666
2018-12-21 09:42:37 +01:00
Bjørn Erik Pedersen 9cd54cab20 Move the emoji parsing to pageparser
This avoids double parsing the page content when `enableEmoji=true`.

This commit also adds some general improvements to the parser, making it in general much faster:

```bash
benchmark                     old ns/op     new ns/op     delta
BenchmarkShortcodeLexer-4     90258         101730        +12.71%
BenchmarkParse-4              148940        15037         -89.90%

benchmark                     old allocs     new allocs     delta
BenchmarkShortcodeLexer-4     456            700            +53.51%
BenchmarkParse-4              28             33             +17.86%

benchmark                     old bytes     new bytes     delta
BenchmarkShortcodeLexer-4     69875         81014         +15.94%
BenchmarkParse-4              8128          8304          +2.17%
```

Running some site benchmarks with Emoji support turned on:

```bash
benchmark                                                                                     old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4     924556797     818115620     -11.51%

benchmark                                                                                     old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4     4112613        4133787        +0.51%

benchmark                                                                                     old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4     426982864     424363832     -0.61%
```

Fixes #5534
2018-12-20 20:08:01 +01:00
Bjørn Erik Pedersen 4d93aca27d
hugolib: Enable Emoji in site benchmark 2018-12-20 17:24:51 +01:00
Bjørn Erik Pedersen d5a0b6bbbc hugolib: Remove "double layout" lookup
Fixes #5390
2018-12-17 11:34:32 +01:00
Bjørn Erik Pedersen 9ce0a1fb70 hugolib: Restore taxonomy term path separation
Fixes #5513
2018-12-14 10:29:32 +01:00
Bjørn Erik Pedersen 7829474088
Add /config dir support
This commit adds support for a configuration directory (default `config`). The different pieces in this puzzle are:

* A new `--environment` (or `-e`) flag. This can also be set with the `HUGO_ENVIRONMENT` OS environment variable. The value for `environment` defaults to `production` when running `hugo` and `development` when running `hugo server`. You can set it to any value you want (e.g. `hugo server -e "Sensible Environment"`), but as it is used to load configuration from the file system, the letter case may be important. You can get this value in your templates with `{{ hugo.Environment }}`.
* A new `--configDir` flag (defaults to `config` below your project). This can also be set with `HUGO_CONFIGDIR` OS environment variable.

If the `configDir` exists, the configuration files will be read and merged on top of each other from left to right; the right-most value will win on duplicates.

Given the example tree below:

If `environment` is `production`, the left-most `config.toml` would be the one directly below the project (this can now be omitted if you want), and then `_default/config.toml` and finally `production/config.toml`. And since these will be merged, you can just provide the environment specific configuration setting in you production config, e.g. `enableGitInfo = true`. The order within the directories will be lexical (`config.toml` and then `params.toml`).

```bash
config
├── _default
│   ├── config.toml
│   ├── languages.toml
│   ├── menus
│   │   ├── menus.en.toml
│   │   └── menus.zh.toml
│   └── params.toml
├── development
│   └── params.toml
└── production
    ├── config.toml
    └── params.toml
```

Some configuration maps support the language code in the filename (e.g. `menus.en.toml`): `menus` (`menu` also works) and `params`.

Also note that the only folders with "a meaning" in the above listing is the top level directories below `config`. The `menus` sub folder is just added for better organization.

We use `TOML` in the example above, but Hugo also supports `JSON` and `YAML` as configuration formats. These can be mixed.

Fixes #5422
2018-12-11 13:08:36 +01:00
Bjørn Erik Pedersen 8378358857 hugolib: Add .Site.Sites
Fixes #5504
2018-12-06 14:37:25 +01:00
Bjørn Erik Pedersen 831d23cb4d Add tpl/site and tpl/hugo
This means that the current `.Site` and ´.Hugo` is available as a globals, so you can do `site.IsServer`, `hugo.Version` etc.

Fixes #5470
Fixes #5467
Fixes #5503
2018-12-06 14:37:25 +01:00
Bjørn Erik Pedersen a9a93d082d hugolib: Fall back to title in ByLinkTitle sort
Fixes #4953
2018-11-28 13:52:26 +01:00
Bjørn Erik Pedersen b09a40333f
hugolib: Improve nil handling in IsDescendant and IsAncestor
Fixes #5461
2018-11-28 12:36:59 +01:00
Bjørn Erik Pedersen 7540a62834
parser/pageparser: Fix handling of commented out front matter
When the page parser was rewritten in 0.51, this was interpreted literally, but commented out front matter is used in the wild to "hide it from GitHub", e.g:

```
<!--
+++
title = "hello"
+++
-->
```

Fixes #5478
2018-11-28 10:28:50 +01:00
Bjørn Erik Pedersen 7e75aeca80
Remove duplicate mapstructure depdendency 2018-11-28 00:17:43 +01:00
Bjørn Erik Pedersen bc337e6ab5 Add inline shortcode support
An inline shortcode's name must end with `.inline`, all lowercase.

E.g.:

```bash
{{< time.inline >}}{{ now }}{{< /time.inline >}}
```

The above will print the current date and time.

Note that an inline shortcode's inner content is parsed and executed as a Go text template with the same context as a regular shortcode template.

This means that the current page can be accessed via `.Page.Title` etc. This also means that there are no concept of "nested inline shortcodes".

The same inline shortcode can be reused later in the same content file, with different params if needed, using the self-closing syntax:

```
{{< time.inline />}}
```

Fixes #4011
2018-11-27 16:14:09 +01:00
Bjørn Erik Pedersen 94ab125b27 parser/pageparser: Fix when only shortcode and then summary
Fixes #5464
2018-11-24 18:23:23 +01:00
Bjørn Erik Pedersen f9b4eb4f39 Handle themes in the new file cache (for images, assets)
In the newly consolidated file cache implementation, we forgot that we also look in the theme(s) for assets (SCSS transformations etc.), which is not good for Netlify and the demo sites.

Fixes #5460
2018-11-23 21:09:00 +01:00
Bjørn Erik Pedersen 5df2b79dd2 Fix Permalink for resource, baseURL with path and canonifyURLs set
Fixes #5226
2018-11-15 16:37:11 +01:00
Bjørn Erik Pedersen 12742bac71 Add tests for permalink on Resource with baseURL with path
See #5226
2018-11-15 08:45:15 +01:00
Bjørn Erik Pedersen 94f0f7e597
cache/filecache: Add a :project placeholder
This allows for "cache per Hugo project", making `hugo --gc` work as expected, even if you have several Hugo projects running on the same PC.

See #5439
2018-11-14 23:14:51 +01:00
Bjørn Erik Pedersen 3c29c5af8e
cache/filecache: Add a cache prune func
Fixes #5439
2018-11-14 23:14:51 +01:00
Bjørn Erik Pedersen f7aeaa6129 Add a consolidated file cache
This commits reworks how file caching is performed in Hugo. Now there is only one way, and it can be configured.

This is the default configuration:

```toml
[caches]
[caches.getjson]
dir = ":cacheDir"
maxAge = -1
[caches.getcsv]
dir = ":cacheDir"
maxAge = -1
[caches.images]
dir = ":resourceDir/_gen"
maxAge = -1
[caches.assets]
dir = ":resourceDir/_gen"
maxAge = -1
```

You can override any of these cache setting in your own `config.toml`.

The placeholders explained:

`:cacheDir`: This is the value of the `cacheDir` config option if set (can also be set via OS env variable `HUGO_CACHEDIR`). It will fall back to `/opt/build/cache/hugo_cache/` on Netlify, or a `hugo_cache` directory below the OS temp dir for the others.
`:resourceDir`: This is the value of the `resourceDir` config option.

`maxAge` is the time in seconds before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off.

This means that if you run your builds on Netlify, all caches configured with `:cacheDir` will be saved and restored on the next build. For other CI vendors, please read their documentation. For an CircleCI example, see 6c3960a8f4/.circleci/config.yml

Fixes #5404
2018-11-13 14:19:42 +01:00
Bjørn Erik Pedersen 3a44920e79 hugolib: Improve error message on duplicate menu items 2018-11-07 10:38:58 +01:00
Bjørn Erik Pedersen b8b8436fcc hugolib: Fix changing paginators in lazy render
Fixes #5406
2018-11-06 11:04:10 +01:00
Bjørn Erik Pedersen d16a7a33ff Fix shortcode directly following a shortcode delimiter
Fixes #5402
2018-11-05 14:14:28 +01:00
Bjørn Erik Pedersen 4b7d3e57a4 Make WARN the new default log log level
This commit also pulls down the log level for a set of WARN statements to INFO. There should be no ERRORs or WARNINGs in a regular Hugo build. That is the story about the Boy Who Cried Wolf.

Since the WARN log is now more visible, this commit also improves on some of them, most notable the "layout not found", which now would look something like this:

```bash
WARN 2018/11/02 09:02:18 Found no layout for "home", language "en", output format "CSS": create a template below /layouts with one of these filenames: index.en.css.css, home.en.css.css, list.en.css.css, index.css.css, home.css.css, list.css.css, index.en.css, home.en.css, list.en.css, index.css, home.css, list.css, _default/index.en.css.css, _default/home.en.css.css, _default/list.en.css.css, _default/index.css.css, _default/home.css.css, _default/list.css.css, _default/index.en.css, _default/home.en.css, _default/list.en.css, _default/index.css, _default/home.css, _default/list.css
```

Fixes #5203
2018-11-03 12:06:23 +01:00
Bjørn Erik Pedersen 0bc4b0246d
hugolib: Remove deprecated useModTimeAsFallback 2018-11-01 21:14:30 +01:00
Bjørn Erik Pedersen faeb55c1d8
hugolib: Bump to ERROR for the deprecated Pages.Sort 2018-11-01 21:11:45 +01:00
Bjørn Erik Pedersen 6c6a6c87ec
hugolib: Deprecate .Site.Ref and .Site.RelRef
Fixes #5386
2018-11-01 21:08:12 +01:00
Bjørn Erik Pedersen 1d18eb0574 Add file (line/col) info to ref/relref errors
See #5371
2018-11-01 21:06:35 +01:00
Bjørn Erik Pedersen 33a7b36fd4 hugolib: Add .Position to shortcode
To allow for better error logging in shortcodes. Note that this may be expensive to calculate, so this is primarily for error situations.

See #5371
2018-11-01 21:06:35 +01:00
Bjørn Erik Pedersen 6180c85fb8 hugolib: Fix REF_NOT_FOUND logging to include page path
Fixes #5371
2018-11-01 21:06:35 +01:00
Bjørn Erik Pedersen b2a676f5f0 hugolib: Fix broken manual summary handling
Fixes #5381
2018-10-31 23:14:37 +01:00
Bjørn Erik Pedersen 3a786a248d tpl: Fix BOM issue in templates
Fixes #4895
2018-10-30 18:12:14 +01:00
Bjørn Erik Pedersen 729593c842 hugolib: Fix deadlock when content building times out
Fixes #5375
2018-10-30 12:18:29 +01:00
Bjørn Erik Pedersen b27ccf34bf resource: Allow .Data.Integrity to be accessed on its own
Fixes #5296
2018-10-27 11:34:15 +02:00
Bjørn Erik Pedersen 78578632f5 Fix archetype handling of directories in theme
Fixes #5318
2018-10-26 12:16:28 +02:00
Bjørn Erik Pedersen acc14b4646 hugolib: Adjust error test to make it pass on Go tip 2018-10-24 15:19:49 +02:00
Bjørn Erik Pedersen 2d7709d155 tpl: Handle truncated identifiers in Go template errors
Long identifiers will give errors on the format:

```bash
 _default/single.html:5:14: executing "main" at <.ThisIsAVeryLongTitl...>: can't evaluate field ThisIsAVeryLongTitle
```

Hugo use this value to match the "base template or not", so we need to strip the "...".

Fixes #5346
2018-10-24 13:54:04 +02:00
Bjørn Erik Pedersen 889aca054a Run gofmt -s 2018-10-24 00:12:32 +02:00
Bjørn Erik Pedersen e3ed4a83b8
hugolib: Rename some page_* files
To make it easier to see/work with the source files that is about the `Page` struct.
2018-10-23 22:21:21 +02:00
Bjørn Erik Pedersen 6636cf1bea
Resolve error handling/parser related TODOs
See #5324
2018-10-23 19:41:22 +02:00
Bjørn Erik Pedersen f669ef6bec
herrors: Improve handling of JSON errors
`*json.UnmarshalTypeError` and `*json.SyntaxError` has a byte `Offset`, so use that.

This commit also reworks/simplifies the errror line matching logic. This also makes the file reading unbuffered, but that should be fine in this error case.

See #5324
2018-10-23 14:35:43 +02:00
Bjørn Erik Pedersen ed7b3e2619
commands, hugolib: Get file context in "config parse failed" errors
Fixes #5325
2018-10-23 08:09:41 +02:00
Bjørn Erik Pedersen 2bf686ee21
hugolib: Improve errors in /i18n handlling
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen 9f74dc2a52
hugolib: Improve errors in /data handlling
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen d1661b823a
hugolib: Continue the file context/line number errors work
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen 7930d2132a
hugolib: Remove the now superflous Source struct
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen eb038cfa0a
Convert the rest to new page parser code paths
And remove some now unused code.

See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen 129c27ee6e
parser/metadecoders: Consolidate the metadata decoders
See #5324
2018-10-22 20:46:13 +02:00
Bjørn Erik Pedersen 44da60d869
hugolib: Redo the summary delimiter logic
Now that we have a proper page parse tree, this can be greatly simplified.

See #5324
2018-10-22 20:46:13 +02:00
Bjørn Erik Pedersen 1e3e34002d
hugolib: Integrate new page parser
See #5324
2018-10-22 20:46:13 +02:00
Bjørn Erik Pedersen 1b7ecfc2e1
hugolib: Use []byte in shortcode parsing
See #5324
2018-10-22 19:57:44 +02:00
Bjørn Erik Pedersen d6c16afde0
Move the shortcode parser to the new pageparser package
See #5324
2018-10-22 19:57:43 +02:00
Bjørn Erik Pedersen 4a366fcfee Prevent stale content in Fast Render Mode
We do that by re-render visited pages that is not already in the stack. This may potentially do some double work, but that small penalty should be well worth it.

Fixes #5281
2018-10-17 10:15:22 +02:00
Bjørn Erik Pedersen 083311d033
hugolib: Fix test on Windows 2018-10-17 08:37:02 +02:00
Bjørn Erik Pedersen 0fe4ff1875
tpl: Improve the Execute panic error message
See #5327
2018-10-17 08:25:57 +02:00
Janus 1f42e47e47 Allow date and slug from filename for leaf bundles
Fixes #4558
2018-10-16 23:51:48 +02:00
Bjørn Erik Pedersen 35fbfb19a1
commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes #5284
Fixes #5290
See #5325
See #5324
2018-10-16 22:10:56 +02:00
Bjørn Erik Pedersen e421696d02 helpers: Consolidate MakeSegment vs MakePathSanitized
In short:

* Avoid double tolower in MakeSegment
* Use MakePathSanitized for taxonomies in pageToPermalinkTitle; this matches what MakeSegment does.
* Move the "double hyphen and space" logic into UnicodeSanitize

The last bullet may be slightly breaking for some that now does not get the "--" in some URLs, but we need to reduce the amount of URL logic.

See #4926
2018-10-03 15:55:25 +02:00
Gregorio Martinez 4b4af2c52e hugolib: Use stdlib context package 2018-10-03 14:59:35 +02:00
Cameron Moore fae48d7457 hugolib: Normalize permalink path segments
When constructing permalinks, ensure that most inputs used as path
segments are normalized with PathSpec.MakeSegment instead of
PathSpec.URLize.

The primary exception to that rule is with taxonomy titles in
pageToPermalinkTitle(). The approach taken here is to use URLize for
taxonomy pages. Everything else will use MakeSegment. The reason for
this exception is that people use taxonomies such as "s1/p1" to generate
URLs precisely they way they wish (see #5223). Tests have been added to
check for this case.

Fixes #4926
2018-10-03 11:02:15 +03:00
Gregorio Martinez 06d28a464d hugolib: Fix FuzzyWordCount test error message 2018-10-03 10:01:30 +03:00
Kaushal Modi 6818170308 Render Markdown in figure shortcode "caption" and "attr" params
Fixes https://github.com/gohugoio/hugo/issues/4406.
2018-10-03 09:55:53 +03:00
Kaushal Modi c5279064df Re-organize the figure shortcode for better readability 2018-10-03 09:55:53 +03:00
Cameron Moore d3b81ee58e hugolib: Improve error message for bad taxonomy weights
Error is now "Unable to convert taxonomy weight []string{"40"} to int
for variables/files.md"
2018-10-03 09:34:45 +03:00
Cameron Moore 1fd30d462e hugolib: Cast taxonomy weight parameters to int
Fixes #4628
2018-10-03 09:34:45 +03:00
Cameron Moore 498d629958 hugolib: Allow nil to be unwrapped as *Page
Previously, calls to *Page.Eq(nil) would always return false because the
unwrapPage func didn't support the nil case.  Add support for unwrapping
nil to a *Page.

Fixes #5043
2018-10-03 09:33:42 +03:00
Bjørn Erik Pedersen 10ac2ec446 tpl/collections: Fix handling of different interface types in Slice
In Hugo `0.49` we improved type support in `slice`. This has an unfortunate side effect in that `resources.Concat` now expects something that can resolve to `resource.Resources`.

This worked for most situations, but when you try to `slice` different `Resource` objects, you would be getting `[]interface {}` and not `resource.Resources`. And `concat` would fail:

```bash
error calling Concat: slice []interface {} not supported in concat.
```

This commit fixes that by simplifying the type checking logic in `Slice`:

* If the first item implements the `Slicer` interface, we try that
* If the above fails or the first item does not implement `Slicer`, we just return the `[]interface {}`

Fixes #5269
2018-10-02 23:54:16 +03:00
Bjørn Erik Pedersen fb732d5322
hugolib: Be a litle more specific in NextPage TODO 2018-09-26 09:26:53 +02:00
Ricardo N Feliciano ad705aac06 hugolib: Introduce Page.NextPage and Page.PrevPage
Introduce new page position variables in order to fix the ordering issue
of `.Next` and `.Prev` while also allowing an upgrade path via
deprecation.

`.NextInSection` becomes `.NextPageInSection`.
`.PrevInSection` becomes `.PrevPageInSection`.

`.Next` becomes a function returning `.PrevPage`.
`.Prev` becomes a function returning `.NextPage`.

Fixes #1061
2018-09-26 09:22:43 +02:00
Bjørn Erik Pedersen 2650fa772b Add directory based archetypes
Given this content:

```bash
archetypes
├── default.md
└── post-bundle
    ├── bio.md
    ├── images
    │   └── featured.jpg
    └── index.md
```

```bash
hugo new --kind post-bundle post/my-post
```

Will create a new folder in `/content/post/my-post` with the same set of files as in the `post-bundle` archetypes folder.

This commit also improves the archetype language detection, so, if you use template code in your content files, the `.Site` you get is for the correct language. This also means that it is now possible to translate strings defined in  the `i18n` bundles,  e.g. `{{ i18n "hello" }}`.

Fixes #4535
2018-09-23 19:27:23 +02:00
Bjørn Erik Pedersen f916814697 Prevent symbolic links from themes 2018-09-23 17:18:35 +02:00
Bjørn Erik Pedersen f1a00b2069
hugolib: Remove deprecated rssURI 2018-09-21 14:30:57 +02:00
Bjørn Erik Pedersen 2e2e34a935
hugolib: Deprecate Pages.Sort
In favour of ByWeight.
2018-09-21 14:23:00 +02:00
Bjørn Erik Pedersen 2eed35c826
hugolib: Minor cleaning in the sorting code 2018-09-21 14:13:45 +02:00
Bjørn Erik Pedersen dd692c710a hugolib: Clear the page cache per render
This is a slight performance optimization with no functional effect.

See #5239
2018-09-21 11:21:15 +02:00
Bjørn Erik Pedersen ed4f1edbd7 hugolib: Compare every element in pages cache
It is slightly slower, but correctnes is, of course, more important:

```bash
benchmark                             old ns/op     new ns/op     delta
BenchmarkSortByWeightAndReverse-4     367           645           +75.75%

benchmark                             old allocs     new allocs     delta
BenchmarkSortByWeightAndReverse-4     2              2              +0.00%

benchmark                             old bytes     new bytes     delta
BenchmarkSortByWeightAndReverse-4     64            64            +0.00%
```

Running the same benchmark without any cache (i.e. resorting the slice on every iteration) and then compare it to the current version shows that it still is plenty worth it:

```bash
▶ benchcmp 2.bench 1.bench
benchmark                             old ns/op     new ns/op     delta
BenchmarkSortByWeightAndReverse-4     1358757       645           -99.95%

benchmark                             old allocs     new allocs     delta
BenchmarkSortByWeightAndReverse-4     17159          2              -99.99%

benchmark                             old bytes     new bytes     delta
BenchmarkSortByWeightAndReverse-4     274573        64            -99.98%
```

Closes #5239
2018-09-21 11:21:15 +02:00
Bjørn Erik Pedersen 058cc6c2c3
Revert "hugolib: Normalize permalink path segments"
This reverts commit 06976ebb87.

Fixes #5223
2018-09-18 09:27:49 +02:00
Cameron Moore 06976ebb87 hugolib: Normalize permalink path segments
When constructing permalinks, ensure that path segments are normalized
with PathSpec.MakeSegment instead of PathSpec.URLize.

Fixes #4926
2018-09-14 11:35:26 +02:00
Bjørn Erik Pedersen 555a5612b2 hugolib: Do not set RSS as Kind in RSS output
Before Hugo this commit  we set the pseudo page kind RSS on the page when output to RSS. This had some unintended side effects, esp. when the only output format for that page  was RSS.

For the page kinds that can have multiple output formats, the Kind should be one of the standard home, page etc.

Fixes #5138
2018-09-14 11:33:02 +02:00
Bjørn Erik Pedersen 75e54345f9 hugolib: Make sure ambiguous lookups in GetPage gets an error
See #5138
2018-09-14 11:33:02 +02:00
Bjørn Erik Pedersen e27fd4c1b8 tpl/collections: Add collections.Append
Before this commit you would typically use `.Scratch.Add` to manually create slices in a loop.

With variable overwrite in Go 1.11, we can do better. This commit adds the `append` template func.

A made-up example:

```bash
{{ $p1 := index .Site.RegularPages 0 }}{{ $p2 := index .Site.RegularPages 1 }}
{{ $pages := slice }}
{{ if true }}
  {{ $pages = $pages | append $p2 $p1 }}
{{ end }}
```

Note that with 2 slices as arguments, the two examples below will give the same result:

```bash
{{ $s1 := slice "a" "b" | append (slice "c" "d") }}
{{ $s2 := slice "a" "b" | append "c" "d" }}
```

Both of the above will give `[]string{a, b, c, d}`.

This commit also improves the type handling in the `slice` template function. Now `slice "a" "b"` will give a `[]string` slice. The old behaviour was to return a `[]interface{}`.

Fixes #5190
2018-09-14 10:12:08 +02:00
Bjørn Erik Pedersen fe6676c775 tpl/collections: Improve type handling in collections.Slice
Fixes #5188
2018-09-10 09:19:01 +02:00
Bjørn Erik Pedersen 6667c6d743 tpl/collections: Add group template func
This extends the page grouping in Hugo with a template function that allows for ad-hoc grouping.

A made-up example:

```
{{ $cool := where .Site.RegularPages "Params.cool" true | group "cool" }}
{{ $blue := where .Site.RegularPages "Params.blue" true | group "blue" }}
{{ $paginator := .Paginate (slice $cool $blue) }}
```

Closes #4865
2018-09-08 20:20:26 +02:00
Vincent Danjean cfda13b363 hugolib: Allow creating page groups from any page collection
This also adjusts the pagination logic to allow for these new collections.

Note that we will follow up with a template function named `group` that will be the end user API. The `.Group` method on `Page` should be considered as internal.

Updates #4865
2018-09-08 11:14:09 +02:00
Thomas Jost be3ae3ec92 hugolib: Do not FirstUpper taxonomy titles
Doing so was probably a mistake. This may be a breaking change for some people,
but it's easy to restore the previous behaviour in the layouts.

Fixes #5172
2018-09-06 20:29:21 +02:00
Cameron Moore e38e881248 hugolib: Simplify some code 2018-08-30 22:31:29 +02:00
Cameron Moore 0665a3951b hugolib: Add missing error checks 2018-08-30 22:31:29 +02:00
Cameron Moore 1ed8c36336 hugolib: Remove extraneous createStaticFs call
This looks like a copy+paste error.  We have the same line a few lines
below this call and we actually handle the err value there.
2018-08-30 22:31:29 +02:00
Cameron Moore e03eb90a36 hugolib: Fix godoc comment 2018-08-30 22:31:29 +02:00
Cameron Moore c915d0d325 hugolib: Fix typo in private func name 2018-08-30 22:31:29 +02:00
Bjørn Erik Pedersen 6022f219f1
hugolib: Only run variable variable overwrite test on Go 1.11 2018-08-28 18:00:19 +02:00
Bjørn Erik Pedersen 0c8a415483
hugolib: Add a test for template variable overwrite
New in Go 1.11.
2018-08-28 17:35:33 +02:00
Bjørn Erik Pedersen ebb56e8bdb Improve minifier MIME type resolution
This commit also removes the deprecated `Suffix` from MediaType. Now use `Suffixes` and put the MIME type suffix in the type, e.g. `application/svg+xml`.

Fixes #5093
2018-08-28 17:00:53 +02:00
Bjørn Erik Pedersen 94d0e79d33
hugolib: Include language code in REF_NOT_FOUND errors
Fixes #5110
2018-08-24 09:23:01 +02:00
Cameron Moore fff132537b Fix handling of taxonomy terms containing slashes
Fixes #4090
2018-08-22 07:18:37 +02:00
Bjørn Erik Pedersen baa62d0abb
hugolib: Adjust tests for shortcode p-issue
See #1642
2018-08-17 09:13:39 +02:00
gllera 78c99463fd
hugolib: Fix shortcode output wrapped in p
Fixes #1642
2018-08-17 09:13:30 +02:00
Bjørn Erik Pedersen 9d973004f5 hugolib: Fix image cache-clearing for sub-languages
Fixes #5084
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen 2247546017 hugolib: Force render of any changed page, even in Fast Render Mode
Fixes #5083
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen d139a037d9 hugoblib: Fix "adding a bundle" in server mode
Before this commit, the live reload logic in `hugo server` got confused when you dropped a new bundle into the project while the server was running. The workaround was to restart the server.

This commit fixes the "live reload bundle detection" in server mode, and also makes sure that the bundle headers are always processed first.

Fixes #5075
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen 0a88741fe8 hugolib: Fix error when deleting a bundle in server mode
Fixes #5077
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen a6f199f7a6
hugolib: Use the interface value when doing Related search
Currently it makes no practical difference, but this is more a protection if we in the future creates index from the content related fields. That will not work from a shortcode.

See #5071
2018-08-14 18:19:45 +02:00
Bjørn Erik Pedersen 0dd06bdac0
hugolib: Fix Related when called from shortcode
Fixes #5071
2018-08-14 18:11:36 +02:00
Bjørn Erik Pedersen 56c61559b2
Only duplicate resource output in multihost mode
Yesterday's commit was a little too agressive.

This commit makes sure that the duplication of resources to public/en etc. is only performed in multihost mode.

See #5058
2018-08-14 12:04:17 +02:00
Bjørn Erik Pedersen 78f8475a05 Fix Resource output in multihost setups
In Hugo 0.46 we made the output of what you get from resources.Get and similar static, i.e. language agnostic. This makes total sense, as it is wasteful and time-consuming to do SASS/SCSS/PostCSS processing for lots of languages when the output is lots of duplicates with different filenames.

But since we now output the result once only, this had a negative side effect for multihost setups: We publish the resource once only to the root folder (i.e. not to the language "domain folder").

This commit removes the language code from the processed image keys. This creates less duplication in the file cache, but it means that you should do a `hugo --gc` to clean up stale files.

Fixes #5058
2018-08-13 19:00:51 +02:00
Bjørn Erik Pedersen 2182ecfd34 hugolib: Fix GitInfo when multiple content dirs
Fixes #5054
2018-08-11 19:51:19 +02:00
Bjørn Erik Pedersen e85833d868 hugolib: Add multiple content dirs to GitInfo test site
See #5054
2018-08-11 19:51:19 +02:00
Bjørn Erik Pedersen 43a5aaa778
Fix broken tests 2018-08-09 20:58:31 +02:00
Christian Oliff 06bd013641 Update alias.go
less markup :-)
2018-08-07 09:42:49 +02:00
Bjørn Erik Pedersen 789ef8c639
Add support for minification of final output
Hugo Pipes added minification support for resources fetched via ´resources.Get` and similar.

This also adds support for minification of the final output for supported output formats: HTML, XML, SVG, CSS, JavaScript, JSON.

To enable, run Hugo with the `--minify` flag:

```bash
hugo --minify
```

This commit is also a major spring cleaning of the `transform` package to allow the new minification step fit into that processing chain.

Fixes #1251
2018-08-06 19:58:41 +02:00
Bjørn Erik Pedersen 88e447c449
tocss/scss: Improve _ prefix handling in SCSS imports
See #5008
2018-07-31 16:53:03 +02:00
Bjørn Erik Pedersen f219ac09f6 tocss/scss: Improve SCSS project vs themes import resolution
Before this commit, only SASS/SCSS  components imported from main.scss at first level can be overwritten by homonymous files in projects or over-preceding theme components.

This commit fixes that by implementing a custom import resolver which will be tried first. This resolver will make sure that the project/theme hierarchy is always respected.

Fixes #5008
2018-07-31 10:54:10 +02:00
Bjørn Erik Pedersen 786f72302f
Fix image cache eviction for sites with subdir in baseURL
Fixes #5006
2018-07-29 14:26:45 +02:00
Bjørn Erik Pedersen 04d4c08dbc hugolib: Fix .Site.GetPage regression
In Hugo 0.44 we simplified the `.Site.GetPage` API and added code to handle the old-style syntax in most cases.

This logic did not handle the lookup of the home page via `.Site.GetPage "section" ""` and similar. This commit fixes that.

Fixes #4989
2018-07-24 11:37:59 +02:00
Bjørn Erik Pedersen 016dd4a69a
Add Page.FirstSection
It was added and then removed by accident some time ago. Let us add it again, as it is useful.
2018-07-23 20:20:19 +02:00
Bjørn Erik Pedersen c6b599a06d hugolib: Fix "borrow content from another language" issue
If a content file contains shortcode(s), we have logic in place to re-render it per output format.

We also have logic in place that avoids making a copy of the content used for this process if we don't need it.

This was before this commit limited to server mode and if the page should be output to multiple formats.

But there is a third case: If a site (language) borrows and renders `.Content` from another language. This would, before this commit, behave oddly for content with shortcodes.

Fixes #4986
2018-07-23 20:12:41 +02:00
Bjørn Erik Pedersen a451c49fde hugolib: Print a WARNING about relative non-relative ref/relref matches
Fixes #4973
2018-07-21 11:42:07 +02:00
Bjørn Erik Pedersen ff16c42ed0
hugolib: Allow untyped nil to be merged in lang.Merge
Fixes #4977
2018-07-21 11:41:23 +02:00
Bjørn Erik Pedersen 166483fe12 resource/scss: Add IncludePaths config option
Takes paths relative to the current working dir.

Fixes #4921
2018-07-20 17:50:44 +02:00
Vas Sudanagunta 8278384b96 Increase refLinker test coverage 2018-07-20 07:02:41 +02:00
Bjørn Erik Pedersen 65e610e161
hugolib: Avoid nilpointer in absoluteSourceRef 2018-07-19 22:39:24 +02:00
Vas Sudanagunta b56d9a1294 Fix typo-logic bug in GetPage 2018-07-19 21:36:10 +02:00
Vas Sudanagunta d6fde8fa13 Enable test case fixed by commit 501543d4 2018-07-19 20:40:50 +02:00
Vas Sudanagunta 2bac371544
Add test coverage for recent ref overhaul
Closes #4969
2018-07-19 18:32:23 +02:00
Bjørn Erik Pedersen 9b4b97a722
hugolib: Create an adapter from old to new getPage
To make sure we confirm that the existing tests run the correct code path.

Updates #4969
2018-07-19 18:26:10 +02:00
Bjørn Erik Pedersen 501543d4b6
hugolib: Only do page-relative getPage for relative paths
Updates #4969
2018-07-19 17:47:05 +02:00
Bjørn Erik Pedersen 6ffa882349
hugolib: Improve error handling in refLink 2018-07-19 17:17:35 +02:00
Bjørn Erik Pedersen e25aa655f4 Add configurable ref/relref error handling and notFoundURL
Two new settings:

* refLinksErrorLevel: ERROR (default) or WARNING. ERROR will fail the build.
* refLinksNotFoundURL: Used as a placeholder when page references cannot be found.

Fixes #4964
2018-07-19 14:32:43 +02:00
Bjørn Erik Pedersen 12679b4083 hugolib: Mark shortcode changes as content changes in server mode
This is unfortunate, but is needed to re-create the taxonomies collections etc. that may be referenced from them.

Fixes #4965
2018-07-19 13:58:18 +02:00
Bjørn Erik Pedersen c39ac5738d
Revert "hugolib: Fix potential server panic with drafts/future enabled"
This had a side-effect. I have understand this better to apply the correct fix. Will come soon.

This reverts commit 1ab4658c0d.
2018-07-19 12:20:01 +02:00
Bjørn Erik Pedersen 1ab4658c0d
hugolib: Fix potential server panic with drafts/future enabled
Fixes #4965
2018-07-18 17:01:50 +02:00
Bjørn Erik Pedersen 75acff5f20
hugolib: Avoid server panic on TOML mistake in i18n
Fixes #4942
2018-07-18 11:53:04 +02:00
Bjørn Erik Pedersen 5c9d5413a4 hugolib/filesystems: Fix theme config for Work Fs
Which was the reason why `PostCSS´ from theme(s) was not picked up.

Fixes #4951
2018-07-18 11:39:04 +02:00
Bjørn Erik Pedersen d741064beb Add optional lang as argument to rel/relref
Fixes #4956
2018-07-18 00:07:20 +02:00
Bjørn Erik Pedersen 3eb313fef4 Simplify .Site.GetPage etc.
This commit is a follow up to a recent overhaul of the GetPage/ref/relref implemenation.

The most important change in this commit is the update to `.Site.GetPage`:

* To reduce the amount of breakage in the wild to its minimum, I have reworked .Site.GetPage with some rules:

* We cannot support more than 2 arguments, i.e. .Site.GetPage "page" "posts" "mypage.md" will now throw an error. I think this is the most uncommon syntax and should be OK. It is an easy fix to change the above to .Site.GetPage "/posts/mypage.md" or similar.
* .Site.GetPage "home", .Site.GetPage "home" "" and .Site.GetPage "home" "/" will give you the home page. This means that if you have page in root with the name home.md you need to do .Site.GetPage "/home.md" or similar

This commit also fixes some multilingual issues, most notable it is now possible to do cross-language ref/relref lookups by prepending the language code to the path, e.g. `/jp/posts/mypage.md`.

This commit also reverts the site building tests related to this to "Hugo 0.44 state", to get better control of the changes made.

Closes #4147
Closes #4727
Closes #4728
Closes #4728
Closes #4726
Closes #4652
2018-07-18 00:07:20 +02:00
Vas Sudanagunta b93417aa1d Unify page lookups
This commit unifies the core internal page index for all page kinds.

This enables the `ref` and `relref` shortcodes to support all pages kinds, and adds a new page-relative  `.GetPage` method with simplified signature.

See #4147
See #4727
See #4728
See #4728
See #4726
See #4652
2018-07-18 00:07:20 +02:00
Garry McNulty 38204c4ab6 hugolib: Only set 'allThemes' if there are themes in the config file
When the 'allThemes' configuration setting is read, it will panic if
there are no themes. This was a regression introduced in Hugo 0.42.

Also updated a unit test to check for this.

Fixes #4851
2018-07-15 16:13:48 +02:00
Bjørn Erik Pedersen beec1fc98e resource: Fix resources.Concat for transformed resources
Fixes #4936
2018-07-12 19:36:12 +02:00
Bjørn Erik Pedersen 306573def0 Improve type support in resources.Concat
This allows the result of `.Resources.Match` and similar to be concatenated.

Fixes #4934
2018-07-12 13:43:27 +02:00
Bjørn Erik Pedersen 80c8f3b81a hugolib: Fix static filesystem for themed multihost sites
Multihost is where each language has its own `baseURL`. In this configuration, static files from the theme was not picked up.

This was a regression in Hugo `0.42`. This commit also adds proper tests for this, so that does not happen again.

Fixes #4929
2018-07-12 12:18:56 +02:00
Bjørn Erik Pedersen 6b6dcb44a0 Flush partialCached cache on rebuilds
Fixes #4931
2018-07-11 20:40:04 +02:00
Bjørn Erik Pedersen b874a1ba7a media: Allow multiple file suffixes per media type
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml").

This had its limitations. For one, it was only possible with one file extension per MIME type.

Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type
identifier:

[mediaTypes]
[mediaTypes."image/svg+xml"]
suffixes = ["svg", "abc ]

In most cases, it will be enough to just change:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffix = "txt"

To:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffixes = ["txt"]

Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release.

Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename.

Fixes #4920
2018-07-10 22:13:52 +02:00
Anthony Fok 4108705934 hugolib: Remove empty resources/ dir after TestNewSiteDefaultLang 2018-07-10 09:01:54 +02:00
Bjørn Erik Pedersen 179de5f5bc
Revert "Consider root and current section's content type if set in front matter"
This reverts commit c790029e1d.
2018-07-09 10:29:18 +02:00
Bjørn Erik Pedersen 5dc1f95b63
Revert "Restrict the new type layout resolution to page and section kinds"
This reverts commit 78e8a744b3.
2018-07-09 10:29:16 +02:00
Yang Li 3cea2932e1 Fix typos 2018-07-07 12:29:56 +02:00
Bjørn Erik Pedersen 2b8d907ab7 Add a newScratch template func
Fixes #4685
2018-07-06 17:51:38 +02:00
Bjørn Erik Pedersen 43338c3a99 hugolib: Do not create paginator pages for the other output formats
This is a recent regression in Hugo, where we have started to produce `/page/30/index.json` when the main output format (usually `HTML`) is set up with pagination.

For JSON this is potentially lot of superflous work and hurts performance.

This commit reinstates the earlier behaviour: We only create paginators if in use in the main output format.

And add a test for it to prevent this from happening again.

Fixes #4890
2018-07-06 14:01:27 +02:00
Bjørn Erik Pedersen dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen 78e8a744b3
Restrict the new type layout resolution to page and section kinds
Will have to test and evaluate this a little.

See  #4891
2018-06-30 16:50:02 +02:00
Bjørn Erik Pedersen c790029e1d
Consider root and current section's content type if set in front matter
This should allow for less duplication of templates. Before this commit it was possible to override the content page of a given page/section, but only one page at a time.

Full "template sets" can now be inherited by setting `type: blog` etc. in the section content page's front matter, and that type will be considered when looking for layouts for all pages in that section.

For nested sections, it will use consider both `type` set in the current section first, then `type` set in the first section below home, e.g. `/docs`.

This commit also adds a new Page method: `FirstSection`. This navigates up to the first section below home (e.g. `/docs`). For the home page it will return itself.

Fixes #4891
2018-06-30 13:10:04 +02:00
Bjørn Erik Pedersen de37455ec7
hugolib: Allow forward slash in shortcode names
Fixes #4886
2018-06-28 16:06:16 +02:00
Anthony Fok 42ed602580 hugolib: Replace deprecated {Get,}ByPrefix with {Get,}Match 2018-06-15 09:03:56 +02:00
Bjørn Erik Pedersen 128f14efad hugolib: Reset the global pages cache on server rebuilds
In Hugo 0.42, this could lead to errors of type `runtime error: invalid memory address or nil pointer dereference` in some rare situations.

Note that this was also an issue before 0.42, but the symptom was then potentially stale list content on rebuilds on content changes.

This commit also improves the above error message.

Fixes #4845
2018-06-13 12:00:30 +02:00
Bjørn Erik Pedersen 80230f26a3
Add support for theme composition and inheritance
This commit adds support for theme composition and inheritance in Hugo.

With this, it helps thinking about a theme as a set of ordered components:

```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```

The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.

So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.

Hugo uses two different algorithms to merge the filesystems, depending on the file type:

* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.

The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are  plans to improve on this and get a URL scheme so this can be resolved automatically.

Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:

* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`

The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.

A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.

Fixes #4460
Fixes #4450
2018-06-10 23:55:20 +02:00
Bjørn Erik Pedersen 6464981adb
hugolib: Make sure that .Site.Taxonomies is always set on rebuilds
Fixes #4838
2018-06-10 20:55:17 +02:00
Christian Oliff 2e6712e281 tpl: Always load GA script over HTTPS 2018-06-09 23:47:44 +02:00
Alexandros 65deb72dc4 tplimpl: Remove speakerdeck shortcode
Fixes #4830
2018-06-09 11:13:36 +02:00
Bjørn Erik Pedersen dc4226a8b2 hugolib: Reset Page's main output on server rebuilds
Not doing so prevents the paginators to be rebuilt on changes.

Fixes #4819
2018-06-05 10:45:24 +03:00
Bjørn Erik Pedersen bf5f10faa9 Reset the "distinct error logger" on rebuilds
Fixes #4818
2018-06-05 10:45:24 +03:00
Alex ceaff7cafc tpl: Remove frameborder attr YT iframe + CSS fixes 2018-05-30 14:06:32 +02:00
Bjørn Erik Pedersen 2174525cec
hugolib: A little more on shortcode ordinal 2018-05-28 10:29:48 +02:00
Vas Sudanagunta 4f0665f476 Enhance Page and Resource String() 2018-05-26 09:43:47 +02:00
Bjørn Erik Pedersen 1f1d955b56
Add anonymizeIP to GA privacy config
See #4751
2018-05-25 10:53:39 +02:00
Bjørn Erik Pedersen 35ccf06dae
Fix some recently broken embedded templates
And add tests for them.

Fixes #4757
2018-05-23 10:03:11 +02:00
Bjørn Erik Pedersen 4ddcf52ccc Add an unified .Site.Config with a services section
Fixes #4751
2018-05-22 18:11:03 +02:00
Bjørn Erik Pedersen 353148c2bc Move the privacy config into a parent
See #4751
2018-05-22 18:11:03 +02:00
Bjørn Erik Pedersen 2f17f9378a Do not return error on .Get "class" and vice versa in shortcodes
The current error handling makes parameter checking in shortcodes too verbose for no good reason.

Fixes #4745
2018-05-21 18:52:13 +02:00
Bjørn Erik Pedersen 5f24a2c047 Add PrivacyEnhanced mode for YouTube to the GDPR Policy
See #4616
2018-05-21 16:49:00 +02:00
Bjørn Erik Pedersen 0bbdef986d config: Add the foundation for GDPR privacy configuration
See #4616
2018-05-21 00:41:42 +02:00
Anthony Fok 9bd4236e1b
Show site build warning in TestPageBundlerSiteRegular
So that a timeout warning does appear if it does happen
especially on a slow machine with soft floating-point CPU.

Special thanks to @bep for the solution.

See #4672
2018-05-14 00:09:39 -06:00
Anthony Fok b2b500f563
hugolib: Use double quotes instead of back quotes
so that '\n' is interpreted in the timeout warning.
2018-05-14 00:03:03 -06:00
Vas Sudanagunta 2fb9af59c1 Improve markup determination logic
Sets Page.markup earlier (as early as possible, when the page is
loaded). Sets it once and only once, removing many redundant calls
to determineMarkupType().

This kills a sleeping bug that was avoided by the parts of the
code depending on this value making those redundant calls.
2018-05-12 20:16:26 +02:00
Bjørn Erik Pedersen 086ae81a98
hugolib: Fix possible .Content cut
There have been one report of a site with truncated `.Content` after the Hugo `0.40.1` release.

This commit fixes this so that race should not be possible anymore. It also adds a stress test with focus on content rendering and multiple output formats.

Fixes #4706
2018-05-08 16:52:51 +02:00
Bjørn Erik Pedersen 34ad9a4f17
tpl/tplimpl: Extract internal templates
Having them in separate files should make maintainance easier.

When adding new or making changes to the templates:

```bash
mage generate
```

This will get the Go code in sync.

Fixes #4457
2018-05-04 23:12:10 +02:00
Bjørn Erik Pedersen 655983a22d Reset .Page.Scratch on live-reload
Fixes #4683
2018-04-30 08:21:22 +02:00
Bjørn Erik Pedersen 001a28c2f9 Fix .WordCount, .FuzzyWordCount, .ReadingTime when summary marker is set
This bug was introduced in Hugo 0.40. It is when you use the `<!--more-->` summary marker.

Note that this affects the word stats only. The related `PlainWords`, `Plain`, `Content` all return correct values.

Fixes #4675
Fixes #4682
2018-04-29 10:58:58 +02:00
Bjørn Erik Pedersen 288c396439
hugolib: Fix some shortcode vs .Content corner cases
This is a follow-up to #4632. There were some assumptions in that implementation that did not hold water in all situations.

This commit simplifies the content lazy initalization making it more robust.

Fixes #4664
2018-04-25 08:56:46 +02:00
Bjørn Erik Pedersen 402f6788ee
hugolib: Add .Page.BundleType
I eturn either:

1. leaf
2. branch
3. empty string

The above sits well with constructs like:

```
{{ with .BundleType }}
// Now we know it is a bundle
{{ end }}
```

Fixes #4662
2018-04-23 08:41:19 +02:00
Bjørn Erik Pedersen 3decf4a327
hugolib: Add zero-based Ordinal to shortcode
The count starts at 0 relative to the shortcode's parent: Either the page or the surrounding shortcode.

Access it in a shortcode like this:

```bash
Ordinal is {{ .Ordinal }}
```

Note that this is a shared ordinal for all shortcodes in the relevant context, so, as an example, you have this in a content page:

```markdown
This is a shortcode:

{{< hello >}}

This is another shortcode:

{{< hugo >}}

The `.Ordinal` you get in the two shortcodes above is 0 and 1.
```

See #3359
2018-04-23 08:09:56 +02:00
Bjørn Erik Pedersen 24c662ce6b
hugolib: Defer the unlock in orderedMap.Add
Just in case someone tries to recover from the potential panic.
2018-04-22 21:32:05 +02:00
Bjørn Erik Pedersen 85535084de
hugolib: Process and render shortcodes in their order of appearance
Fixes #3359
2018-04-22 17:40:51 +02:00
Bjørn Erik Pedersen 19084eaf74 hugolib: Init the content and shortcodes early
In most cases we could delay the content init until rendering time, but there could be use cases where the templates would depend on state set in the shortcodes (.Page.Scratch.Set), so we  need to do this early.

See #4632
2018-04-22 12:03:01 +02:00
Bjørn Erik Pedersen 4d26ab33dc
Make .Content (almost) always available in shortcodes
This resolves some surprising behaviour when reading other pages' content from shortcodes. Before this commit, that behaviour was undefined. Note that this has never been an issue from regular templates.

It will still not be possible to get **the current shortcode's  page's rendered content**. That would have impressed Einstein.

The new and well defined rules are:

* `.Page.Content` from a shortcode will be empty. The related `.Page.Truncated` `.Page.Summary`, `.Page.WordCount`, `.Page.ReadingTime`, `.Page.Plain` and `.Page.PlainWords` will also have empty values.
* For _other pages_ (retrieved via `.Page.Site.GetPage`, `.Site.Pages` etc.) the `.Content` is there to use as you please as long as you don't have infinite content recursion in your shortcode/content setup. See below.
* `.Page.TableOfContents` is good to go (but does not support shortcodes in headlines; this is unchanged)

If you get into a situation of infinite recursion, the `.Content` will be empty. Run `hugo -v` for more information.

Fixes #4632
Fixes #4653
Fixes #4655
2018-04-21 22:02:56 +02:00
Bjørn Erik Pedersen d6a2024e6b
Revert "Improve .Content vs shortcodes"
This reverts commit e590cc26eb.
2018-04-19 17:40:57 +02:00
Bjørn Erik Pedersen d6982ac0ac
Revert "hugolib: Prepare child page resources before the page itself"
This reverts commit 3238e14fdf.
2018-04-19 17:40:54 +02:00
Bjørn Erik Pedersen 3238e14fdf
hugolib: Prepare child page resources before the page itself
To make sure that the child resources' content is ready before any shortcode is processed in their owner.

See #4632
2018-04-19 16:35:21 +02:00
Bjørn Erik Pedersen 47c05c47e0 Add language merge support for Pages in resource.Resources
Fixes #4644
2018-04-19 16:23:00 +02:00
Bjørn Erik Pedersen e590cc26eb
Improve .Content vs shortcodes
For the content from other pages in shortcodes there are some chicken and
egg dependencies that is hard to get around. But we can improve on this  by preparing the pages in a certain order:

 1. The headless bundles goes first. These are page typically page and image collections..
 2. Leaf bundles
 3. Regular single pages
 4. Branch bundles

Fixes #4632
2018-04-19 14:46:50 +02:00
cmal e2b277bba5 Update missing positional parameter test for .Get 2018-04-17 15:39:33 +02:00
cmal 236f0c840b .Get doesn't crash on missing positional param
fixes #4619
2018-04-17 15:39:33 +02:00
Bjørn Erik Pedersen d681ea55a0
Improve error message in metadata parse
Fixes #3696
2018-04-17 15:37:05 +02:00
Bjørn Erik Pedersen a96ad5c211
hugolib: Fix broken test 2018-04-17 10:28:32 +02:00
Bjørn Erik Pedersen 61d52f1462
hugolib: Do not reset .Page.Scratch on rebuilds
Fixes #4627
2018-04-15 19:31:42 +02:00
Bjørn Erik Pedersen 417c5e2b67 Make Page.Content a method that returns interface{}
To prepare for a `Resource.Content` method.

See #4622
2018-04-15 18:08:06 +02:00
Bjørn Erik Pedersen f3775877c6
hugolib: Fix livereload of bundled pages
Fixes #4607
2018-04-12 12:19:46 +02:00
Bjørn Erik Pedersen 14c35c8a56
Allow "*/" inside commented out shortcodes
Fixes #4608
2018-04-11 22:41:48 +02:00
Bjørn Erik Pedersen 080302eb87
Fix handling of --contentDir etc. flag
We need to revisit the commands package re globals and tests, but this should fix the init order of flags and languages.

Fixes #4589
2018-04-07 16:40:45 +02:00
Anthony Fok e84b7ade65
Revert "Disable failing test on Travis"
This reverts commit 2170943ae6
and commit 623c9afa84.
2018-04-07 06:53:47 -06:00
Bjørn Erik Pedersen 623c9afa84
And now really disable failing test on Travis
See #4584
2018-04-06 18:14:11 +02:00
Bjørn Erik Pedersen 2170943ae6
Disable failing test on Travis
See #4584
2018-04-06 17:31:13 +02:00
Bjørn Erik Pedersen 6792d86ad0
hugolib: Set .Parent in bundled pages to its owner
Fixes #4582
2018-04-06 10:41:28 +02:00
Bjørn Erik Pedersen f87239e4ca
Fix livereload for the home page bundle
Fixes #4576
2018-04-05 18:03:33 +02:00
Bjørn Erik Pedersen a4deaeff0c
Fix some livereload content regressions
Introduced in Hugo 0.38.

Fixes #4566
2018-04-05 14:28:45 +02:00
Brian Fitzgerald 26f34fd59d Fix two tests that are broken on Windows
Put guards around TestPageBundlerCaptureSymlinks and
TestPageBundlerSiteWitSymbolicLinksInContent so that they aren't
run on Windows (they both use symbolic links and the Go library
implementation requires administrator mode on Windows).
2018-04-05 08:35:06 +02:00
Bjørn Erik Pedersen eb42774e58
Add support for a content dir set per language
A sample config:

```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true

[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"

[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```

The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.

The content files will be assigned a language by

1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.

The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.

Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.

If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.

Fixes #4523
Fixes #4552
Fixes #4553
2018-04-02 08:06:21 +02:00
Ricardo N Feliciano 1823c053c8 Add .Site.IsServer
Fixes #4478
2018-03-30 20:15:22 +02:00
Bjørn Erik Pedersen 93e24a03ce
hugolib: Fix freeze in invalid front matter error case
Fixes #4526
2018-03-24 10:06:58 +01:00
Bjørn Erik Pedersen e9c7b6205f
Allow themes to define output formats, media types and params
This allows a `config.toml` (or `yaml`, ´yml`, or `json`)  in the theme to set:

1) `params` (but cannot override params in project. Will also get its own "namespace", i.e. `{{ .Site.Params.mytheme.my_param }}` will be the same as `{{ .Site.Params.my_param }}` providing that the main project does not define a param with that key.
2) `menu` -- but cannot redefine/add menus in the project. Must create its own menus with its own identifiers.
3) `languages` -- only `params` and `menu`. Same rules as above.
4) **new** `outputFormats`
5) **new** `mediaTypes`

This should help with the "theme portability" issue and people having to copy and paste lots of setting into their projects.

Fixes #4490
2018-03-21 09:22:19 +01:00
Bjørn Erik Pedersen 3d1a6e109c
hugolib: Add ConfigSourceDescriptor
To prepare for config in themes

See #4490
2018-03-20 21:30:43 +01:00
Bjørn Erik Pedersen da88015776
Spring test cleaning, take 2 2018-03-18 09:54:32 +01:00
Bjørn Erik Pedersen debd3663dd hugolib: Test cleaning #1 2018-03-17 18:59:07 +01:00
Bjørn Erik Pedersen af67ad8af1 hugolib: Trim some now superflous tests
These were written as a development aid in some kind of structural change at some point.

They served their purpose then, but these are tests covered elsewhere and is deleted to reduce maintainance.

The reported test covrage is not reduced because of this.
2018-03-17 18:59:07 +01:00
cmal e46ab29bd2 hugolib: Add Reset method to delete key from Scratch 2018-03-17 00:13:23 +01:00
Bjørn Erik Pedersen ffaec4ca8c
Add a way to merge pages by language
As an example:

```html
{{ $pages := .Site.RegularPages | lang.Merge $frSite.RegularPages | lang.Merge $enSite.RegularPages }}
```

Will "fill in the gaps" in the current site with, from left to right, content from the French site, and lastly the English.

Fixes #4463
2018-03-16 10:10:01 +01:00
Bjørn Erik Pedersen e427ba4268
hugolib: Remove superflous debug file 2018-03-12 23:36:20 +01:00
Bjørn Erik Pedersen 79dd7cb31a
hugolib: Simplify Prev/Next 2018-03-12 22:13:25 +01:00
Bjørn Erik Pedersen f12ad7d56a
hugolib: Adjust GitInfo author date test
Now that we have a commit to assert against.

See #4495
2018-03-11 21:32:15 +01:00
Bjørn Erik Pedersen ce6e4310fe
Refactor the GitInfo into the date handlers
Fixes #4495
2018-03-11 21:32:05 +01:00
Bjørn Erik Pedersen 68bf1511f2
hugolib: Extract date and slug from filename
This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter.

This should make it easier to move content from Jekyll to Hugo.

To enable, put this in your `config.toml`:

```toml
[frontmatter]
date  = [":filename", ":default"]
```

This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc.

So, if you want to use the `file modification time`, this can be a good configuration:

 ```toml
[frontmatter]
date = [ "date",":fileModTime", ":default"]
lastmod = ["lastmod" ,":fileModTime", ":default"]
```

The current `:default` values for the different dates are

```toml
[frontmatter]
date = ["date","publishDate", "lastmod"]
lastmod = ["lastmod", "date","publishDate"]
publishDate = ["publishDate", "date"]
expiryDate = ["expiryDate"]
```

The above will now be the same as:

```toml
[frontmatter]
date = [":default"]
lastmod = [":default"]
publishDate = [":default"]
expiryDate = [":default"]
```

Note:

* We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate.
* If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved.
* All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`).
* The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values.
* The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list.

Fixes #285
Closes #3310
Closes #3762
Closes #4340
2018-03-11 18:26:18 +01:00
Bjørn Erik Pedersen f8dc47eeff Allow partial redefinition of the ouputs config
Fixes #4487
2018-03-10 16:57:25 +01:00
Bjørn Erik Pedersen e7fda289b1
Remove defaultLayout config
Not in use.
2018-03-01 17:19:49 +01:00
Bjørn Erik Pedersen 9f740b37cf
hugolib: Fix paginator URL for sections with URL in front matter
Fixes #4415
2018-02-26 17:45:51 +01:00
Bjørn Erik Pedersen e39797fa72
hugolib: Avoid scanning entire site to find the home
See #4447
2018-02-25 10:50:44 +01:00
Vas Sudanagunta 00868081f6 Fix bug in Site.assembleSections method
Site.assembleSections logic assumes that the the home page would always be the first in the Site's list of pages. This is not in fact guaranteed to be true. When it is not, the method can fail to set the parent for some or all root-level pages. 

Fixes #4447
2018-02-25 10:34:24 +01:00
Bjørn Erik Pedersen e9750d831f
hugolib: Continue GitInfo lookup on error
The current logic stops looking after the first failure to connect a page with a Git commit. This implies a fatal error, but that may not be the case.
2018-02-22 17:21:09 +01:00
Bjørn Erik Pedersen 0602135fd4
Make ge, le etc. work with the Hugo Version number
This means that you can do something ala:

```html
{{ if ge .Hugo.Version "0.36" }}Reasonable new Hugo version!{{ end }}
```

The intented use is feature toggling, but please note that it will take some time and Hugo versions until this can be trusted. It does not work in older Hugo versions.

Fixes #4443
2018-02-22 09:15:12 +01:00
Bjørn Erik Pedersen 772128485a Run gofmt -s with Go 1.10
See #4434
2018-02-21 09:59:33 +01:00
Bjørn Erik Pedersen c2c4cb1f9f
hugolib: Fix broken footnote test
See #4433
2018-02-21 09:25:32 +01:00
Bjørn Erik Pedersen 4edb4bcc65
hugolib: Improve error message in .Render 2018-02-13 09:01:47 +01:00
Vas Sudanagunta 1fa2417777 Add support for YAML array data files
* Unmarshaled YAML arrays indistinguishable from JSON arrays.
* Fixes #3890
2018-02-12 17:14:40 +01:00
Vas Sudanagunta bb549a0d57 Account for array type data in data dir merge/override logic
* Fixes #4366

* Error message to console for unsupported data types
2018-02-12 17:14:40 +01:00
Vas Sudanagunta 82eefded13 hugolib: Refactor tests for JSON, YAML and TOML equivalency, add coverage
The same code now cycles through equivalent JSON, YAML and TOML data sets,
verifying output both proper and identical. Coverage increased by applying
previous tests for one format to the others.

More DRY. Added tests for numeric and bool value types.
2018-02-12 17:14:40 +01:00
Vas Sudanagunta e172834993 hugolib: Re-enable YAML data tests disabled in f554503f
Also gave basic tests for JSON, YAML and TOML identical inputs and expected outputs, a step toward JSON, YAML and TOML equivalency (see https://github.com/gohugoio/hugo/issues/4393#issuecomment-364437785).
2018-02-12 17:14:40 +01:00
Kaushal Modi 2e95ec6844 Add "target" and "rel" parameters to figure shortcode
Also:

- Remove unnecessary space from `figure` tag if no class is specified.
  - Update related tests.
- Add test cases for the changes made to the figure shortcode.
- Document the newly added target and rel parameters
- Add more detail to the documentation of all figure shortcode parameters.
2018-02-09 22:27:58 +01:00
Bjørn Erik Pedersen f554503fef
hugolib: Temp. disable some YAML data tests
They fail.

We will have to think about this.

See #4393
2018-02-09 12:32:45 +01:00
Cameron Moore fc06d5c18b hugolib: Add additional test to TestTemplateLookupOrder
Add an additional test to "Variant 4, theme, use site base" to also test
for the index.html base (from by testing of #3505).  Also add a "name"
field to the test cases to make it easier to know which test is failing
versus just getting a slice index.
2018-02-05 08:38:04 +01:00
Cameron Moore 9a367d9d06 hugolib: Fix broken TestTemplateLookupOrder
It looks like we left some debugging code in place that caused all but
one test case to run.
2018-02-05 08:38:04 +01:00
Vas Sudanagunta 4402c07775 Fix JSON array-based data file handling regression
This bug was introduced in Hugo 0.35.

Fixes #4361
2018-02-02 09:14:37 +01:00
Vas Sudanagunta 4743de0d3c Increase data directory test coverage
*  Adds retro-coverage for #4361
*  Verifies open issues #4138, #3890, #4366, 4083
*  Removes test reliance on the very code it is testing (hugo/parser package).
   Expected results are now all built manually / are more precise.
   Tests can run against different versions (no linkage errs)
2018-02-02 09:14:37 +01:00