Merge pull request #2 from tychoish/yaml-rst-support

Switching to YAML and adding support for optional restructured text parsing
This commit is contained in:
Steve Francia 2013-07-08 12:59:44 -07:00
commit 94e577740d
22 changed files with 200 additions and 169 deletions

View file

@ -1,4 +0,0 @@
{
"Indexes" : {"tag": "tags"},
"BaseUrl" : "http://localhost"
}

4
docs/config.yaml Normal file
View file

@ -0,0 +1,4 @@
Indexes:
tag: 'tags'
BaseUrl: 'http://localhost'
...

View file

@ -1,19 +1,21 @@
{
"title": "Configuring Hugo",
"Pubdate": "2013-07-01"
}
---
title: "Configuring Hugo"
pubdate: "2013-07-01"
...
The directory structure and templates provide the majority of the
configuration for a site. In fact a config file isn't even needed for many websites
since the defaults used follow commonly used patterns.
The following is an example of a config file with the default values
The following is an example of a config file with the default values:
SourceDir: "content"
LayoutDir: "layouts"
PublishDir: "public"
BuildDrafts: false
Tags:
category: "categories"
tag: "tags"
BaseUrl: "http://yourSite.com/"
...
{
"SourceDir" : "content",
"LayoutDir" : "layouts",
"PublishDir" : "public",
"BuildDrafts" : false,
"Tags" : { "category" : "categories", "tag" : "tags" },
"BaseUrl" : "http://yourSite.com/"
}

View file

@ -1,7 +1,7 @@
{
"title": "Contributing to Hugo",
"Pubdate": "2013-07-01"
}
---
title: "Contributing to Hugo"
Pubdate: "2013-07-01"
...
1. Fork it from https://github.com/spf13/hugo
2. Create your feature branch (`git checkout -b my-new-feature`)

View file

@ -1,7 +1,7 @@
{
"title": "Contributors",
"Pubdate": "2013-07-01"
}
---
title: "Contributors"
Pubdate: "2013-07-01"
...
Hugo was built with love and golang by:

View file

@ -1,22 +1,22 @@
{
"title": "Example Content File",
"Pubdate": "2013-07-01"
}
---
title: "Example Content File"
Pubdate: "2013-07-01"
...
Somethings are better shown than explained. The following is a very basic example of a content file:
**mysite/project/nitro.md <- http://mysite.com/project/nitro.html**
{
"Title": "Nitro : A quick and simple profiler for golang",
"Description": "",
"Keywords": [ "Development", "golang", "profiling" ],
"Tags": [ "Development", "golang", "profiling" ],
"Pubdate": "2013-06-19",
"Topics": [ "Development", "GoLang" ],
"Slug": "nitro",
"project_url": "http://github.com/spf13/nitro"
}
---
Title: "Nitro : A quick and simple profiler for golang"
Description": ""
Keywords": [ "Development", "golang", "profiling" ]
Tags": [ "Development", "golang", "profiling" ]
Pubdate": "2013-06-19"
Topics": [ "Development", "GoLang" ]
Slug": "nitro"
project_url": "http://github.com/spf13/nitro"
...
# Nitro

View file

@ -1,22 +1,25 @@
{
"title": "Front Matter",
"Pubdate": "2013-07-01"
}
---
title: "Front Matter"
Pubdate: "2013-07-01"
...
The front matter is one of the features that gives Hugo it's strength. It enables
you to include the meta data of the content right with it. Hugo supports a few
different formats. The main format supported is JSON. Here is an example:
different formats. The main format supported is YAML. Here is an example:
{
"Title": "spf13-vim 3.0 release and new website",
"Description": "spf13-vim is a cross platform distribution of vim plugins and resources for Vim.",
"Tags": [ ".vimrc", "plugins", "spf13-vim", "vim" ],
"Pubdate": "2012-04-06",
"Categories": [ "Development", "VIM" ],
"Slug": "spf13-vim-3-0-release-and-new-website"
}
---
Title: "spf13-vim 3.0 release and new website"
Description: "spf13-vim is a cross platform distribution of vim plugins and resources for Vim."
Tags: [ ".vimrc", "plugins", "spf13-vim", "vim" ]
Pubdate: "2012-04-06"
Categories:
- "Development"
- "VIM"
Slug: "spf13-vim-3-0-release-and-new-website"
...
### Variables
There are a few predefined variables that Hugo is aware of and utilizes. The user can also create
any variable they want to. These will be placed into the `.Params` variable available to the templates.
@ -31,6 +34,8 @@ any variable they want to. These will be placed into the `.Params` variable avai
**Draft** If true the content will not be rendered unless `hugo` is called with -d<br>
**Type** The type of the content (will be derived from the directory automatically if unset).<br>
**Markup** (Experimental) Specify "rst" for reStructuredText (requires
`rst2html`,) or "md" (default) for the Markdown.<br>
**Slug** The token to appear in the tail of the url.<br>
*or*<br>
**Url** The full path to the content from the web root.<br>

View file

@ -1,7 +1,8 @@
{
"title": "Installing Hugo",
"Pubdate": "2013-07-01"
}
---
title: "Installing Hugo"
Pubdate: "2013-07-01"
...
Hugo is written in GoLang with support for Windows, Linux, FreeBSD and OSX.
The latest release can be found at [hugo releases](https://github.com/spf13/hugo/releases).

View file

@ -1,7 +1,7 @@
{
"title": "License",
"Pubdate": "2013-07-01"
}
---
title: "License"
Pubdate: "2013-07-01"
...
Hugo is released under the Simple Public License.

View file

@ -1,7 +1,7 @@
{
"title": "Organization",
"Pubdate": "2013-07-01"
}
---
title: "Organization"
Pubdate: "2013-07-01"
...
Hugo uses markdown files with headers commonly called the front matter. Hugo respects the organization
that you provide for your content to minimize any extra configuration, though this can be overridden

View file

@ -1,8 +1,7 @@
{
"title": "Release Notes",
"Pubdate": "2013-07-01"
}
---
title: "Release Notes"
Pubdate: "2013-07-01"
...
* **0.7.0** July 4, 2013
* Hugo now includes a simple server

View file

@ -1,7 +1,7 @@
{
"title": "Roadmap",
"Pubdate": "2013-07-01"
}
---
title: "Roadmap"
Pubdate: "2013-07-01"
...
In no particular order, here is what I'm working on:

14
docs/content/doc/rst.rst Normal file
View file

@ -0,0 +1,14 @@
---
Markup: 'rst'
...
==============
This is a Test
==============
Really
------
text *here* and **HERE**.

View file

@ -1,7 +1,7 @@
{
"title": "Shortcodes",
"Pubdate": "2013-07-01"
}
---
title: "Shortcodes"
Pubdate: "2013-07-01"
...
Because Hugo uses markdown for it's content format, it was clear that there's a lot of things that
markdown doesn't support well. This is good, the simple nature of markdown is exactly why we chose it.

View file

@ -1,7 +1,7 @@
{
"title": "Source Directory Organization",
"Pubdate": "2013-07-01"
}
---
title: "Source Directory Organization"
Pubdate: "2013-07-01"
...
Hugo takes a single directory and uses it as the input for creating a complete website.

View file

@ -1,7 +1,7 @@
{
"title": "Templates",
"Pubdate": "2013-07-01"
}
---
title: "Templates"
Pubdate: "2013-07-01"
...
Hugo uses the excellent golang html/template library for it's template engine. It is an extremely
lightweight engine that provides a very small amount of logic. In our

View file

@ -1,7 +1,7 @@
{
"title": "Using Hugo",
"Pubdate": "2013-07-01"
}
---
title: "Using Hugo"
Pubdate: "2013-07-01"
...
Make sure either hugo is in your path or provide a path to it.

View file

@ -1,7 +1,7 @@
{
"title": "Variables",
"Pubdate": "2013-07-01"
}
---
title: "Variables"
Pubdate: "2013-07-01"
...
Hugo makes a set of values available to the templates. Go templates are context based. The following
are available in the context for the templates.

View file

@ -14,7 +14,7 @@
package hugolib
import (
"encoding/json"
"launchpad.net/goyaml"
"fmt"
"io/ioutil"
"os"
@ -55,7 +55,7 @@ func SetupConfig(cfgfile *string, path *string) *Config {
file, err := ioutil.ReadFile(configPath)
if err == nil {
if err := json.Unmarshal(file, &c); err != nil {
if err := goyaml.Unmarshal(file, &c); err != nil {
fmt.Printf("Error parsing config: %s", err)
os.Exit(1)
}

View file

@ -81,7 +81,7 @@ func interfaceToBool(i interface{}) bool {
case bool:
return b
default:
Error("Only Boolean values are supported for this JSON key")
Error("Only Boolean values are supported for this YAML key")
}
return false
@ -106,7 +106,7 @@ func interfaceToString(i interface{}) string {
case string:
return s
default:
Error("Only Strings are supported for this JSON key")
Error("Only Strings are supported for this YAML key")
}
return ""

View file

@ -16,11 +16,13 @@ package hugolib
import (
"bytes"
"encoding/json"
"launchpad.net/goyaml"
"fmt"
"github.com/theplant/blackfriday"
"html/template"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
@ -40,6 +42,7 @@ type Page struct {
contentType string
Draft bool
Tmpl *template.Template
Markup string
PageMeta
File
Position
@ -80,6 +83,7 @@ func initializePage(filename string) (page Page) {
page.Extension = "html"
page.Params = make(map[string]interface{})
page.Keywords = make([]string, 10, 30)
page.Markup = "md"
page.setSection()
return page
@ -141,36 +145,47 @@ func (p *Page) analyzePage() {
}
// TODO //rewrite to use byte methods instead
func (page *Page) parseYamlMetaData(data []byte) ([]string, error) {
var err error
datum, lines := splitPageContent(data, "---", "...")
err = page.handleMetaData(page.handleYamlMetaData([]byte(strings.Join(datum, "\n"))))
return lines, err
}
func (page *Page) parseJsonMetaData(data []byte) ([]string, error) {
var err error
datum, lines := splitPageContent(data, "{", "}")
err = page.handleMetaData(page.handleJsonMetaData([]byte(strings.Join(datum, "\n"))))
return lines, err
}
func splitPageContent(data []byte, start string, end string) ([]string, []string) {
lines := strings.Split(string(data), "\n")
datum := lines[0:]
// go through content parse between "{" and "}"
// must be on their own lines (for now)
var found = 0
for i, line := range lines {
line = strings.TrimSpace(line)
if line == "{" {
if strings.HasPrefix(line, start) {
found += 1
}
if line == "}" {
if strings.HasPrefix(line, end) {
found -= 1
}
if found == 0 {
datum = lines[0 : i+1]
datum = lines[1: i+1]
lines = lines[i+1:]
break
}
}
err = page.handleJsonMetaData([]byte(strings.Join(datum, "\n")))
return lines, err
return datum, lines
}
func (p *Page) Permalink() template.HTML {
@ -185,12 +200,24 @@ func (p *Page) Permalink() template.HTML {
}
}
func (page *Page) handleJsonMetaData(datum []byte) error {
func (page *Page) handleYamlMetaData(datum []byte) interface{} {
m := map[string]interface{}{}
if err := goyaml.Unmarshal(datum, &m); err != nil {
return fmt.Errorf("Invalid YAML in %s \nError parsing page meta data: %s", page.FileName, err)
}
return m
}
func (page *Page) handleJsonMetaData(datum []byte) interface{} {
var f interface{}
if err := json.Unmarshal(datum, &f); err != nil {
return fmt.Errorf("Invalide JSON in $v \nError parsing page meta data: %s", page.FileName, err)
}
return f
}
func (page *Page) handleMetaData(f interface{}) error {
m := f.(map[string]interface{})
for k, v := range m {
@ -216,6 +243,8 @@ func (page *Page) handleJsonMetaData(datum []byte) error {
page.Draft = interfaceToBool(v)
case "layout":
page.layout = interfaceToString(v)
case "markup":
page.Markup = interfaceToString(v)
case "status":
page.Status = interfaceToString(v)
default:
@ -236,8 +265,8 @@ func (page *Page) handleJsonMetaData(datum []byte) error {
}
}
}
//Printer(page.Params)
return nil
}
func (page *Page) GetParam(key string) interface{} {
@ -256,47 +285,6 @@ func (page *Page) GetParam(key string) interface{} {
return nil
}
func (page *Page) parseFileMetaData(data []byte) ([]string, error) {
lines := strings.Split(string(data), "\n")
// go through content parse from --- to ---
var found = 0
for i, line := range lines {
line = strings.TrimSpace(line)
if found == 1 {
// parse line for param
colonIndex := strings.Index(line, ":")
if colonIndex > 0 {
key := strings.TrimSpace(line[:colonIndex])
value := strings.TrimSpace(line[colonIndex+1:])
value = strings.Trim(value, "\"") //remove quotes
switch key {
case "title":
page.Title = value
case "layout":
page.layout = value
case "extension":
page.Extension = "." + value
default:
page.Params[key] = value
}
}
} else if found >= 2 {
// params over
lines = lines[i:]
break
}
if line == "---" {
found += 1
}
}
return lines, nil
}
func (page *Page) Err(message string) {
fmt.Println(page.FileName + " : " + message)
}
@ -306,11 +294,11 @@ func (page *Page) parseFileHeading(data []byte) ([]string, error) {
if len(data) == 0 {
page.Err("Empty File, skipping")
} else {
if data[0] == '-' {
return page.parseFileMetaData(data)
}
if data[0] == '{' {
return page.parseJsonMetaData(data)
}
return page.parseYamlMetaData(data)
}
return nil, nil
}
@ -352,7 +340,12 @@ func (page *Page) buildPageFromFile() error {
return err
}
switch page.Markup {
case "md":
page.convertMarkdown(content)
case "rst":
page.convertRestructuredText(content)
}
return nil
}
@ -379,3 +372,20 @@ func (page *Page) convertMarkdown(lines []string) {
page.Content = template.HTML(content)
page.Summary = template.HTML(TruncateWordsToWholeSentence(StripHTML(StripShortcodes(content)), summaryLength))
}
func (page *Page) convertRestructuredText(lines []string) {
page.RawMarkdown = strings.Join(lines, " ")
cmd := exec.Command("rst2html.py", "--template=/tmp/template.txt")
cmd.Stdin = strings.NewReader(page.RawMarkdown)
var out bytes.Buffer
cmd.Stdout = &out
if err := cmd.Run(); err != nil {
fmt.Println(err)
}
content := out.String()
page.Content = template.HTML(content)
page.Summary = template.HTML(TruncateWordsToWholeSentence(StripHTML(StripShortcodes(content)), summaryLength))
}

View file

@ -14,10 +14,10 @@
package main
import (
"./hugolib"
"flag"
"fmt"
"github.com/howeyc/fsnotify"
"github.com/spf13/hugo/hugolib"
"net/http"
"os"
"path/filepath"
@ -26,12 +26,12 @@ import (
)
const (
cfgFiledefault = "config.json"
cfgFiledefault = "config.yaml"
)
var (
baseUrl = flag.String("b", "", "hostname (and path) to the root eg. http://spf13.com/")
cfgfile = flag.String("c", cfgFiledefault, "config file (default is path/config.json)")
cfgfile = flag.String("c", cfgFiledefault, "config file (default is path/config.yaml)")
checkMode = flag.Bool("k", false, "analyze content and provide feedback")
draft = flag.Bool("d", false, "include content marked as draft")
help = flag.Bool("h", false, "show this help")