Hyde Press

Simple • Static • Blog-aware

Jekyll Importers 'n' Syndicators (Book Edition)

by Tom Preston-Werner, Nick Quaranto, Parker Moore, et al

Import your old 'n' busted site or blog for use with Jekyll

Contents

Preface

Note: The book edition is still an early release and a work-in-progess.

This is the importers and syndicators documentation for the Jekyll static site builder / generator reformatted in a single-page book edition.

See the source repo for how the book gets auto-built with "plain" Jekyll - of course - and hosted on GitHub Pages.

Onwards.

Acknowledgments

Thanks to all Jekyll importers and syndicators contributors for making it all possible.

Part 1 - Getting Started

Chapter 1.1 - Welcome

If you’re switching to Jekyll from another blogging system, Jekyll’s importers can help you with the move. Most methods listed on this page require read access to the database from your old system to generate posts for Jekyll. Each method generates .markdown posts in the _posts directory based on the entries in the foreign system.

Other Systems

If you have a system for which there is currently no migrator, consider writing one and sending us a pull request.

Chapter 1.2 - Installation

Because the importers have many of their own dependencies, they are made available via a separate gem called jekyll-import. To use them, all you need to do is install the gem, and they will become available as part of Jekyll’s standard command line interface.

$ gem install jekyll-import
Jekyll-import requires you to manually install some dependencies.

Most importers require one or more dependencies. In order to keep jekyll-import's footprint small, we don't bundle the gem with every plausible dependency. Instead, you will see a nice error message describing any missing dependency and how to install it. If you're especially savvy, take a look at the require_deps method in your chosen importer to install all of the deps in one go.

Chapter 1.3 - Usage

You should now be all set to run the importers with the following incantation:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::MyImporter.run({
      # options for this importer
    })'

Where MyImporter is the name of the specific importer.

Note: Always double-check migrated content

Importers may not distinguish between published or private posts, so you should always check that the content Jekyll generates for you appears as you intended.

Part 2 - Importers A-Z

Chapter 2.1 - Behance

To import your posts from your Behance, generate an API token for your user account and run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Behance.run({
      "user"      => "my_username",
      "api_token" => "my_api_token"
    })'

Both user and api_token are required.

Chapter 2.2 - Blogger

To import your posts from your Blogger, you first have to export the blog to a XML file (blog-MM-DD-YYYY.xml), and run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Blogger.run({
      "source"                => "/path/to/blog-MM-DD-YYYY.xml",
      "no-blogger-info"       => false, # not to leave blogger-URL info (id and old URL) in the front matter
      "replace-internal-link" => false, # replace internal links using the post_url liquid tag.
    })'

The only required field is source. The other fields default to their above values.

“Labels” will be included in export as “Tags”.

Chapter 2.3 - CSV

To import your posts from a CSV file, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::CSV.run({
      "file" => "my_posts.csv"
    })'

Your file CSV file will be read in with the following columns:

  1. title
  2. permalink
  3. body
  4. published_at
  5. filter (e.g. markdown, textile)

If you wish to specify custom front matter for each of your posts, you can use the no-front-matter option to prevent the default front matter from being written to the imported files:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::CSV.run({
      "file" => "my_posts.csv",
      "no-front-matter" => true
    })'

Chapter 2.4 - Drupal 6

To import your posts from a Drupal 6 installation, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Drupal6.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "prefix"   => "mytableprefix",
      "types"    => ["blog", "story", "article"]
    })'

The only required fields are dbname and user. password defaults to "", host defaults to "localhost", and prefix defaults to "".

By default, this will pull in nodes of type blog, story, and article. To specify custom types, you can use the types option when you run the importer:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Drupal6.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "prefix"   => "mytableprefix",
      "types"    => ["blog", "post"]
    })'

That will import nodes of type blog and post only.

Chapter 2.5 - Drupal 7

To import your posts from a Drupal 7 installation, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Drupal7.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "prefix"   => "mytableprefix",
      "types"    => ["blog", "story", "article"]
    })'

The only required fields are dbname and user. password defaults to "", host defaults to "localhost", and prefix defaults to "".

By default, this will pull in nodes of type blog, story, and article. To specify custom types, you can use the types option when you run the importer:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Drupal7.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "prefix"   => "mytableprefix",
      "types"    => ["blog", "post"]
    })'

That will import nodes of type blog and post only.

Chapter 2.6 - EasyBlog

To import your posts from a EasyBlog installation, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Easyblog.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "prefix"   => "mytableprefix"
    })'

The only required fields are dbname and user. password defaults to "", host defaults to "localhost" prefix defaults to "jos_". This will export all articles (in any state). Category and tags will be included in export.

Chapter 2.7 - Enki

To import your posts from a Enki installation, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Enki.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost"
    })'

The only required fields are dbname and user. password defaults to "" and host defaults to "localhost".

Chapter 2.8 - Ghost

To import your posts from your self-hosted Ghost instance, you first have to download your ghost.db from your server and run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Ghost.run({
      "dbfile"   => "/path/to/your/ghost.db"
    })'

There are no required fields. dbfile defaults to "ghost.db".

If you have a Ghost backup file, consider using another tool called jekyll_ghost_importer to import your content. It is a separate gem and docs can be found at the link provided.

Chapter 2.9 - Joomla

To import your posts from a Joomla installation, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Joomla.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "section"  => "thesection",
      "prefix"   => "mytableprefix"
    })'

The only required fields are dbname and user. password defaults to "", host defaults to "localhost", and section defaults to "1" and prefix defaults to "jos_".

Chapter 2.10 - Joomla 3

To import your posts from a Joomla installation, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Joomla.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "section"  => "thesection",
      "prefix"   => "mytableprefix"
    })'

The only required fields are dbname and user. password defaults to "", host defaults to "localhost", and section defaults to "1" and prefix defaults to "jos_".

Chapter 2.11 - Jrnl

To import your posts from Jrnl, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Jrnl.run({
      "file"        => "~/journal.txt",
      "time_format" => "%Y-%m-%d %H:%M",
      "extension"   => "md",
      "layout"      => "post"
    })'

None of the fields are mandatory. The default to the values in the example block above.

Chapter 2.12 - Marley

To import your posts from Marley, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Marley.run({
      "marley_data_dir" => "my_marley_data_dir"
    })'

The marley_data_dir field is required and points to the directory in which your Marley data resides.

Chapter 2.13 - Mephisto

To import your posts from Mephisto, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Mephisto.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost"
    })'

The only required fields are dbname and user. password defaults to "" and host defaults to "localhost".

Chapter 2.14 - Movable Type

To import your posts from Movable Type, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::MT.run({
      "engine"   => "mysql", # "postgres" is also supported
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost",
      "blog_id"   => nil,  # Set to specific ID to iimport just one blog
      "categories" => true, # Set to false to not save categories to front matter
      "src_encoding" => "UTF-8",
      "dest_encoding" => "UTF-8",
      "comments" => true
    })'

Posts will be generated and placed in _posts directory.

The only required fields are dbname and user. password defaults to "" and host defaults to "localhost".

comments, which defaults to false, control the generation of comment. If comments set to true, posts will be generated and placed in _comments directory.

All of the posts and comments will include post_id in YAML front matter to link a post and its comments.

To include imported comments as part of a post, use the yet to merge fork of mt-static-comments to include statically generate comments in your post. Fork and provide feedback if necessary.

Chapter 2.15 - Posterous

To import your posts from Posterous, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Posterous.run({
      "email"     => "myemail",
      "password"  => "mypassword",
      "api_token" => "mytoken"
    })'

All three fields are required.

There is also an alternative Posterous migrator that maintains permalinks and attempts to import images too.

Chapter 2.16 - RSS

To import your posts from an RSS feed (local or remote), run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::RSS.run({
      "source" => "my_file.xml"
    })'

The source field is required and can be either a local file or a remote one.

Chapter 2.17 - S9Y

To import your posts from an S9Y feed, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::S9Y.run({
      "source" => "http://blog.example.com/rss.php?version=2.0&all=1"
    })'

The source field is required.

Chapter 2.18 - Textpattern

To import your posts from Textpattern, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::TextPattern.run({
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost"
    })'

The only required fields are dbname and user. password defaults to "" and host defaults to "localhost".

You will need to run the above from the parent directory of your _import folder. For example, if _import is located in /path/source/_import, you will need to run this code from /path/source. The hostname defaults to localhost, all other variables are required. You may need to adjust the code used to filter entries. Left alone, it will attempt to pull all entries that are live or sticky.

Chapter 2.19 - Tumblr

To import your posts from Tumblr, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Tumblr.run({
      "url"            => "http://myblog.tumblr.com",
      "format"         => "html", # or "md"
      "grab_images"    => false,  # whether to download images as well.
      "add_highlights" => false,  # whether to wrap code blocks (indented 4 spaces) in a Liquid "highlight" tag
      "rewrite_urls"   => false   # whether to write pages that redirect from the old Tumblr paths to the new Jekyll paths
    })'

The only required field is url. The other fields default to their above values.

Chapter 2.20 - Typo

To import your posts from Typo (now Publify), run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Typo.run({
      "server"   => "mysql",
      "dbname"   => "name",
      "user"     => "myuser",
      "password" => "mypassword",
      "host"     => "myhost"
    })'

The only required fields are server, dbname, and user. password defaults to "" and host defaults to "localhost".

This code has only been tested with Typo version 4+.

Chapter 2.21 - WordPress

Install additional gems

To use this importer, you need to install these additional gems: `gem install unidecode sequel mysql2 htmlentities`

To import your posts from a self-hosted WordPress installation, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::WordPress.run({
      "dbname"   => "",
      "user"     => "",
      "password" => "",
      "host"     => "localhost",
      "socket"   => "",
      "table_prefix"   => "wp_",
      "site_prefix"    => "",
      "clean_entities" => true,
      "comments"       => true,
      "categories"     => true,
      "tags"           => true,
      "more_excerpt"   => true,
      "more_anchor"    => true,
      "extension"      => "html",
      "status"         => ["publish"]
    })'

None of the fields are required. Their defaults are as you see above.

This only imports post & page data & content

This importer only converts your posts and creates YAML front-matter. It does not import any layouts, styling, or external files (images, CSS, etc.).

Chapter 2.22 - WordPress.com

To import your posts from a WordPress.com blog, run:

$ ruby -rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::WordpressDotCom.run({
      "source" => "wordpress.xml",
      "no_fetch_images" => false,
      "assets_folder" => "assets"
    })'

The source, no_fetch_images, and assets_folder fields are not required. Their default values are what you see above.

ProTip™: WordPress.com Export Tool

If you are migrating from a WordPress.com account, you can access the export tool at the following URL: https://YOUR-USER-NAME.wordpress.com/wp-admin/export.php.

Further WordPress migration alternatives

While the above method works, it does not import much of the metadata that is usually stored in WordPress posts and pages. If you need to export things like pages, tags, custom fields, image attachments and so on, the following resources might be useful to you:

Part 3 - Syndicators 'n' Aggregators

Chapter 3.1 - Planet

Add articles, blogs to your site via web feeds (and planet pluto).

Usage

Step 1: Planet Pluto Configuration

Use the Planet Pluto machinery to setup your planet and feed list. Example planet.ini:

title = Planet Ruby

[rubyflow]
  title  = Ruby Flow
  link   = http://rubyflow.com
  feed   = http://feeds.feedburner.com/Rubyflow?format=xml

[rubyonrails]
  title = Ruby on Rails Blog
  link  = http://weblog.rubyonrails.org
  feed  = http://weblog.rubyonrails.org/feed/atom.xml

[viennarb]
  title = vienna.rb Blog
  link  = http://vienna-rb.at
  feed  = http://vienna-rb.at/atom.xm

Use the pluto command line tool and pass in the planet configuration. Example:

$ pluto update planet.ini

This will

1) fetch all feeds listed in planet.ini and

2) store all entries in a local database, that is, planet.db in your working folder.

See the Planet Pluto docs for more ».

Step 2: Generate Planet Posts for Jekyll

Use Jekyll Planet Ruby script to generate the blog posts for jekyll. Example:

$ ruby -r 'jekyll/planet' -e 'JekyllPlanet.main'

This will

1) fetch the latest entries from the local database, that is, planet.db and

2) generate a blog story for every entry in _posts/

jekyll-planet/0.2.1 on Ruby 2.1.4 (2014-10-27) [i686-linux]
db settings:
{:adapter=>"sqlite3", :database=>"./planet.db"}
[1] [ANN] Rails 4.2.0.rc3 has been released!
[2] I made a Tic-tac-toe for the Terminal in Ruby
[3] slideshow (S9) RubyGem Update - Adds npm Template Pack Support - $ npm install slideshow-impress.js
[4] Picks / What the vienna.rb Team Thinks Is Worth Sharing This Week
[5] Rack/Rails middleware that will add rel="nofollow" to all your links
...

For example

<p>Happy Friday everyone!</p>

<p>The Rails team has just released <a href="http://rubygems.org/gems/rails/versions/4.2.0.rc3">
the third Release Candidate of Rails 4.2.0</a>
today. For an overview of all the major changes in the 4.2 series, please refer
to the <a href="http://edgeguides.rubyonrails.org/4_2_release_notes.html">release notes</a>.</p>
...

becomes _posts/2014-12-13-ann_rails_420rc3_has_been_released.html:

---
title:      "[ANN] Rails 4.2.0.rc3 has been released!"
created_at: 2014-12-13 03:00:00 UTC
autor:      Ruby on Rails Blog
layout:     post
---
<p>Happy Friday everyone!</p>

<p>The Rails team has just released <a href="http://rubygems.org/gems/rails/versions/4.2.0.rc3">
the third Release Candidate of Rails 4.2.0</a>
today. For an overview of all the major changes in the 4.2 series, please refer
to the <a href="http://edgeguides.rubyonrails.org/4_2_release_notes.html">release notes</a>.</p>
...

Step 3: Use Jekyll to Generate Your Site

Nothing new. Use Jekyll as usual to build your site. Example:

$ jekyll build

That’s it.

Install

Just install the gem:

$ gem install jekyll-planet

Part 4 - Appendix

Chapter 4.1 - Contributing

jekyll-import is entirely open-source, which means we need your help to make it better!

Ran into an Issue?

Found an issue with one of the importers? Sorry about that! In order to better assist you and make sure the problem never happens, again, we would love for you to do a few things:

  1. Collection information about your system: operating system and version, Ruby version, Jekyll version, jekyll-import version.
  2. Which importer are you using? Note this.
  3. Collect the relevant data. This may be data from your database or input file. This will help us diagnose where the issue occurred.
  4. Ensure the --trace option is specified if you’re running jekyll import from the command-line.
  5. Open a new issue describing the four above points, as well as what you expected the outcome of your incantation to be.

You should receive help soon. As always, check out our help repo if you just have a question.

Creating a New Importer

So you have a new system you want to be able to import from? Great! It’s pretty simple to add a new importer to jekyll-import. In this example, we’ll be creating the Blag importer.

First thing’s first: create the file where the importer will go. In this case, that will be lib/jekyll-import/importers/blag.rb. Inside this file, we’ll add this template:

module JekyllImport
  module Importers
    class Blag < Importer
      def self.require_deps
        JekyllImport.require_with_fallback(%w[
          safe_yaml
          mysql2
        ])
      end

      def self.specify_options(c)
        c.option 'dbname', '--dbname DB', 'Database name (default: "")'
        c.option 'user', '--user USER', 'Database user name (default: "")'
        c.option 'password', '--password PW', "Database user's password (default: "")"
        c.option 'host', '--host HOST', 'Database host name (default: "localhost")'
      end

      def self.process(opts)
        options = {
          :dbname   => opts.fetch('dbname', ''),
          :user     => opts.fetch('user', ''),
          :password => opts.fetch('password', ''),
          :host     => opts.fetch('host', '')
        }

        # Do the magic!
      end
    end
  end
end

Let’s go through this quickly.

self.require_deps

This function is called before you run your importer to make sure all the necessary gem dependencies are installed on the user’s system.

self.specify_options

The specify_options function is passed c, which is the Mercenary::Command instance for this importer. It allows you to specify the right options for your importer to be used with the command-line interface for your importer. jekyll-import sets up everything else – just specify these options and you’re golden.

self.process

Where the magic happens! This method should read from your Blag source, then output a Jekyll site.

Optional: self.validate

This function is entirely optional, but allows for some validation of the options. This method allows you to validate the options any way you wish. For example:

def self.validate(opts)
  abort "Specify a username!" if opts["username"].nil?
  abort "Your username must be a number." unless opts["username"].match(/\A\d+\z/)
end

Once you have your importer working (test with script/console), then you’re ready to add documentation. Add your new file: site/_importers/blag.md. Take a look at one of the other importers as an example. You just add basic usage and you’re golden.

All set? Add everything to a branch on your fork of jekyll-import and submit a pull request. Thank you!

Chapter 4.2 - History

0.10.0 / 2016-01-16

0.9.0 / 2015-11-04

0.8.0 / 2015-09-13

0.7.1 / 2015-05-06

0.7.0 / 2015-05-06

0.6.0 / 2015-03-07

Minor Enhancements

Bug Fixes

Development Fixes

0.5.3 / 2014-12-29

Bug Fixes

Site Enhancements

0.5.2 / 2014-12-03

Bug Fixes

0.5.1 / 2014-11-03

Bug Fixes

0.5.0 / 2014-09-19

Minor Enhancements

Development Fixes

0.4.1 / 2014-07-31

Bug Fixes

Site Enhancements

0.4.0 / 2014-06-29

Minor Enhancements

Bug Fixes

Site Enhancements

Development Fixes

0.3.0 / 2014-05-23

Minor Enhancements

Bug Fixes

0.2.0 / 2014-03-16

Major Enhancements

Minor Enhancements

Bug Fixes

Site Enhancements

Development Fixes

0.1.0 / 2013-12-18

Major Enhancements

Minor Enhancements

Bug Fixes

Site Enhancements

Development Fixes