« 25 Days of Ruby Gems - Ruby Advent Calendar 2020, December 1st - December 25th

Day 23 - json-next Gem - Read Next Generation JSON Versions (HanSON, SON, JSONX/JSON11, Etc) with Comments, Unquoted Keys, Multi-Line Strings, Trailing Commas, Optional Commas, and More

Written by geraldb Gerald Bauer

A code monkey and enthusiastic collector (and publisher) of open football and beer data. Skills include Ruby, SQLite and CSV. Spec lead of CSV <3 JSON.

Let’s start with a quiz:

Q: What’s your favorite text (“human”/non-binary) data exchange format?

If you picked (A) JSON - let’s continue with a question:

What’s missing in JSON?

  1. Comments, Comments, Comments
  2. Unquoted Keys
  3. Multi-Line Strings
    • a) Folded – Folds Newlines
    • b) Unfolded
  4. Trailing Commas in Arrays and Objects

More:

Discuss - Fixing JSON - Is it too late?

Fixing JSON. Request for comments, please!

We can easily agree on what’s wrong with JSON, and I can’t help wondering if it’d be worth fixing it.

– Tim Bray (Fixing JSON)

XML already does everything JSON does! And there’s no way to differentiate between nodes and attributes! And there are no namespaces! And no schemas! What’s the point of JSON?

– Anonymous

We need to fix engineers that try to ‘fix JSON’, absolutely nothing is broken with JSON.

– Anonymous

And what’s your take? Enter the json-next gem.

What’s the json-next gem?

The json-next gem lets you convert and read (parse) next generation json versions including: HanSON e.g. HANSON.parse, SON e.g. SON.parse, JSONX e.g. JSONX.parse.

HanSON

HanSON - JSON for Humans by Tim Jansen et al

HanSON is an extension of JSON with a few simple additions to the spec:

Example:

{
  listName: "Sesame Street Monsters", // note that listName needs no quotes
  content: [
    {
      name: "Cookie Monster",
      /* Note the template quotes and unescaped regular quotes in the next string */
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    // don't worry, the trailing comma will be ignored
   ]
}

Use HANSON.convert to convert HanSON text to ye old’ JSON text:

{
  "listName": "Sesame Street Monsters",
  "content": [
    { "name": "Cookie Monster",
       "background": "Cookie Monster used to be a\n ... to eat \"healthy\" food."
    },
    { "name": "Herry Monster",
      "background": "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Use HANSON.parse instead of JSON.parse to parse text to ruby hash / array / etc.:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

SON

SON - Simple Object Notation by Aleksander Gurin et al

Simple data format similar to JSON, but with some minor changes:

JSON is compatible with SON in a sense that JSON data is also SON data, but not vise versa.

Example:

{
  # Personal information

  "name": "Alexander Grothendieck"
  "fields": "mathematics"
  "main_topics": [
    "Etale cohomology"
    "Motives"
    "Topos theory"
    "Schemes"
  ]
  "numbers": [1 2 3 4]
  "mixed": [1.1 -2 true false null]
}

Use SON.convert to convert SON text to ye old’ JSON text:

{
  "name": "Alexander Grothendieck",
  "fields": "mathematics",
  "main_topics": [
    "Etale cohomology",
    "Motives",
    "Topos theory",
    "Schemes"
  ],
  "numbers": [1, 2, 3, 4],
  "mixed": [1.1, -2, true, false, null]
}

Use SON.parse instead of JSON.parse to parse text to ruby hash / array / etc.:


{
  "name" => "Alexander Grothendieck",
  "fields" => "mathematics",
  "main_topics" =>
    ["Etale cohomology", "Motives", "Topos theory", "Schemes"],
  "numbers" => [1, 2, 3, 4],
  "mixed" => [1.1, -2, true, false, nil]
}

JSONX

JSON with Extensions or JSON v1.1 (a.k.a. JSON11 or JSON XI or JSON II)

Includes all JSON extensions from HanSON:

Plus all JSON extensions from SON:

Plus some more extra JSON extensions:

Example:

{
  #  use shell-like (or ruby-like) comments

  listName: "Sesame Street Monsters"   # note: comments after key-value pairs are optional
  content: [
    {
      name: "Cookie Monster"
      // note: the template quotes and unescaped regular quotes in the next string
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    /* don't worry, the trailing comma will be ignored  */
   ]
}

Use JSONX.convert (or JSONXI.convert or JSON11.convert or JSONII.convert) to convert JSONX text to ye old’ JSON text:

{
  "listName": "Sesame Street Monsters",
  "content": [
    { "name": "Cookie Monster",
       "background": "Cookie Monster used to be a\n ... to eat \"healthy\" food."
    },
    { "name": "Herry Monster",
      "background": "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Use JSONX.parse (or JSONXI.parse or JSON11.parse or JSONII.parse) instead of JSON.parse to parse text to ruby hash / array / etc.:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Live Examples

require 'json/next'

text1 =<<TXT
{
  listName: "Sesame Street Monsters", // note that listName needs no quotes
  content: [
    {
      name: "Cookie Monster",
      /* Note the template quotes and unescaped regular quotes in the next string */
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    // don't worry, the trailing comma will be ignored
   ]
}
TXT

pp HANSON.parse( text1 )  # note: is the same as JSON.parse( HANSON.convert( text ))

resulting in:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

and


text2 =<<TXT
{
  # Personal information

  "name": "Alexander Grothendieck"
  "fields": "mathematics"
  "main_topics": [
    "Etale cohomology"
    "Motives"
    "Topos theory"
    "Schemes"
  ]
  "numbers": [1 2 3 4]
  "mixed": [1.1 -2 true false null]
}
TXT

pp SON.parse( text2 )  # note: is the same as JSON.parse( SON.convert( text ))

resulting in:

{
  "name" => "Alexander Grothendieck",
  "fields" => "mathematics",
  "main_topics" =>
    ["Etale cohomology", "Motives", "Topos theory", "Schemes"],
  "numbers" => [1, 2, 3, 4],
  "mixed" => [1.1, -2, true, false, nil]
}

and

text3 =<<TXT
{
  #  use shell-like (or ruby-like) comments

  listName: "Sesame Street Monsters"   # note: comments after key-value pairs are optional
  content: [
    {
      name: "Cookie Monster"
      // note: the template quotes and unescaped regular quotes in the next string
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    /* don't worry, the trailing comma will be ignored  */
   ]
}
TXT

pp JSONX.parse( text3 )   # note: is the same as JSON.parse( JSONX.convert( text ))
pp JSONXI.parse( text3 )  # note: is the same as JSON.parse( JSONXI.convert( text ))
pp JSON11.parse( text3 )  # note: is the same as JSON.parse( JSON11.convert( text ))
pp JSONII.parse( text3 )  # note: is the same as JSON.parse( JSONII.convert( text ))

resulting in:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Bonus: More JSON Formats

See the Awesome JSON - What’s Next? collection / page.

Find out more

References

Built with Ruby (running Jekyll) on 2021-07-25 15:15:02 +0000 in 0.371 seconds.
Hosted on GitHub Pages. </> Source on GitHub. (0) Dedicated to the public domain.