Skip to content

Latest commit

Β 

History

History
645 lines (475 loc) Β· 8.07 KB

File metadata and controls

645 lines (475 loc) Β· 8.07 KB

tinyjson Roadmap

🎯 Version 1 β€” Primitive JSON Parser

Goal: Parse valid JSON containing only primitive values.

Supported:

null
true
false
numbers
strings

Not supported yet:

arrays []
objects {}

Milestone 1 β€” Project foundation

tinyjson/
β”œβ”€β”€ CMakeLists.txt
β”œβ”€β”€ Makefile
β”œβ”€β”€ build.sh
β”œβ”€β”€ include/
β”‚   └── tinyjson/
β”œβ”€β”€ src/
└── tests/

Tasks:

  • Set up CMake
  • Set up Makefile
  • Set up build script
  • Set up test framework/simple test runner
  • Establish include/ and src/ structure

Milestone 2 β€” Character Scanner

Understand and implement the basic JSON scanner.

Example:

const char *json = "{\"temperature\": 28.5}";
const char *current = json;

Tasks:

  • Walk through characters
  • Detect '\0'
  • Advance pointer
  • Read current character
  • Skip whitespace

You'll have:

skip_whitespace()

Milestone 3 β€” Tokenizer

Turn characters into tokens.

Your tokenizer.h will contain:

TokenType
Token
tokenizer_next()

Tasks:

  • {
  • }
  • [
  • ]
  • :
  • ,
  • strings
  • numbers
  • true
  • false
  • null
  • EOF
  • error

For v1, arrays and objects can be recognized as tokens but the parser does not need to support them yet.

Example:

"hello"
   ↓
TOKEN_STRING

123.45
   ↓
TOKEN_NUMBER

true
   ↓
TOKEN_TRUE

null
   ↓
TOKEN_NULL

Milestone 4 β€” String Parsing

Implement JSON string handling.

Examples:

"hello"
"hello world"

Eventually:

"hello\nworld"

Tasks:

  • Find opening "
  • Find closing "
  • Calculate string length
  • Handle escaped quotes
  • Handle basic escape sequences
  • Detect unterminated strings

Milestone 5 β€” Number Parsing

Support JSON numbers.

Examples:

0
10
-10
3.14
-42.5
1e10
-2.5e-3

Tasks:

  • Positive integers
  • Negative numbers
  • Decimal numbers
  • Exponents
  • Reject invalid numbers

Eventually the tokenizer should produce:

Token token;

with:

token.type == TOKEN_NUMBER

and the parser can convert the token to:

double

Milestone 6 β€” Literals

Implement:

true
false
null

For example:

true
 ↓
TOKEN_TRUE
false
 ↓
TOKEN_FALSE
null
 ↓
TOKEN_NULL

Also detect invalid input:

tru
fals
nul

Milestone 7 β€” JsonValue

Create the internal representation for primitive JSON values.

Your current design is a good starting point:

typedef enum
{
    JSON_NULL,
    JSON_BOOL,
    JSON_NUMBER,
    JSON_STRING,
    JSON_ARRAY,
    JSON_OBJECT
} JsonType;

For v1, the important types are:

JSON_NULL
JSON_BOOL
JSON_NUMBER
JSON_STRING

Then:

typedef struct JsonValue
{
    JsonType type;

    union
    {
        int boolean;
        double number;
        char *string;
    } data;

} JsonValue;

We can leave JSON_ARRAY and JSON_OBJECT in the enum as reserved for v2, or remove them from the v1 API until v2.


Milestone 8 β€” Parser

Now connect everything:

JSON
 ↓
Tokenizer
 ↓
Tokens
 ↓
Parser
 ↓
JsonValue

The parser should understand:

null    β†’ JsonValue(JSON_NULL)
true    β†’ JsonValue(JSON_BOOL)
false   β†’ JsonValue(JSON_BOOL)
123.4   β†’ JsonValue(JSON_NUMBER)
"hello" β†’ JsonValue(JSON_STRING)

Implement:

parse_value()

This becomes the central function.

Conceptually:

switch (token.type)
{
    case TOKEN_NULL:
        // create JSON_NULL

    case TOKEN_TRUE:
    case TOKEN_FALSE:
        // create JSON_BOOL

    case TOKEN_NUMBER:
        // create JSON_NUMBER

    case TOKEN_STRING:
        // create JSON_STRING

    default:
        // error
}

Milestone 9 β€” Memory Management

Strings require dynamic memory.

Implement things like:

json_value_free()

For example:

JsonValue
   β”‚
   β”œβ”€β”€ type = JSON_STRING
   β”‚
   └── string β†’ "hello"

When finished:

json_value_free(&value);

should release anything allocated by the parser.

This becomes especially important in v2 when arrays and objects introduce much more dynamic memory.


Milestone 10 β€” Error Handling

Make invalid JSON fail cleanly.

Examples:

"hello
12.3.4
tru
nullx

You could eventually have:

typedef enum
{
    JSON_SUCCESS,
    JSON_ERROR_UNEXPECTED_TOKEN,
    JSON_ERROR_INVALID_NUMBER,
    JSON_ERROR_INVALID_STRING,
    JSON_ERROR_UNEXPECTED_END
} JsonError;

Milestone 11 β€” Tests

Build tests around each component.

Scanner tests

whitespace
empty input
character traversal

Tokenizer tests

{}
[]
:
,
true
false
null
123
"hello"

Parser tests

null
true
123.45
"hello"

And invalid JSON:

tru
"hello
12.3.4

πŸš€ Version 1 Definition of Done

I'd define tinyjson v1 as:

                  tinyjson v1
                      β”‚
             β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”
             β”‚                 β”‚
         Tokenizer           Parser
             β”‚                 β”‚
             β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                      ↓
                  JsonValue
                      β”‚
          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
          ↓           ↓           ↓
        null        bool       number
                                  β”‚
                               string

It should be able to:

json_parse("null");
json_parse("true");
json_parse("false");
json_parse("123.45");
json_parse("\"hello\"");

and reject malformed primitive JSON.


πŸ—οΈ Version 2 β€” Compound JSON

This is where your non-primitive types come in.

Milestone 12 β€” Arrays

Support:

[]
[1, 2, 3]
[true, false, null]

Eventually:

[1, "hello", true, null]

And later nested structures:

[[1, 2], [3, 4]]

Milestone 13 β€” Objects

Support:

{}
{"name": "John"}
{
    "name": "John",
    "age": 25
}

Milestone 14 β€” Nested Values

Support:

{
    "name": "John",
    "scores": [10, 20, 30]
}

and:

[
    {"name": "John"},
    {"name": "Jane"}
]

This is where the recursive nature of JSON becomes important.


Milestone 15 β€” Complete Memory Management

Handle freeing:

JsonValue
   β”‚
   β”œβ”€β”€ string
   β”‚
   β”œβ”€β”€ array
   β”‚    β”œβ”€β”€ JsonValue
   β”‚    β”œβ”€β”€ JsonValue
   β”‚    └── JsonValue
   β”‚
   └── object
        β”œβ”€β”€ key β†’ JsonValue
        β”œβ”€β”€ key β†’ JsonValue
        └── key β†’ JsonValue

json_value_free() will need to recursively free everything.


🌟 Version 3 β€” Quality & Features

After v2 works, you can consider:

v3
β”œβ”€β”€ Better error messages
β”œβ”€β”€ Line/column error locations
β”œβ”€β”€ UTF-8 / Unicode handling
β”œβ”€β”€ More escape sequences
β”œβ”€β”€ Pretty printing
β”œβ”€β”€ JSON serialization
β”œβ”€β”€ Streaming parser
β”œβ”€β”€ Custom allocators
└── Performance improvements

The overall journey

                    tinyjson
                       β”‚
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚                     β”‚
           V1                    V2
            β”‚                     β”‚
     Primitive JSON        Compound JSON
            β”‚                     β”‚
     β”Œβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”        β”Œβ”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”
     ↓      ↓      ↓        ↓           ↓
    null   bool   number   arrays     objects
                   β”‚          β”‚           β”‚
                   ↓          β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜
                 string              ↓
                              nested JSON