Showing revision 2

JData specifications: a general-purpose data storage and interchange interchange format

Status of This Document
This document is current under the state of "Request for Comments". It does not specify a standard of any kind. Distribution of this memo is unlimited.

Copyright Notice
Copyright (C) Qianqian Fang (2015).

Version
RFC.pre.0

Last Update
Jan. 02, 2015

Abstract
JData is a general-purpose data interchange format. It was derived from the JavaScript Object Notation (JSON) [RFC 4627] and Universal Binary JSON specifications. JData defines a list of dedicated "name" fields to represent a wide range of data structures, including scalars, arrays, tables, structures, hashes, trees and graphs.

1. Introduction

Data are the digital representations of our world. Generating and processing data are the essential parts of our daily lives, and are at the very foundations of modern sciences, information technologies, businesses, and the functions of our global societies.

Data take many different forms. Some data can be represented by simple scalars; some others have complex forms with hierarchical structures. An efficient representation of data also strongly depends on the application needs. In some cases, plain text files with white-space separated fields are sufficient; however, for performance-sensitive applications, binary formats are often preferred. The abilities to store and parse complex data structures are particularly important to the scientific communities.

It is a challenging task to encapsulate the wide varieties of data forms in a single data interchange format. There has been many previous efforts in designing a general-purpose data storage specification. Some of them have become popular choices in one or multiple categories of applications. Extensible Markup Language (XML), for example, is used as a popular data-exchange format, but the verbosity of the syntax, moderate complexity for parsing, impeded readability and inefficiency in expressing structured data suggested rooms for alternative formats. Comma Separated Value (CSV), a rather simple plain-text format, is used among some applications to exchange a tabular data structure (such as a spreadsheet); yet, its inability to encode more complex data forms, lack of flexibility and precision restrict it to specific applications.

Hierarchical Data Format (HDF) is a format targeting at the broad needs from the scientific communities. It has an extensible hierarchical data model with a large capacity to represent complex binary data. However, to effectively use HDF require skillful implementation and in-depth understanding to the underlying data models. For small projects with non-critical performance needs, using such such a non-intuitive data format may requires additional development maintenance efforts.

1.1. Conventions Used in This Document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119].

The grammatical rules in this document are to be interpreted as described in [RFC 4234].

2. JSON Grammar

JSON grammar used in JData is defined in [RFC 4627].

3. JData Keywords

3.1 Data Array Storage Keywords

In this document, we consider the following general data storage models:
  • a full array (empty, 1D, 2D, 3D or in higher dimensions)
  • a sparse array (empty, 1D, 2D, 3D or in higher dimensions)

 _ArraySize_, _ArrayType_, _ArrayData_,
 _ArrayIsComplex_, _ArrayIsSparse_

3.1 Data Representing Keywords

All JData keywords are case sensitive.

In this document, we define the meanings for the following JData keywords that represents mesh-related data structures:

3.1 Meta-data Keywords

In this document, we defines the following JMesh keywords to represent non-geometric meta-data:

 Author, CreationTime, Comment
Powered by Habitat