JData specifications: a general-purpose data storage and interchange interchange format

Status of This Document

This document is current under the state of "Request for Comments". It does not specify a standard of any kind. Distribution of this memo is unlimited.

Copyright Notice

Copyright (C) Qianqian Fang (2015).

Version

RFC.pre.0

Last Update

Jan. 02, 2015

Abstract

JData is a general-purpose data interchange format. It was derived from the JavaScript Object Notation (JSON) [RFC 4627] and Universal Binary JSON specifications. JData defines a list of dedicated "name" fields to represent a wide range of data structures, including scalars, arrays, tables, structures, hashes, trees and graphs.

1. Introduction

Data are the digital representations of our world. Generating and processing data are the essential parts of our daily lives, and are at the very foundations of modern sciences, information technologies, businesses, and the functions of our global societies.

Data take many different forms. Some data can be represented by simple scalars; some others have complex forms with hierarchical structures. An efficient representation of data also strongly depends on the application needs. In some cases, plain text files with white-space separated fields are sufficient; however, for performance-sensitive applications, binary formats are often preferred. The abilities to store and parse complex data structures are particularly important to the scientific communities.

It is a challenging task to encapsulate the wide varieties of data forms in a single data interchange format. There has been many previous efforts in designing a general-purpose data storage specification. Some of them have become popular choices in one or multiple categories of applications. Extensible Markup Language (XML), for example, is used as a popular data-exchange format, but the verbosity of the syntax, moderate complexity for parsing, impeded readability and inefficiency in expressing structured data suggested rooms for alternative formats. Comma Separated Value (CSV), a rather simple plain-text format, is used among some applications to exchange a tabular data structure (such as a spreadsheet); yet, its inability to encode more complex data forms, lack of flexibility and precision restrict it to specific applications.

Hierarchical Data Format (HDF) is a format targeting at the broad needs from the scientific communities. It has an extensible hierarchical data model with a large capacity to represent complex binary data. However, to effectively use HDF require skillful implementation and in-depth understanding to the underlying data models. For small projects with non-critical performance needs, using and advanced data format such as HDF may requires additional development maintenance efforts. Similar arguments can be made to the Common Data Format (CDF) or Network Common Data Format (netCDF) that are partly derived from HDF. In addition, the MATLAB mat-file format and Tecplot data format are also used among the research communities. Because of the requirement of propitiatory software or libraries, these formats also have difficulties to find wide-spread use outside of the user communities of the associated software.

The JavaScript Object Notation (JSON) format is a text-based data format that is known for its capability of storing complex data, excellent portability and human-readability. JSON is widely adopted among modern web applications, and is gradually becoming popular in local applications.

1.1. Conventions Used in This Document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119].

The grammatical rules in this document are to be interpreted as described in [RFC 4234].

2. JSON Grammar

JSON grammar used in JData is defined in [RFC 4627].

3. JData Keywords

3.1 Data Representing Keywords

All JData keywords are case sensitive.

In this document, we define the meanings for the following JData keywords that represents mesh-related data structures:

3.1 Data Array Storage Keywords

In this document, we consider the following general data storage models:

• a full array (empty, 1D, 2D, 3D or in higher dimensions)
• a sparse array (empty, 1D, 2D, 3D or in higher dimensions)

 _ArraySize_, _ArrayType_, _ArrayData_,
 _ArrayIsComplex_, _ArrayIsSparse_

3.1 Meta-data Keywords

In this document, we defines the following JMesh keywords to represent non-geometric meta-data:

 Author, CreationTime, Comment