BERT.JS

INTRO

The bert.js module provides JavaScript encoder/decoder for External Term Format used in Erlang distribution protocol. That means your JavaScript applications talk to Erlang natively. This module uses DataView and getUint8, getUint16, getUint32 word accessors which is the fastest way of dealing with binaries in JavaScript. The library is used in Erang and Haskell N2O versions. The size of bert.js is 4654 bytes.

Note that this library is low-level, fast version of BERT encoder/decoder. If you want to generate JavaScript SDK with field names as in Erlang include files, please take RPC as high-level counterpart library that generates wrappers around bert.js and provides more slick JavaScript experience.

Erlang structure that compiles to BERT format:

#io{ code = login, data = { 'Token', 42 } }

Target structure of bert.js — fast low-level handcoded parser from BERT format.

{ t: 104, v: [ { t: 100, v: "io" }, { t: 100, v: "login" }, { t: 104, v: [ { t: 100, v: "Token" }, { t: 97, v: 42 } ] } ] }

Target structure of generated parser with RPC parse transform:

{ tup: 'io', code: 'login', data: { tup: '$', 0: 'Token', 1: 42 } },

The following practical subset of BERT format is supported:

70

IEEE-754 float encoding. A float is stored as 8 bytes in big-endian IEEE format. This term is used in minor version 1 of the external format.

1 8 70 IEEE-754 float

97

Byte encoding. Unsigned 8-bit integer.

1 1 97 Int

98

Integer encoding. Signed 32-bit integer in big-endian format.

1 4 98 Int

99

Float POSIX encoding. A float is stored in string format. The format used in sprintf to format the float is "%.20e" (there are more bytes allocated than necessary). To unpack the float, use sscanf with format "%lf".

1 31 99 Float POSIX string.

100

Atom encoding. An atom is stored with a 2 byte unsigned length in big-endian order, followed by N numbers of 8-bit Latin-1 characters that forms the name. The maximum allowed value for Len is 255.

1 2 N 100 N Characters

104

Tuple encoding. The Arity field is an unsigned byte N that determines how many elements that follows in section Elements.

1 1 N 104 N Elements

105

Large tuple encoding. Same as 104 except that N is an unsigned 4 byte integer in big-endian format.

1 4 N 105 N Elements

106

Nil.

1 106

107

String encoding. As field N is an unsigned 2 byte integer (big-endian), implementations must ensure that lists longer than 65535 elements are encoded as 108.

1 2 N 107 N Characters

108

List encoding. N is the number of elements that follows in section Characters. Tail is the final tail of the list; it is 106 for a proper list, but can be any type if the list is improper (for example, [a|b]).

1 4 N 108 N Characters Tail

109

Binary encoding. Binaries are generated with bit syntax expression or with erlang:list_to_binary/1, erlang:term_to_binary/1, or as input from binary ports. The N length field is an unsigned 4 byte integer (big-endian).

1 4 N 109 N Bytes

110

Small bignum encoding. Bignums are stored in unary form with a Sign byte, that is, 0 if the binum is positive and 1 if it is negative. The digits are stored with the least significant byte stored first. To calculate the integer, the following formula can be used:

1 1 1 n 110 n Sign d(0)...d(n-1)

B = 256,
d0*B0 + d1*B1 + d2*B2 + ... dN-1*B(n-1)

111

Large bignum encoding. Same as 110 except that the length field is an unsigned 4 byte integer.

1 4 1 n 111 n Sign d(0)...d(n-1)

115

Small atom encoding. An atom is stored with a 1 byte unsigned length, followed by N numbers of 8-bit Latin-1 characters that forms the Characters.

1 1 N 115 N Characters

116

Maps encoding. The N field is an unsigned 4 byte integer in big-endian format that determines the number of key-value pairs in the map. Key and value pairs (Ki => Vi) are encoded in section Pairs in the following order: K1, V1, K2, V2,..., Kn, Vn. Duplicate keys are not allowed within the same map. Requires OTP 17.

1 4 N 116 N K1,V1,K2,V2,...

118

UTF8 atom encoding. An atom is stored with a 2 byte unsigned length in big-endian order, followed by N bytes containing the Characters encoded in UTF-8.

1 2 N 118 N Characters

119

UTF8 small atom encoding. An atom is stored with a 1 byte unsigned length, followed by N bytes containing the Characters encoded in UTF-8. Longer atoms encoded in UTF-8 can be represented using 118.

1 1 N 119 N Characters

API

enc(json)

Encodes internal JSON to binary buffer.

> enc({t: 119, v: "日本"}) Uint8Array([131,119,6,230,151,165,230,156,172])

dec(buffer)

Decodes binary buffer to internal JSON.

> dec((new Uint8Array([131,119,6,230,151,165,230,156,172])).buffer) {t: 119, v: "日本"}

bin(x)

> dec(enc(bin('N2O,')).buffer)) {t: 109, v: "N2O,"} [131,109,0,0,0,4,78,50,79,44]

Creates JSON for binary encoding.

atom(x)

Creates JSON for Latin-1 atom encoding.

> dec(enc(atom('ok')).buffer)) {t: 100, v: "ok"} [131,100,0,2,111,107]

string(x)

Creates JSON for string encoding.

> dec(enc(string('ok')).buffer)) {t: 107, v: "ok"} [131,107,0,2,111,107]

float(x)

Creates JSON for IEEE-754 float encoding.

> dec(enc(float('123.13')).buffer) {t: 70, v: 123.13} [131,70,64,94,200,81,235,133,30,184]

number(x)

Creates JSON for integers and GMP big numbers encoding.

> dec(enc(number('1')).buffer) {t: 97, v: 1} [131,97,1] > dec(enc(number('100000000')).buffer) {t: 98, v: 100000000} [131,98,5,245,225,0] > dec(enc(number('10000000000000000000000')).buffer) {t: 110, v: 1e+22} [131,110,10,0,0,0,64,178,186,201,224,25,30,2]

list(x,...)

Creates JSON for list encoding.

> dec(enc(list(atom('1'),number('1'),bin('1'))).buffer) {t: 108, v: [{t: 100, v: "1"}, {t: 97, v: 1}, {t: 109, v: "1"}]} [131,108,0,0,0,3,100,0,1,49,97,1,109,0,0,0,1,49,106]

tuple(x,...)

Creates JSON for tuple encoding.

> dec(enc(tuple(atom('1'),number('1'),bin('1'))).buffer) {t: 104, v: [{t: 100, v: "1"}, {t: 97, v: 1}, {t: 109, v: "1"}]} [131,104,3,100,0,1,49,97,1,109,0,0,0,1,49]

map(x,...)

Creates JSON for map encoding.

> dec(enc(map( {k:bin('rent'),v:float(1.2)}, {k:atom('ok'), v:list(number(1),float(1.0),bin('1'))})).buffer) {t:116, v:[{k:{t:109,v:"rent"},v:{t:70, v:1.2}}, {k:{t:100,v:"ok"}, v:{t:108,v:[{t:97, v:1}, {t:70, v:1}, {t:109,v:"1"}]}}]} [131,116,0,0,0,2,109,0,0,0,4,114,101,110, 116,70,63,243,51,51,51,51,51,51,100,0,2, 111,107,108,0,0,0,3,97,1,70,63,240,0,0,0, 0,0,0,109,0,0,0,1,49,106]

You may also want to read: utf8.js, ieee754.js, heart.js, nitro.js, mq.js, n2o.js.