gluonts.core.serde package#

gluonts.core.serde#

Simple Serialization/Deserialization framework that uses human-readable json-encodings to represent data.

In some ways, it is an alternative to pickle, but instead of using a binary encoding, serde outputs json-compatible text. This has two main advantages: It is human-readable and the encoding works across different Python versions.

Values which can be directly represented in json are not changed, thus serde.encode is virtually the same as json.dumps for values such as 1, [“a”, “b”, “c”] or {“foo”: “bar”}.

However, more complex values are encoded differently. For these, we create an object which uses a special __kind__ attribute, which can be one of:

class Kind(str, Enum):
    Type = "type"
    Instance = "instance"
    Stateful = "stateful"

A type is just a path to a class or function, an instance is an object which can be re-constructed by passing arguments to its constructor, and stateful represents objects that are decoded by setting the __dict__ attribute on an otherwise empty instance of the type.

serde.encode uses functools.singledispatch to encode a given object. It is implemented for a variety of existing types, such as named-tuples, paths, and pydantic.BaseModel.

In addition, one can derive from serde.Stateful or serde.Stateless to opt into one of the behaviours. The latter requires that the class supports __getnewargs_ex__ from the pickle protocol.

To encode custom values, one can use the serde.encode.register decorator. For example, support for numpy’s arrays are added by:

@encode.register(np.ndarray)
def encode_np_ndarray(v: np.ndarray) -> Any:
    return {
        "__kind__": Kind.Instance,
        "class": "numpy.array",
        "args": encode([v.tolist(), v.dtype]),
    }

There is no need to implement decode for a given type since the encoding should contain the information on how the object can be constructed.

Similarly to json, serde does not support object-identity. This means that if an object exists twice in an object graph, it is encoded at each of the occurances. Consequently, ciruclar references do not work.

dump_json and load_json are simple helpers, which use encode and decode internally.

class gluonts.core.serde.Eventual(value: T)[source]#

Bases: Generic[T]

set(value: T) → None[source]#
set_default(value: T) → None[source]#
unwrap() → T[source]#
value: T#
class gluonts.core.serde.OrElse(fn: Callable)[source]#

Bases: object

A default field for a dataclass, that uses a function to calculate the value if none was passed.

The function can take arguments, which are other fields of the annotated dataclass, which can also be OrElse fields. In case of circular dependencies an error is thrown.

@serde.dataclass
class X:
    a: int
    b: int = serde.OrElse(lambda a: a + 1)
    c: int = serde.OrEsle(lambda c: c * 2)

x = X(1)
assert x.b == 2
assert x.c == 4

```

fn: Callable#
parameters: dict#
class gluonts.core.serde.Stateful[source]#

Bases: object

class gluonts.core.serde.Stateless(*args, **kwargs)[source]#

Bases: object

gluonts.core.serde.dataclass(cls=None, *, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False)[source]#

Custom dataclass wrapper for serde.

This works similar to the dataclasses.dataclass and pydantic.dataclasses.dataclass decorators. Similar to the pydantic version, this does type checking of arguments during runtime.

gluonts.core.serde.decode(r: Any) → Any[source]#

Decodes a value from an intermediate representation r.

Parameters:

r – An intermediate representation to be decoded.

Returns:

A Python data structure corresponding to the decoded version of r.

Return type:

Any

See also

encode

Inverse function.

gluonts.core.serde.dump_json(o: Any, indent: Optional[int] = None) → str[source]#

Serializes an object to a JSON string.

Parameters:

  • o – The object to serialize.

  • indent – An optional number of spaced to use as an indent.

Returns:

A string representing the object in JSON format.

Return type:

str

See also

load_json

Inverse function.

gluonts.core.serde.encode(v: Any) → Any[source]#
gluonts.core.serde.encode(v: Stateful) → Any
gluonts.core.serde.encode(v: PurePath) → Any
gluonts.core.serde.encode(v: BaseModel) → Any
gluonts.core.serde.encode(v: partial) → Any
gluonts.core.serde.encode(v: dtype) → Any
gluonts.core.serde.encode(v: ndarray) → Any
gluonts.core.serde.encode(v: datetime64) → Any
gluonts.core.serde.encode(v: inexact)
gluonts.core.serde.encode(v: integer)
gluonts.core.serde.encode(v: Timestamp) → Any
gluonts.core.serde.encode(v: Period) → Any
gluonts.core.serde.encode(v: BaseOffset) → Any
gluonts.core.serde.encode(v: Context) → Any
gluonts.core.serde.encode(v: NDArray) → Any
gluonts.core.serde.encode(v: Periods)

Transforms a value v as a serializable intermediate representation (for example, named tuples are encoded as dictionaries). The intermediate representation is then recursively traversed and serialized either as Python code or as JSON string.

This function is decorated with singledispatch() and can be specialized by clients for families of types that are not supported by the basic implementation (explained below).

Examples

The conversion logic implemented by the basic implementation is used as a fallback and is best explained by a series of examples.

Lists (as lists).

>>> encode([1, 2.0, '3'])
[1, 2.0, '3']

Dictionaries (as dictionaries).

>>> encode({'a': 1, 'b': 2.0, 'c': '3'})
{'a': 1, 'b': 2.0, 'c': '3'}

Named tuples (as dictionaries with a '__kind__': <Kind.Instance: 'instance'> member).

>>> from pprint import pprint
>>> from typing import NamedTuple
>>> class ComplexNumber(NamedTuple):
...     x: float = 0.0
...     y: float = 0.0
>>> pprint(encode(ComplexNumber(4.0, 2.0)))
{'__kind__': <Kind.Instance: 'instance'>,
 'class': 'gluonts.core.serde._base.ComplexNumber',
 'kwargs': {'x': 4.0, 'y': 2.0}}

Classes with a validated() initializer (as dictionaries with a '__kind__': <Kind.Instance: 'instance'> member).

>>> from gluonts.core.component import validated
>>> class ComplexNumber:
...     @validated()
...     def __init__(self, x: float = 0.0, y: float = 0.0) -> None:
...         self.x = x
...         self.y = y
>>> pprint(encode(ComplexNumber(4.0, 2.0)))
{'__kind__': <Kind.Instance: 'instance'>,
 'args': [],
 'class': 'gluonts.core.serde._base.ComplexNumber',
 'kwargs': {'x': 4.0, 'y': 2.0}}

Classes with a __getnewargs_ex__ magic method (as dictionaries with a '__kind__': <Kind.Instance: 'instance'> member).

>>> from gluonts.core.component import validated
>>> class ComplexNumber:
...     def __init__(self, x: float = 0.0, y: float = 0.0) -> None:
...         self.x = x
...         self.y = y
...     def __getnewargs_ex__(self):
...         return [], {'x': self.x, 'y': self.y}
>>> pprint(encode(ComplexNumber(4.0, 2.0)))
{'__kind__': <Kind.Instance: 'instance'>,
 'args': [],
 'class': 'gluonts.core.serde._base.ComplexNumber',
 'kwargs': {'x': 4.0, 'y': 2.0}}

Types (as dictionaries with a '__kind__': <Kind.Type: 'type'> member).

>>> encode(ComplexNumber)
{'__kind__': <Kind.Type: 'type'>,
 'class': 'gluonts.core.serde._base.ComplexNumber'}
Parameters:

v – The value to be encoded.

Returns:

An encoding of v that can be serialized to Python code or JSON string.

Return type:

Any

See also

decode

Inverse function.

dump_json

Serializes an object to a JSON string.

gluonts.core.serde.load_json(s: str) → Any[source]#

Deserializes an object from a JSON string.

Parameters:

s – A string representing the object in JSON format.

Returns:

The deserialized object.

Return type:

Any

See also

dump_json

Inverse function.

Submodules#