Serde version

Versioning support for serde.

When software are developped and used at the same time the data formats may change from one version to another and persisting data may be produced by a specific version and loaded by another version.

Serde version provide a versioning feature for serde for the main use cases.

Note 1: Requires the specialization feature. Note 2: Use the derive feature to generate the DeserializeVersioned implementation

We aim at solving the case were a type or a set of types in a deserializer's data needs to be upgraded to their latest format. This is the case when a mandatory property was added or removed, or an existing property changed.

Note: There already is support for added optional properties in serde. (Use the default feature of serde)

Example: Let's have a file containing these structure with those version number: A: 1, B: 1, C: 2 and the current version numbers are: A: 3, B: 2, C: 4.

Then in latest code version, we have the former data structures versions, let's call them: Av1, Av2, Bv1, Cv1, Cv2, Cv3.

Deserializing, whenever a structure A, B or C is ran into, then it is loaded with the appropriate format (in our case it will be Av1, Bv1 and Cv2) and then converted to A, B or C using the From trait.

Non goals

This is based on types that can be upgraded individually. Types that needs to be upgraded together is way more complex to handle and usually relies on domain specific deserializer.

So, these data format should be handle with specific Deserialize traits implementations.

Guide

The way serde-version works is pretty simple.

The best practice is to have a version header which contains enough data to know which version to use for each type to deserialize.

There are several strategies to handle this situation.

At the lowest level, serde-version expect to have a VersionMap with a version number for each deserialized type. See Versioned Types or the versioned_types example.

But you can also choose to define a unique version for a set of types. We name this a version group defined by a version uri. See Versioned Groups or the versioned_groups example.

Versioned groups

Motivation

Types barely change alone, usually a set of types will change during the release of a software. So keeping track of a single version number that will define the schema to use for a set of types.

Version group and uris

The version group is identified by a uri (VersionGroupURI). This uri is defined by an api group and a version, like "api_group:version"

You can also define an enum that maps to the appropriate uri with version_group_enum.

Resolving the version groups

When deserializing with serde-version, you need to provide a way to find the version group for each version uri defined in the version header.

This is the purpose of the VersionGroupResolver.

You can define one statically with the macro version_group_resolver_static.

See the versioned_group example.

Example in Toml

# Version header with 2 version uris
v = ["org.my.company:1.0.2", "org.my.plugin:1.3.2"] 

[config]
name = "my config name"

[my_plugin]
plugin_name = "plugin name"

Versioned types

In order to deserialize properly, serde-version expect a VersionMap with a version number for each deserialized types. (If a version is not defined, then the default deserialization occurs).

You can use the macros version_map_new! and version_map_static! to help you create VersionMap.

Deriving `DeserializeVersioned`

You can generate the implementation of DeserializeVersioned with the derive macro.


#![allow(unused_variables)]
fn main() {
// Only Deserialize is required for previous version
#[derive(Deserialize)]
#[serde(rename(deserialize = "A"))]
struct Av1 {
    a: u8,
}

#[derive(Deserialize)]
#[serde(rename(deserialize = "A"))]
struct Av2 {
    b: u8,
}

// This type has 3 version:
// - 1 = Av1
// - 3 = Av2
// - 4 = current
#[derive(Deserialize, PartialEq, DeserializeVersioned, Debug)]
#[serde(rename(deserialize = "A"))]
#[versions(
    v(index = 1, type = "Av1"),
    version(index = 3, type = "Av2", default),
    v(index = 4, self)
)]
struct A {
    c: u8,
}
}

Unsupported Serde feature with versioning

`deserialize_in_place` is not supported

Deserializing in place with versioning support is way more complicated, so we don't deal with this in this crate.

Not supported with `deserialize_with` callback

You must take care of the versioning in your callback

Versioning is only supported for structs and enums

There is no use case where versioning tuples and the unit type is useful.

Design

Versioned deserialization

A type can be composed of multiple other types and each of those can be versioned. Thus we need to know how to migrate any types encountered during the deserialization.

In the serde model, there 4 case were we actually want to migrate the type:

when deserializing an element in a sequence
when deserializing a key of a map
when deserializing a value of a map
when deserializing a variant of an enum

So, we introduce a new trait DeserializeVersioned to handle those cases. DeserializeVersioned::next_element, DeserializeVersioned::next_value, DeserializeVersioned::next_key, DeserializeVersioned::variant are the functions that will handles thoses case.

The trait have a default implementation which is the serde implementation without versioning.

To have the actual implementation, we use a derive macro that will implement those functions as a specialization.

Versioned groups

During software development, we barely version a single type, usually a set of types are versioned together. It makes simpler to declare the versioned used of a file.

So instead of declaring the version number of each type, we can instead define a version group id and deduce the version number of a set of types.

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

README for crates.io

0.5.0

Added

API to aggregate version maps from multiple version uris into a single version map
API to easily deserialize toml with versioning support (feature gate toml-support)

Changed

The VersionMap is now used as a generic parameter and not as &dyn VersionMap.
VersionMap new requires Clone. Users should provide a reference instead of an owned type when providing a VersionMap to avoid cloning during the deserialization.

0.4.2

Changed

Refactored travis CI

0.4.1

Changed

Refactored documentation files and travis CI

0.4.0

Added

Version group apis: macros and utilities to defined version maps, version groups and version uris.

Changed

The VersionMap is now used as a &dyn VersionMap and not a generic parameter in types.

0.3.1

Changed

To define the version for the current type, use self attribute instead of type = "path to self"
Use the type_name of a type as the key to find its version.

0.3.0

Added

Added changelog
The v attribute is an alias for the version attribute.
Added the versioned group pattern with example

Changed

The version attribute now requires an explicit index.
The versions attribute now requires an entry for the current version.
Use a trait for VersionMap instead of a HashMap<String, usize>

Removed

The #[versions("Av1", "Av2")] syntax is not supported anymore, instead use the more explicit version #[versions(v(index = 1, type = "Av1"), v(index = 2, type = "Av2"))]

0.2.3 - 2019-09-08

Added

Added an InvalidVersionError when the version number provided is not handled by current code.

0.2.2 - 2019-09-08

Added

Added proper README file

0.1.0 - 2019-09-06

Added

Versioning feature for serde with additional trait DeserializeVersioned and deserializer VersionedDeserialized

Serde Version Manual

0.2.3 - 2019-09-08

0.2.2 - 2019-09-08

0.1.0 - 2019-09-06