Add how-to guide for storage data migration

[leighmcculloch]

What problem does your feature solve?

When contracts upgrade and their data structures change (e.g., adding new fields), developers need clear guidance on how to handle existing stored data. The intuitive approaches don't work as expected, and developers often discover this the hard way.

Intuitive approach 1 - Just read with the new type:

#[contracttype]
pub struct DataV1 { a: i64, b: i64 }

#[contracttype]
pub struct DataV2 { a: i64, b: i64, c: Option<i64> }

// Developer expects c = None for old entries
let data: DataV2 = env.storage().persistent().get(&key).unwrap();
// TRAPS with Error(Object, UnexpectedSize)

Intuitive approach 2 - Use try_from_val fallback:

let raw: Val = env.storage().persistent().get(&key).unwrap();
if let Ok(v2) = DataV2::try_from_val(&env, &raw) {
    v2
} else {
    // Never reached - host traps before returning Err
    let v1 = DataV1::try_from_val(&env, &raw).unwrap();
    DataV2 { a: v1.a, b: v1.b, c: None }
}

Both approaches trap at the host level before the SDK can handle the mismatch. There is no documentation explaining this behavior or the correct pattern to use.

What would you like to see?

A new how-to guide added to https://developers.stellar.org/docs/build/guides/storage explaining storage data migration using the version marker pattern:

#[contracttype]
pub struct DataV1 { a: i64, b: i64 }

#[contracttype]
pub struct DataV2 { a: i64, b: i64, c: Option<i64> }

#[contracttype]
pub enum DataKey {
    DataVersion(u32),  // version marker keyed by id
    Data(u32),         // data keyed by id
}

fn read_data(env: &Env, id: u32) -> DataV2 {
    let version: u32 = env.storage().persistent()
        .get(&DataKey::DataVersion(id))
        .unwrap_or(1);  // default to v1 for entries without version marker
    
    match version {
        1 => {
            let v1: DataV1 = env.storage().persistent().get(&DataKey::Data(id)).unwrap();
            DataV2 { a: v1.a, b: v1.b, c: None }
        }
        _ => env.storage().persistent().get(&DataKey::Data(id)).unwrap(),
    }
}

fn write_data(env: &Env, id: u32, data: &DataV2) {
    env.storage().persistent().set(&DataKey::DataVersion(id), &2u32);
    env.storage().persistent().set(&DataKey::Data(id), data);
}

The guide should cover:

1. Why intuitive approaches fail - Host validates field count before SDK can handle mismatches
2. Version marker pattern - Store version alongside data, check before reading
3. Lazy vs eager migration - Convert on read (lazy) vs batch migration (eager), noting that lazy migration is generally preferred on blockchains since batch migrations can be prohibitively expensive or hit resource limits
4. Testing migrations - How to test upgrade paths

What alternatives are there?

• Developers currently discover this through trial and error

Related:

• Improving Storage Migration with Changes to `map_new_from_slices`, `map_unpack_to_slice`, and `contracttype` stellar-protocol#1877

[tupui]

Nice snippet, that would help 👍

Are you considering adding some helpers directly into the SDK? If we want to push some best practices that could help. All data entries are versioned (with the ledger) so if we were to register converters to go from v1 to v2 to vn, and make them ledger aware, that could be abstracted away. (Sort of the Stripe way with their API and forward compatibility policy.)

(I still think we need another way to migrate fully the data if wanted, a DB migration but that would have to be able to run on the side and not pollute the contract. A migration contract that can access the same data. I think my ideal solution would look like the upgrade function which takes a wasm. It would only run once atomically during the upgrade.)

[khomiakmaxim]

@leighmcculloch, I see that it's possible to use the following snippet as well:

#[contracttype]
pub enum Data {
    V1(DataV1),
    V2(DataV2),
}

impl Data {
    pub fn into_v2(self) -> DataV2 {
        match self {
            Data::V1(v1) => DataV2 { a: v1.a, b: v1.b, c: None },
            Data::V2(v2) => v2,
        }
    }
}

#[contracttype]
pub enum DataKey {
    Data(u64),
}

#[contractimpl]
impl Contract {
    pub fn write_data(e: Env, id: u64, data: DataV2) {
        e.storage().instance().set(&DataKey::Data(id), &Data::V2(data));
    }

    pub fn read_data(e: Env, id: u64) -> Option<DataV2> {
        let data_enum: Data = e.storage().instance().get(&DataKey::Data(id))?;

        Some(data_enum.into_v2())
    }

    pub fn upgrade(e: Env, new_wasm_hash: BytesN<32>) {
        e.deployer().update_current_contract_wasm(new_wasm_hash);
    }
}

[leighmcculloch]

Yes that pattern is also a good one. It requires thinking ahead to use an enum wrapper around each type. But this approach should be documented as well.