MAAP

I wish to implement versioning where I save full data as reference and subsequent changes upon that I wish to implement changing data instances between any two versions of their base class We could define transforms between data that are used both when applying changes to instances due to a detected change in their base class as well as using these transforms to step between versions

Do these transforms only go towards newer versions or both directions?

So to save the first version is easy, I save the whole data. How do I now save changes? I want it unique to certain types So if the base data type is a string then I want to use a custom function to produce the change data If we cant find such a function for that type or if the base data type has changed then we must store a new copy It would be good to go down into a hierarchy structure to determine the highest level in the tree changes So for any runtime editable class that doesnt have its own implementation here, we can test for changes upon the data contained within it

class T():
	tree: int
	nice: str
	poy: list[ T]

So we may start off with an instance:

t= T(
	tree= 4,
	nice= "hit"
	poy= { T( 6, "f", []), T( 2, "r", [])}
)

So this instance may have their tree field change to 9, we could loop over the attributes and do equality checks here we need to store the varName and the new value. So varChange: ( "tree", 9) In order to make this bidirectional we would either need to store both values or we would work out the value prior in a transform chain

So we store no integer data transforms but we do for a string So one changes nice to "hiyt" We loop over and do equality checks and find that the value for nice is unequal. We go to nice and because we can see that str supports data transforms we can query how the nice string has transformed using the string data transform functions. In the string transform functions we find that a y has been added at index 2 maybe i will only do line based string transforms but for this example it could be: addition: ( 2, "y") this is bidirectional we need to store the final result as a walk down the hierarchy so we could use a passthrough name: pass: ( "nice", addition: ( 2, y)) This remains bidirectional The transformation names are unique to the type that is providing them. With the example directly above, it could be infered what the type of nice at the prior stage is from calculating the stack before but this is not bidirectional. Instead we can store the type that provides that transformation so: ( ( "Some.Module", "RuntimeEditable"), "pass", ( "nice", [ ( ( "builtins", int), "addition", ( 2, y))]))

maybe we could also add an option for denying a transform so the above layer just defaults to a sweeping change

As a bigger example, lets say we then change the poy var, handling additions and removals from looking at the before and after is very difficult and I dont want to implement it, it is a very similar problem to the string difference one and so if all elements of the list could be converted to bytes then we could use the ndiff function on a base64 ascii string and then use that arrangement information on the original objects. I think for now a list shouldnt have custom transforms so lets use a set in this example instead

So we add one more T instance and we change an existing one

ADD:
T( 9, "o", [])

CHANGE:
T(
  6, --> 44
  "f",
  []
)

with sets we could loop through the before, if they arent in the after then theyre "notPresent" and ( with the remaining in the before, if there are extra in the after set the those extra are "newlyPresent" So here we would be saying:

( ( "Some.Module", "RuntimeEditable"), [ ( "varDelta", ( "poy", [ ( ( "builtins", "set"), "notPresent", [ T( 6, "f", []), ( ( "builtins", "set"), "newlyPresent", [ T( 9, "o", []), T( 44, "f", [])])])]))])

We can store transforms as a list

operation= tuple[ operationId, operationData]
deltaData= tuple[ type| None, [ operation]]

Like the above example, some

For runtime editables we can loop through attributes and check for ones that notPresent and ones that are newlyPresent and use them similar to the set varChange, notPresent, newlyPresent work on dynamically allocable types, what of non dynamically allocable types?, these all still work as on slotted classes we cannot add non specified attributes but we can still remove existing ones

w So for a type implementing data transforms we need:

• A function that takes two memory instances and produces an ordered list of bidirectional transforms from one to another Transforms be like ( typeSerialisation, transformId, transformData)
• A function that takes an object instance and a transform list and a direction and can transform the given object instance using the transform list and direction

When storing we need to know if we are looking at a list of transforms or if its just a restore of the data We can give an indication of this for every full store, maybe a bool called full store next to the data This frees us up to do all sorts of full store deletions if possible to save space, update latest to full to increase speed, all sorts of cool shit We can solve the ordered list backwards to go ( to -> from)

How will in memory instanciation change

Currently when a code aspect wants a python object in all store implementations, i check if it is in the memory dictionary. If it is then I return that and if it isnt then I load from the store into memory, save it to the dictionary and then return that

I think our loading function should take an optional version id parameter, if it isnt specified then the store should provide the latest version Then we still have a instanciatedData dict, non versioned acts as before but if we are versioned, instead of storing the memory instance as the value, we store another dict which is accessed by the version id

The saving function should have the same interface, it will have to use knowledge about the versioned status contained on the ltsd to determine how to save it, it should have a toggle for including commiter data set to off by default, ( username, pcname)

I think a function on the store to get all of the versions in chronological creation order should be good Other functions which return data related to ltsd need an option to specify version

maybe just do fsjson store first

Extra data on ltsd

Version number, version commit info, special: movement funcs