diff --git a/FPM.ftd b/FPM.ftd index 151c1e7f6..d8ca62593 100644 --- a/FPM.ftd +++ b/FPM.ftd @@ -55,6 +55,9 @@ implements: fifthtry.github.io/theme - `fpm diff`: /diff/ - `fpm update`: /update/ - `fpm serve`: /serve/ + - `fpm add`: /cli/add/ + - `fpm rm`: /cli/rm/ + - `fpm edit`: /cli/edit/ diff --git a/cli/add.ftd b/cli/add.ftd new file mode 100644 index 000000000..ef7d9fc00 --- /dev/null +++ b/cli/add.ftd @@ -0,0 +1,29 @@ +-- ft.page: `fpm add ` + + +This would create entry in workspace.ftd and only the files added would be +consider for sync. + +-- ft.h1: `fpm add -//` + +This command should check that a file in cr `-//` exists. + +-- ft.code: `.client-workspace/workspace.ftd` +lang: ftd + +\-- fpm.client-workspace: +cr: + + +-- ft.h1: Error handling + +Adding a file that exists in latest history would suggest to use `fpm edit` +instead. + + + + + + + + diff --git a/cli/edit.ftd b/cli/edit.ftd new file mode 100644 index 000000000..e3e58f710 --- /dev/null +++ b/cli/edit.ftd @@ -0,0 +1,32 @@ +-- ft.page: `fpm edit --cr ` + +This command works with cr number only, else it fails +This would create entry in workspace.ftd and only the files added would be +consider for sync. + +This command will copy the file in cr directory `-/` and also +start tracking with the latest version. + + +-- ft.code: `.client-workspace/workspace.ftd` +lang: ftd + +\-- fpm.client-workspace: -// +version: + + +-- ft.code: `.tracks/-//` +lang: ftd + +\-- fpm.tracks: +version: + + +-- ft.h1: Error handling + +If file is already in `-/` directory, throw error +If the file doesn't exist in the latest history then throw error + + + + diff --git a/cli/rm.ftd b/cli/rm.ftd new file mode 100644 index 000000000..9278a30d6 --- /dev/null +++ b/cli/rm.ftd @@ -0,0 +1,42 @@ +-- ft.page: `fpm rm` + + +-- ft.h1: `fpm rm ` + +This would mark the file deleted in workspace.ftd. +This would delete the file. + +-- ft.code: `.client-workspace/workspace.ftd` +lang: ftd + +\-- fpm.client-workspace: +version: +cr: +deleted: true + +-- ft.h1: `fpm rm --cr ` + +This would add an entry in -//-/delete.ftd + +-- ft.code: `-//-/delete.ftd` +lang: ftd + +\-- fpm.deleted: +version: + + +-- ft.code: `.client-workspace/workspace.ftd` +lang: ftd + +\-- fpm.client-workspace: -//-/delete.ftd +version: +cr: + + +-- ft.h1: Error Handling + +If the file doesn't exist in the latest history then throw error + + + + diff --git a/dev/cr.ftd b/dev/cr.ftd new file mode 100644 index 000000000..36190cbde --- /dev/null +++ b/dev/cr.ftd @@ -0,0 +1,188 @@ +-- ft.page: How CR Works + +-- ft.h1: Reserved CR IDs + +Each clone has a unique id, stored in `.client-state/clone-id.ftd`. The id is +created by the server. + +On clone we create `.client-state/reserved-crs.ftd`, and store the 5 CR numbers +we get from server. + +Server will have `.server-state/reserved-crs.ftd` which will mention the CRs +handed out to each "clone". + +-- ft.code: `.server-state/reserved-crs.ftd` +lang: ftd + +\-- client: +reserved-cr: 1 +reserved-cr: 2 +reserved-cr: 3 +reserved-cr: 4 +reserved-cr: 5 + +\-- client: +reserved-cr: 6 +reserved-cr: 7 +reserved-cr: 8 +reserved-cr: 9 +reserved-cr: 10 + +-- ft.markdown: + +On every sync, we replenish our reserved CR pool. + +-- ft.h1: Create A New CR + +`fpm create-cr ""` can be used to create a CR. + + +-- ft.markdown: + +We find the smallest CR number from reserved CR, and create +`/-/<cr-id>/-/meta.ftd`: + +-- ft.code: +lang: ftd + +\-- import: fpm + +\-- fpm.cr: <title> +status: open + +-- ft.markdown: + +On next `fpm sync`, the `meta.ftd` file would be synced and everyone will have +access to it. Server will also replenish the reserved cr pool as we have +consumed one CR number from there. + +-- ft.h1: Edit CR Description + +The description of a CR is stored in `/-/<cr-id>/-/about.ftd`. + +`fpm cr <cr-number> edit` launched `$EDITOR` with the content of `about.ftd`. + +If the file does not exist it will be initialised with `FPM/default-cr-about.ftd` if +present, else, `THEME/default-cr-about.ftd`. + + +-- ft.h1: Edit CR Title + +`fpm cr <cr-number> edit-title` will launch `$EDITOR` with current title. + +Note: Only the first line would be used as CR title. Content before the first +new line character is first line. + + + +-- ft.h1: Add a new file in CR + +`fpm cr <cr-number> --add <filename>` + +To add a new file, create the file `/-/<cr-id>/<file.ftd>`, and let user edit +the file. + +On sync the file would be synced. From sync's perspective this is just a normal +file. Before sync `workspace.ftd` will not have a version number for this file, +and post sync it will. + +-- ft.h1: Edit an existing file in CR + +`fpm cr <cr-number> --add <filename>` + +If we want to edit `foo.ftd` in CR 1, we have to create `-/1/foo.ftd` and make +it [start tracking](/dev/track/) `foo.ftd`. + +On the sync the two files will be synced with server. One every update server +will update the `self-version` in the `.track/-/1/foo.ftd`. + +-- ft.h1: Editing a file both on client and server + +Say you started editing the file, but someone else also started editing the +file at the same time and their sync landed first. If we handle this as regular +conflict (both sides added a file) we ask end users to do a lot. We can make +users life better by implementing a special case for this scenario, and use the +smaller version number from both tracking files, and do a three way sync. If the +sync succeeds there is no conflict. + +-- ft.h1: Delete a file in CR + +`fpm rm <filename> --cr <cr-number>` + +We will have a file called `/-/1/-/deleted.ftd` which stores all the files and +their versions that were deleted by CR number 1. + +-- ft.code: +lang: ftd + +\-- fpm.deleted: file/name.ftd +version: v1 + +-- ft.markdown: + +This file gets synced with server as normal file. + +-- ft.h1: Detect conflict in CR + +We can detect if a file `/-/1/foo.ftd` in CR 1 is conflicted with corresponding +`foo.ftd` in main by looking at `.track/-/1/foo.ftd.track` to get the version +number, and finding the content of that version of `foo.ftd` from +`.server-state/foo.<version>.ftd`, and using it as "base", and content of +`foo.ftd` as "theirs" and content of `/-/1/foo.ftd` as ours and do three way +merge. + +If `/-/1/foo.ftd` does not exist, we check for `/-/1/-/deleted.ftd`, and see if +the version for `foo.ftd` in that file is different than the latest version of +`foo.ftd`. + +-- ft.h1: Merge changes in main to CR + +`fpm merge --src=main --dest=1 foo.ftd` # --src=main is default, if no +file/folder name, merge all + + +Merge only happens on server for now + +We perform the three way merge as described in previous section. We use content +of `foo.ftd` as "theirs" and content of `/-/1/foo.ftd` as "ours". If there is +no conflict we update `/-/1/foo.ftd` with the merged content, and update the +tracking information corresponding to `/-/1/foo.ftd` against `foo.ftd` (store +the version of `foo.ftd` we just merged in `.track/-/1/foo.ftd`. + +-- ft.h1: Merge CR changes to main + + +`fpm merge --src=1 --dest=main foo.ftd` + +Merge only happens on server for now + +We perform the three way merge as described in previous section. We use content +of `foo.ftd` as "ours" and content of `/-/1/foo.ftd` as "theirs". If there is +no conflict we update `foo.ftd` with the merged content, delete `/-/1/foo.ftd`, +and `.track/-/1/foo.ftd` and increment the version number of all three files in +`history.ftd`. + +-- ft.h1: Resolve conflict in CR when merging from main Or from CR into main + +In both these scenarios, we will create a temporary file with conflict markers +as described in previous two steps and launch `$EDITOR`, and once user has +resolved conflict we will do what is described in previous two steps. + +-- ft.h1: Close a CR + +To close a CR we update the `about.ftd` and set `status: closed`. + +`fpm close-cr <cr-number>` + +This sets the `close` flag in `-/<cr-number>/-/about.ftd`. + + +-- ft.h1: Update CR About + + +-- ft.h1: `fpm status --cr=1` +-- ft.h1: `fpm status` (sync status) +-- ft.h1: `fpm status --versions` +-- ft.h1: `fpm status --translation` + + diff --git a/dev/sync.ftd b/dev/sync.ftd new file mode 100644 index 000000000..9ea40571f --- /dev/null +++ b/dev/sync.ftd @@ -0,0 +1,248 @@ +-- ft.page: How Sync Works + +We have two states, "server state" and "client state". + +-- ft.h1: Server state + +The server state is stored inside the `.server-state` folder. It comprises all +files' history. + +-- ft.code: `.server-state/history.ftd` +lang: ftd + +\-- record server-state: +file-history list history: + + + + +-- ft.h2: History + +File history captures every modification to a file. It also included information +about changes that happened due to merges, if an action happened to the "main +branch" via merge from a CR, we store the CR where the change came from in the +`src-cr` field. + + +-- ft.code: +lang: ftd + +\-- record file-history: +caption filename: +file-edit list history: + +\-- or-type file-edit: +\--- added: +caption or body message: +integer timestamp: +integer version: +optional string author: +option integer src-cr: + +\--- updated: +caption or body message: +integer timestamp: +integer version: +optional string author: +option integer src-cr: + +\--- deleted: +caption or body message: +integer timestamp: +integer version: +optional string author: +option integer src-cr: + + +-- ft.markdown: + +Or we can have file-edit as a record + + +-- ft.code: +lang: ftd + +\-- record file-edit: +caption or body message: +integer timestamp: +integer version: +optional string author: +option integer src-cr: +string operation: + +-- ft.markdown: + +Actual content is stored in the file system based on +`.server-state/<file-name-without-last-extension>.<version>.<extension>`. + + +-- ft.h2: Global Versioning + +For a FPM project, there is a single version number that represents current state of +the server. This number is incremented by one on every successful sync. + +-- ft.h2: Consistent Read + +Since sync operation modifies multiple files in the package, if we do not take +care we can indavertantly read inconsistent state from file system. We have to +ensure reads are always consistent, even in the middle of a large sync. + +-- ft.h3: Locked `version-number` + +First we use global read-write lock mutex, with single writer, and multiple +readers. Since fpm server is single server, in the server's memory there exists +a variable called `version-number`, which is can be read by any number of threads +but can be updated by only one thread. All fpm server's read operations take a +read lock to read the `version-number`, and sync operation takes a write lock and +updates `version-number`. + +Most important thing is that the `version-number` is only updated after successful +sync operation, at the very end, after everything else has happened. + +-- ft.h3: Read From History + +Till the `version-number` has been updated, all reads read operations do not read +from the "workspace" on the server, but from the history folder. If current version +number is `v1`, and sync operation is going on, some `v2` files may exist in history +folder. But the readers ignore any file with version greater than current value of +`version-number`. + + + + + +-- ft.h1: Client state + +On the client, the state is largely two: workspace content and the unsynced +tracking information. + + +-- ft.code: +lang: ftd + +\-- record client-state: +workspace-entry list workspace: + +\-- record workspace-entry: +caption filename: +optional boolean deleted: +integer version: + +-- ft.markdown: + +Client state is stored in `.client-state` folder. Workspace information is +stored in `.client-state/workspace.ftd`. + + +-- ft.h1: Sync Scenarios + +Given the current server and client state, a bunch of operations are there, +which either print some output or update either the client or server state. + +-- ft.h2: A file was updated locally, no changes on remote + +We detect a file is updated locally by finding the version for every file in +`workspace.ftd`, and then comparing the local file with the +`.history/<filename>.<version>.<ext>`. If the content is different file is +considered modified. + +Say file is file.ftd, and its current version on server is v1. On sync server +will get file.ftd, v1 and new content of file. Server sees their version is also +v1, so server computes v2, and stored the new content of the file. + +Server returns latest `file.v2.ftd` content, and client stores it in +`.server-state`, and updates the `.client-state/workspace.ftd` and `file.ftd` +in workspace. + +-- ft.h2: A file was updated locally, with non conflicting changes on server + +If `file.ftd` was modified by someone else in the meanwhile, and the server +state has already moved to v2, server does a three way merge between v1 version +of file.ftd as "base", and v2 as "theirs" and the current content as "ours". + +In non conflicting state the file content is simply updated. Server computes +a v3, and returns the merged content to client, client adds it to .server-state +folder, and updates the workspace.ftd and file.ftd's content in the workspace. + +-- ft.h2: A file was updated locally with conflicting changes on server + +If on server, v2 and our changes do not merge cleanly with v1 version as based, +server notifies the client that the sync for that file failed. Server sends the +v2 version of file which client stored in `.server-state` folder. `workspace.ftd` +and `file.ftd` in workspace are not modified. + +-- ft.h2: Only a file with no known conflicts are sent to server + +Before sending a file we check if the file has known conflict. To do this we +use the version in `workspace.ftd` and the content of `file.ftd` for that +version as "base", and the latest version of `file.ftd` in `.server-state` as +"theirs", and the content of `file.ftd` in the workspace as "ours" and perform +three way merge, and if the merge fails, we know there is conflict. + +-- ft.h2: Conflict Resolution + +if there is any conflict, we have a local command `fpm resolve-conflict +file.ftd`, which does the three way merge, and generates the merge with conflict +marker string, and launches `$EDITOR`. User has to manually resolve the conflict, +and when they confirm they have done it by saving it, and we do not find any +remaining conflict markers, we update the `workspace.ftd` with the latest +version number, and the content of `file.ftd` with the content in EDITOR. + +We also have `fpm resolve-conflict --pick-ours file.ftd` and `fpm resolve-conflict +--pick-theirs file.ftd` commands, these commands will not launch `$EDITOR`. + +The resolve-conflict will update the version number to latest in `workspace.ftd` +in the end. + +-- ft.h2: `fpm revert file.ftd` + +We will have this command which can be used to discard any local changes done to +a file. This command will find the latest version of `file.ftd` from the +`.server-state` and update both `.client-state/workspace.ftd` and `file.ftd` in +the workspace with the latest version. + +-- ft.h2: Both sides have added a file conflict + +In this case we have a conflict, and server will detect it, and send the new +file added on server. And `fpm sync` will show conflict message to user. + +On next `fpm sync` we will not pick `file.ftd` as our conflict detector will +now see the file has a version in `.server-state`, but in `workspace.ftd` there +is no entry for that file, so we are adding a file that already exists, and will +show conflict message. + +-- ft.h2: Deleting a file, no edits on server case + +If a file was deleted from workspace, and since it already has an entry in +`workspace.ftd` we know that file is being deleted by us, we send this +information to server on `fpm sync`, and our version number. Server will see our +version numbers match, so server will create a new version number to mark +deletion of the file, and update the `.server-state/history.ftd`. And the client +will delete the entry from `.client-state/workspace.ftd`. + +-- ft.h2: Deleted by us, edited on server + +If the file we deleted has been updated on server, server will see different +version number, and return a conflict message along with the latest content of +the file. Our conflict detector will spot that we have deleted a file +corresponding to a version number in workspace.ftd that is no longer the latest +version of that file, and treat is as conflict and not send it to server on next +fpm sync. + +In the meanwhile user will have to review the diff of changes from their version +till the latest version of that file and decide if the file should be still +deleted (using `fpm resolve-conflict file.ftd`) or the file should be reverted +(using `fpm revert-changes file.ftd`). + + +-- ft.h2: Edited by us, deleted on server + +Server will again see two different version numbers, on delete we increase the +version number, and reject the changes, as delete and edit are not cleanly +merge-abe using three way merge operation. + +Client will show the conflict message and user will have to `fpm +resolve-conflict` or `fpm revert-changes`. + + + diff --git a/dev/track.ftd b/dev/track.ftd new file mode 100644 index 000000000..7b51404e3 --- /dev/null +++ b/dev/track.ftd @@ -0,0 +1,54 @@ +-- ft.page: How track works + +Say `a.ftd` is tracking `b.ftd`. We have some operations that mark `a.ftd` up +to date with `b.ftd`. This can happen when the author performs fpm +mark-up-to-date action if `a` is tracking `b` in an ad-hoc fashion, or if `a` is +a translation of `b`. This can also happen when a merge happens from CR, if `a` +was the CR version of `b`, and was being merged into `b`. + +`version` represents the version of `b.ftd` , and `self-version` represents the +version of `a.ftd`. + +Every file that is tracking something has a corresponding file: +`.tracks/<filename>.track`, which contains a list of `tracking-info`. + +-- ft.code: +lang: ftd + +\-- record tracking-info: +caption filename: +integer version: +option integer self-version: + +-- ft.markdown: + +Sample track file: + +-- ft.code: `.tracks/a.ftd.track` +lang: ftd + +\-- import: fpm +\-- fpm.tracks: b.ftd +version: 2 +self-version: 1 + +-- ft.h1: To Start Tracking A File + +Tracking is initialised by creating a `.tracks/<filename>.track` file. + +`self-version` may be missing if the file is being added and tracked at the +same time. + +The job of `fpm sync` is to also populate the missing `self-version` in every +`.track` file on successfully syncing the corresponding file. This will be done +by server. + +If `a.ftd` wants to track `b.ftd`, `b.ftd` must be first synced with the server +else we do not have version number. + +`fpm start-tracking a.ftd --target b.ftd` + + +-- ft.h1: `fpm stop-tracking a.ftd --target b.ftd` + +-- ft.h1: `fpm mark-up-to-date a.ftd --target b.ftd` \ No newline at end of file diff --git a/draft-cr.ftd b/draft-cr.ftd new file mode 100644 index 000000000..4f1985779 --- /dev/null +++ b/draft-cr.ftd @@ -0,0 +1,306 @@ +-- ft.page: How Sync And CR Works? + +We have two states, "server state" and "client state". + +-- ft.h1: Server state + +The server state is stored inside the `.server-state` folder. It comprises all +files' history and tracking information. + +-- ft.code: +lang: ftd + +\-- record server-state: +file-history list history: +file-tracking list tracks: + +-- ft.markdown: + +History-related information is stored in the `.server-state/history.ftd` and +track-related information is stored in the `server-state/tracks.ftd` + + + + + +-- ft.h2: History + +File history captures every modification to a file. It also included information +about changes that happened due to merges, if an action happened to the "main +branch" via merge from a CR, we store the CR where the change came from in the +`src-cr` field. + + +-- ft.code: +lang: ftd + +\-- record file-history: +caption filename: +file-edit list history: + +\-- or-type file-edit: +\--- added: +caption or body message: +integer timestamp: +integer version: +string author: +option integer src-cr: + +\--- updated: +caption or body message: +integer timestamp: +integer version: +string author: +option integer src-cr: + +\--- deleted: +caption or body message: +integer timestamp: +integer version: +string author: +option integer src-cr: + + +-- ft.markdown: + +Or we can have file-edit as a record + + +-- ft.code: +lang: ftd + +\-- record file-edit: +caption or body message: +integer timestamp: +integer version: +string author: +option integer src-cr: +string operation: + +-- ft.markdown: + +Actual content is stored in the file system based on +`.server-state/<file-name-without-last-extension>.<version>.<extension>`. + +Also, the version number starts with 1 for every file, and is incremented by +one for every change in that file. Delete also increments the version number. + + + +-- ft.h2: Tracks + +A file can track one or more files. Any file that is tracking another file has +an entry in `server-state.track`, represented by file-tracking. + + +-- ft.code: +lang: ftd + +\-- record file-tracking: +caption filename: +tracking-info list tracks: + +TODO: split one big track file into one track file per file. filename which is +caption here, will become `<filename>.track`. + +-- ft.markdown: + +Every file that is being tracked is recorded using `tracking-info`. + +-- ft.code: +lang: ftd + +\-- record tracking-info: +caption filename: +integer version: +option integer self-version: + + +-- ft.code: In `fpm.ftd` +lang: ftd + +\-- file-tracking list tracker: + +-- ft.markdown: + +Say `a.ftd` is tracking `b.ftd`. We have some operations that mark `a.ftd` up +to date with `b.ftd`. This can happen when we sync with remote, say if `a` was +local and `b` was a remote version of the same file. This can happen when the +author performs fpm mark-up-to-date action if `a` is tracking `b` in an ad-hoc +fashion, or if `a` is a translation of `b`. This can also happen when a merge +happens from CR, if `a` was the CR version of `b`, and was being merged into `b`. + +`version` represents the version of `b.ftd` , and `self-version` represents the +version of `a.ftd`. + + + +-- ft.h1: Client state + + +On the client, the state is largely two: workspace content and the unsynced +tracking information. + + +-- ft.code: +lang: ftd + +\-- record client-state: +workspace-entry list workspace: +file-tracking list unsynced-tracks: + +\-- record workspace-entry: +caption filename: +optional boolean deleted: +integer version: + +-- ft.markdown: + +Client state is stored in `.client-state` folder. Workspace information is +stored in `.client-state/workspace.ftd` and unsynced changes are stored in +`.client-state/unsynced.tracks.ftd`. + + +-- ft.h1: Operations + +Given the current server and client state, a bunch of operations are there, +which either print some output or update either the client or server state. + +-- ft.h2: Sync Vs Tracking + +The state we keep are of two kinds, history, which is used for syncing. + +We should be able to do syncing with just the content of the `history.ftd` and +the `workspace.ftd` files only. + +Track files from syncing perspective are just another files in the package. Since +track files are generated by us, we may be able to do a better job of helping the +users with conflict resolutions, beyond what we do for other regular files. +Special UI, or maybe some heuristic which can decide the conflict is not a real +conflict, maybe earlier or later version wins. + +-- ft.h2: `fpm clone` + +This command starts with a remote state and no client state, and fetches the +client state and lays things out. + +It also stores the server state in `.server-state` folder. This folder contains +`history.ftd`, `tracks.ftd` and `index.1.ftd` etc. + + +-- ft.h2: `fpm start-tracking a.ftd --target b.ftd` + +With this command, `a.ftd` starts tracking `b.ftd`. If command is executed +directly on server, this creates an entry in `.server-state/tracks.ftd`. In case +of client, this commands creates the entry in `.client-state/tracks.ftd` + +In case, if the files (`a.ftd` and `b.ftd`) are not synced i.e. latest snapshot +is not created or is not in sync with the workspace, this command does nothing, +instead, it throws a warning asking to sync first. Let's say the `a.ftd` version +is 1 and the `b.ftd` version is 2 then the entry looks something like this. + + + +-- ft.code: +lang: ftd + +\-- fpm.tracker : a.ftd + +\-- tracks: b.ftd +version: 2 +self-version: 1 + + + +-- ft.h2: `fpm status` + +This shows the changes in local. Sync-related and track-related information + + + +-- ft.h2: `fpm sync` + +On the client, some files are updated, if there is a merge-able change on the +server, along with their entries in the workspace file. + +On every sync, we also copy and update the server-state to the `.server-state` +folder. + + +-- ft.h2: `fpm create-cr --title <optional title> --description <optional description>` + +This command creates a new CR. In case, if the package is not a client (not a +clone package) then this does nothing, rather gives a warning that this command +is not supported. It achieves this in the following steps: + +- It calls the server/remote post API `create-cr/` with title and description, + both are optional fields. +- The server acquires the lock and reads the file `.server-state/cr`. This file + contains the current CR number. It then increments the CR number by one and + then releases the lock. +- The server then creates a file `-/<cr-number>/about.ftd`. This file stores the + title and description of the CR. In case the title is not specified, then the + CR number becomes the title + +-- ft.code: +lang: ftd + +\-- record cr-about-data: +caption title: +optional body description: + +-- ft.markdown: + +- Afterwards, the server returns the cr-number. (Probably also the file +`-/<cr-number>/about.ftd` content. Not sure though). +- On the client side, the file `-/<cr-number>/about.ftd` is created with +data similar to the server. + + +-- ft.h2: `fpm cr <cr-number> <operation> <file-name>` + +The operations available are `add` and `delete` + +-- ft.h3: `add` + +This operation adds a new file or existing file (with some modification) in CR +(let’s say the CR number is 1). + +In case of adding a new file, let's say `foo.ftd`, just create a new file in a +current file system, do some modification if needed and then run this command +`fpm cr 1 add foo.ftd`. + +Similarly, in the case of adding an existing file with some modification in CR, +just do the modification and run the command. + +On running this command, it first checks if the file is different from the +latest version in the history or if it’s a new file altogether, else this fails +with a warning that no changes detected. It then clones the file from the +current file system to the `-/<cr-number>/<file-name>`. + +Q: Should it also creates the entry in `.client-state/-/<cr-number>/workspace.ftd` +file? + + +-- ft.h3: `delete` + +This operation deletes the existing file in the CR. It first checks if the file exists +in the latest history else this fails with the warning that the file doesn’t exist. +It creates an entry in the `.client-state/-/<cr-number>/workspace.ftd`. + +-- ft.code: +lang: ftd + +\-- workspace-entry list entry: + +\-- entry: <file-name> +deleted: true +version: 1 + + + + + + + + diff --git a/how-sync-works.ftd b/how-sync-works.ftd new file mode 100644 index 000000000..c8a97f953 --- /dev/null +++ b/how-sync-works.ftd @@ -0,0 +1,337 @@ +-- ft.page: How sync & CR works + + +-- ft.h1: State + +State is represented by what is with the server and what is with the client. + +-- ft.h2: Server State + +Server state is composed of history of all files, and the tracking information +about all files. + +-- ft.code: +lang: ftd + +\-- record server-state: +file-history list history: +file-tracking list tracks: + +-- ft.markdown: + +File history captures every modification to a file. It also included information +about changes that happened due to merges, if an action happened to "main +branch" via merge from a CR, we store the CR where the change came from in the +`src-cr` field. + +-- ft.code: +lang: ftd + +\-- record file-history: +caption filename: +file-edit list history: + +\-- or-type file-edit: +\--- added: +caption or body message: +integer timestamp: +integer version: +string author: +option integer src-cr: + +\--- updated: +caption or body message: +integer timestamp: +integer version: +string author: +option integer src-cr: + +\--- deleted: +caption or body message: +integer timestamp: +integer version: +string author: +option integer src-cr: + +-- ft.markdown: + +Actual content is stored in the file system based on +`<file-name-without-last-extension>.<version>.<extension>`. + +Also `version` starts with 1 for every file, and is incremented by one for every +change in that file. Delete also increments the version number. + +A file can track one or more files. Any file that is tracking another file has +an entry in `server-state.track`, represented by `file-tracking`. + +-- ft.code: +lang: ftd + +\-- record file-tracking: +caption filename: +tracking-info list tracks: + +-- ft.markdown: + +Every file that is being tracked is recorded using `tracking-info`. + +-- ft.code: +lang: ftd + +\-- record tracking-info: +caption filename: +integer version: +option integer self-version: + +-- ft.markdown: + +Say `a.ftd` is tracking `b.ftd`. We have some operation that mark `a.ftd` up to +date with `b.ftd`. This can happen when a we sync with remote, say if a was +local and b was remote version of same file. This can happen when the author +performs `fpm mark-up-to-date` action if a is tracking b in an ad-hoc fashion, +or if a is a translation of b. This can also happen when a merge happens from +CR, if a was CR version of b, and was being merged into b. + +`version` represents the version of `b.ftd` , and `self-version` +represents the version of `a.ftd`. + +-- ft.h1: Client State + +On the client the state is largely two: workspace content and the unsynced +tracking information. + +-- ft.code: +lang: ftd + +\-- record client-state: +workspace-entry list workspace: +file-tracking list unsynced-tracks: + +\-- record workspace-entry: +caption filename: +optional boolean deleted: +integer version: + + +-- ft.markdown: + +Client state is stored in `.client-state` folder. Workspace information is +stored in `.client-state/workspace.ftd` and unsynced changes are stored in +`.client-state/unsynced.tracks.ftd`. + +-- ft.h1: Operations + +Given current server and client state, a bunch of operations are there, which +either print some output, or update either the client or server state. + +-- ft.h2: `fpm clone` + +This command starts with remote state and no client state, and fetches client +state and lays things out. + +It also stores the server state in `.server-state` folder. This folder contains +`history.ftd`, `tracks.ftd` and `index.1.ftd` etc. + +-- ft.h2: `fpm start-tracking a.ftd b.ftd` + + + +-- ft.h2: `fpm status` + +This shows the changes on local. + +-- ft.h2: `fpm sync` + +On client, some files are updated, if there is a mergable change on server, +along with their entries in workspace file. + +On every sync we also copy and update the `server-state` to `.server-state` +folder. + + +-- ft.h1: `.fpm/workspace.ftd` + +After fpm clone, the .fpm/workspace.ftd created. This contains entry for all the +files in the file system. It is hashmap of filename to version. + +The only time the workspace is updated is when we do a sync or resolve a conflict. + + +-- ft.code: +lang: ftd + +\-- fpm.workspace-file: <filename> +timestamp: <timestamp> + + + +-- ft.h1: `.history/history.ftd` + +This contains the history related information for the files + + +-- ft.code: +lang: ftd + + +\-- fpm.history-file: <filename> + +\--- action.add: +timestamp: <timestamp> + + + + + + + +-- ft.h1: `new-file-added` + +Before sync: + +Create file in file system + +After sync: +- .history/index.v1.ftd +- Create entry in .latest.ftd +- Create entry .history/history.ftd + +-- ft.code: +lang: ftd + +\-- fpm.history: index.ftd + +\--- fpm.status: Added +timestamp: v1 + +-- ft.markdown: + +- Create entry in `.fpm/workspace.ftd` + + +-- ft.h1: `new-file-added-after-already-added` + +Before sync: + +Create file in file system + +After sync: +- Return conflict (index.v1.ftd: content and version) +- Create entry in .fpm/workspace.ftd + +-- ft.code: +lang: ftd + +\-- fpm.workspace: index.ftd +status: conflicted +version: v1 + +-- ft.markdown: + +- Don't modify index.ftd in fs + + + +-- ft.h1: `file-deleted` + +Before sync: + +Delete file in file system + +After sync: +- Remove entry from .history/.latest.ftd +- Create entry in .history/history.ftd + +-- ft.code: +lang: ftd + +\-- fpm.history: index.ftd + +\--- fpm.status: Deleted +timestamp: v1 + +-- ft.markdown: + +- Remove entry from .fpm/workspace.ftd + + + + +-- ft.h1: `file-deleted-after-already-modified` + +Before sync: + +Delete the file + +After sync: +- Return conflict (index.v1.ftd: content and version) +- Create entry in .fpm/workspace.ftd + +-- ft.code: +lang: ftd + +\-- fpm.workspace: index.ftd +version: v2 + +-- ft.markdown: + +- It won't restore the file in fs + + + + + +-- ft.h1: `file-deleted-after-already-deleted` + + +Before sync: + +Delete file in file system + +After sync: + +Nothing + +- Remove entry from `.fpm/workspace.ftd` + + + + +-- ft.h1: `file-modified` + +Before sync: + +modify the file + +After sync: + +- Create new snapshot .history/index.v2.ftd +- Create entry in .latest.ftd +- Create entry in .history/history.ftd + + + +-- ft.h1: `file-modified-after-already-deleted` + +Before sync: + +modify the file + +After sync: + +Nothing + + + +-- ft.h1: `file-modified-after-already-modified` + +Before sync: + +modify the file + +After sync: + +Nothing + + +