Erase organization command in server CLI - #12172
Conversation
c58fc2c to
9fd277c
Compare
| encrypted workspaces metadata (e.g. content of folder, list of blocks for each file version). | ||
| - A blockstore (e.g. S3) containing the blocks (i.e. encrypted pieces of data that compose the files). | ||
| - On top of that, each Parsec client having access to the organization has an encrypted local database | ||
| containing a copy of the certificates, metadata for the workspaces its has access to, and a subset |
There was a problem hiding this comment.
| containing a copy of the certificates, metadata for the workspaces its has access to, and a subset | |
| containing a copy of the certificates, metadata for the workspaces it has access to, and a subset |
|
|
||
| Blockstores have their own backup strategy. Typically AWS S3 allows for a bucket to | ||
| have an history so that a data removal can be cancelled. | ||
| You should pay attention to this to ensure the blocks have actually been removed. |
There was a problem hiding this comment.
Do we expect them to be removed from the history too ?
There was a problem hiding this comment.
The goal here is to permanently erase all data from a given organization, so we expect the administrator to remove the history.
However this is not Parsec responsibility since blockstore configuration is specific to the administrator needs (see #12172 (comment)).
| organization using the local cache (e.g. creating a new file in a workspace or | ||
| reading an existing file that is in cache). | ||
|
|
||
| To prevent the users from accessing the local cache, the local configuration and data should be manually removed: |
There was a problem hiding this comment.
by the user themselves ?
There was a problem hiding this comment.
This can also be done by an administrator that has remote access to the machine. Here we don't specify how things should be done but what should be done.
There was a problem hiding this comment.
I'd rephrase it like this:
| To prevent the users from accessing the local cache, the local configuration and data should be manually removed: | |
| The local cache can be safely erased by manually deleting the local configuration and data for the device having access to the organization: |
IMO we should add a note or warning to explain how to identify which device is linked to the organization (device id is displayed in the "My Devices" when logged-in... maybe we should consider displaying it in the Home page as tooltip or detail button ?)
There was a problem hiding this comment.
Actually, in other sections of the docs we suggest using the CLI for that, e.g.:
The
<DEVICE_ID>mentioned below refer to the device used to access the
organization. You can find out which is your device ID by running:.. code-block:: bash
parsec-cli device list
|
|
||
| .. warning:: | ||
|
|
||
| This operation cannot be undone. Make sure you have a backup of any data you may need before |
There was a problem hiding this comment.
nit: maybe add the command to make a backup in the warning message ?
There was a problem hiding this comment.
Again, the administrator is responsible for the backup strategy, so we cannot provide a one-size-fits-all command here (typically "backup any data" may refer to do a PostgreSQL snapshot and storing it on cold storage, or it may refer to simply have a user start a Parsec client and copy/paste the files still needed)
There was a problem hiding this comment.
There is already a section describing how to make a backup, we should link to it. See my next comment/suggestion.
|
|
||
| .. note:: | ||
|
|
||
| After erasing an organization, it is possible to create a new organization with the same name |
There was a problem hiding this comment.
What would be the expected error on the client side if they haven't deleted their local data and a new organization with the same name is created ?
If their parsec client would just look offline that may be confusing if the same entity re creates an organization and expect to enroll the same people.
There was a problem hiding this comment.
The client will get an error complaining the organization is not found on the server (even if a new organization with the same name is created, since they don't have the same root verify key)
I've added a word about this in the note 👍
There was a problem hiding this comment.
Otherwise one may think that it is OK to create an org with same name just after step 1 (postgresql), before step 2 (blockstore cleanup).
IMO, regardless of Parsec internals, the erasing procedure should be treated almost as a" transaction", step 1 to 3 should be performed as a whole before attempting to create a new organization with same name.
| "- No trace of the organization will remain, so it will be possible to re-create another organization with the same name." | ||
| ) | ||
| click.echo( | ||
| f"- The organization's blocks won't be erased from the blockstore, you should instead manually remove the {display_bucket_path} top level directory from it." |
There was a problem hiding this comment.
| f"- The organization's blocks won't be erased from the blockstore, you should instead manually remove the {display_bucket_path} top level directory from it." | |
| f"- The organization's blocks won't be erased from the blockstore, you should manually remove the {display_bucket_path} top level directory from it." |
9fd277c to
8e8f3d3
Compare
mmmarcos
left a comment
There was a problem hiding this comment.
Some suggestions regarding the docs, mostly about wording or rephrasing.
I didn't review the code.
| Where & how the data are stored | ||
| ------------------------------- |
There was a problem hiding this comment.
| Where & how the data are stored | |
| ------------------------------- | |
| Where is the organization's data stored? | |
| ---------------------------------------- |
| - A PostgreSQL database containing the certificates (e.g. user/devices/workspaces) and | ||
| encrypted workspaces metadata (e.g. content of folder, list of blocks for each file version). | ||
| - A blockstore (e.g. S3) containing the blocks (i.e. encrypted pieces of data that compose the files). | ||
| - On top of that, each Parsec client having access to the organization has an encrypted local database | ||
| containing a copy of the certificates, metadata for the workspaces it has access to, and a subset | ||
| of the blocks (depending on local cache configuration). |
There was a problem hiding this comment.
| - A PostgreSQL database containing the certificates (e.g. user/devices/workspaces) and | |
| encrypted workspaces metadata (e.g. content of folder, list of blocks for each file version). | |
| - A blockstore (e.g. S3) containing the blocks (i.e. encrypted pieces of data that compose the files). | |
| - On top of that, each Parsec client having access to the organization has an encrypted local database | |
| containing a copy of the certificates, metadata for the workspaces it has access to, and a subset | |
| of the blocks (depending on local cache configuration). | |
| - A PostgreSQL database containing certificates (e.g. users, devices, workspaces) and | |
| encrypted workspace metadata (e.g. folder contents, block list for each file version). | |
| - An S3 blockstore containing the encrypted file blocks. | |
| - On each Parsec client having access to the organization: An encrypted local database | |
| containing a copy of the certificates, workspace metadata it has access to, and a subset | |
| of the encrypted file blocks (depending on local cache configuration). |
| When no longer in use (or for legal reason) an organization can be erased from the Parsec server. | ||
|
|
||
| In practice this means: |
There was a problem hiding this comment.
| When no longer in use (or for legal reason) an organization can be erased from the Parsec server. | |
| In practice this means: | |
| When no longer in use (or for legal reason) an organization can be erased from the Parsec server | |
| following these steps: |
| 1. Removing everything (certificates & metadata) related to the organization in the PostgreSQL database. | ||
| 2. Removing the blocks from the blockstore. | ||
| 3. Removing the remaining data from the clients. |
There was a problem hiding this comment.
I'd use "delete" (or erase) since "remove" may indicate persistence.
| 1. Removing everything (certificates & metadata) related to the organization in the PostgreSQL database. | |
| 2. Removing the blocks from the blockstore. | |
| 3. Removing the remaining data from the clients. | |
| 1. Delete certificates & metadata from the PostgreSQL database. | |
| 2. Delete the encrypted file blocks from the blockstore. | |
| 3. Delete remaining local data from Parsec clients. |
| Step 1: remove from PostgreSQL | ||
| ------------------------------ |
There was a problem hiding this comment.
I'd make these steps (1, 2, 3) subsections of "Erasing data"
| Step 1: remove from PostgreSQL | |
| ------------------------------ |
| Step 1: remove from PostgreSQL | |
| ------------------------------ | |
| Step 1: Delete certificates from PostgreSQL | |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
|
|
||
| .. note:: | ||
|
|
||
| After erasing an organization, it is possible to create a new organization with the same name |
There was a problem hiding this comment.
Otherwise one may think that it is OK to create an org with same name just after step 1 (postgresql), before step 2 (blockstore cleanup).
IMO, regardless of Parsec internals, the erasing procedure should be treated almost as a" transaction", step 1 to 3 should be performed as a whole before attempting to create a new organization with same name.
| Step 3: Clients cleanup | ||
| ----------------------- |
There was a problem hiding this comment.
| Step 3: Clients cleanup | |
| ----------------------- | |
| Step 3: delete local data from Parsec clients | |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| For this reason, an end-user is still able to use his Parsec client to work on the | ||
| organization using the local cache (e.g. creating a new file in a workspace or | ||
| reading an existing file that is in cache). |
There was a problem hiding this comment.
| For this reason, an end-user is still able to use his Parsec client to work on the | |
| organization using the local cache (e.g. creating a new file in a workspace or | |
| reading an existing file that is in cache). | |
| For this reason, users are still able to use the Parsec client to work on the | |
| organization for actions relying on the local cache such as importing a new file | |
| or reading an existing file that is in cache. |
| - config: ``$XDG_DATA_HOME or $HOME/.local/share/parsec3/<device_id>`` (e.g. ``/home/alice/.local/share/parsec3/e68b7131394749a4bbd279bd087e6ae6``) | ||
| - data: ``$XDG_CONFIG_HOME or $HOME/.config/parsec3/libparsec/devices/<device_id>`` (e.g. ``/home/alice/.config/parsec3/libparsec/devices/e68b7131394749a4bbd279bd087e6ae6``) |
There was a problem hiding this comment.
| - config: ``$XDG_DATA_HOME or $HOME/.local/share/parsec3/<device_id>`` (e.g. ``/home/alice/.local/share/parsec3/e68b7131394749a4bbd279bd087e6ae6``) | |
| - data: ``$XDG_CONFIG_HOME or $HOME/.config/parsec3/libparsec/devices/<device_id>`` (e.g. ``/home/alice/.config/parsec3/libparsec/devices/e68b7131394749a4bbd279bd087e6ae6``) | |
| - config: ``$XDG_DATA_HOME`` or ``$HOME/.local/share/parsec3/<device_id>`` (e.g. ``/home/alice/.local/share/parsec3/e68b7131394749a4bbd279bd087e6ae6``) | |
| - data: ``$XDG_CONFIG_HOME`` or ``$HOME/.config/parsec3/libparsec/devices/<device_id>`` (e.g. ``/home/alice/.config/parsec3/libparsec/devices/e68b7131394749a4bbd279bd087e6ae6``) |
| organization using the local cache (e.g. creating a new file in a workspace or | ||
| reading an existing file that is in cache). | ||
|
|
||
| To prevent the users from accessing the local cache, the local configuration and data should be manually removed: |
There was a problem hiding this comment.
I'd rephrase it like this:
| To prevent the users from accessing the local cache, the local configuration and data should be manually removed: | |
| The local cache can be safely erased by manually deleting the local configuration and data for the device having access to the organization: |
IMO we should add a note or warning to explain how to identify which device is linked to the organization (device id is displayed in the "My Devices" when logged-in... maybe we should consider displaying it in the Home page as tooltip or detail button ?)
No description provided.