mycofs server

[despiegk]

• remove s3, zdb, and directory as backend only support mycofs server
• mycofs stores the data on a folder based structure (aa/bb/cccccccccc. these are the blobs) ...???
• a mycofs server has following features
  • quickly define which blobs are there (e.g. give list of 1000 blobs, we get to know which ones we have)
  • verify a list of blobs (hash recalculated)
  • download a list of blobs as tar
  • upload a list of blobs as tar
  • get a blob
  • delete a list of blobs
  • copy a list of blobs to a remote server (rsync like) (so we can instruct one server to also copy the blobs to another server)
  • manage flists (upload, delete)
  • people can list and download flists, they are groupd per account
• is part of mycofs cmdline
• to manage mycofs server is openrpc on local socket (ask maxim how to do in Be), this can be reached from mycelium
• make rust client using the openrpc over mycelium as well as over local socket
• we log all actions (how much data uploaded, downloaded, ...)
• a mycofslist can be exposed over a http optionally webdav interface (so an flist can be a webserver)
  • the exposes mycofslist or short mylist can be attached to prefix or domain name (only http)
  • so basically a mylist become a website (http/html/files...)
• fuse...

management

simplified for phase 1

• there are guests, users & admins
• users & admins get authenticated based on a signature on a pubkey (see our webassembly stuff), so basically I use my private key to prove that I own the pubkey
• guests dont need authentication, they can download anything, only for exposed mylists which are public
• the pubkey is what allows me to login, once logged in I get a unique token which allows me to do my actions
• users can download, we track what they download
• admins need authentication (just based on a key) they can upload, delete, expose mylists as http...

step 1

• rfs server: should be able download an flist, download a tar of all of lists blocks, upload a targz as an flist
  • GET /flist/<id>/download
  • POST a targz /flist/upload
  • DELETE /flist/<id> (should delete the flist and all of its blocks if not referenced -> can be done later if not easily calculated

step 2

• rfs server: should be able to retrive certain block or a list of blocks
  • GET /blocks/<blockids> if one it retrieves one block
  • GET /blocks/<blockids> if comma separated, retrieves multiple blocks (in tar)
• rfs server: should be able to sync with other servers
  • /sync/<serverurl>/<blockids>

[xmonader]

• need clarify the meaning of blob, is it a file in the file system structure or is it a chunk
• if you are using content based addressing based on the content hash, then there's no meaning to update a content to a new value
• what do we mean by list of blobs, list of files, or list of chunks? because if list of files tar.gzed then it's fine that the normal behavior