Feat: Verification with MerkleDAG #88
Description
Problem statement
To use File CID directly, we need to add additional interaction between the data provider and consumers.
Expectation proposal
Data Structure for MerkleDAG
- A HashMap of File CID to the corresponding DAG
- A recursive structure to represent DAG. A block, identified by block hash, also contains a list of blocks, potentially an enum that is a parent block, or data content (leave blocks)
- Function to return DAG without leave blocks
- Function to return leave blocks by File CID and position
File Hasher
- Create MerkleDAG for files, matching the same schemes as IPFS files
- Validate a DAG from root to all children blocks
- Validate specific data content to a specific block of the DAG
File Service
- Expose a function or a dedicated endpoint for providing DAG to clients for a File CID (serialize in json object)
- Returns specific data content for a request that contains a File CID and a range
Downloader
- Given a File CID, request for the DAG to available file services
- Once receiving a DAG, de-serialize it and validate top to bottom without leave blocks
- For each request, predetermine the position, data range, and the correct hash from DAG
- For each content response, validate the content with respect to the correct hash
Make sure that File CID created matches exactly with IPFS CID v0
Alternative considerations
Reference existing implementation