-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
feat(fs): implement client-side multipart upload with File-Path header #1877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements a new client-side multipart upload API that enables chunked file uploads with concurrent upload support and resumable capabilities. The implementation provides a two-step workflow (upload chunks → complete) with automatic session initialization and cleanup.
Key Changes:
- New PUT
/api/fs/multipartendpoint supportinguploadandcompleteactions - Session-based chunk management with 2-hour expiration
- Idempotent chunk uploads allowing retry of failed chunks
- Support for concurrent chunk uploads via shared session state
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
| server/router.go | Adds the new /multipart endpoint with upload rate limiting and authentication middleware |
| server/handles/multipart.go | Implements HTTP handlers for chunk upload and completion actions with header-based API design |
| internal/fs/multipart.go | Core multipart session manager with chunk storage, merging, and cleanup logic |
| internal/model/multipart.go | Data structures for multipart sessions, chunk metadata, and API responses |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: DnsLin <49158572+dnslin@users.noreply.github.com>
|
@copilot open a new pull request to apply changes based on the comments in this thread |
KirCute
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
能不能开始上传的时候就构建FileStream并调用驱动的上传,这样做至少有两个好处:
- 可以在首个分片的上传请求里携带文件哈希,在驱动触发秒传,如果秒传成功,后续分片就都不用上传了
- 可以同时进行
用户终端->OpenList和OpenList->网盘两个上传过程,这两个过程分别吃服务器的下行和上行带宽,不冲突,可以很大程度上减少总的上传时间
Description / 描述
Implement a new multipart upload API that supports client-side chunked uploads with concurrent upload capability.
实现新的分片上传接口,支持客户端分片并发上传。
API Endpoint: PUT /api/fs/multipart?action=upload|complete
Features / 功能:
Request Headers:
Motivation and Context / 背景
The existing upload APIs (/api/fs/put, /api/fs/form) don't support resumable uploads for large files. This new multipart upload API allows clients to:
现有的上传接口不支持大文件的断点续传。新的分片上传接口允许客户端:
Relates to #XXXX (如果有相关 issue 请填写)
How Has This Been Tested? / 测试
使用 Python 异步客户端进行手动测试,测试了并发上传、幂等性、会话过期清理等功能。
Checklist / 检查清单