Feature: conditional upload API #178
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…, inmem providers Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
alsenz
force-pushed
the
feat_conditions_API_129
branch
from
c21b930 to
1bb876b
Compare
Author
|
@bwplotka - would you be the appropriate maintainer to submit this for a first review? Greatly appreciated. |
yuchen-db
reviewed
Oct 17, 2025
| } | ||
| } | ||
|
|
||
| func tryOpenFile(name string, ifNotExists bool) (exists bool, err error) { |
| } | ||
|
|
||
| // ValidateUploadOptions ensures that only supported options are passed as options. | ||
| func ValidateUploadOptions(supportedOptions []ObjectUploadOptionType, opts ...ObjectUploadOption) error { |
There was a problem hiding this comment.
maybe we should also validate IfNotExists and IfMatch/IfNotMatch aren't used together
yuchen-db
reviewed
Oct 17, 2025
| b.mtx.Lock() | ||
| defer b.mtx.Unlock() | ||
|
|
||
| params := ApplyObjectUploadOptions(opts...) |
yuchen-db
approved these changes
Oct 17, 2025
|
What's the progress on this PR? Any chance it gets merged? |
Signed-off-by: Alsenz <7210552+alsenz@users.noreply.github.com>
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
alsenz
force-pushed
the
feat_conditions_API_129
branch
from
a5d77ae to
b395c1c
Compare
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Author
|
Thanks @yuchen-db for your review. I agree with all your points and have pushed some changes accordingly. This is an old PR but hopefully we can get it across the line. |
Author
|
@anarcher this is an older PR that didn't get much attention, hopefully the maintainers will be able to find some time to review and accept. I have brought the feature branch back in line with main and responded to comments, so it should be ready to go. |
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
alsenz
force-pushed
the
feat_conditions_API_129
branch
from
cbdcc69 to
9ee1894
Compare
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #129
Adds support for conditional object uploads.
Extends
ObjectAttributeswith optionalObjectVersion(if supported), andUpload(...)with three newObjectUploadOption, depending on provider support.IfMatch: the object is written only if the existing object matches the provided versionIfNotMatch: the object is written only if the existing object does not match the provided versionIfNotExists: the object is written only if no object already exists for the provided keySupports two forms of
ObjectVersion:Not all providers support conditional write. The following providers are supported:
IfNotMatchis not yet supported (as it is not supported by AWS).Clients can check conditional API by calling
SupportedObjectUploadOptionson theBucketinterface.Changes
API changes
ObjectUploadOptions:IfMatch,IfNotMatchandIfNotExists1.1 Made
ObjectUploadOptiontyped, repeating the existingIterOptionpattern.1.2 Added three new parameter modifiers-
WithIfMatch,WithIfNotExistsandWithIfNotMatch, andWithContentTypeto support the previous un-typedObjectUploadOptionObjectVersionfield toObjectAttributesIfMatch,IfNotMatch,IfNotExistsand forAttributesAPIBucketprovider implementations for:1.1 filesystem: this now uses Extended Attributes, if supported by the host system
1.2 inmem
1.3 GCS
1.4 Azure
1.5 S3, excepting
IfNoneMatchwhich is not fully supported (by AWS) yet1.6 Wrappers
Backwards compatibility
API changes are additive and should be backwards compatible.
The
UploadAPI changes use a trailing variadic parameter so should be backwards compatibleThe
ObjectAttributesadd new field so should be backwards compatibleThe
SupportedObjectUploadOptionsinterface method potentially breaks existingBucketimplementations. Library users who implement their own providers will need to implement this method. Full or stub implementations are provided for all provider implementations in the libary.Dependencies
Verification
Added acceptance tests for
IfMatchIfNotMatchIfExistsDid not run integration tests on other object store providers, as I do not have access to test environments, but these should be unchanged.