-
Notifications
You must be signed in to change notification settings - Fork 14
oci: Extend with more OCI-native manifest/config storage #216
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Update doc/repository.md and repository.rs module docs: - Remove "experimental" disclaimer - the layout is stable - Fix CLI flag: --path → --repo - Fix hash length: 64 chars → 64/128 for sha256/sha512 - Add comprehensive module-level rustdoc to repository.rs with ASCII diagram of the repository structure - Document images/ vs streams/ distinction and security rationale - Expand name_stream() documentation to explain GC root behavior Assisted-by: OpenCode (Claude claude-opus-4-5-20250514) Signed-off-by: Colin Walters <walters@verbum.org>
a229b24 to
0ece424
Compare
Previously, composefs-oci only stored the config and layer data but
discarded the OCI manifest after pulling. This forced downstream
consumers (like bootc) to implement manifest storage internally,
which doesn't really make sense.
My opinion is that while the true "composefs core" is today and
should remain logically flexible, there's absolutely not reason for
us not to have a very opinionated mechanism to store OCI. Very very
key to this is that OCI is not just container images (with tar media
types) - there's nowadays OCI artifacts which allow storing arbitrary
data easily - and I think mapping "foreign" types to OCI makes a
ton of sense.
So as part of this the composefs-oci layer also learns how to handle
non-tar layers from manifests.
- Manifests stored at `oci-manifest-{digest}` with refs to config/layers
- Named tags under `streams/refs/oci/{name}` act as GC roots
- New `OciImage` type provides high-level access to stored images
- `pull_image()` returns `PullResult` with both manifest and config info
- Original `pull()` maintained for backward compatibility
CLI additions (cfsctl oci):
- `images` - list all tagged OCI images
- `inspect <image>` - show image details (arch, labels, layers, etc.)
- `tag <digest> <name>` - tag an image by manifest digest
- `untag <name>` - remove a tag
The storage model follows OCI semantics: everything is content-addressed,
manifests reference their components, and tags are just named pointers
to manifests. This enables offline access to all image metadata and
proper garbage collection of unreferenced content.
See-also: bootc-dev/bootc#1830
Assisted-by: OpenCode (Opus 4.5)
Signed-off-by: Colin Walters <walters@verbum.org>
0ece424 to
10bb53f
Compare
|
Something I want to highlight in this PR that I think is quite cool and important is that this project is now more directly/obviously supporting non-container OCI types i.e. OCI artifacts. Often when people say "OCI" they think of course about the classic tar layer container images, but that's not the only use case for OCI. And it makes an equal amount of sense to store e.g. AI model training weights or disk images in a composefs repo as it does OCI. So again "OCI" != "OCI container images" in general. |
|
I looked briefly. This seems like pretty cool/useful stuff... There's a bit of tension here with #218. What's the eventual plan? Import via containers-image-proxy-rs, or skip skopeo and use the cstor-rs stuff (via jsonrpc) directly? |
The two PRs are 100% compatible AFAICS and complementary. This is just extending things so we store the manifest too etc., it's orthogonal to how we fetch the data.
I think for bootc we will likely always fetch via podman/skopeo for the near term future by default because of the "containers/image" part of container-libs (talking to registries, registries.conf auth file handling etc.) into a containers-storage, and then reflink from there into composefs (and then reflink from there into ostree if enabled). |
Previously, composefs-oci only stored the config and layer data but
discarded the OCI manifest after pulling. This forced downstream
consumers (like bootc) to implement manifest storage internally,
which doesn't really make sense.
My opinion is that while the true "composefs core" is today and
should remain logically flexible, there's absolutely not reason for
us not to have a very opinionated mechanism to store OCI. Very very
key to this is that OCI is not just container images (with tar media
types) - there's nowadays OCI artifacts which allow storing arbitrary
data easily - and I think mapping "foreign" types to OCI makes a
ton of sense.
So as part of this the composefs-oci layer also learns how to handle
non-tar layers from manifests.
oci-manifest-{digest}with refs to config/layersstreams/refs/oci/{name}act as GC rootsOciImagetype provides high-level access to stored imagespull_image()returnsPullResultwith both manifest and config infopull()maintained for backward compatibilityCLI additions (cfsctl oci):
images- list all tagged OCI imagesinspect <image>- show image details (arch, labels, layers, etc.)tag <digest> <name>- tag an image by manifest digestuntag <name>- remove a tagThe storage model follows OCI semantics: everything is content-addressed,
manifests reference their components, and tags are just named pointers
to manifests. This enables offline access to all image metadata and
proper garbage collection of unreferenced content.
See-also: bootc-dev/bootc#1830
Assisted-by: OpenCode (Opus 4.5)