AI mask

[tobiasBora]

Is is often really helpful to select a subject precisely (to apply an effect only to the subject, or to remove the subject from the another mask etc), and is done in so many lightroom tutorials that I feel a bit lost when using any open-source software (see e.g. https://www.youtube.com/watch?v=FMaB03h1rQk). Would it be possible to add an AI-masking tool in ART? Some possible implementations details are discussed in darktable (darktable-org/darktable#12295) that mention, e.g., Segment Anything https://segment-anything.com/, https://github.com/danielgatis/rembg , SAM2, FAST-SAM, Mobile SAM…

[agriggio]

See https://github.com/artpixls/SMART
Combined with the new "external mask" feature, it basically gives you what you want.

HTH

[tobiasBora]

Thanks, it's fairly cool to see progress here, but couldn't it be more integrated so that I could just click a single button instead of jumping from software 1 to software 2? (this click may just open SMART with parameters explaining where to save the mask after etc)

It would be great also to create an extension system (like each extension would provide a configuration file in the $XDG_DATA_DIRS that explains how to start the external program with a given input image and output mask) that allows us to plug different AI system (for what I see in https://d2x313g9lpht1q.cloudfront.net/original/3X/c/3/c3b267d5a11c870675503dab7638d3c2b82441a3.mp4 the output of SMART is still not perfect).

EDIT: also, what is the version that features external mask?

[agriggio]

It will never be fully integrated (at least by me), it's just too complicated for too little benefit (imho of course). But using the "user commands" feature of art you can automate most of it. I can show you how to do that at least for Unix-like environments, but if you are interested the forum is a better place than GitHub to have such conversation.

Best

[agriggio]

FWIW I put some instructions here: https://artraweditor.github.io/AItools

[tobiasBora]

Thanks a lot for this! Out of curiosity, which step of the pipeline do you export for denoising/ai mask?

[agriggio]

For denoising, basically all steps except demosaicing, raw corrections and white balancing are turned off.
For ai masking, I'm using the same settings as in the sidecar (so the full pipeline), except for the crop.