[QUESTION] <about scale_jitter sizes customization>

[dingliangxiansheng]

Hi, thanks for your great work!

I have a question about the scale_jitter augmentation:

"scale_jitter": {
"divisible_by": null,
"max_scale": 2.0,
"min_scale": 0.5,
"num_scales": 20,
"prob": 1.0,
"seed_offset": 0,
"sizes": null,
"step_seeding": false
}

Currently, the scales are automatically generated based on the model input resolution (e.g., 518×518).

My question is:
Is it possible to manually define custom resizing sizes， use them in the scale jitter stage, but still keep the final model input size fixed (518×518)?

So essentially:

The scale jitter resize uses my manually defined sizes

The final crop/resize to the model remains 518×518

No change to the patch size or model input shape

Is this supported or does the current implementation force the scales to be generated from the model input size?

Thanks!

[stegmuel]

Hi,
Yes ScaleJitter accepts a sizes argument that allows you to achieve this.
By default scale jittering is followed by random cropping to the model's input size (518x518) during.
I assumed you were using semantic segmentation based on the image resolution you mentioned. Let me know if this is incorrect.

[dingliangxiansheng]

Hi, Yes ScaleJitter accepts a sizes argument that allows you to achieve this. By default scale jittering is followed by random cropping to the model's input size (518x518) during. I assumed you were using semantic segmentation based on the image resolution you mentioned. Let me know if this is incorrect.

Hi, thanks for your previous reply!

I have one follow-up question regarding the sizes field in scale_jitter.

I am using semantic segmentation training (train_semantic_segmentation), and after debugging the transformation pipeline (semantic_segmentation_transform.py), I noticed that the resize operation inside ScaleJitter is computed as:

transform += [
                OneOf(
                    [
                        Resize(
                            height=int(scale * transform_args.image_size[0]),
                            width=int(scale * transform_args.image_size[1]),
                        )
                        for scale in scales
                    ],
                    p=transform_args.scale_jitter.prob,
                )
            ]

This means the scale jitter always uses transform_args.image_size
(e.g. the default 518 × 518) as its base resolution.

However, even if I set:

transform_args={    
            "scale_jitter": {
                "divisible_by": None,
                "max_scale": 4.0,
                "min_scale": 1.0,
                "num_scales": 40,
                "prob": 1.0,
                "seed_offset": 0,
                "sizes": [(540, 960)],
                "step_seeding": False
            },
                
            "val": {    # Override validation parameters
                "image_size": (1080, 1920), # (height, width)
            },  
        },

the provided sizes list is never used anywhere in the semantic segmentation transform pipeline.

So my questions are:

1. In semantic segmentation, is scale_jitter.sizes actually supported?

2. If segmentation does not use sizes currently, is this the expected behavior?

Or is this a missing implementation?

3. Is there a recommended way to manually define custom resize sizes

(e.g., 540×960, 720×1280, etc.) before the final crop to the model’s input size
while keeping the model input size fixed?
Additional Question

The default image_size for semantic segmentation is 518 × 518.

However, 518 is not divisible by 16, while many vision backbones (ViT, ConvNeXt, Swin, etc.) usually recommend input sizes that align with the patch size or stride.

So my question is:

4. Why is the default segmentation input size set to 518×518?

Thanks for the clarification! It would really help me better understand the design choices behind the semantic segmentation pipeline.

[stegmuel]

Hi,

1-2. If you specify sizes, neither of target_size, scale_range or num_scales must be set. I'm surprised an error was not raised.
3. I unfortunately do not have recommendations with regards to the sizes to set, I personally would stick with the default behavior using scaling in range [0.5 - 2.0].
4. It's because we initially started with DINOv2 backbones that have patch_size=14 and did not change it for the DINOv3 backbones. This can also be set to 512 if needed.