feat(eap): Sample by trace #7158

phacops · 2025-05-05T16:09:54Z

This PR will change the materialized views so we now sample by trace_id in order to have complete traces and be sure a trace included in a lower tier is also present in the tiers above.

github-actions · 2025-05-05T16:37:33Z

This PR has a migration; here is the generated SQL for ./snuba/migrations/groups.py ()

-- start migrations

-- forward migration events_analytics_platform : 0051_items_sample_by_trace_id
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_items_1_downsample_8_mv_5 TO eap_items_1_downsample_8_local (organization_id UInt64, project_id UInt64, item_type UInt8, timestamp DateTime CODEC (DoubleDelta, ZSTD(1)), trace_id UUID, item_id UInt128, sampling_weight UInt64 CODEC (ZSTD(1)), sampling_factor Float64 CODEC (ZSTD(1)), retention_days UInt16 CODEC (T64, ZSTD(1)), attributes_bool Map(String, Bool), attributes_int Map(String, Int64), attributes_array JSON(max_dynamic_paths=128) CODEC (ZSTD(1)), attributes_string_0 Map(String, String) CODEC (ZSTD(1)), attributes_string_1 Map(String, String) CODEC (ZSTD(1)), attributes_string_2 Map(String, String) CODEC (ZSTD(1)), attributes_string_3 Map(String, String) CODEC (ZSTD(1)), attributes_string_4 Map(String, String) CODEC (ZSTD(1)), attributes_string_5 Map(String, String) CODEC (ZSTD(1)), attributes_string_6 Map(String, String) CODEC (ZSTD(1)), attributes_string_7 Map(String, String) CODEC (ZSTD(1)), attributes_string_8 Map(String, String) CODEC (ZSTD(1)), attributes_string_9 Map(String, String) CODEC (ZSTD(1)), attributes_string_10 Map(String, String) CODEC (ZSTD(1)), attributes_string_11 Map(String, String) CODEC (ZSTD(1)), attributes_string_12 Map(String, String) CODEC (ZSTD(1)), attributes_string_13 Map(String, String) CODEC (ZSTD(1)), attributes_string_14 Map(String, String) CODEC (ZSTD(1)), attributes_string_15 Map(String, String) CODEC (ZSTD(1)), attributes_string_16 Map(String, String) CODEC (ZSTD(1)), attributes_string_17 Map(String, String) CODEC (ZSTD(1)), attributes_string_18 Map(String, String) CODEC (ZSTD(1)), attributes_string_19 Map(String, String) CODEC (ZSTD(1)), attributes_string_20 Map(String, String) CODEC (ZSTD(1)), attributes_string_21 Map(String, String) CODEC (ZSTD(1)), attributes_string_22 Map(String, String) CODEC (ZSTD(1)), attributes_string_23 Map(String, String) CODEC (ZSTD(1)), attributes_string_24 Map(String, String) CODEC (ZSTD(1)), attributes_string_25 Map(String, String) CODEC (ZSTD(1)), attributes_string_26 Map(String, String) CODEC (ZSTD(1)), attributes_string_27 Map(String, String) CODEC (ZSTD(1)), attributes_string_28 Map(String, String) CODEC (ZSTD(1)), attributes_string_29 Map(String, String) CODEC (ZSTD(1)), attributes_string_30 Map(String, String) CODEC (ZSTD(1)), attributes_string_31 Map(String, String) CODEC (ZSTD(1)), attributes_string_32 Map(String, String) CODEC (ZSTD(1)), attributes_string_33 Map(String, String) CODEC (ZSTD(1)), attributes_string_34 Map(String, String) CODEC (ZSTD(1)), attributes_string_35 Map(String, String) CODEC (ZSTD(1)), attributes_string_36 Map(String, String) CODEC (ZSTD(1)), attributes_string_37 Map(String, String) CODEC (ZSTD(1)), attributes_string_38 Map(String, String) CODEC (ZSTD(1)), attributes_string_39 Map(String, String) CODEC (ZSTD(1)), attributes_float_0 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_1 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_2 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_3 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_4 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_5 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_6 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_7 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_8 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_9 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_10 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_11 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_12 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_13 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_14 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_15 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_16 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_17 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_18 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_19 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_20 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_21 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_22 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_23 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_24 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_25 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_26 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_27 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_28 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_29 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_30 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_31 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_32 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_33 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_34 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_35 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_36 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_37 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_38 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_39 Map(String, Float64) CODEC (ZSTD(1))) AS SELECT organization_id, project_id, item_type, timestamp, trace_id, item_id, attributes_bool, attributes_int, attributes_array, attributes_string_0, attributes_string_1, attributes_string_2, attributes_string_3, attributes_string_4, attributes_string_5, attributes_string_6, attributes_string_7, attributes_string_8, attributes_string_9, attributes_string_10, attributes_string_11, attributes_string_12, attributes_string_13, attributes_string_14, attributes_string_15, attributes_string_16, attributes_string_17, attributes_string_18, attributes_string_19, attributes_string_20, attributes_string_21, attributes_string_22, attributes_string_23, attributes_string_24, attributes_string_25, attributes_string_26, attributes_string_27, attributes_string_28, attributes_string_29, attributes_string_30, attributes_string_31, attributes_string_32, attributes_string_33, attributes_string_34, attributes_string_35, attributes_string_36, attributes_string_37, attributes_string_38, attributes_string_39, attributes_float_0, attributes_float_1, attributes_float_2, attributes_float_3, attributes_float_4, attributes_float_5, attributes_float_6, attributes_float_7, attributes_float_8, attributes_float_9, attributes_float_10, attributes_float_11, attributes_float_12, attributes_float_13, attributes_float_14, attributes_float_15, attributes_float_16, attributes_float_17, attributes_float_18, attributes_float_19, attributes_float_20, attributes_float_21, attributes_float_22, attributes_float_23, attributes_float_24, attributes_float_25, attributes_float_26, attributes_float_27, attributes_float_28, attributes_float_29, attributes_float_30, attributes_float_31, attributes_float_32, attributes_float_33, attributes_float_34, attributes_float_35, attributes_float_36, attributes_float_37, attributes_float_38, attributes_float_39, downsampled_retention_days AS retention_days, sampling_weight * 8 AS sampling_weight, sampling_factor / 8 AS sampling_factor, client_sample_rate / 8 AS client_sample_rate, server_sample_rate / 8 AS server_sample_rate FROM eap_items_1_local WHERE (cityHash64(trace_id + 8)  % 8) = 0;
Local op: DROP TABLE IF EXISTS eap_items_1_downsample_8_mv_4 SYNC;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_items_1_downsample_64_mv_5 TO eap_items_1_downsample_64_local (organization_id UInt64, project_id UInt64, item_type UInt8, timestamp DateTime CODEC (DoubleDelta, ZSTD(1)), trace_id UUID, item_id UInt128, sampling_weight UInt64 CODEC (ZSTD(1)), sampling_factor Float64 CODEC (ZSTD(1)), retention_days UInt16 CODEC (T64, ZSTD(1)), attributes_bool Map(String, Bool), attributes_int Map(String, Int64), attributes_array JSON(max_dynamic_paths=128) CODEC (ZSTD(1)), attributes_string_0 Map(String, String) CODEC (ZSTD(1)), attributes_string_1 Map(String, String) CODEC (ZSTD(1)), attributes_string_2 Map(String, String) CODEC (ZSTD(1)), attributes_string_3 Map(String, String) CODEC (ZSTD(1)), attributes_string_4 Map(String, String) CODEC (ZSTD(1)), attributes_string_5 Map(String, String) CODEC (ZSTD(1)), attributes_string_6 Map(String, String) CODEC (ZSTD(1)), attributes_string_7 Map(String, String) CODEC (ZSTD(1)), attributes_string_8 Map(String, String) CODEC (ZSTD(1)), attributes_string_9 Map(String, String) CODEC (ZSTD(1)), attributes_string_10 Map(String, String) CODEC (ZSTD(1)), attributes_string_11 Map(String, String) CODEC (ZSTD(1)), attributes_string_12 Map(String, String) CODEC (ZSTD(1)), attributes_string_13 Map(String, String) CODEC (ZSTD(1)), attributes_string_14 Map(String, String) CODEC (ZSTD(1)), attributes_string_15 Map(String, String) CODEC (ZSTD(1)), attributes_string_16 Map(String, String) CODEC (ZSTD(1)), attributes_string_17 Map(String, String) CODEC (ZSTD(1)), attributes_string_18 Map(String, String) CODEC (ZSTD(1)), attributes_string_19 Map(String, String) CODEC (ZSTD(1)), attributes_string_20 Map(String, String) CODEC (ZSTD(1)), attributes_string_21 Map(String, String) CODEC (ZSTD(1)), attributes_string_22 Map(String, String) CODEC (ZSTD(1)), attributes_string_23 Map(String, String) CODEC (ZSTD(1)), attributes_string_24 Map(String, String) CODEC (ZSTD(1)), attributes_string_25 Map(String, String) CODEC (ZSTD(1)), attributes_string_26 Map(String, String) CODEC (ZSTD(1)), attributes_string_27 Map(String, String) CODEC (ZSTD(1)), attributes_string_28 Map(String, String) CODEC (ZSTD(1)), attributes_string_29 Map(String, String) CODEC (ZSTD(1)), attributes_string_30 Map(String, String) CODEC (ZSTD(1)), attributes_string_31 Map(String, String) CODEC (ZSTD(1)), attributes_string_32 Map(String, String) CODEC (ZSTD(1)), attributes_string_33 Map(String, String) CODEC (ZSTD(1)), attributes_string_34 Map(String, String) CODEC (ZSTD(1)), attributes_string_35 Map(String, String) CODEC (ZSTD(1)), attributes_string_36 Map(String, String) CODEC (ZSTD(1)), attributes_string_37 Map(String, String) CODEC (ZSTD(1)), attributes_string_38 Map(String, String) CODEC (ZSTD(1)), attributes_string_39 Map(String, String) CODEC (ZSTD(1)), attributes_float_0 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_1 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_2 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_3 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_4 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_5 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_6 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_7 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_8 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_9 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_10 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_11 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_12 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_13 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_14 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_15 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_16 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_17 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_18 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_19 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_20 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_21 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_22 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_23 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_24 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_25 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_26 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_27 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_28 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_29 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_30 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_31 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_32 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_33 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_34 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_35 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_36 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_37 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_38 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_39 Map(String, Float64) CODEC (ZSTD(1))) AS SELECT organization_id, project_id, item_type, timestamp, trace_id, item_id, attributes_bool, attributes_int, attributes_array, attributes_string_0, attributes_string_1, attributes_string_2, attributes_string_3, attributes_string_4, attributes_string_5, attributes_string_6, attributes_string_7, attributes_string_8, attributes_string_9, attributes_string_10, attributes_string_11, attributes_string_12, attributes_string_13, attributes_string_14, attributes_string_15, attributes_string_16, attributes_string_17, attributes_string_18, attributes_string_19, attributes_string_20, attributes_string_21, attributes_string_22, attributes_string_23, attributes_string_24, attributes_string_25, attributes_string_26, attributes_string_27, attributes_string_28, attributes_string_29, attributes_string_30, attributes_string_31, attributes_string_32, attributes_string_33, attributes_string_34, attributes_string_35, attributes_string_36, attributes_string_37, attributes_string_38, attributes_string_39, attributes_float_0, attributes_float_1, attributes_float_2, attributes_float_3, attributes_float_4, attributes_float_5, attributes_float_6, attributes_float_7, attributes_float_8, attributes_float_9, attributes_float_10, attributes_float_11, attributes_float_12, attributes_float_13, attributes_float_14, attributes_float_15, attributes_float_16, attributes_float_17, attributes_float_18, attributes_float_19, attributes_float_20, attributes_float_21, attributes_float_22, attributes_float_23, attributes_float_24, attributes_float_25, attributes_float_26, attributes_float_27, attributes_float_28, attributes_float_29, attributes_float_30, attributes_float_31, attributes_float_32, attributes_float_33, attributes_float_34, attributes_float_35, attributes_float_36, attributes_float_37, attributes_float_38, attributes_float_39, downsampled_retention_days AS retention_days, sampling_weight * 64 AS sampling_weight, sampling_factor / 64 AS sampling_factor, client_sample_rate / 64 AS client_sample_rate, server_sample_rate / 64 AS server_sample_rate FROM eap_items_1_local WHERE (cityHash64(trace_id + 64)  % 64) = 0;
Local op: DROP TABLE IF EXISTS eap_items_1_downsample_64_mv_4 SYNC;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_items_1_downsample_512_mv_5 TO eap_items_1_downsample_512_local (organization_id UInt64, project_id UInt64, item_type UInt8, timestamp DateTime CODEC (DoubleDelta, ZSTD(1)), trace_id UUID, item_id UInt128, sampling_weight UInt64 CODEC (ZSTD(1)), sampling_factor Float64 CODEC (ZSTD(1)), retention_days UInt16 CODEC (T64, ZSTD(1)), attributes_bool Map(String, Bool), attributes_int Map(String, Int64), attributes_array JSON(max_dynamic_paths=128) CODEC (ZSTD(1)), attributes_string_0 Map(String, String) CODEC (ZSTD(1)), attributes_string_1 Map(String, String) CODEC (ZSTD(1)), attributes_string_2 Map(String, String) CODEC (ZSTD(1)), attributes_string_3 Map(String, String) CODEC (ZSTD(1)), attributes_string_4 Map(String, String) CODEC (ZSTD(1)), attributes_string_5 Map(String, String) CODEC (ZSTD(1)), attributes_string_6 Map(String, String) CODEC (ZSTD(1)), attributes_string_7 Map(String, String) CODEC (ZSTD(1)), attributes_string_8 Map(String, String) CODEC (ZSTD(1)), attributes_string_9 Map(String, String) CODEC (ZSTD(1)), attributes_string_10 Map(String, String) CODEC (ZSTD(1)), attributes_string_11 Map(String, String) CODEC (ZSTD(1)), attributes_string_12 Map(String, String) CODEC (ZSTD(1)), attributes_string_13 Map(String, String) CODEC (ZSTD(1)), attributes_string_14 Map(String, String) CODEC (ZSTD(1)), attributes_string_15 Map(String, String) CODEC (ZSTD(1)), attributes_string_16 Map(String, String) CODEC (ZSTD(1)), attributes_string_17 Map(String, String) CODEC (ZSTD(1)), attributes_string_18 Map(String, String) CODEC (ZSTD(1)), attributes_string_19 Map(String, String) CODEC (ZSTD(1)), attributes_string_20 Map(String, String) CODEC (ZSTD(1)), attributes_string_21 Map(String, String) CODEC (ZSTD(1)), attributes_string_22 Map(String, String) CODEC (ZSTD(1)), attributes_string_23 Map(String, String) CODEC (ZSTD(1)), attributes_string_24 Map(String, String) CODEC (ZSTD(1)), attributes_string_25 Map(String, String) CODEC (ZSTD(1)), attributes_string_26 Map(String, String) CODEC (ZSTD(1)), attributes_string_27 Map(String, String) CODEC (ZSTD(1)), attributes_string_28 Map(String, String) CODEC (ZSTD(1)), attributes_string_29 Map(String, String) CODEC (ZSTD(1)), attributes_string_30 Map(String, String) CODEC (ZSTD(1)), attributes_string_31 Map(String, String) CODEC (ZSTD(1)), attributes_string_32 Map(String, String) CODEC (ZSTD(1)), attributes_string_33 Map(String, String) CODEC (ZSTD(1)), attributes_string_34 Map(String, String) CODEC (ZSTD(1)), attributes_string_35 Map(String, String) CODEC (ZSTD(1)), attributes_string_36 Map(String, String) CODEC (ZSTD(1)), attributes_string_37 Map(String, String) CODEC (ZSTD(1)), attributes_string_38 Map(String, String) CODEC (ZSTD(1)), attributes_string_39 Map(String, String) CODEC (ZSTD(1)), attributes_float_0 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_1 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_2 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_3 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_4 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_5 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_6 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_7 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_8 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_9 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_10 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_11 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_12 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_13 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_14 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_15 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_16 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_17 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_18 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_19 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_20 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_21 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_22 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_23 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_24 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_25 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_26 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_27 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_28 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_29 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_30 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_31 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_32 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_33 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_34 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_35 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_36 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_37 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_38 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_39 Map(String, Float64) CODEC (ZSTD(1))) AS SELECT organization_id, project_id, item_type, timestamp, trace_id, item_id, attributes_bool, attributes_int, attributes_array, attributes_string_0, attributes_string_1, attributes_string_2, attributes_string_3, attributes_string_4, attributes_string_5, attributes_string_6, attributes_string_7, attributes_string_8, attributes_string_9, attributes_string_10, attributes_string_11, attributes_string_12, attributes_string_13, attributes_string_14, attributes_string_15, attributes_string_16, attributes_string_17, attributes_string_18, attributes_string_19, attributes_string_20, attributes_string_21, attributes_string_22, attributes_string_23, attributes_string_24, attributes_string_25, attributes_string_26, attributes_string_27, attributes_string_28, attributes_string_29, attributes_string_30, attributes_string_31, attributes_string_32, attributes_string_33, attributes_string_34, attributes_string_35, attributes_string_36, attributes_string_37, attributes_string_38, attributes_string_39, attributes_float_0, attributes_float_1, attributes_float_2, attributes_float_3, attributes_float_4, attributes_float_5, attributes_float_6, attributes_float_7, attributes_float_8, attributes_float_9, attributes_float_10, attributes_float_11, attributes_float_12, attributes_float_13, attributes_float_14, attributes_float_15, attributes_float_16, attributes_float_17, attributes_float_18, attributes_float_19, attributes_float_20, attributes_float_21, attributes_float_22, attributes_float_23, attributes_float_24, attributes_float_25, attributes_float_26, attributes_float_27, attributes_float_28, attributes_float_29, attributes_float_30, attributes_float_31, attributes_float_32, attributes_float_33, attributes_float_34, attributes_float_35, attributes_float_36, attributes_float_37, attributes_float_38, attributes_float_39, downsampled_retention_days AS retention_days, sampling_weight * 512 AS sampling_weight, sampling_factor / 512 AS sampling_factor, client_sample_rate / 512 AS client_sample_rate, server_sample_rate / 512 AS server_sample_rate FROM eap_items_1_local WHERE (cityHash64(trace_id + 512)  % 512) = 0;
Local op: DROP TABLE IF EXISTS eap_items_1_downsample_512_mv_4 SYNC;
-- end forward migration events_analytics_platform : 0051_items_sample_by_trace_id




-- backward migration events_analytics_platform : 0051_items_sample_by_trace_id
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_items_1_downsample_8_mv_4 TO eap_items_1_downsample_8_local (organization_id UInt64, project_id UInt64, item_type UInt8, timestamp DateTime CODEC (DoubleDelta, ZSTD(1)), trace_id UUID, item_id UInt128, sampling_weight UInt64 CODEC (ZSTD(1)), sampling_factor Float64 CODEC (ZSTD(1)), retention_days UInt16 CODEC (T64, ZSTD(1)), attributes_bool Map(String, Bool), attributes_int Map(String, Int64), attributes_array JSON(max_dynamic_paths=128) CODEC (ZSTD(1)), attributes_string_0 Map(String, String) CODEC (ZSTD(1)), attributes_string_1 Map(String, String) CODEC (ZSTD(1)), attributes_string_2 Map(String, String) CODEC (ZSTD(1)), attributes_string_3 Map(String, String) CODEC (ZSTD(1)), attributes_string_4 Map(String, String) CODEC (ZSTD(1)), attributes_string_5 Map(String, String) CODEC (ZSTD(1)), attributes_string_6 Map(String, String) CODEC (ZSTD(1)), attributes_string_7 Map(String, String) CODEC (ZSTD(1)), attributes_string_8 Map(String, String) CODEC (ZSTD(1)), attributes_string_9 Map(String, String) CODEC (ZSTD(1)), attributes_string_10 Map(String, String) CODEC (ZSTD(1)), attributes_string_11 Map(String, String) CODEC (ZSTD(1)), attributes_string_12 Map(String, String) CODEC (ZSTD(1)), attributes_string_13 Map(String, String) CODEC (ZSTD(1)), attributes_string_14 Map(String, String) CODEC (ZSTD(1)), attributes_string_15 Map(String, String) CODEC (ZSTD(1)), attributes_string_16 Map(String, String) CODEC (ZSTD(1)), attributes_string_17 Map(String, String) CODEC (ZSTD(1)), attributes_string_18 Map(String, String) CODEC (ZSTD(1)), attributes_string_19 Map(String, String) CODEC (ZSTD(1)), attributes_string_20 Map(String, String) CODEC (ZSTD(1)), attributes_string_21 Map(String, String) CODEC (ZSTD(1)), attributes_string_22 Map(String, String) CODEC (ZSTD(1)), attributes_string_23 Map(String, String) CODEC (ZSTD(1)), attributes_string_24 Map(String, String) CODEC (ZSTD(1)), attributes_string_25 Map(String, String) CODEC (ZSTD(1)), attributes_string_26 Map(String, String) CODEC (ZSTD(1)), attributes_string_27 Map(String, String) CODEC (ZSTD(1)), attributes_string_28 Map(String, String) CODEC (ZSTD(1)), attributes_string_29 Map(String, String) CODEC (ZSTD(1)), attributes_string_30 Map(String, String) CODEC (ZSTD(1)), attributes_string_31 Map(String, String) CODEC (ZSTD(1)), attributes_string_32 Map(String, String) CODEC (ZSTD(1)), attributes_string_33 Map(String, String) CODEC (ZSTD(1)), attributes_string_34 Map(String, String) CODEC (ZSTD(1)), attributes_string_35 Map(String, String) CODEC (ZSTD(1)), attributes_string_36 Map(String, String) CODEC (ZSTD(1)), attributes_string_37 Map(String, String) CODEC (ZSTD(1)), attributes_string_38 Map(String, String) CODEC (ZSTD(1)), attributes_string_39 Map(String, String) CODEC (ZSTD(1)), attributes_float_0 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_1 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_2 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_3 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_4 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_5 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_6 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_7 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_8 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_9 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_10 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_11 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_12 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_13 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_14 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_15 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_16 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_17 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_18 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_19 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_20 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_21 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_22 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_23 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_24 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_25 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_26 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_27 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_28 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_29 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_30 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_31 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_32 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_33 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_34 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_35 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_36 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_37 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_38 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_39 Map(String, Float64) CODEC (ZSTD(1))) AS SELECT organization_id, project_id, item_type, timestamp, trace_id, item_id, attributes_bool, attributes_int, attributes_array, attributes_string_0, attributes_string_1, attributes_string_2, attributes_string_3, attributes_string_4, attributes_string_5, attributes_string_6, attributes_string_7, attributes_string_8, attributes_string_9, attributes_string_10, attributes_string_11, attributes_string_12, attributes_string_13, attributes_string_14, attributes_string_15, attributes_string_16, attributes_string_17, attributes_string_18, attributes_string_19, attributes_string_20, attributes_string_21, attributes_string_22, attributes_string_23, attributes_string_24, attributes_string_25, attributes_string_26, attributes_string_27, attributes_string_28, attributes_string_29, attributes_string_30, attributes_string_31, attributes_string_32, attributes_string_33, attributes_string_34, attributes_string_35, attributes_string_36, attributes_string_37, attributes_string_38, attributes_string_39, attributes_float_0, attributes_float_1, attributes_float_2, attributes_float_3, attributes_float_4, attributes_float_5, attributes_float_6, attributes_float_7, attributes_float_8, attributes_float_9, attributes_float_10, attributes_float_11, attributes_float_12, attributes_float_13, attributes_float_14, attributes_float_15, attributes_float_16, attributes_float_17, attributes_float_18, attributes_float_19, attributes_float_20, attributes_float_21, attributes_float_22, attributes_float_23, attributes_float_24, attributes_float_25, attributes_float_26, attributes_float_27, attributes_float_28, attributes_float_29, attributes_float_30, attributes_float_31, attributes_float_32, attributes_float_33, attributes_float_34, attributes_float_35, attributes_float_36, attributes_float_37, attributes_float_38, attributes_float_39, downsampled_retention_days AS retention_days, sampling_weight * 8 AS sampling_weight, sampling_factor / 8 AS sampling_factor FROM eap_items_1_local WHERE (cityHash64(item_id + 8)  % 8) = 0;
Local op: DROP TABLE IF EXISTS eap_items_1_downsample_8_mv_5 SYNC;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_items_1_downsample_64_mv_4 TO eap_items_1_downsample_64_local (organization_id UInt64, project_id UInt64, item_type UInt8, timestamp DateTime CODEC (DoubleDelta, ZSTD(1)), trace_id UUID, item_id UInt128, sampling_weight UInt64 CODEC (ZSTD(1)), sampling_factor Float64 CODEC (ZSTD(1)), retention_days UInt16 CODEC (T64, ZSTD(1)), attributes_bool Map(String, Bool), attributes_int Map(String, Int64), attributes_array JSON(max_dynamic_paths=128) CODEC (ZSTD(1)), attributes_string_0 Map(String, String) CODEC (ZSTD(1)), attributes_string_1 Map(String, String) CODEC (ZSTD(1)), attributes_string_2 Map(String, String) CODEC (ZSTD(1)), attributes_string_3 Map(String, String) CODEC (ZSTD(1)), attributes_string_4 Map(String, String) CODEC (ZSTD(1)), attributes_string_5 Map(String, String) CODEC (ZSTD(1)), attributes_string_6 Map(String, String) CODEC (ZSTD(1)), attributes_string_7 Map(String, String) CODEC (ZSTD(1)), attributes_string_8 Map(String, String) CODEC (ZSTD(1)), attributes_string_9 Map(String, String) CODEC (ZSTD(1)), attributes_string_10 Map(String, String) CODEC (ZSTD(1)), attributes_string_11 Map(String, String) CODEC (ZSTD(1)), attributes_string_12 Map(String, String) CODEC (ZSTD(1)), attributes_string_13 Map(String, String) CODEC (ZSTD(1)), attributes_string_14 Map(String, String) CODEC (ZSTD(1)), attributes_string_15 Map(String, String) CODEC (ZSTD(1)), attributes_string_16 Map(String, String) CODEC (ZSTD(1)), attributes_string_17 Map(String, String) CODEC (ZSTD(1)), attributes_string_18 Map(String, String) CODEC (ZSTD(1)), attributes_string_19 Map(String, String) CODEC (ZSTD(1)), attributes_string_20 Map(String, String) CODEC (ZSTD(1)), attributes_string_21 Map(String, String) CODEC (ZSTD(1)), attributes_string_22 Map(String, String) CODEC (ZSTD(1)), attributes_string_23 Map(String, String) CODEC (ZSTD(1)), attributes_string_24 Map(String, String) CODEC (ZSTD(1)), attributes_string_25 Map(String, String) CODEC (ZSTD(1)), attributes_string_26 Map(String, String) CODEC (ZSTD(1)), attributes_string_27 Map(String, String) CODEC (ZSTD(1)), attributes_string_28 Map(String, String) CODEC (ZSTD(1)), attributes_string_29 Map(String, String) CODEC (ZSTD(1)), attributes_string_30 Map(String, String) CODEC (ZSTD(1)), attributes_string_31 Map(String, String) CODEC (ZSTD(1)), attributes_string_32 Map(String, String) CODEC (ZSTD(1)), attributes_string_33 Map(String, String) CODEC (ZSTD(1)), attributes_string_34 Map(String, String) CODEC (ZSTD(1)), attributes_string_35 Map(String, String) CODEC (ZSTD(1)), attributes_string_36 Map(String, String) CODEC (ZSTD(1)), attributes_string_37 Map(String, String) CODEC (ZSTD(1)), attributes_string_38 Map(String, String) CODEC (ZSTD(1)), attributes_string_39 Map(String, String) CODEC (ZSTD(1)), attributes_float_0 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_1 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_2 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_3 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_4 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_5 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_6 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_7 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_8 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_9 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_10 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_11 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_12 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_13 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_14 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_15 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_16 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_17 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_18 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_19 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_20 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_21 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_22 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_23 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_24 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_25 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_26 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_27 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_28 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_29 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_30 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_31 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_32 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_33 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_34 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_35 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_36 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_37 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_38 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_39 Map(String, Float64) CODEC (ZSTD(1))) AS SELECT organization_id, project_id, item_type, timestamp, trace_id, item_id, attributes_bool, attributes_int, attributes_array, attributes_string_0, attributes_string_1, attributes_string_2, attributes_string_3, attributes_string_4, attributes_string_5, attributes_string_6, attributes_string_7, attributes_string_8, attributes_string_9, attributes_string_10, attributes_string_11, attributes_string_12, attributes_string_13, attributes_string_14, attributes_string_15, attributes_string_16, attributes_string_17, attributes_string_18, attributes_string_19, attributes_string_20, attributes_string_21, attributes_string_22, attributes_string_23, attributes_string_24, attributes_string_25, attributes_string_26, attributes_string_27, attributes_string_28, attributes_string_29, attributes_string_30, attributes_string_31, attributes_string_32, attributes_string_33, attributes_string_34, attributes_string_35, attributes_string_36, attributes_string_37, attributes_string_38, attributes_string_39, attributes_float_0, attributes_float_1, attributes_float_2, attributes_float_3, attributes_float_4, attributes_float_5, attributes_float_6, attributes_float_7, attributes_float_8, attributes_float_9, attributes_float_10, attributes_float_11, attributes_float_12, attributes_float_13, attributes_float_14, attributes_float_15, attributes_float_16, attributes_float_17, attributes_float_18, attributes_float_19, attributes_float_20, attributes_float_21, attributes_float_22, attributes_float_23, attributes_float_24, attributes_float_25, attributes_float_26, attributes_float_27, attributes_float_28, attributes_float_29, attributes_float_30, attributes_float_31, attributes_float_32, attributes_float_33, attributes_float_34, attributes_float_35, attributes_float_36, attributes_float_37, attributes_float_38, attributes_float_39, downsampled_retention_days AS retention_days, sampling_weight * 64 AS sampling_weight, sampling_factor / 64 AS sampling_factor FROM eap_items_1_local WHERE (cityHash64(item_id + 64)  % 64) = 0;
Local op: DROP TABLE IF EXISTS eap_items_1_downsample_64_mv_5 SYNC;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS eap_items_1_downsample_512_mv_4 TO eap_items_1_downsample_512_local (organization_id UInt64, project_id UInt64, item_type UInt8, timestamp DateTime CODEC (DoubleDelta, ZSTD(1)), trace_id UUID, item_id UInt128, sampling_weight UInt64 CODEC (ZSTD(1)), sampling_factor Float64 CODEC (ZSTD(1)), retention_days UInt16 CODEC (T64, ZSTD(1)), attributes_bool Map(String, Bool), attributes_int Map(String, Int64), attributes_array JSON(max_dynamic_paths=128) CODEC (ZSTD(1)), attributes_string_0 Map(String, String) CODEC (ZSTD(1)), attributes_string_1 Map(String, String) CODEC (ZSTD(1)), attributes_string_2 Map(String, String) CODEC (ZSTD(1)), attributes_string_3 Map(String, String) CODEC (ZSTD(1)), attributes_string_4 Map(String, String) CODEC (ZSTD(1)), attributes_string_5 Map(String, String) CODEC (ZSTD(1)), attributes_string_6 Map(String, String) CODEC (ZSTD(1)), attributes_string_7 Map(String, String) CODEC (ZSTD(1)), attributes_string_8 Map(String, String) CODEC (ZSTD(1)), attributes_string_9 Map(String, String) CODEC (ZSTD(1)), attributes_string_10 Map(String, String) CODEC (ZSTD(1)), attributes_string_11 Map(String, String) CODEC (ZSTD(1)), attributes_string_12 Map(String, String) CODEC (ZSTD(1)), attributes_string_13 Map(String, String) CODEC (ZSTD(1)), attributes_string_14 Map(String, String) CODEC (ZSTD(1)), attributes_string_15 Map(String, String) CODEC (ZSTD(1)), attributes_string_16 Map(String, String) CODEC (ZSTD(1)), attributes_string_17 Map(String, String) CODEC (ZSTD(1)), attributes_string_18 Map(String, String) CODEC (ZSTD(1)), attributes_string_19 Map(String, String) CODEC (ZSTD(1)), attributes_string_20 Map(String, String) CODEC (ZSTD(1)), attributes_string_21 Map(String, String) CODEC (ZSTD(1)), attributes_string_22 Map(String, String) CODEC (ZSTD(1)), attributes_string_23 Map(String, String) CODEC (ZSTD(1)), attributes_string_24 Map(String, String) CODEC (ZSTD(1)), attributes_string_25 Map(String, String) CODEC (ZSTD(1)), attributes_string_26 Map(String, String) CODEC (ZSTD(1)), attributes_string_27 Map(String, String) CODEC (ZSTD(1)), attributes_string_28 Map(String, String) CODEC (ZSTD(1)), attributes_string_29 Map(String, String) CODEC (ZSTD(1)), attributes_string_30 Map(String, String) CODEC (ZSTD(1)), attributes_string_31 Map(String, String) CODEC (ZSTD(1)), attributes_string_32 Map(String, String) CODEC (ZSTD(1)), attributes_string_33 Map(String, String) CODEC (ZSTD(1)), attributes_string_34 Map(String, String) CODEC (ZSTD(1)), attributes_string_35 Map(String, String) CODEC (ZSTD(1)), attributes_string_36 Map(String, String) CODEC (ZSTD(1)), attributes_string_37 Map(String, String) CODEC (ZSTD(1)), attributes_string_38 Map(String, String) CODEC (ZSTD(1)), attributes_string_39 Map(String, String) CODEC (ZSTD(1)), attributes_float_0 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_1 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_2 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_3 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_4 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_5 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_6 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_7 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_8 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_9 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_10 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_11 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_12 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_13 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_14 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_15 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_16 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_17 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_18 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_19 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_20 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_21 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_22 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_23 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_24 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_25 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_26 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_27 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_28 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_29 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_30 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_31 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_32 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_33 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_34 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_35 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_36 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_37 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_38 Map(String, Float64) CODEC (ZSTD(1)), attributes_float_39 Map(String, Float64) CODEC (ZSTD(1))) AS SELECT organization_id, project_id, item_type, timestamp, trace_id, item_id, attributes_bool, attributes_int, attributes_array, attributes_string_0, attributes_string_1, attributes_string_2, attributes_string_3, attributes_string_4, attributes_string_5, attributes_string_6, attributes_string_7, attributes_string_8, attributes_string_9, attributes_string_10, attributes_string_11, attributes_string_12, attributes_string_13, attributes_string_14, attributes_string_15, attributes_string_16, attributes_string_17, attributes_string_18, attributes_string_19, attributes_string_20, attributes_string_21, attributes_string_22, attributes_string_23, attributes_string_24, attributes_string_25, attributes_string_26, attributes_string_27, attributes_string_28, attributes_string_29, attributes_string_30, attributes_string_31, attributes_string_32, attributes_string_33, attributes_string_34, attributes_string_35, attributes_string_36, attributes_string_37, attributes_string_38, attributes_string_39, attributes_float_0, attributes_float_1, attributes_float_2, attributes_float_3, attributes_float_4, attributes_float_5, attributes_float_6, attributes_float_7, attributes_float_8, attributes_float_9, attributes_float_10, attributes_float_11, attributes_float_12, attributes_float_13, attributes_float_14, attributes_float_15, attributes_float_16, attributes_float_17, attributes_float_18, attributes_float_19, attributes_float_20, attributes_float_21, attributes_float_22, attributes_float_23, attributes_float_24, attributes_float_25, attributes_float_26, attributes_float_27, attributes_float_28, attributes_float_29, attributes_float_30, attributes_float_31, attributes_float_32, attributes_float_33, attributes_float_34, attributes_float_35, attributes_float_36, attributes_float_37, attributes_float_38, attributes_float_39, downsampled_retention_days AS retention_days, sampling_weight * 512 AS sampling_weight, sampling_factor / 512 AS sampling_factor FROM eap_items_1_local WHERE (cityHash64(item_id + 512)  % 512) = 0;
Local op: DROP TABLE IF EXISTS eap_items_1_downsample_512_mv_5 SYNC;
-- end backward migration events_analytics_platform : 0051_items_sample_by_trace_id

davidtsuk · 2025-05-05T17:42:05Z

snuba/snuba_migrations/events_analytics_platform/0041_items_sample_by_trace_id.py

+        f"sampling_weight * {sampling_weight} AS sampling_weight, "
+        f"sampling_factor / {sampling_weight} AS sampling_factor "
+        "FROM eap_items_1_local "
+        f"WHERE (cityHash64(trace_id)  % {sampling_weight}) = 0"


This will break extrapolation. For example, imagine you have 2 traces:

trace 1 has 1000 spans

trace 2 has 5 spans

Let's assume the sampling weight is 2. If we sample only trace 1, we will have 2000 spans extrapolated, if we only sample trace 2, we will have 10 spans extrapolated. This leads to insane variance and thus the extrapolated numbers cannot be trusted. In general, our extrapolation formula assumes that events are sampled independently and here they are not sampled independently.

OK, so how can we change the formula to not assume this?

phacops · 2025-10-21T18:41:41Z

We're not doing this.

phacops · 2025-11-14T19:01:52Z

We're re-opening the discussion on this.

We're discussing doing this again.

codecov · 2025-11-14T19:11:42Z

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
806	1	805	8

View the top 1 failed test(s) by shortest run time

tests.migrations.test_runner::test_run_and_reverse_all

Stack Traces | 13.2s run time

Traceback (most recent call last):
  File ".../snuba/clickhouse/native.py", line 202, in execute
    result_data = query_execute()
                  ^^^^^^^^^^^^^^^
  File ".../snuba/clickhouse/native.py", line 185, in query_execute
    return conn.execute(  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 382, in execute
    rv = self.process_ordinary_query(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 580, in process_ordinary_query
    return self.receive_result(with_column_types=with_column_types,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.../sentry_sdk/integrations/clickhouse_driver.py", line 110, in _inner_end
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 212, in receive_result
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.../site-packages/clickhouse_driver/result.py", line 50, in get_result
    for packet in self.packet_generator:
  File "/.venv/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 228, in packet_generator
    packet = self.receive_packet()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 245, in receive_packet
    raise packet.exception
clickhouse_driver.errors.ServerException: Code: 524.
DB::Exception: Trying to ALTER DROP column attributes_array which is referenced by materialized view ['eap_items_1_downsample_8_mv_4','eap_items_1_downsample_512_mv_4','eap_items_1_downsample_64_mv_4']. Stack trace:

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f52449b
1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000009fc506c
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000009fc4a2b
3. DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000001468b55b
4. DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000001359ac49
5. DB::InterpreterAlterQuery::execute() @ 0x00000000135982ad
6. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr<DB::IAST>&) @ 0x00000000139f96ab
7. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000139f40c4
8. DB::TCPHandler::runImpl() @ 0x0000000014d2d60c
9. DB::TCPHandler::run() @ 0x0000000014d4b7f9
10. Poco::Net::TCPServerConnection::start() @ 0x000000001847f067
11. Poco::Net::TCPServerDispatcher::run() @ 0x000000001847f4b9
12. Poco::PooledThread::run() @ 0x000000001844b81b
13. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000018449cfd
14. ? @ 0x00007f58f86e2ac3
15. ? @ 0x00007f58f8773a04


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/.venv/lib/python3.11........./site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/.venv/lib/python3.11........./site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11....../site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11....../site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.../site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/.venv/lib/python3.11....../site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/.venv/lib/python3.11....../site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/.venv/lib/python3.11....../site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/.venv/lib/python3.11....../site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/.venv/lib/python3.11....../site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/.venv/lib/python3.11....../site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/.venv/lib/python3.11.../site-packages/_pytest/capture.py", line 880, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/.venv/lib/python3.11.../site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11........./site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File "/.venv/lib/python3.11....../site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/.venv/lib/python3.11....../site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11....../site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/.venv/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11....../site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../tests/migrations/test_runner.py", line 233, in test_run_and_reverse_all
    runner.reverse_migration(migration, force=True)
  File ".../snuba/migrations/runner.py", line 386, in reverse_migration
    self._reverse_migration_impl(migration_key)
  File ".../snuba/migrations/runner.py", line 507, in _reverse_migration_impl
    migration.backwards(context, dry_run)
  File ".../snuba/migrations/migration.py", line 198, in backwards
    op.execute()
  File ".../snuba/migrations/operations.py", line 159, in execute
    super().execute()  # type: ignore
    ^^^^^^^^^^^^^^^^^
  File ".../snuba/migrations/operations.py", line 100, in execute
    connection.execute(self.format_sql(), settings=self._settings)
  File ".../snuba/clickhouse/native.py", line 281, in execute
    raise ClickhouseError(e.message, code=e.code) from e
snuba.clickhouse.errors.ClickhouseError: DB::Exception: Trying to ALTER DROP column attributes_array which is referenced by materialized view ['eap_items_1_downsample_8_mv_4','eap_items_1_downsample_512_mv_4','eap_items_1_downsample_64_mv_4']. Stack trace:

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f52449b
1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000009fc506c
2. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000009fc4a2b
3. DB::MergeTreeData::checkAlterIsPossible(DB::AlterCommands const&, std::shared_ptr<DB::Context const>) const @ 0x000000001468b55b
4. DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000001359ac49
5. DB::InterpreterAlterQuery::execute() @ 0x00000000135982ad
6. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr<DB::IAST>&) @ 0x00000000139f96ab
7. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000139f40c4
8. DB::TCPHandler::runImpl() @ 0x0000000014d2d60c
9. DB::TCPHandler::run() @ 0x0000000014d4b7f9
10. Poco::Net::TCPServerConnection::start() @ 0x000000001847f067
11. Poco::Net::TCPServerDispatcher::run() @ 0x000000001847f4b9
12. Poco::PooledThread::run() @ 0x000000001844b81b
13. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000018449cfd
14. ? @ 0x00007f58f86e2ac3
15. ? @ 0x00007f58f8773a04

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

feat(eap): Sample by trace

d6847dd

phacops requested review from a team as code owners May 5, 2025 16:09

github-actions bot added the migrations label May 5, 2025

Use hard-coded column list

8af7422

phacops added 2 commits May 5, 2025 12:42

Fix typing

5aeb9d2

Fix column name

5f2acd4

davidtsuk previously requested changes May 5, 2025

View reviewed changes

phacops force-pushed the master branch from 7fca603 to c8950ad Compare September 7, 2025 04:38

phacops closed this Oct 21, 2025

phacops reopened this Nov 14, 2025

Merge branch 'master' into pierre/eap-sample-by-trace

fadef7f

Update migration to latest state

e3d8618

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(eap): Sample by trace #7158

feat(eap): Sample by trace #7158

Uh oh!

phacops commented May 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented May 5, 2025 •

edited

Loading

Uh oh!

davidtsuk May 5, 2025 •

edited

Loading

Uh oh!

phacops May 5, 2025

Uh oh!

phacops commented Oct 21, 2025

Uh oh!

phacops commented Nov 14, 2025

Uh oh!

codecov bot commented Nov 14, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

feat(eap): Sample by trace #7158

Are you sure you want to change the base?

feat(eap): Sample by trace #7158

Uh oh!

Conversation

phacops commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidtsuk May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

phacops May 5, 2025

Choose a reason for hiding this comment

Uh oh!

phacops commented Oct 21, 2025

Uh oh!

phacops commented Nov 14, 2025

Uh oh!

codecov bot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 1 Tests Failed:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

phacops commented May 5, 2025 •

edited

Loading

github-actions bot commented May 5, 2025 •

edited

Loading

davidtsuk May 5, 2025 •

edited

Loading

codecov bot commented Nov 14, 2025 •

edited

Loading