Skip to content

Conversation

@guswynn
Copy link
Contributor

@guswynn guswynn commented Mar 21, 2024

I was being wrong with this option:

Roshan found #26189, and #26131 made me realize that future clusters wont get the new updates.

Motivation

  • This PR fixes a recognized bug.

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • This PR includes the following user-facing behavior changes:

@guswynn guswynn added the release-blocker Critical issue that should block *any* release if not fixed label Mar 21, 2024
@guswynn guswynn requested review from a team, danhhz, rjobanp and teskje March 21, 2024 19:26
@guswynn guswynn force-pushed the fix-dyncfg-storage branch from 7db9d54 to 20032ed Compare March 21, 2024 19:28
@guswynn guswynn requested a review from a team as a code owner March 21, 2024 19:28
.storage_configuration
.update(storage_parameters);

// Clear out the updates as we no longer forward them to anyone else to process.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rjobanp maybe I should just not do this, it only saves one set of updates per-worker

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh actually we clone self.storage_state.storage_configuration a bunch during rendering...

@guswynn guswynn force-pushed the fix-dyncfg-storage branch from 20032ed to e54db90 Compare March 21, 2024 19:31
Copy link
Contributor

@rjobanp rjobanp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM to get in for this release. It would be great to add some tests around this that validate dyncfg update propagation across clusters and to new clusters, etc in a future PR

@guswynn guswynn enabled auto-merge March 21, 2024 19:33
@guswynn
Copy link
Contributor Author

guswynn commented Mar 21, 2024

@rjobanp definitely, but we would need to think of a way to expose parameter values in an testable way...maybe through the internal http endpoint?

@danhhz
Copy link
Contributor

danhhz commented Mar 21, 2024

@rjobanp definitely, but we would need to think of a way to expose parameter values in an testable way...maybe through the internal http endpoint?

Metrics!

@guswynn
Copy link
Contributor Author

guswynn commented Mar 21, 2024

Oh @danhhz thats a good idea, a configset metric with a name label

@guswynn guswynn merged commit 3c8ecd1 into MaterializeInc:main Mar 21, 2024
@guswynn guswynn deleted the fix-dyncfg-storage branch March 22, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-blocker Critical issue that should block *any* release if not fixed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants