modify handling shift network source conflicts #2509
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes #2350
This PR is to test a new method for handling shift network lowerings where multiple sources map to the same target, for example with ciphertexts of repeated content. The current method uses minimal distance between target and potential sources to select sources.
This code is still under development as it isn't running in reasonable time for the example MLIR and needs to be tested more if it improves current method. I'm submitting to see if there's any input on approach or implementation
The method used here is:
Shuffle target slots to fill.
Fill a target slot by sorting potential sources by 1) the count of targets the source has already been mapped to, and 2) the number of slots it would have a rotation conflict with ascending.
Any source with the minimal count can be picked at random.
Fill all the targets and track number of rotations needed and random seed used.
Repeat the process with a different random seed, here 50 times, and keep the best mapping.
This was implemented with these changes:
In ImplementShiftNetwork::findShiftScheme(), targetToSources is checked for multiple sources and proceeds if any.
All the potential sources are added as vertices to an all conflicts graph.
The edges needed to come from a strategy where all potential sources are included.
ShiftStrategy::evaluate() runs with all targetToSources instead of targetToSource
Sourceshifts is filled with each potential source and caculated shift to a potential target.
The collisions across rounds were calculated, this is where a bottleneck is for the original 4x4 duplicates 64 times ciphertext
The edges are added to the all conflicts graph.
Tests are run 50 times - shuffle targets and for each choose a minimal source, calculate new conflict graph & number of rotation groups, keep the mapping and conflict graph with lowest number rotation groups
To run use
This is the shorter example I used for tests/Dialect/TensorExt/Transforms/implement_shift_network_issue_2350.mlir