Skip to content

Feature/bert #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
pgimenes wants to merge 299 commits into
base: main
Choose a base branch
from
Open

Feature/bert #186

pgimenes wants to merge 299 commits into from

Conversation

@pgimenes
Copy link
Collaborator

@pgimenes pgimenes commented May 15, 2024

No description provided.

J-u1i0 and others added 30 commits March 10, 2024 18:45
@J-u1i0
Made LUT parameterisable by width but not by size
21ba407
@J-u1i0
Fixed testbench and reduced register sizes of nr stage
7e0bfe6
@J-u1i0
Updated the testbenches to import software module parts from a single…
d7c247d 
… source
@J-u1i0
All isqrt test passing
7ae18b9
@J-u1i0
Fixed assert stmt
8670894
@dereklai1
factored out lut parameter dict
60a8a32
@dereklai1
invsqrt working but group norm sometimes has 1 bit error, fix is WIP
0e52760
@dereklai1
wip: groupnorm2d still failing test, have no clue how it is expecting…
d2670a0 
… unknown data??
remove nettype none for vivado bug
aa896a3
added tcl script for vivado
88e434a
@dereklai1
changed all software quantized modules
8619dda
@dereklai1
added an error threshold monitor, groupnorm2d still has very weird hi…
54fc5f8 
…gh error bug in streaming test
@dereklai1
updated error threshold monitor to support no check
2dfbab0
@dereklai1
major fix to fifo, groupnorm2d now passing
781b688
@dereklai1
add instance norm and fixed software emit verilog
94f4123
@dereklai1
added batchnorm2d, groupnorm and instance norm quantizable modules in…
366d54c 
… mase
@dereklai1
removed fixed signed cast
f5a71bc
@dereklai1
added lut common element
4aff8e9
@dereklai1
removed local path
26150c1
@dereklai1
integrated new lut into fixed isqrt
ad5ae52
@dereklai1
fixed streaming mon, added new lut isqrt into groupnorm 2d, added mor…
e0eef35 
…e tests for groupnorm 2d
@dereklai1
added unified norm testbench and removed old temporary inv sqrt
a9400c7
@dereklai1
updated emit verilog tb, top level norm still failing
744d69f
@dereklai1
fixed isqrt linting error
1eba479
@dereklai1
removed unused params from tb
d069332
@dereklai1
pipelined isqrt module and integrated into group norm 2d
609963a
@dereklai1
added new rms norm extension layers, hardware not passing tb
36886f0
@dereklai1
fixed rms norm implementation and added tests for hw
524073a
@dereklai1
added rms norm to software stack
3183671
@dereklai1
refactored rms_norm and test_emit_verilog_norm test
69d1b11
pgimenes and others added 30 commits May 23, 2024 13:59
@pgimenes
pass linear synth check
c3b130d
@pgimenes
update linear tb
d5f6c19
@pgimenes
emit verilog linear fixes (for galapagos)
419471a
@pgimenes
match sequence length with parallelism in emit bert smoke test
9577f09
@pgimenes
mount board files in make shell
0bc0d74
@pgimenes
Merge branch 'feature/bert' of https://github.com/DeepWok/mase into f…
c9548f8 
…eature/bert
@Aaron-Zhao123
fix reg_slicer related stuff when calling python src/mase_components/…
9ea719b 
…linear/test/test_lint_linear.py
@Aaron-Zhao123
Merge remote-tracking branch 'refs/remotes/origin/feature/bert' into …
0c6f92e 
…feature/bert
@Aaron-Zhao123
now passed python3 test/passes/graph/transforms/verilog/test_emit_ver…
7e08de4 
…ilog_bert.py for real
@pgimenes
nas bert
308aca3
@pgimenes
Merge branch 'feature/bert' of https://github.com/DeepWok/mase into f…
b8c2ff3 
…eature/bert
@pgimenes
[NAS-BERT] layerwise search space working
0ac62fd
@pgimenes
[NAS-BERT] include num heads/hidden dim in search space, plot pareto
17d4a24
@pgimenes
compatibility changes for questasim
28b5b4e
@pgimenes
comparator fix to pass larger config
911cfe7
@pgimenes
fix mount for beholder and register old gelu removal
340f9b7
initial commit
b5bdad3
@pgimenes
make simulator choice explicit
8158db1
@pgimenes
initialize llama, mistral
568a822
@pgimenes
auto pipeline for emit verilog
9db2122
@pgimenes
changes for emitting llama
bcb9112
@pgimenes
cvxpy bug for resharding cost term in intra operator pass
b67d712
@pgimenes
update resharding cost model and ILP constraints
82b706e
@pgimenes
some refactoring
e1cd29a
@pgimenes
attach to runtime
b3ecaac
@pgimenes
handle resharding between nodes, improve logging
df4a02b
@pgimenes
make sharding decision dict based instead of positional
74c2712
@pgimenes
Merge branch 'research/alpa-light' into feature/bert
6d2a9b9
@pgimenes
run autosharding on huggingface bert
63b633b
@pgimenes
emit verilog for bert doc
661aec8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@pgimenes @J-u1i0 @dereklai1 @btarifi10 @Aaron-Zhao123