Rewrite collect_tokens implementations to use a flattened buffer
#77250
+254
−167
collect_tokens implementations to use a flattened buffer
#77250