I initially assumed that the window you get when running :LspInfo is a floating window. However after displaying border on it, it looks like this:
So it's definitely not following the floating window highlight groups. We need to investigate what type of window it is and the related highlight groups to have it match the floating window colors.
For those wondering, I displayed the border with:
require("lspconfig.ui.windows").default_options = { border = "rounded" }