Skip to content

Add SVGD #98

@SamDuffield

Description

@SamDuffield

We should definitely add Stein variational gradient descent (paper, code)

SVGD requires a kernel specification. IMO we don't need to supply a suite of kernels ourselves (aside from maybe a default Gaussian kernel).

I think we should enforce a kernel signature like

eval = kernel(params1, params2, aux1, aux2, batch, **kernel_params)

where kernel_params are any kernel hyperparameters such as bandwidth.

To future-proof against more sophisticated kernels that e.g. could use info from the model call via aux1 and aux2.

Also we should think about how to support adaptive kernel_params updates like the median heuristic used in the SVGD [paper](https://proceedings.neurips.cc/paper/2016/file/b3ba8f1bee1238a2f37603d90b58898d-Paper.pdf.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions