Http stats

[gogasca]

Add support for HTTP stats server that initialize a HTTP server and display HTTP codes and backend latencies. It also allows contact for localhost:<>/debug/vars to see internal server information including requests. This is disabled by default. Also fixes a race condition found during testing.

Build local server

   go build -o ~/bin/inverting-proxy ./server
   go build -o ~/bin/proxy-forwarding-agent ./agent
   go build -o ~/bin/inverting-proxy-run-local testing/runlocal/main.go
   GOPATH=$HOME ~/bin/inverting-proxy-run-local --port 8081

Testing

    cd ~/inverting-proxy
         go test -race -v ./agent/sessions/...
         go test -race -v ./agent/stats/...
         go test -race -v ./agent/metrics/...
         go test -race -v ./agent/utils/...
         go test -race -v ./agent/websockets/...
         go test -race -v ./agent/...

To watch local page, used lynx:

watch -n 5 "curl -s http://localhost:3000 | lynx -stdin -dump"

and hey

hey -z 1m -q 10 http://127.0.0.1:8081/

to generate requests.

Example:

curl -s http://localhost:3000 | lynx -stdin -dump
                          Inverting Proxy Agent Stats

   Backend ID: testBackend

   Proxy URL: http://localhost:8081/

Response Codes

   Code Count
   200  70001
   307  69991

Response Times (ms)

   Percentile Time (ms)
   p50        36.62
   p90        63.86
   p99        93.06

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[ojarjur]

Can you give more background on what the goal is with this change?

In particular, I'm surprised that we would host these metrics on a local server rather than pushing them to StackDriver monitoring.

[gogasca]

@ojarjur the goal was to use the existing Health agent that pushes metrics via OpenTelemetry and can keep track of Proxy connection. The goal is to solve 2 problems: 1) Currently only by querying docker status for Proxy Agent we were able to know if Proxy agent was up or not, but there is not Health endpoint that allow us to know if Proxy agent is running and if its correctly connected to Proxy Server or not. 2) Use single health agent to push metrics to Cloud Monitoring. I agree #2 sounds redudant but can help centralize logs/metrics.