Running shinyproxy in a resource constrained environment

[oker1]

I've been deploying ShinyProxy in a Kubernetes environment where the namespace has a resource quota set up. I've tried to set up container resources and proxy.max-total-instances to make sure that the resources can be fully utilized if needed but config rollouts can also succeed.

During testing I noticed that the scaling can overshoot above what I would expect based on the proxy.max-total-instances setting, it's easy to trigger it with minimum-seats-available: 1, allow-container-re-use: false, proxy.max-total-instances: 2 with a single user restarting the same app quickly. If there are proxies to clean up but a scale up is needed and can't succeed because of hitting the Kubernetes quota the proxy gets deadlocked in ScaleUp state because the released instances are not cleaned up in ScaleUp state.

I tried a small patch in containerproxy to consider instances waiting for cleanup against the proxy.max-total-instances setting in the decision whether to scale up or not so there will be no overshoot in scale up but the ToRemove instances are cleaned up first to make room for the scale up. It works in our environment not shooting above the quota.

This trades off increased latency of scaling up for not going above the limit so I think it could be a config option to toggle this behavior if most users are fine with the current overshoots. The latency in this case could be improved if the scheduling of the cleanup could be configurable instead of the current 20 seconds.

The patch:

diff --git a/src/main/java/eu/openanalytics/containerproxy/backend/dispatcher/proxysharing/ProxySharingScaler.java b/src/main/java/eu/openanalytics/containerproxy/backend/dispatcher/proxysharing/ProxySharingScaler.java
index 84cc62a2..ba91b6bf 100644
--- a/src/main/java/eu/openanalytics/containerproxy/backend/dispatcher/proxysharing/ProxySharingScaler.java
+++ b/src/main/java/eu/openanalytics/containerproxy/backend/dispatcher/proxysharing/ProxySharingScaler.java
@@ -323,7 +323,8 @@ public class ProxySharingScaler implements AutoCloseable {
             return;
         }
         long numPendingSeats = getNumPendingSeats();
-        long num = seatStore.getNumUnclaimedSeats() + numPendingSeats - pendingDelegatingProxies.size();
+        long numSeatsToRemove = delegateProxyStore.getAllDelegateProxies(DelegateProxyStatus.ToRemove).count();
+        long num = seatStore.getNumUnclaimedSeats() + numSeatsToRemove + numPendingSeats - pendingDelegatingProxies.size();
         debug(String.format("Status: %s, Unclaimed: %s + PendingDelegate: %s - PendingDelegating: %s = %s -> minimum: %s",
             lastReconcileStatus, seatStore.getNumUnclaimedSeats(), numPendingSeats,
             pendingDelegatingProxies.size(), num, specExtension.minimumSeatsAvailable));