Is there a critical threshold number of collaborators?

[tremblerz]

An interesting parameter appears in decentralized machine learning systems that is completely missing in a centralized FL system -- number of collaborators per node. This parameter can have a significant impact on the convergence of models in a decentralized setting.

A research question I am interested in understanding --

1. How does performance (convergence) scale as we increase this number. Is the improvement linear, exponential or logarithmic?
2. How does 1. change as we increase the number of users. Is there a scaling law here?
3. How do the above two results compare across different topologies?
4. In a truly decentralized system, number of collaborators per node will be different for different nodes. Therefore, what would be a good metric for comparison? Average degree of the graphs, centrality, clustering coefficient etc.?

There are too many moving pieces in this problem so a systematic research study will be a pretty impactful paper. The results will nicely corroborate with a strong body of empirical and theoretical work in graph theory and complex systems.