Add cross site evaluation to FedAvgRecipe #3695
Conversation
holgerroth
requested review from
YuanTingHsieh,
ZiyueXu77 and
chesterxgchen
|
/build |
…into fedavg_crosssiteeval
|
/build |
| parser.add_argument("--n_clients", type=int, default=2) | ||
| parser.add_argument("--num_rounds", type=int, default=2) | ||
| parser.add_argument("--batch_size", type=int, default=16) | ||
| parser.add_argument("--train_script", type=str, default="client.py") |
There was a problem hiding this comment.
why make this an argument, are we keep changing this file ?
There was a problem hiding this comment.
Makes it easier to use. Previous instructions required the user to change or "overwrite" the code in client.py...
chesterxgchen
left a comment
chesterxgchen
left a comment
There was a problem hiding this comment.
I think we should be consistent, if we have cross-site eval, should add to all recipes particulate pt, tensorflow, numpy, adding to just one doesn't seem serve the purpose.
nvflare/app_opt/pt/recipes/fedavg.py
Outdated
| job.to_server(controller) | ||
|
|
||
| if self.cross_site_eval: | ||
| model_locator_id = job.to_server(PTFileModelLocator(pt_persistor_id=job.comp_ids["persistor_id"])) |
There was a problem hiding this comment.
I realize this adds a dependency on PT. Having a utility function to add cross-site eval to an existing recipe, similar to how we do experiment tracking, might be a better option @chesterxgchen .
There was a problem hiding this comment.
I moved it to utils as
add_cross_site_evaluation(recipe, model_locator_type="pytorch")
and added option for numpy locator using a registry similar to tracking @YuanTingHsieh
|
/build |
|
/build |
|
@holgerroth lets not adding this 2.7.0 TP release, we are too late for this one |
Without cross-site evaluation workflow, this script will be not functional https://github.com/NVIDIA/NVFlare/blob/main/examples/hello-world/hello-pt/client_with_eval.py |
|
then remove it for next release, it’s too late.
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Holger Roth ***@***.***>
Sent: Saturday, September 20, 2025 9:03:37 PM
To: NVIDIA/NVFlare ***@***.***>
Cc: Chester Chen ***@***.***>; Mention ***@***.***>
Subject: Re: [NVIDIA/NVFlare] Add cross site evaluation to FedAvgRecipe (PR #3695)
[https://avatars.githubusercontent.com/u/6304754?s=20&v=4]holgerroth left a comment (NVIDIA/NVFlare#3695)<#3695 (comment)>
@holgerroth<https://github.com/holgerroth> lets not adding this 2.7.0 TP release, we are too late for this one
Without cross-site evaluation workflow, this script will be not functional https://github.com/NVIDIA/NVFlare/blob/main/examples/hello-world/hello-pt/client_with_eval.py
—
Reply to this email directly, view it on GitHub<#3695 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAD5FQYXHBKPYZWPR6DEJYD3TYPRTAVCNFSM6AAAAACGWXW2NSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJVGQ4DEMZXG4>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
@holgerroth is this still in draft status ? |
2 participants
Fixes # .
Description
Adds an option for cross-site evaluation to FedAvgRecipe. Tested with existing client_with_eval.py in hello-pt example.
Types of changes
./runtest.sh.