【OSCP】在 SecretFlow 中添加基于torch后端的fed_pac策略 #1276

Dorothy21650 · 2024-04-28T07:59:41Z

Type of change

Add new papers (Please tell us why you think this paper is awesome!)
Fix the category of an existing paper/papers (Please tell us the reasons)
[ x] Add a new tool/primitive/application with a new markdown page (Thank you! Also, please tell us more about this awesome thing!)

Description

Fixed #1280

add fedpac server

github-actions · 2024-04-28T07:59:52Z

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

Dorothy21650 · 2024-04-28T08:01:11Z

I have read the CLA Document and I hereby sign the CLA

wyanbing · 2024-04-28T08:03:43Z

I have read the CLA Document and I hereby sign the CLA

secretflow/ml/nn/core/torch/module.py

zhaocaibei123

加个单测，参考 tests/ml/nn/fl/test_fl_model_torch.py

secretflow/ml/nn/core/torch/module.py

zhaocaibei123 · 2024-05-06T11:18:13Z

secretflow/ml/nn/fl/backend/torch/fedpac_sampler.py

+from torch.utils.data import DataLoader
+
+
+def cifar10(stage='train'):


数据集处理写到单测，或者测试脚本

+1 挪到单测里使用custom databuilder模式，或者直接使用datasets里面的逻辑secretflow/utils/simulation/datasets.py

zhaocaibei123 · 2024-05-06T11:18:53Z

secretflow/ml/nn/fl/backend/torch/fl_base.py

@@ -49,6 +51,7 @@ def __init__(
        self.train_set = None
        self.eval_set = None
        self.skip_bn = skip_bn
+        # self.dataset_size = torch.tensor(len(self.train_set.dataset)).to(self.exe_device)


无用代码删掉

zhaocaibei123 · 2024-05-06T11:21:38Z

secretflow/ml/nn/fl/backend/torch/fl_base.py

@@ -214,11 +217,11 @@ def next_batch(self, stage="train"):
    def get_rows_count(self, filename):
        return int(rows_count(filename=filename)) - 1  # except header line

-    def get_weights(self, return_numpy=True):
+    def get_weights(self):


这里为什么删掉参数 return_numpy ？

secretflow/ml/nn/fl/backend/torch/fl_base.py

zhaocaibei123 · 2024-05-06T11:49:45Z

secretflow/ml/nn/fl/backend/torch/fl_base.py

+        self.num_classes = kwargs.get("num_classes", 10)
+        self.criterion = nn.CrossEntropyLoss()
+        self.local_model = self.model
+        self.last_model = deepcopy(self.model)


self.last_model 用来？

这里是照搬源码的，源码是在本地训练开始之前保存的模型的深度副本，可能是考虑到了回滚，但是源码中也没有明确使用 self.last_model，现在已经删除掉这行了。

zhaocaibei123 · 2024-05-06T12:01:17Z

secretflow/ml/nn/fl/backend/torch/fl_base.py

+from copy import deepcopy
+
+
+class FedPACTorchModel(BaseTorchModel):


这个类可以合到 strategy/fed_pac.py FedPAC

zhaocaibei123 · 2024-05-06T12:07:12Z

secretflow/ml/nn/fl/backend/torch/strategy/fed_pac.py

+        lr = kwargs.get("lr", 0.01)
+
+        epoch_classifier = 1
+        optimizer = torch.optim.SGD(


optimizer loss 之类的可以写到 TorchModel 里吗

zhaocaibei123 · 2024-05-06T12:10:06Z

secretflow/ml/nn/fl/backend/torch/strategy/fed_pac.py

+            momentum=0.5,
+            weight_decay=0.0005,
+        )
+        for ep in range(epoch_classifier):


每个 step 训练一个 epoch 吗

zhaocaibei123 · 2024-05-06T12:14:18Z

secretflow/security/aggregation/fedpac_aggregator.py

+from secretflow.security.aggregation.aggregator import Aggregator
+
+
+class FedPACAggregator(Aggregator):


average sum 逻辑没变的话可以继承 PlainAggregator

add fed_pac to init

…原有的一致

zhaocaibei123 · 2024-05-15T06:27:42Z

secretflow/ml/nn/fl/backend/torch/fl_base.py

@@ -216,9 +219,9 @@ def get_rows_count(self, filename):

    def get_weights(self, return_numpy=True):
        if self.skip_bn:
-            return self.model.get_weights_not_bn(return_numpy=return_numpy)
+            return self.model.get_weights_not_bn(return_numpy=True)


这里 return_numpy 为什么写死？

zhangxingmeng · 2024-05-17T09:52:13Z

secretflow/ml/nn/core/torch/module.py

@@ -23,7 +23,6 @@

 from .mixins import ParametersMixin

-


格式化配置一下，这里不要做修改

zhangxingmeng · 2024-05-17T09:53:21Z

secretflow/ml/nn/fl/backend/torch/fedpac_sampler.py

+    return dataset
+
+
+def batch_sampler(


batch_sampler已有实现，直接调用即可secretflow/ml/nn/fl/backend/torch/sampler.py

zhangxingmeng · 2024-05-17T09:54:25Z

secretflow/ml/nn/fl/backend/torch/fedpac_sampler.py

+from torch.utils.data import DataLoader
+
+
+def cifar10(stage='train'):


+1 挪到单测里使用custom databuilder模式，或者直接使用datasets里面的逻辑secretflow/utils/simulation/datasets.py

Dorothy21650 and others added 3 commits April 28, 2024 07:09

"fed_pac client part"

a9fe1a9

add fedpac server

d9761d4

Merge pull request #1 from wyanbing/main

3a3a800

add fedpac server

Dorothy21650 requested review from ian-huu, Jamie-Cui and warriorpaw as code owners April 28, 2024 07:59

wyanbing and others added 3 commits April 28, 2024 08:32

add fedpac in __init__.py

b92fcf0

Merge branch 'Dorothy21650:main' into main

bdc9334

add PYUFedPAC to init file

396deb9

ian-huu requested a review from zhaocaibei123 April 30, 2024 02:30

ian-huu reviewed Apr 30, 2024

View reviewed changes

secretflow/ml/nn/core/torch/module.py Outdated Show resolved Hide resolved

zhaocaibei123 requested changes May 6, 2024

View reviewed changes

Merge pull request #2 from wyanbing/main

da49808

add fed_pac to init

Dorothy21650 changed the title ~~add fed_pac strategy based on torch backend~~ 【OSCP】在 SecretFlow 中添加基于torch后端的fed_pac策略 May 10, 2024

Dorothy21650 added 3 commits May 10, 2024 13:38

删掉模型定义，改成写在测试脚本中

745f338

1. 将flbase中新加的关于fedpac方法的类合并到fed_pac.py中。 2. 修改对eval_set的处理，改成和flbase…

4460a85

…原有的一致

修改flmodel_fedpac中对应的evaluate接口

12add29

zhaocaibei123 reviewed May 15, 2024

View reviewed changes

zhangxingmeng reviewed May 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【OSCP】在 SecretFlow 中添加基于torch后端的fed_pac策略 #1276

【OSCP】在 SecretFlow 中添加基于torch后端的fed_pac策略 #1276

Dorothy21650 commented Apr 28, 2024 •

edited

github-actions bot commented Apr 28, 2024 •

edited

Dorothy21650 commented Apr 28, 2024

wyanbing commented Apr 28, 2024

zhaocaibei123 left a comment

zhaocaibei123 May 6, 2024

zhangxingmeng May 17, 2024

zhaocaibei123 May 6, 2024

Dorothy21650 May 10, 2024

zhaocaibei123 May 6, 2024

zhaocaibei123 May 6, 2024

Dorothy21650 May 10, 2024

zhaocaibei123 May 6, 2024

Dorothy21650 May 10, 2024

zhaocaibei123 May 6, 2024

zhaocaibei123 May 6, 2024

zhaocaibei123 May 6, 2024

zhaocaibei123 May 15, 2024

zhangxingmeng May 17, 2024

zhangxingmeng May 17, 2024

zhangxingmeng May 17, 2024

		from torch.utils.data import DataLoader


		def cifar10(stage='train'):

		from copy import deepcopy


		class FedPACTorchModel(BaseTorchModel):

		from secretflow.security.aggregation.aggregator import Aggregator


		class FedPACAggregator(Aggregator):

【OSCP】 在 SecretFlow 中添加基于torch后端的fed_pa​​c策略 #1276

Are you sure you want to change the base?

【OSCP】 在 SecretFlow 中添加基于torch后端的fed_pa​​c策略 #1276

Conversation

Dorothy21650 commented Apr 28, 2024 • edited

github-actions bot commented Apr 28, 2024 • edited

Dorothy21650 commented Apr 28, 2024

wyanbing commented Apr 28, 2024

zhaocaibei123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

【OSCP】在 SecretFlow 中添加基于torch后端的fed_pac策略 #1276

【OSCP】在 SecretFlow 中添加基于torch后端的fed_pac策略 #1276

Dorothy21650 commented Apr 28, 2024 •

edited

github-actions bot commented Apr 28, 2024 •

edited