Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,15 @@ SecretNote SF 现在隐语实训平台提供云端版本,开箱即用,无需

SecretNote SCQL 提供了 P2P SCQL 的产品化封装,降低了开发者配置 CCL 和编写 Query 的难度。

**版本兼容性说明:** SecretNote 当前适配 SCQL 1.0.0b1 及以上版本。如需使用旧版本 SCQL,请参考对应的 SecretNote 历史版本。

- 安装 [SecretNote Python 包](https://pypi.org/project/secretnote),Python 版本需在 3.10 及以上、3.12 以下

```sh
pip install secretnote
```

- 使用 [P2P 方式](https://www.secretflow.org.cn/zh-CN/docs/scql/main/topics/deployment/how-to-deploy-p2p-cluster) 拉起 SCQL 环境,参考 [scql/examples/p2p-tutorial](https://github.com/secretflow/scql/tree/main/examples/p2p-tutorial),下载该文件夹,执行
- 使用 [P2P 方式](https://www.secretflow.org.cn/zh-CN/docs/scql/1.0.0b1/topics/deployment/how-to-deploy-p2p-cluster) 拉起 SCQL 环境,参考 [scql/examples/p2p-tutorial](https://github.com/secretflow/scql/tree/main/examples/p2p-tutorial),下载该文件夹,执行

```sh
bash setup.sh && docker compose up -d
Expand Down
4 changes: 3 additions & 1 deletion docs/SCQL-INTRO.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@

下面演示在一台机器上使用 SCQL + SecretNote 来模拟不同参与方完成一次 SCQL Query。

**版本兼容性说明:** 本文档适配 SCQL 1.0.0b1 及以上版本。如需使用旧版本 SCQL,请参考对应的 SecretNote 历史版本。

### 准备环境

使用 docker 以 P2P 模式部署 SCQL 系统,可以参考[官网文档](https://www.secretflow.org.cn/zh-CN/docs/scql/0.5.0b2/topics/deployment/how-to-deploy-p2p-cluster)。简单起见,我们直接使用 [scql p2p examples](https://github.com/secretflow/scql/tree/main/examples/p2p-tutorial)。
使用 docker 以 P2P 模式部署 SCQL 系统,可以参考[官网文档](https://www.secretflow.org.cn/zh-CN/docs/scql/1.0.0b1/topics/deployment/how-to-deploy-p2p-cluster)。简单起见,我们直接使用 [scql p2p examples](https://github.com/secretflow/scql/tree/main/examples/p2p-tutorial)。

![p2p-tutorial.jpg](./images/p2p-tutorial.png)

Expand Down
10 changes: 8 additions & 2 deletions packages/secretnote-scql/src/modules/scql-broker/service.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// Service for interacting with backend's broker APIs only without states.
// @see https://www.secretflow.org.cn/zh-CN/docs/scql/0.9.0b1/reference/broker-api
// @see https://www.secretflow.org.cn/zh-CN/docs/scql/1.0.0b1/reference/broker-api
// @see pyprojects/secretnote/secretnote/scql/server/services/broker_manager.py
// The schema follows the document, best effort.

Expand Down Expand Up @@ -133,18 +133,24 @@ export type Tensor = {
shape: TensorShape;
elem_type: 'PrimitiveDataType_UNDEFINED' | TableColumnDesc['dtype'];
option: 'VALUE' | 'REFERENCE' | 'VARIABLE';
status:
status?:
| 'TENSORSTATUS_UNKNOWN'
| 'TENSORSTATUS_PRIVATE'
| 'TENSORSTATUS_SECRET'
| 'TENSORSTATUS_CIPHER'
| 'TENSORSTATUS_PUBLIC';
annotation?: any;
// Legacy fields for backwards compatibility with SCQL < 1.0.0b1
int32_data?: number[];
int64_data?: number[];
float_data?: number[];
double_data?: number[];
string_data?: string[];
bool_data?: boolean[];
// New field for SCQL >= 1.0.0b1
ss?: {
ss?: string[];
};
};

// scql.pb.QueryResult
Expand Down
8 changes: 8 additions & 0 deletions packages/secretnote-scql/src/modules/scql-editor/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,14 @@ export class QueryService {

getRow(row: Tensor) {
let res;

// For SCQL >= 1.0.0b1, check the new ss field first
const ssData = row.ss?.ss;
if (ssData && ssData.length) {
return ssData;
}

// For backwards compatibility with SCQL < 1.0.0b1, check legacy fields
(
[
'int32_data',
Expand Down
4 changes: 3 additions & 1 deletion packages/secretnote-scql/src/modules/welcome/welcome.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

下面演示在一台机器上使用 SCQL + SecretNote 来模拟不同参与方完成一次 SCQL Query。

**版本兼容性说明:** SecretNote 当前适配 SCQL 1.0.0b1 及以上版本。

---

## 准备环境

使用 docker 以 P2P 模式部署 SCQL 系统,可以参考[官网文档](https://www.secretflow.org.cn/zh-CN/docs/scql/0.5.0b2/topics/deployment/how-to-deploy-p2p-cluster)。简单起见,我们直接使用 [scql p2p examples](https://github.com/secretflow/scql/tree/main/examples/p2p-tutorial)。
使用 docker 以 P2P 模式部署 SCQL 系统,可以参考[官网文档](https://www.secretflow.org.cn/zh-CN/docs/scql/1.0.0b1/topics/deployment/how-to-deploy-p2p-cluster)。简单起见,我们直接使用 [scql p2p examples](https://github.com/secretflow/scql/tree/main/examples/p2p-tutorial)。

## 创建项目

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This is the manager that interacts with SCQL's broker, just like "broker's broker".
# SCQL itself exposes a set of APIs to manage projects, tables, and column control lists (CCLs).
# @see https://www.secretflow.org.cn/zh-CN/docs/scql/0.9.0b1/reference/broker-api
# @see https://www.secretflow.org.cn/zh-CN/docs/scql/1.0.0b1/reference/broker-api
# APIs here are consistent with those of SCQL's broker.

from typing import Any, Dict, List, Union
Expand All @@ -17,7 +17,9 @@
"db_type":"mysql","ref_table":"bob.user_stats","table_name":"tb","table_owner":"bob"}
ColumnControlList{"col":{"column_name":"ID","table_name":"ta"},"constraint":"PLAINTEXT",
"party_code":"alice"}
OutColumns{"elem_type":"STRING","name":"ID","option":"VALUE","shape":{"dim":[{"dim_value":"2"},
OutColumns (SCQL >= 1.0.0b1): {"elem_type":"STRING","name":"ID","option":"VALUE","shape":{"dim":[{"dim_value":"2"},
{"dim_value":"1"}]},"ss":{"ss":["alice","bob"]}}
OutColumns (SCQL < 1.0.0b1): {"elem_type":"STRING","name":"ID","option":"VALUE","shape":{"dim":[{"dim_value":"2"},
{"dim_value":"1"}]},"string_data":["alice","bob"]}
"""
Project = Dict[str, Any]
Expand Down