dask.compute()函数怎么使用distributed调度器进行并行计算?
时间: 2024-05-13 19:15:05 浏览: 195
`dask.compute()`函数是用于触发Dask图计算的方法。如果要使用Distributed调度器进行并行计算,可以按照以下步骤:
1. 创建一个`Client`对象,指定Distributed调度器的地址:
```python
from dask.distributed import Client
client = Client('scheduler_address:8786')
```
2. 构建Dask图,并使用`client.submit()`方法将任务提交到Distributed调度器上:
```python
from dask import delayed
@delayed
def inc(x):
return x + 1
@delayed
def double(x):
return x * 2
data = [1, 2, 3, 4, 5]
incs = [inc(i) for i in data]
dbls = [double(i) for i in incs]
results = sum(dbls)
futures = client.compute(results)
```
3. 使用`dask.distributed.wait()`等待所有任务完成:
```python
from dask.distributed import wait
wait(futures)
```
4. 使用`dask.distributed.gather()`方法收集任务结果:
```python
from dask.distributed import gather
results = gather(futures)
```
5. 关闭`Client`对象:
```python
client.close()
```
这些步骤可以确保Dask图在Distributed调度器上进行并行计算。
阅读全文