Shardy Pass Doc Diff
创建于 差异永不过期
<!-- Autogenerated by mlir-tblgen; don't manually edit -->
<!-- Autogenerated by mlir-tblgen; don't manually edit -->
### `-sdy-aggressive-propagate`
### `-sdy-aggressive-propagate`
_Runs the aggressive sharding propagation algorithm._
_Runs the aggressive sharding propagation algorithm._
Propagate shardings with an aggressive factor propagation strategy. The
Propagate shardings with an aggressive factor propagation strategy. The
basic strategy only propagates shardings without conflicts, while the
basic strategy only propagates shardings without conflicts, while the
aggressive strategy resolves conflicts. Higher aggressiveness can reduce the
aggressive strategy resolves conflicts. Higher aggressiveness can reduce the
memory footprint at the cost of potential communication.
memory footprint at the cost of potential communication.
Options:
Options:
* All options from `BasicPropagationPass`
* All options from `BasicPropagationPass`
* `-propagation-strategy` : which factor propagation strategy to use
* `-propagation-strategy` : which factor propagation strategy to use
### `-sdy-basic-propagate`
### `-sdy-basic-propagate`
_Runs the basic sharding propagation algorithm._
_Runs the basic sharding propagation algorithm._
The basic propagation algorithm is the lowest strategy of propagation in the
The basic propagation algorithm is the lowest strategy of propagation in the
hierarchy, that doesn't do any conflict resolution, and instead propagates
hierarchy, that doesn't do any conflict resolution, and instead propagates
axes that are compatible between all operands and results.
axes that are compatible between all operands and results.
Options:
Options:
* `-keep-sharding-rules` : whether to keep existing and created op
* `-keep-sharding-rules` : whether to keep existing and created op
sharding rules
sharding rules
* `-module-dump-directory` : where to dump any rewritten modules for
* `-module-dump-directory` : where to dump any rewritten modules for
debugging
debugging
* `-conservative-propagation` : whether to disallow split axes and
* `-conservative-propagation` : whether to disallow split axes and
non-divisible sharding axes during propagation
non-divisible sharding axes during propagation
### `-sdy-op-priority-propagate`
### `-sdy-op-priority-propagate`
_Runs the op-priority propagation algorithm._
_Runs the op-priority propagation algorithm._
Propagates shardings in iterations, based on a set of op heuristics. Each
Propagates shardings in iterations, based on a set of op heuristics. Each
heuristic determines whether to propagate in forward, backward, or both
heuristic determines whether to propagate in forward, backward, or both
directions for a given op.
directions for a given op.
This starts at op-priority 0 and increases until all op heuristics have been
This starts at op-priority 0 and increases until all op heuristics have been
exhausted and the program has reached a fixed-point.
exhausted and the program has reached a fixed-point.
For a priority p, all op heuristics are considered from 0 <= i < p. The most
For a priority p, all op heuristics are considered from 0 <= i < p. The most
expressive direction is chosen for each op, with
expressive direction is chosen for each op, with
`BOTH > BACKWARD == FORWARD > NONE`. If during this iteration,
`BOTH > BACKWARD == FORWARD > NONE`. If during this iteration,
`FORWARD`/`BACKWARD` is seen first then `BACKWARD`/`FORWARD` is seen, the
`FORWARD`/`BACKWARD` is seen first then `BACKWARD`/`FORWARD` is seen, the
direction is `BOTH`.
direction is `BOTH`.
This propagation strategy extends the aggressive propagation strategy, which
This propagation strategy extends the aggressive propagation strategy, which
means that at each op-priority iteration, a full aggressive propagation is
means that at each op-priority iteration, a full aggressive propagation is
applied (see `AggressivePropagationPass`).
applied (see `AggressivePropagationPass`).
Options:
Options:
* All options from `AggressivePropagationPass`
* All options from `AggressivePropagationPass`
* `-run-op-priority-propagation` : whether to run (or skip) op-priority
* `-run-op-priority-propagation` : whether to run (or skip) op-priority
propagation
propagation
### `-sdy-populate-op-sharding-rules`
### `-sdy-populate-op-sharding-rules`
_Populates all registered ops with an `OpShardingRuleAttr`._
_Populates all registered ops with an `OpShardingRuleAttr`._
Populates all registered ops with an `OpShardingRuleAttr`, which is used for
Populates all registered ops with an `OpShardingRuleAttr`, which is used for
debugging/testing the registered sharding rules. Propagation already does
debugging/testing the registered sharding rules. Propagation already does
this just-in-time, but this pass does it all at once.
this just-in-time, but this pass does it all at once.
#### Options
#### Options
```
```
-conservative-propagation : whether to disllow rules that can propagate non-divisible sharding axes
-conservative-propagation : whether to disllow rules that can propagate non-divisible sharding axes
```
```
### `-sdy-user-priority-propagate`
### `-sdy-user-priority-propagate`
_Runs the user-priority propagation algorithm._
_Runs the user-priority propagation algorithm._
Propagates shardings in iterations, starting from the highest priority
Propagates shardings in iterations, starting from the highest priority
(lowest number, normally p0) to the lowest priority (highest number). In
(lowest number, normally p0) to the lowest priority (highest number). In
other words, at the i-th iteration, only dimension shardings with priority
other words, at the i-th iteration, only dimension shardings with priority
<=i are propagated.
<=i are propagated.
This propagation strategy extends the op-priority propagation strategy,
This propagation strategy extends the op-priority propagation strategy,
which means that at each user-priority iteration, a full op-priority
which means that at each user-priority iteration, a full op-priority
propagation is applied (see `OpPriorityPropagationPass`).
propagation is applied (see `OpPriorityPropagationPass`).
Options:
Options:
* All options from `OpPriorityPropagationPass`
* All options from `OpPriorityPropagationPass`