DTRConfig

class DTRConfig(eviction_threshold=0, evictee_minimum_size=1 << 20, recomp_memory_factor=1, recomp_time_factor=1)

Configuration for DTR memory optimization.

Parameters:

• eviction_threshold (int) – eviction threshold in bytes. When GPU memory usage exceeds this value, DTR will heuristically select and evict resident tensors until the amount of used memory falls below this threshold.

• evictee_minimum_size (int) – memory threshold of tensors in bytes. Only tensors whose size exceeds this threshold will be added to the candidate set. A tensor that is not added to the candidate set will never be evicted during its lifetime. Default: 1048576.

• recomp_memory_factor (float) – hyperparameter of the estimated memory of recomputing the tensor. The larger this value is, the less memory-consuming tensor will be evicted in heuristic strategies. This value is greater than or equal to 0. Default: 1.

• recomp_time_factor (float) – hyperparameter of the estimated time of recomputing the tensor. The larger this value is, the less time-consuming tensor will be evicted in heuristic strategies. This value is greater than or equal to 0. Default: 1.