Deeploy.TilingExtension.TilerExtension.Tiler

class Deeploy.TilingExtension.TilerExtension.Tiler(memoryHierarchy: MemoryHierarchy, testName: str | None = None, workDir: str | None = None)

Bases: object

Tiler for a computation graphs with memory-aware optimization.

The Tiler class provides functionality for tiling operations to fit within memory constraints of target hardware platforms. It performs memory allocation, constraint solving, and scheduling to optimize execution within hierarchical memory systems.

Parameters:

memoryHierarchy (MemoryHierarchy) – The memory hierarchy specification defining available memory levels and their capacities.

Variables:

arenaName (str) – Name prefix for memory arena buffers.
memorySchedulerClass (Type[MemoryScheduler]) – Class type for memory scheduler instances.
memoryHierarchy (MemoryHierarchy) – The memory hierarchy configuration.
tilerModel (Optional[TilerModel]) – The constraint solver model for tiling optimization.
innerMemoryScheduler (MemoryScheduler) – Scheduler for inner memory level allocation.
outerMemoryScheduler (MemoryScheduler) – Scheduler for outer memory level allocation.
symbolicMemoryConstraints (Optional[List[PatternMemoryConstraints]]) – Symbolic memory constraints for the tiling problem.
visualizeMemoryAlloc (bool) – Flag to enable memory allocation visualization.
memoryAllocStrategy ({"TetrisRandom", "TetrisCo-Opt", "MiniMalloc"}) – Strategy for memory allocation.
searchStrategy ({"min", "max", "random-max"}) – Search strategy for constraint solving.

Examples

>>> L3 = MemoryLevel(name = "L3", neighbourNames = ["L2"], size = 1024000)
>>> L2 = MemoryLevel(name = "L2", neighbourNames = ["L3", "L1"], size = 512000)
>>> L1 = MemoryLevel(name = "L1", neighbourNames = ["L2"], size = 128000)
>>> memoryHierarchy = MemoryHierarchy([L3, L2, L1])
>>> memoryHierarchy.setDefaultMemoryLevel("L3")
>>> tiler = Tiler(hierarchy)
>>> tiler.memoryAllocStrategy = "MiniMalloc"
>>> solution = tiler.computeTilingSchedule(context)

Methods

__init__(memoryHierarchy: MemoryHierarchy, testName: str | None = None, workDir: str | None = None)

Initialize the Tiler with a memory hierarchy.

Parameters:

memoryHierarchy (MemoryHierarchy) – The memory hierarchy specification defining available memory levels.
testName (Optional[str], optional) – Optional name for the test case, used for file naming. Defaults to None.
workDir (Optional[str], optional) – Optional working directory for temporary files. Defaults to None.

`__init__`(memoryHierarchy[, testName, workDir])	Initialize the Tiler with a memory hierarchy.
`annotateMemoryLevel`(ctxt, tilingSolution, ...)	Annotate memory constraints with actual address space allocations.
`assertLayerWiseTiling`(schedule)	Assert that the schedule uses layer-wise tiling (one node per pattern).
`assertUniformMemoryLevelAllocation`(ctxt, ...)	Assert that all local buffers are allocated to the default memory level.
`computeMemoryMap`(ctxt, tilingSolution)	Compute memory allocation map from the tiling solution.
`computeTilingSchedule`(ctxt)	Compute the optimal tiling schedule for the network.
`minimalloc`(memoryMap, ctxt, ...)	Perform memory allocation using the MiniMalloc external tool.
`multiBufferStrategy`(tilerModel, ctxt, ...)	Determine multi-buffering coefficient for a tensor in the tiling strategy.
`plotMemoryAlloc`(memoryMap, ctxt, ...)	Generate interactive visualization of memory allocation patterns.
`propagateIOBufferStrategy`(...)	Propagate I/O buffer strategy across the tiling pattern.
`setupModel`(ctxt, schedule, layerBinding, ...)	Set up the constraint optimization model for tiling.
`testMemoryMapCorrectness`(memoryMap, graph, ...)	Test the correctness of a computed memory map.
`testTilingSolutionCorrectness`(tilingSolution)	Test the correctness of a computed tiling solution.

Attributes

`arenaName`
`worstCaseBufferSize`	Get the worst-case buffer sizes for each memory level.

property worstCaseBufferSize

Get the worst-case buffer sizes for each memory level.

Returns:: Dictionary mapping memory level names to their worst-case buffer sizes in bytes.
Return type:: Dict[str, int]

plotMemoryAlloc(memoryMap: Dict[str, List[List[MemoryBlock]]], ctxt: NetworkContext, deeployStateDir: str, memoryHierarchy: MemoryHierarchy)

Generate interactive visualization of memory allocation patterns.

Creates an HTML file with Plotly visualizations showing memory allocation over time for each memory level in the hierarchy.

Parameters:

memoryMap (Dict[str, List[List[MemoryBlock]]]) – Memory allocation map containing blocks for each memory level and time step.
ctxt (NetworkContext) – Network context containing buffer information.
deeployStateDir (str) – Directory path where the visualization HTML file will be saved.
memoryHierarchy (MemoryHierarchy) – Memory hierarchy configuration for the visualization.

Notes

Generates a file named ‘memory_alloc.html’ in the specified directory. Each memory level is visualized as a separate subplot showing buffer lifetimes and address space usage.

minimalloc(memoryMap, ctxt, nodeMemoryConstraint, capacity: int, memoryLevel: str)

Perform memory allocation using the MiniMalloc external tool.

Interfaces with the external MiniMalloc memory allocator to compute optimal memory allocation for the given memory blocks and constraints.

Parameters:

memoryMap (List[MemoryBlock]) – List of memory blocks to be allocated.
ctxt (NetworkContext) – Network context containing buffer information.
nodeMemoryConstraint (Optional[NodeMemoryConstraint]) – Memory constraints for the current node, if available.
capacity (int) – Total memory capacity available for allocation.
memoryLevel (str) – Name of the memory level being allocated.

Returns:

Updated memory blocks with assigned address spaces.

Return type:

List[MemoryBlock]

Raises:

KeyError – If MINIMALLOC_INSTALL_DIR environment variable is not set.
subprocess.CalledProcessError – If the MiniMalloc tool fails to execute successfully.

Notes

Requires the MiniMalloc tool to be installed and the MINIMALLOC_INSTALL_DIR environment variable to be set to the installation directory.

computeTilingSchedule(ctxt: NetworkContext) → List[PatternMemoryConstraints]

Compute the optimal tiling schedule for the network.

Solves the constraint optimization problem to find the best tiling solution that satisfies memory and computational constraints.

Parameters:: ctxt (NetworkContext) – Network context containing the computational graph and constraints.
Returns:: The computed tiling solution with memory constraints for each pattern.
Return type:: TilingSolution
Raises:: AssertionError – If the tiler model or symbolic memory constraints are not initialized.

Notes

This method requires that setupModel() has been called previously to initialize the constraint model and symbolic memory constraints.

computeMemoryMap(ctxt: NetworkContext, tilingSolution: List[PatternMemoryConstraints]) → Dict[str, List[List[MemoryBlock]]]

Compute memory allocation map from the tiling solution.

Generates a concrete memory allocation map that assigns specific memory addresses to each buffer based on the tiling solution.

Parameters:

ctxt (NetworkContext) – Network context containing buffer information.
tilingSolution (TilingSolution) – The computed tiling solution.

Returns:

Dictionary mapping memory level names to lists of memory blocks for each time step.

Return type:

MemoryMap

Notes

The memory allocation strategy (TetrisRandom, TetrisCo-Opt, or MiniMalloc) determines how the actual memory addresses are assigned.

annotateMemoryLevel(ctxt: NetworkContext, tilingSolution: List[PatternMemoryConstraints], memoryMap: Dict) → NetworkContext

Annotate memory constraints with actual address space allocations.

Updates the memory constraints in the tiling solution with the actual address spaces computed during memory allocation.

Parameters:

ctxt (NetworkContext) – Network context containing buffer information.
tilingSolution (TilingSolution) – The tiling solution to be annotated.
memoryMap (Dict[str, List[List[MemoryBlock]]]) – Memory allocation map with assigned address spaces.

Returns:

Updated network context (returned for consistency).

Return type:

NetworkContext

Notes

This method modifies the tiling solution in-place by adding address space information to memory constraints.

setupModel(ctxt: NetworkContext, schedule: List[List[Node]] | List[Node], layerBinding: OrderedDict[str, ONNXLayer], targetMemoryLevelMapping: TargetMemoryLevelMapping) → NetworkContext

Set up the constraint optimization model for tiling.

Initializes the tiler model with geometric constraints, memory constraints, and optimization objectives based on the network schedule and layer bindings.

Parameters:

ctxt (NetworkContext) – Network context containing the computational graph.
schedule (Schedule) – Execution schedule defining the order of operations.
layerBinding (OrderedDict[str, ONNXLayer]) – Mapping from node names to their layer implementations.
targetMemoryLevelMapping (TargetMemoryLevelMapping) – Mapping defining which memory levels to use for each tensor.

Returns:

The network context (returned for consistency).

Return type:

NetworkContext

Notes

This method must be called before computeTilingSchedule() to initialize the constraint model and symbolic memory constraints.

multiBufferStrategy(tilerModel: TilerModel, ctxt: NetworkContext, pattern: List[Node], path: List[str], hop: str, tensorName: str) → int | IntVar

Determine multi-buffering coefficient for a tensor in the tiling strategy.

Computes the buffering factor (e.g., double buffering = 2) for a given tensor based on its type and usage pattern in the computation graph. This coefficient is used to determine how many copies of the tensor should be kept in memory.

Parameters:

tilerModel (TilerModel, (unused)) – The constraint solver model.
ctxt (NetworkContext) – Network context containing buffer information.
pattern (SubGraph, (unused)) – The computation pattern being analyzed.
path (List[str], (unused)) – Memory hierarchy path for the tensor.
hop (str, (unused)) – Current memory level in the path.
tensorName (str) – Name of the tensor to analyze.

Returns:

Buffering coefficient (typically 1 for transient buffers, 2 for others).

Return type:

Union[int, IntVar]

Notes

The multi-buffering strategy helps overlap computation with data movement by maintaining multiple copies of buffers at different memory levels.

propagateIOBufferStrategy(tileConstraintPattern: PatternMemoryConstraints, pattern: List[Node], ctxt: NetworkContext) → PatternMemoryConstraints

Propagate I/O buffer strategy across the tiling pattern.

Implements static n-tuple buffering strategy by propagating border tensor constraints across all steps in the tiling pattern.

Parameters:

tileConstraintPattern (PatternMemoryConstraints) – Memory constraints for the tiling pattern.
pattern (SubGraph) – The computation subgraph being tiled.
ctxt (NetworkContext) – Network context containing buffer information.

Returns:

Updated pattern memory constraints with propagated I/O buffer strategy.

Return type:

PatternMemoryConstraints

Notes

This method ensures that border tensors (inputs/outputs of the pattern) maintain consistent memory allocation across all computation steps.

assertLayerWiseTiling(schedule: List[List[Node]]) → bool

Assert that the schedule uses layer-wise tiling (one node per pattern).

Verifies that each pattern in the schedule contains exactly one node, which is required for certain memory allocation strategies.

Parameters:: schedule (List[List[gs.Node]]) – The execution schedule to validate.
Returns:: True if all patterns contain exactly one node, False otherwise.
Return type:: bool

Notes

Layer-wise tiling is required when using the MiniMalloc memory allocation strategy.

assertUniformMemoryLevelAllocation(ctxt: NetworkContext, defaultMemoryLevel: str) → bool

Assert that all local buffers are allocated to the default memory level.

Verifies that all local buffers in the network context are assigned to the specified default memory level.

Parameters:

ctxt (NetworkContext) – Network context containing buffer information.
defaultMemoryLevel (str) – Name of the default memory level to check against.

Returns:

True if all local buffers use the default memory level, False otherwise.

Return type:

bool

Notes

Uniform memory level allocation is required when using the MiniMalloc memory allocation strategy.

testTilingSolutionCorrectness(tilingSolution: List[PatternMemoryConstraints]) → None

Test the correctness of a computed tiling solution.

Validates that buffer sizes in the tiling solution are properly aligned according to memory alignment requirements.

Parameters:: tilingSolution (TilingSolution) – The tiling solution to validate.
Raises:: AssertionError – If any buffer is not properly aligned or if multi-buffer coefficients are not integers.

Notes

Checks that all allocated buffers meet the byte alignment requirements specified in MemoryScheduler.byteAlignment.

testMemoryMapCorrectness(memoryMap: Dict[str, List[List[MemoryBlock]]], graph: Graph, schedule: List[List[Node]] | List[Node]) → None

Test the correctness of a computed memory map.

Validates that the memory map correctly represents buffer lifetimes and ensures all required buffers are alive when needed.

Parameters:

memoryMap (Dict[str, List[List[MemoryBlock]]]) – The memory map to validate.
graph (gs.Graph) – The computation graph.
schedule (Schedule) – The execution schedule.

Raises:

AssertionError – If output buffers are not alive until the end, input buffers are not alive at the beginning, or required buffers are not alive during computation steps.

Notes

Performs comprehensive validation of buffer lifetimes to ensure the memory map is consistent with the computation requirements.