spark-StorageMemoryPool.adoc

StorageMemoryPool

StorageMemoryPool is a MemoryPool that…​FIXME

StorageMemoryPool is created along with MemoryManager (as onHeapStorageMemoryPool and offHeapStorageMemoryPool pools).

Table 1. StorageMemoryPool’s Internal Properties (e.g. Registries, Counters and Flags)

Name	Description
poolName	FIXME Used when…​FIXME
_memoryUsed	The amount of memory in use for storage (caching) Used when…​FIXME
_memoryStore	MemoryStore Used when…​FIXME

memoryFree Method

memoryFree: Long

memoryFree…​FIXME

Note	memoryFree is used when…​FIXME

acquireMemory Method

acquireMemory(blockId: BlockId, numBytes: Long): Boolean  // (1)
acquireMemory(
  blockId: BlockId,
  numBytesToAcquire: Long,
  numBytesToFree: Long): Boolean

1. Calls acquireMemory with numBytesToFree as a difference between numBytes and memoryFree

acquireMemory…​FIXME

Note	acquireMemory is used when: StaticMemoryManager is requested to acquireUnrollMemory and acquireStorageMemory UnifiedMemoryManager is requested to acquireStorageMemory

freeSpaceToShrinkPool Method

freeSpaceToShrinkPool(spaceToFree: Long): Long

freeSpaceToShrinkPool…​FIXME

Note	freeSpaceToShrinkPool is used exclusively when UnifiedMemoryManager is requested to acquireExecutionMemory.

Creating StorageMemoryPool Instance

StorageMemoryPool takes the following when created:

• Lock

• MemoryMode (either ON_HEAP or OFF_HEAP)

StorageMemoryPool initializes the internal registries and counters.