spark-TaskDescription.adoc

TaskDescription — Metadata of Single Task

TaskDescription is a metadata of a single task with the following properties:

• Task ID

• Task attempt number

• Executor ID

• Task name

• Task index (within the TaskSet)

• Added files (as Map[String, Long])

• Added JAR files (as Map[String, Long])

• Properties

• Serialized task (as ByteBuffer)

The name of the task is of the format:

task [taskID] in stage [taskSetID]

TaskDescription is created when:

• TaskSetManager is requested to find a task ready for execution (given a resource offer)

• TaskDescription is requested to decode (a TaskDescription from a serialized format as a ByteBuffer)

The textual representation of a TaskDescription is as follows:

TaskDescription(TID=[taskId], index=[index])

Decoding TaskDescription (from Serialized Format) — decode Method

decode(byteBuffer: ByteBuffer): TaskDescription

decode…​FIXME

Note	decode is used when: CoarseGrainedExecutorBackend is requested to handle a LaunchTask message Spark on Mesos' MesosExecutorBackend is requested to launch a task

Encoding TaskDescription (to Serialized Format) — encode Method

encode(taskDescription: TaskDescription): ByteBuffer

encode…​FIXME

Note	encode is used when: DriverEndpoint (of CoarseGrainedSchedulerBackend) is requested to launchTasks Spark on Mesos' MesosFineGrainedSchedulerBackend is requested to createMesosTask