spark-history-server-HistoryServer.adoc

HistoryServer — WebUI For Active And Completed Spark Applications

HistoryServer is the web UI for running (active) and completed Spark applications.

HistoryServer is created when…​FIXME

HistoryServer uses the HttpServlet to handle requests to /* URI that FIXME.

Table 1. HistoryServer’s Internal Properties (e.g. Registries, Counters and Flags)

Name	Description
appCache	ApplicationCache Used when…​FIXME
loaderServlet	Java Servlets' HttpServlet Used exclusively when HistoryServer is requested to initialize (and attaches the servlet to the web UI to handle /* URI)

Tip	Enable INFO logging level for org.apache.spark.deploy.history.HistoryServer logger to see what happens inside. Add the following line to conf/log4j.properties: log4j.logger.org.apache.spark.deploy.history.HistoryServer=INFO Refer to Logging.

attachSparkUI Method

attachSparkUI(
  appId: String,
  attemptId: Option[String],
  ui: SparkUI,
  completed: Boolean): Unit

Note	attachSparkUI is part of ApplicationCacheOperations Contract to…​FIXME.

attachSparkUI…​FIXME

Initializing HistoryServer — initialize Method

initialize(): Unit

Note	initialize is part of WebUI Contract to initialize web components.

initialize…​FIXME

Launching HistoryServer Standalone Application — main Method

main(argStrings: Array[String]): Unit

main…​FIXME

Creating HistoryServer Instance

HistoryServer takes the following when created:

• SparkConf

• ApplicationHistoryProvider

• SecurityManager

• Port number

HistoryServer initializes the internal registries and counters.

Requesting Spark Application UI — getAppUI Method

getAppUI(appId: String, attemptId: Option[String]): Option[LoadedAppUI]

Note	getAppUI is part of ApplicationCacheOperations Contract to…​FIXME.

getAppUI…​FIXME

withSparkUI Method

withSparkUI[T](appId: String, attemptId: Option[String])(fn: SparkUI => T): T

Note	withSparkUI is part of UIRoot Contract to…​FIXME.

withSparkUI…​FIXME

loadAppUi Internal Method

loadAppUi(appId: String, attemptId: Option[String]): Boolean

loadAppUi…​FIXME

Note	loadAppUi is used exclusively when HistoryServer is created.

doGet Method

doGet(req: HttpServletRequest, res: HttpServletResponse): Unit

Note	doGet is part of Java Servlet’s HttpServlet to handle HTTP GET requests.

doGet…​FIXME

Note	doGet is used when…​FIXME