Invoking stop() from shutdown hook 2023-06-08 20:00:25,905 INFO (AbstractConnector.java:318)- Stopped Spark@4978bcd5{HTTP/1.1,[http/1.1]}{0.0.0.0:4040} 2023-06-08 20:00:25,906 INFO (Logging.scala:54)- Stopped Spark web UI at http://DESKTOP-UN228O4:4040 2023-06-08 20:00:25,919 INFO (Logging.scala:54)- MapOutputTrackerMasterEndpoint stopped! 2023-06-08 20:00:25,929 INFO (Logging.scala:54)- MemoryStore cleared 2023-06-08 20:00:25,929 INFO (Logging.scala:54)- BlockManager stopped 2023-06-08 20:00:25,935 INFO (Logging.scala:54)- BlockManagerMaster stopped 2023-06-08 20:00:25,937 INFO (Logging.scala:54)- OutputCommitCoordinator stopped! 2023-06-08 20:00:25,950 INFO (Logging.scala:54)- Successfully stopped SparkContext 2023-06-08 20:00:25,950 INFO (Logging.scala:54)- Shutdown hook called 2023-06-08 20:00:25,951 INFO (Logging.scala:54)- Deleting directory C:\Users\HUAWEI\AppData\Local\Temp\spark-87e84c73-3cf7-4db0-bf7d-ab42db0a5a54
时间: 2024-03-18 19:45:56 浏览: 64
hekate:hekate-基于GUI的Nintendo Switch引导程序
5星 · 资源好评率100%
这段日志是SparkContext成功停止并退出的日志。在Spark应用程序运行结束时,SparkContext会调用一个shutdown hook来清理资源和关闭SparkContext。这个过程中,Spark会停止web UI、MapOutputTrackerMasterEndpoint、MemoryStore、BlockManager、BlockManagerMaster、OutputCommitCoordinator等组件,并删除临时目录。
如果你看到这个日志,说明你的Spark应用程序已经正常结束了,不需要再进行其他操作。如果你想在Spark应用程序结束后进行其他操作,可以通过在Spark应用程序中添加shutdown hook来实现。这个shutdown hook会在SparkContext关闭之前被调用,可以用来清理资源、关闭连接等操作。
阅读全文