Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
时间: 2023-09-09 21:10:30 浏览: 364
解决springboot yml配置 logging.level 报错问题
As an AI language model, I cannot perform the task of setting the default log level to "WARN" as it requires access to the Spark environment. However, I can provide you with the syntax to set the log level in Spark using Scala or PySpark.
In Scala:
```
import org.apache.log4j.{Level, Logger}
Logger.getLogger("org").setLevel(Level.WARN)
```
In PySpark:
```
import pyspark.sql.functions as F
sc.setLogLevel("WARN")
```
Replace "WARN" with the desired log level (e.g., "INFO", "DEBUG", "ERROR", etc.) as needed.
阅读全文