Using Sparkling water and PySpark to log console output

Here is the command Option #1: ./pyspark –deploy-mode client –conf spark.dynamicAllocation.enabled=false –packages com.databricks:spark-csv_2.11:1.4.0 –py-files ../../sparkling-water-1.6.7/py/dist/h2o_pysparkling_1.6-1.6.7-py2.7.egg Here is the command Option #2: ./pyspark –deploy-mode client –conf spark.dynamicAllocation.enabled=false –packages com.databricks:spark-csv_2.11:1.4.0,ai.h2o:sparkling-water-core_2.10:1.6.7 –py-files ../../sparkling-water-1.6.7/py/dist/h2o_pysparkling_1.6-1.6.7-py2.7.egg We must make sure that both h2o backend and python version are calling same Version of API. This parameter is using H2O API backend version […]

Continue reading