Apache Spark
Apache Spark Jump into Programming: 1. Spark Config: Configuration for a spark Application and used to set various spark parameters as key and value pairs. SparkConf conf = new SparkConf() .setAppName( "Line count" ) .setMaster( "local[2]" ) .set( "spark.executor.memory" , "1g" ); 2.Java Spark Context: It is Java version of the Spark Context and only one Spark Context active per JVM. JavaSparkContext jsc = new JavaSparkContext( conf ); JavaSparkContext jsc = new JavaSparkContext() It has some useful methods: File related: addFile() binaryFiles(),binaryRecords() clearFiles() textFile()----> returns Java RDD sequenceFi...
Comments
Post a Comment