Java

Apache Spark Jump into Programming: 1. Spark Config:     Configuration for a spark Application and used to set various spark parameters as key  and value pairs. SparkConf conf = new SparkConf()                      .setAppName( "Line count" )                      .setMaster( "local[2]" )                      .set( "spark.executor.memory" , "1g" ); 2.Java Spark Context:    It is Java version of the Spark Context and only one Spark Context active per JVM.    JavaSparkContext jsc = new JavaSparkContext( conf );    JavaSparkContext  jsc = new  JavaSparkContext()    It has some useful methods:     File related: addFile() binaryFiles(),binaryRecords() clearFiles() textFile()----> returns Java RDD sequenceFi...

Streams Introduction: In Java8, new feature ' Stream'  was introduced into the java.util.stream package. Why do we need Streams? In Previous versions of Java, Developer has to loop through the Collections and  needs to have  repeated checks for nulls, which was error prone and more tedious process. Oracle Team came up with this Stream Approach. What is Stream? A stream is not a data structure that stores elements, instead, it conveys elements from a source such as a data structure, an array, a generator function, or an I/O channel, through a pipeline of computational operations. Simple words " Stream represents a sequence of objects from a source, which is used for Computational operations " •  Stream is not a Data Structure that stores elements. •  An operation on a stream produces a result, but it does not modify its source. •  Stream operations are divided into two intermediate (Stream-producing) operations and...