- Install spark locally install#
- Install spark locally series#
Install spark locally series#
There is a fantastic series of videos where one of them really walked through the installation in detail.
It gave me the “ memory error” a few times until I switched off most of my apps and increased the Java heapsize.
Spark is not a small application, which means that you need to optimize your jvm limit. I followed this tutorial where downloaded the latest maven and modified the softlink of /usr/bin/mvn to point to the latest maven. I am not a frequent Java developer so my maven version got out of date 3.0.3 where it requires minimum 3.0.4 to work. I was installing Spark from source using Maven. Spark was originally written in Scala which need JVM, so be ready to set up a Java development environment and you also need the Scala interpreter if you want to build everything from source code. 
There are a few things that you need to know and prepare beforehand:
Install spark locally install#
I want to install Spark locally, in that way, I can easily try out the SparkR package without worrying about breaking the team cluster.
0 kommentar(er)
