Install Spark on AWS EC2 Ubuntu instances

It has been annoying to install spark a couple of times, finally, I decide to document the process.

After the new instance is spin-up, here is the list, you might want to skip some if already installed.

sudo apt-get update
sudo apt-get install git

#install java 8
#sudo apt-get install software-properties-common
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

#install scala
wget http://www.scala-lang.org/files/archive/scala-2.11.4.deb
sudo dpkg -i scala-2.11.4.deb
sudo apt-get update
sudo apt-get install scala

#install sbt
wget http://dl.bintray.com/sbt/debian/sbt-0.13.6.deb
sudo dpkg -i sbt-0.13.6.deb
sudo apt-get update
sudo apt-get install sbt

#may need change to the latest and stable version if needed, it is the latest as of December 19, 2014
git clone https://github.com/apache/spark.git
cd spark
sbt/sbt clean assembly
#it took about 8 minutes on my laptop

#Is it ready?

./bin/spark-shell

Comments are closed.