[go: up one dir, main page]

Skip to content

A simple boilerplate Spark word count application using sbt and scala.

License

Notifications You must be signed in to change notification settings

DevipriyaSarkar/SparkWordCount

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SparkWordCount

A simple boilerplate Spark word count application using sbt and scala.

Pre-requisites

  1. Java
  2. Python
  3. Scala
  4. Spark

How to use?

  1. Install Scala plugin for Intellij IDEA.
  2. Install SBT plugin for Intellij IDEA.
  3. Import this project as a SBT project into your IntelliJ IDEA.
  4. Build and run the project.

How to build JAR?

> cd <project-directory>
> sbt compile
> sbt run
> sbt package

How to submit the Spark job to the local master?

spark-submit --class WordCount --master local <path-to-the-jar>/<jar-file>.jar

Usually the path to the jar file is <project-directory>/target/scala-<scala-version>

About

A simple boilerplate Spark word count application using sbt and scala.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages