On: Command-line tools can be faster than your Hadoop ...

[sgt101]

There's a perception that hadoop commands are terribly complex. If you run

$spark-shell

you can execute (interactively)

val file = spark.textFile("hdfs://...") val errors = file.filter(line => line.contains("ERROR")) errors.count()

And wordcount a file - ok the wget is not there, but this is really not complex!