People are doing all sorts of great things with Scala.
Scala Notebooks, which can be classified as a learning resource https://github.com/n8han/scala-notebook/ - It is an in-work conversion of iPython notebooks using d3 instead of matplotlib.
Another great analysis resource is Saddle, a scala data munging library (analogous to pandas, around the 0.8 revision). http://saddle.github.io/
If you guys really into Scala, I really recommend trying Finagle (https://github.com/twitter/finagle). It was also created by Twitter. It was a lightweight HTTP server and client for Scala. It's my favorite. With just a "single" line of code, you can create an HTTP server in Scala.
I usually combine it with Rogue (https://github.com/foursquare/rogue), an "ORM"-like for MongoDB. I used Finagle and Rogue to build a fast and lightweight REST API.
Finagle doesn't tend to be so well integrated with the rest of the ecosystem - IIRC there isn't even a scala 2.10 release out yet. I'd recommend spray (http://spray.io ) if you want to do async http in scala.
Does it support Swagger now? For my last implementation, I wanted a good automatic API documentation, and I'd rather have the client code in a generated way, and of the three Scala microframeworks that I had a look at, only Scalatra had kinda solid Swagger support. Neither Spray nor Finagle did. Though in hindsight the Swagger release in the current Scalatra is a bit old and thus it has tons of bugs and issues that are fixed in current Swagger but I found it impossible to just update Swagger.
I believe that Twitter tends to define their services as Thrift interfaces, which forms it's own documentation for the application rather than having to crawl through code to find which REST URL endpoints are being listened to.
I could be wrong but I think Finagle reflects this in that Thrift is more of a first-class citizen than REST (although I'm sure the latter is possible).
As someone getting into scala, where can I find the list of top available libs? For clojure I just browsed github, but I feel there probably is some resource out there I'm missing.
Whenever possible I avoid using Java libraries in Scala. Scala and Java have very different programming styles, which are codified in the APIs for libraries. You lose the benefit of Scala style when you are forced to program against a Java library.
Very differently (I'm the original author of Casbah, though I gave up maintainership a few months ago when I joined Typesafe).
Casbah was always created to be a straight MongoDB driver, with no ORM features. It has no dependencies on "external" projects other than the MongoDB Java driver, so that you can use it where-ever and write standard queries.
It has been awhile since I played with Rogue, so forgive any mistakes but the big one:
Rogue is specifically built as an ORM, with integration into Lift.
Really, you are going to use Rogue vs. Lift to accomplish very different tasks. There is Salat (https://github.com/novus/salat), to provide ORM features on top of Casbah.
Pick which tool works best for your use case! I happen to think quite highly of Rogue as well, as there are some tremendously impressive features in it including detecting potential Table scans. Rogue is fairly well battle tested on the Foursquare systems, and the authors are both very, very good at what they do.
I've finished Martin Odersky's Scala Course. I've built a few small play2.0 apps. I've used Scala to solve ~20 Project Euler problems, and I'm fiddling around with parsers and scalaz.
However, all the Scala positions I've found are looking for engineers with experience using Scala in production. How can I make that jump? Are there any opportunities out there for junior engineers who are proficient in Scala?
If finding a Scala position in the US (I assume) is already this difficult, here in Brazil it's infinitely harder. I guess our startup is one of the very few that has a fairly large system in production running on Scala, Play, and Akka. It's next to impossible to find people with experience yet alone experience in production. The key is to find people genuinely interested in learning.
I worked with Scala at EPFL during my 2 year post doc (in Martin's group). I later started working for Microsoft, and found the transition to C# to be initially painful but not incredibly so. I'm now a happy C# programmer, though I take a lot of liberties [1] with the system, and I also use C# mostly to build programming systems of my own [2] (though Scala has had an everlasting influence on my own work).
C# is a bit more advanced in Java (they have lambdas), and in someways better than Scala (reified generics). But the best thing about C# is that...it forces you to to settle on less elegant but functional designs in a "worse is better" style. I found myself obsessing over my Scala code to come up with perfect solutions because...I could...while in C# perfect solutions obviously don't exist. C# relieves the burden of choice, which is an interesting trade off to keep in mind when designing PLs.
That's actually a very fair point - when it comes to usability with respect to the functional style, sometimes less is more. I think C# is actually a pretty good example of a pragmatic language that offers FP to make your life easier, but doesn't beat you over the head with it. If I could choose between Java and C# purely on the basis of the language, there is no question that I'd choose C#.
In my (limited) experience, for expressions make for some of the most concise code I've seen. So many problems can be represented as a chain of Options or Futures mixed with standard {val x = expression y} expressions.
But why would you want to? Even in Scala? Use Futures only when bothered with async IO, and even then be very careful about it. Options always made debugging more difficult because they defer where you get your NPE.
Isn't the point of options to statically rule out the possibility of NPE's?
def f1:Option[Int] = Some(1)
def f2(a: Int):Option[Int] = Some(a + 1)
val r = for {
a <- f1
b <- f2(a)
c = a * b
} yield c
r.getOrElse("generic error")
Individual error messages with scalaz's Validations
def f1:Option[Int] = Some(1)
def f2(a: Int):Option[Int] = Some(a + 1)
val r: Validation[String, Int] = for {
a <- f1.toSuccess(e = "f1 failed")
b <- f2(a).toSuccess(e = "f2 failed")
c = a * b
} yield c
r.fold(e => s"error: $e", r => s"result: $r")
The problem is that on the JVM, you are given a debugger based on control flow, not data flow. Once you go into heavy option code, debugging that code becomes a PITA because your tools are simply wrong. So ya, you can come up with what basically amounts to a printf for data flow code, but you are stuck doing everything on your own at that point.
The problem really stands that no one knows how to build decent data-flow debuggers. Haskell programmers prefer static safety over debugging just as much as because their debuggers suck as the type system is so powerful. In Scala, you have the option to write crash-early strict code: do it, you will make your life so much easier as a result.
Is it realistic to split long chains of Option into subfunctions returning Option? If long chains of Options are hard to effectively reason about, breaking them up could ameliorate the problem.
I suppose that conceptually, I consider returning None inside a chain of flatmapped options as the equivalent of terminating early. It's not hard to pass along an object describing the error instead of None, which gets you the equivalent of a thrown exception.
Also: can you point me towards any examples of this antipattern? I 'd like to see it in action, if only so I can avoid it in my own code.
I found the best way to debug a complex piece of scala code was to get rid of as much laziness and deferral as possible. Creating a huge lazy sequence that propagated through multiple call-layers was absolutely evil. Also, options, although strict, are troublesome across more than a couple of lines of code given the way they crash. In general, only use option if failure is expected, otherwise null is your friend!
For simple programs that do not need to be debugged, laziness is great, you can write some very concise code. Once you are debugging however, you want to expose as much control flow as possible to break and step through. I say this from writing 10s of thousands of Scala code in a fairly complicated context (Scala compiler + Eclipse) with non-trivial control flow (mostly event driven).
I went the other way, started working at Twitter without knowing Scala and it wasn't hard. We also have great internal Scala classes. Feel free to email me for more info (see profile). We're just looking for smart people, you don't need to have experience with Scala.
Coursera has a class as well. Martin Odersky tweeted a while ago about a possible advanced course in the fall but it doesn't look like it's happening yet.
I finished the course when it first came out last year. This appears to be the third session of the same class. I'm also eagerly looking for an advanced course. I used Scala in "Algorithms: Design and Analysis, Part 1" programming problems. Even as some algorithms are described in recursive style, using tail recursive implementations fail for large data sets and I have to fallback to imperative styles with lots of mutable data structures to complete the assignments. A Scala based algorithms course would be very useful.
It's worth mentioning that I got into Scala because of Akka. Amazing features like remote actors, supervision and more. I also appreciate how active the project is :) https://github.com/akka/akka
Akka is great I just launched an app into beta with Play( websocket game) and Akka , currently I'm trying to get a hang of the clustering features so I can distribute my connections out.
The danger of that approach is that, in my experience, it's more likely to turn someone away from Scala. At my last job, some of the engineers did write Java in Scala. In what I'm sure is not a coincidence, they were also the ones who disliked Scala the most, and felt that it was detrimental to our codebase. By not learning its differentiating features, all they saw was the burden of having two languages, library support, spotty IDE, painful Scala upgrades, etc. In fact, since I've left, I've heard that they are migrating a lot of the Scala back to Java; the effort is being led by those who never wrote idiomatic Scala. Again, I'm sure this is not a coincidence.
I'm leading a scala transition at a javashop. I think you need a crital mass of people who understand scala as well as a lot of respect for your colleagues for this to go smoothly, 10x so if you have legacy code. There will be some very tense discussions about how things ought to be implemented, and if not well managed, it can feel like "this approach is wrong, your code is bad and you are a bad engineer" which is toxic to any team. Sorry on mobile or I would write more. There is a lot of blog activity on this subject both from successful teams and unsuccessful that your leads absolutely must understand
It should be evident to anyone with a modicum of understanding of what types are for, that Scala is a superior language to Java. (And Haskell even much more so, but Scala is friendlier to your existing investment in a Java codebase.)
> There will be some very tense discussions about how things things ought to be implemented (...)
Encoding as much as possible of the problem domain's logical constraints in types. Any property enforced by a type checker (and not subverted via a type system escape hatch) is a property that you do not have to test.
> (...) and if not well managed, it can feel like "this approach is wrong. your code is bad and you are a bad engineer"
My experience with most programmers using Java/C#/Python/whatever is precisely that: "Their approach is wrong. Their code is bad. And they are bad engineers." I have to waste my precious time subverting their brittle type systems (downcasts, reflection, assuming stuff is not null, etc.) and enforcing correctness the hard way (extensive testing or even good old non-computer-assisted brain usage), when I could simply read type signatures, get free theorems, run a couple of tests for checking stuff that cannot be inferred from types, and move on to the next thing.
You omitted the part where the previous poster described that quote as toxic.
If you see your time as much more precious than your colleagues, and default to treating them with contempt, you're going to have a hard time finding colleagues. Which you may not care about.
But what should matter to you is that it's basically a self-fulfilling prophecy. If you treat your colleagues like that, they're never going to get better, and they're going to be the ones who eventually vote to throw out all your beautiful code and go back to Java, where they at least understand what's going on and nobody condescends to them all day.
raw tactical coding abilities is one axis of many that a business might value. More important than that, to our business, is that several of our engineers have extensive pharma domain knowledge, two of them we recruited away from our customers. It seems to me that most businesses are like this; a monad stack outside the context of a business is just generating heat.
Pattern matching, immutability, option types, combinators, traits are not advanced features in Scala. They are fundamental concepts. To be productive in Scala, write in Scala. Otherwise, you will needlessly waste a huge chunk of your life, including having to go back and clean up your messes.
That way you can delude yourself into thinking you are productive, because you are writing code immediately. However, productivity is not to be measured by the number of lines of code; rather, by 1. the amount of actual functionality implemented, 2. the elegance of the design (which saves time refactoring later on).
It's easy to forget the value of delivering functionality immediately. It's all too easy to spend a long time on an elegant design that doesn't actually serve business needs. I like to follow an agile approach where if something can't deliver value in 2 weeks then it's not worth doing (or, more likely, needs to be split into smaller pieces). That's plenty of time to learn enough Scala to be more productive than you were in Java, even though it will take months or years to learn the full language.
Learning a new programming language is an intellectual, exploratory endeavor, even if it might result in adopting that language for future programming projects. Writing software professionally is a pragmatic engineering job constrained by non-intellectual considerations such as customer expectations (and lack of knowledge thereof) and limited availability of resources (time, money, skills, etc.). Suggesting that both activities should be approached the same way is a huge mistake.
I learnt scala very effectively simply by delivering a program in it at my day job. I'm not sure about "should", but in my personal experience I've found that the best way to learn.
In my experience, jumping straight to writing code results, if anything, in a huge mess. For anything more complex than proof of concept programs for specific language or library features, this quickly becomes unmanageable.
I have obtained much better results by first developing a nice logical model (not necessarily 100% formal, but at least rigorous enough to be amenable to formalization), and only then picking a programming language into which the model can be translated with a minimum amount of effort. In this regard, a good language is one that allows translating as many logical constraints as possible into statically verifiable constructs (such as types). The nice thing about static verification is that errors in the translation can be detected as soon as you make them. Heck, in some cases, even errors in the original conceptual design can be detected.
Scala is a better language than Java because its type system is so much more powerful. Using Scala as if it were Java nullifies this advantage, and leaves you only with the inconvenience of having to learn a new syntax. So, what is the point?
The new syntax is similar but nicer. If you just use Scala as "java with type inference, no semicolons, and a shorter form of final" it's already an improvement over java.
Scala only has very limited inference. In fact, so limited that, based on this criterion alone, I would take Java's powerful refactoring tools over Scala's inference anytime. (One of my main uses of inference is enabling fearless aggressive refactoring.)
But let's say Scala had a type system amenable to global inference with a nice principal types property. This does not prevent you from having to annotate types when, at a module boundary, you want a term to have a more specialized type than its principal type. (Within module boundaries, it is admittedly no big deal to have terms whose types are more generic than their use would warrant.)
> "no semicolons, and a shorter form of final"
You cannot tell me with a straight face that cosmetic syntactic changes are a particularly compelling reason to switch languages. (Well, you can, but then you cannot convince me you are a pragmatic engineer.)
> "it's already an improvement over java."
Yes, but, without taking advantage of Scala's powerful type system, is the improvement over Java substantial enough to justify the investment in learning a new language?
>Yes, but, without taking advantage of Scala's powerful type system, is the improvement over Java substantial enough to justify the investment in learning a new language?
What investment? You just write the same code that you were writing in Java. You can do that on day 1 (at least, assuming your IDE and build system have scala support already) - on day 1 your productivity is just as high (or slightly higher, because of the cosmetic syntax improvements) as it was in Java. On day 2 your productivity is higher, as you pick up a little bit of scala. So there's no initial cost - just an improvement. I know this is possible because I have done it.
Timely. I'm interested in scala. Would love a couple opinions from those in the know ... (1) As a (very) experienced java guy, what's really the best learning approach: comparative? clean slate? I'd "like" to do Odersky's course, but I can't commit to scheduled tasks; (2) A minimal framework seems like a good idea to me - rather than Play!, what do folks think about Scalatra as a starting point?
if you want smoother transition, try my approach: code with Java style (OOP, imperative) but using Scala syntax. once you're familar with the new syntax, naturally you will wonder, "this seems clunky, there's got to be a way to do this the Scala way". and you'll start googling "how to do X in a list" etc.
to lower the bar, don't use framework. i even coded http://ngajakjalan.com (one of my projects written in Scala) without framework, just plain servlet+JSP. again, eventually you'll think, "i keep doing the same thing, there's got to be some library out there already doing this for me", and start exploring.
I am learning Scala last couple of months. Twitter scala page is really nice for starting up. Another place with discussion of more advanced topics I find useful:
This looks great. My favorite resources are those that assume prior programming knowledge and aim to get the reader up to speed quickly with the specifics of the language, not programming in general. If anything, I think I'll start studying pattern matching :)
I've been interested in venturing into Scala but have not found a good resource for getting a development environment setup. Should I be using an IDE or will sbt/Sublime be sufficient?
I'm using Scala with Ensime in Emacs. It is not as good as the Scala IDE, but it is quite good. The reason I'm doing it is that I like to use the same editor for all the languages that I'm using, so that any additional plugins / modifications I do are not just for this one language, but can be leveraged for everything I do.
However, If that is not a requirement for you, you should really be using the Scala IDE (Eclipse plugin), it has great autocompletion, refactoring, and more.
I have to say I also started with Scala IDE because, when starting with a completely new language, I tend to just install the default IDE so that I don't have to fight two battles in the beginning but can concentrate fully on the language. Otherwise, you'll loose interest / give up even faster.
Take a look at Typesafe Activator - www.typesafe.com/activator - it's a pretty slick way to get up and started without all the IDE setup costs by simply running an IDE for you in your browser.
Ensime in Sublime is kinda pointless I think since it only updates on file change. The REPL and SBT integration is nice.
I use Sublime for quick examples in Scala, or as a VIM alternative for larger projects (as opposed to my main editor like with Ruby).
For most of my Scala work I use IntelliJ though. The lack of half-decent theming in Eclipse and Preferences/Settings being all over the place really puts me off ScalaIDE personally.
Going from Zero to Code in IntelliJ and SBT (on OSX) is pretty trivial. Given a command-line "Hello World" with a traditional Maven layout:
First, install IntelliJ, navigate to "Plugins" under preferences and install the Scala plugin.
Now make sure you have Homebrew installed (http://brew.sh)
$ brew update
$ brew install sbt --devel # currently 0.13.0-RC5
$ cd ~/src/
$ mkdir hello-world
$ cd hello-world
$ mkdir project
$ echo "sbt.version=0.13.0-RC5" | tee project/build.properties
$ cat <<EOS | tee project/plugins.sbt
resolvers += "Sonatype snapshots" at "http://oss.sonatype.org/content/repositories/snapshots/"
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.5.0-SNAPSHOT")
EOS
$ cat <<EOS | tee build.sbt
name := "hello-world"
version := "1.0-SNAPSHOT"
EOS
$ mkdir -p src/main/scala
$ cat <<EOS | tee src/main/scala/Whatever.scala
object Whatever extends App {
println("Hello World!")
}
EOS
$ sbt
> compile
> gen-idea
> run
And there you go. It could be simpler if we were having a LOC competition, but this is actually very close to the exact setup I use for projects day in and day out, including getting a jump on the next version of SBT and best-practicey stuff like setting the SBT version in build.properties.
If this were a Play app, you might have a "project/plugins.sbt" that looked something like this (to get a jump on the upcoming Scala 2.10 version of Play, that integrates with SBT 0.13.x):
// Comment to get more information during initialization
logLevel := Level.Warn
resolvers := Seq("Maven Central" at "http://repo1.maven.org/maven2/",
"Typesafe Snapshots" at "http://repo.typesafe.com/typesafe/snapshots/",
"Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases/",
"Sonatype snapshots" at "http://oss.sonatype.org/content/repositories/snapshots/")
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.2.0-M2")
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.5.0-SNAPSHOT")
This just adds a few more common resolvers you might use for dependencies, and locks you to the latest Play milestone snapshot. You can pretty much otherwise copy/paste files/folders from a sample Play app, like you might see with the Typesafe Activator for example.
there's this https://github.com/n8han/giter8 project that's supposed to create scala project templates for you from the command line, but never really got into it so i can't really comment.
after years of java coding, i started following the scala "trend" but had no chance of actually using it where i used to work. nonetheless, i did coursera's odersky course and started using it here and there for (very little) personal projects. with that in my toolbelt i was able to find the job i now have. they were using scala/akka/play (but also scalding and other stuff) and they were okay with somebody with little experience but willing to commit to it. i accepted and now have spent 4 months away from java and diving into scala and i just love every bit of it (even SBT that at first looked very little interesting).
I've been looking for examples of applications built with scala and akka that I can learn from. I've built small, toy-ish stuff but I feel I learn a lot by looking at examples. Any have any recommendations?
- a "JSFiddle" like editor for Scala: http://scalakata.com
- Interactive tours: http://scalatutorials.com/tour, http://www.scala-tour.com, http://www.simplyscala.com
- Learn Scala in X minutes: http://learnxinyminutes.com/docs/scala/
- Typesafe Activator: http://www.typesafe.com/activator
- Official docs of course: http://docs.scala-lang.org/