Honest question: with tooling (IDE) and build tools (SBT) to my understanding remaining as-is despite of developer feedback, and Java 8 presenting itself as a nice comeback (in my opinion) - is Scala adoption for big companies (Facebook, Google, Twitter) still going up as it were in the high times of it?
I see Java 8 more as an even better gateway drug to Scala than Java 6, so I expect adoption to accelerate. (Then again, I'm biased.)
Java-8-the-language is a pretty timid comeback. Functional programming is not just about lightweight syntax for function literals and some more type inference for polymorphic method calls. That's just the start.
An FP language should also:
* encourage immutability;
* make it easy to write and refactor code that uses combinators (generic higher-order methods) --> local type inference is essential (IDE support for type inference doesn't help, unless other refactorings automatically update the synthetic type annotations)
* encapsulating logic in functions rather than using dynamic dispatch (where it makes sense), naturally leads to pattern matching.
Also, Scala has great support for asynchronous/event-driven/reactive programming through extensible for-comprehensions, and the macro-based scala-async framework.
Then there's the OO side: interfaces with default methods are much more limited than traits,...
I'm very excited about the VM improvements and the potential of more functional libraries in our eco-system, though!
Having recently worked with a project employing Java 8, I whole-heartedly agree. The FP in Java 8 is so crippled as to be useless. Every time you use a lambda, you need to worry about checked exceptions and wrapping and unwrapping them. Also there is no Unit type, so you have to add `return null` everywhere.
Moreover, the Java collection library is now a hodge-podge of ancient cruft haphazardly mixed with a half-hearted FP interface. Java seriously needs to drop pre 1.5 compatibility, and revamp the whole API.
(Scala tech lead here). We're doing everything within our means to improve tooling -- we agree it's crucial. Which changes would you like to see that aren't happening?
One engineer on the Scala team (Grzegorz Kossakowski) spent all of last year on a much better algorithm for incremental compilation (name hashing), sbt 0.13 was a giant leap in usability IMO, and both IDE plugins have made enormous progress over the last years (again, IMO).
I'm just starting with Scala and I find the IntelliJ Scala plugin immensely non-intuitive (and I'm a big JetBrains fan normally).
Why do I have three different, apparently non-equivalent ways to create a new project (Java->Scala, SBT, Scala Module)?
The tooling and documentation seems to assume familiarity with the Java ecosystem. I'm not a Java developer (nor do I want to be), I'm a Python guy. Am I really expected to learn Java first just so I can learn Scala?
You might want to give the Eclipse Scala IDE a go. It's pretty easy to get started even if you're unfamiliar with Eclipse (watch the getting started video): http://scala-ide.org/download/current.html
Disclaimer: I'm the tech lead of the Scala IDE for Eclipse project
Choose File/New Project/Scala (sbt) and it should create an sbt project for you. Most of the scala open source projects and examples you will find on the web, use SBT so try to learn that instead of using maven or letting your IDE manage its own project.
After over a year of working with Scala and IntelliJ I don't know how to create a project from scratch through the IDE.
I just create a build.sbt, project/build.properties, and then open my SBT project in IntelliJ. Since they added SBT import support a few months ago that works really well for me.
I always wondered how that worked with scala. Giving how complex some libraries could be with use of implicits and not explicitly defining the return time I always thought it would be a bit troublesome to try to program without any use of intellisense. Then again I guess the same would apply to dynamic languages such as javascript and python so maybe my concerns are not really concerns.
As a counterpoint to this popular meme: Most developers I know are VIM converts (from TextMate).
I've never seen their choice of editor improve their coding speed. Haven't seen it on videos online. The idea that being fluent in VIM makes you an all around faster/better coder and the IDE is a crutch just isn't supported by evidence IME.
I use IntelliJ. The startup is a bit slower. And the minute or so pause if I change up dependencies can be annoying (though that's rare). On the other hand I always have method signatures at my fingertips, depend a lot less on the web version (of the same documentation), can jump into compiled libraries for exploring interfaces, and instead of a fuzzy "ack-like" symbol search, I have true jump-to-definition/open-class, etc. I have access to the FSC which provides continuous feedback as well. Ensime is a pale shadow of IntelliJ IME.
Maybe those things aren't important to some people. I find it makes me very productive though, and I haven't met a VIM user who can outpace me yet (though I did see some true VS.NET/RAD-tooling masters back in the day that could put me to shame).
All this just to say: Use what works for you. If that's IntelliJ, cool. If it's VIM, great. But there's a lot of anti-IDE sentiment out there. It's just another tool. Use it if it makes sense. Judge it on it's own merits. You are not less (or more) of a coder for using IntelliJ.
As long as you stay away from Eclipse at least. I think we can all agree on that. ;-) (Seriously Eclipse... It's 2014, and you still can't easily theme your IDE with a couple clicks?)
Well, I haven't tried Intellij, but in my experience the best coding environment is:
1. Visual Studio
2. gEdit
Visual Studio has incredibly good fuzzy code completion and gEdit has a total lack of features to be confused by and good font rendering. I just look at all the menus and buttons in Eclipse and I can't stand it. All this stuff can't possibly be necessary.
Since VS is not available for neither Scala nor Linux I'm left with gEdit.
IntelliJ keeps most of it's buttons hidden (in v13 at least). Might be worth a shot. I'm not entirely sure what you mean by "fuzzy code completion", but IntelliJ supports case-insensitive-in-order-partial-symbol matching.
For example, "jsfmt" with match "jsonFormat", and display the matches as I type so I can arrow-down and hit ENTER if I see what I want (or just keep typing until the list filters down to the one I want).
Implicit invocations get underlined by default as well. Which is nice since I'm rarely at loss for understanding what the compiler is doing to make this code work.
With fuzzy I mean that I can search for "tunnel" and get "channel". It does sound fuzzy, unless it's hardcoded. Code completion in Eclipse isn't fuzzy, so "temrinal" doesn't get you "terminal". In Eclipse it's basically just a regex.
With ensime [1], I use my editor (Emacs) and get much more sophisticated tools than you might think possible. It's not perfect, but it provides pretty nice inspection tooling.
Making scalac aggressively incremental was a problem solved long ago. I'm presenting a paper about the continuation of that work at this year's onward.
Hi, thanks for taking the time to reply. My personal opinion of SBT is one that represent almost every developer I've discussed it with. Jumping to the bottom line - an experience like: rubygems, npm, lein (clojure), and that kind of build and package family is much preferred - one example is Buildr.
It's hard to pinpoint a developer 'instinct', but I think a build/dependency tool like rubygems or npm is one of the first things a developer bumps into, and becomes a central part of the experience _very early_, even before the experience of the language. With that in mind, SBT present a cryptic (some times even non-scala) like feel to it.
Speaking of rubygems and npm, SBT also solves issues that neither of them are solving.
Actually npm doesn't even solve the issue of dependency management well for client-side dependencies - people apparently use Bower for that - and then just wait until you get the idea of building mixed apps. And then building stuff may not be so required for one-off JS scripts, but if you want to run a linter over your code, or to package dependencies in a single binary based on Common.js or AMD and maybe to do minification for distribution, then you're in for a world of hurt. And in case you think it's unfair to compare the clusterfuck that is Javascript's ecosystem against SBT/Scala, you may be interested to know that Scala.js (the Scala to Javascript compiler) is a pleasure to work with, one reason being that it just works with SBT.
The Ruby ecosystem is somewhat close to sane, but beginners still have to deal with gemspec versus bundler, rubygems versus Rails, rvm vs rbenv, plus the issue of native dependencies (for Scala/SBT, the only native dependency is the JVM itself and until 2016 at least Java SE 6 is all you need). And Python for example - is another example of a clusterfuck.
It's easy to forget as experienced developers that we did experience pain when we started with our current platform. Lets face it - when it comes to dependency management and build systems, we should be ashamed of ourselves. And now after having worked with other environments, SBT may suck (it does, it really does), but to me it sucks less than everything else I tried, because at least it solves most of my needs out of the box and for the edge cases there's usually a plugin available.
As a former Ruby developer: Please for the love of all that is holy do not make SBT more "Ruby-ish". Rubygems, Bundler, RVM and friends don't do anything SBT doesn't already do. And SBT does it with less set up and ceremony, fewer tools, more reliability, and less DSL "magic".
Rake is a cancer. SBT Tasks are far from straightforward, but they're certainly no more complex once you start throwing in dependencies. Tasks could use some work. But dependencies, project definitions, build settings, cross compilation, all light years ahead of anything in Ruby-land.
You ask that like tooling and build tools are a big problem. I know there is a lot of vocal opposition to the state of the scala toolchain, but for those who use it as a daily driver it works great. Both the ScalaIDE eclipse plugin and the Intelliej Scala plugin are great and quite capable and as for the build tools, the problem has never been with the tools its been with the compile times which are slow across large code bases. That's not getting addressed, but there are ways around mitigating the effect of the compile times such as splitting the code base into smaller components and using the incremental compiler. I can't speak for large corporations, but I do see Scala adoption continuing to go up in the consulting space.
Scala 2.11 went through a major refactoring of the compiler, it saw significant improvements in both total compilation time and incremental compilation and further improvements are planned in follow-up versions: http://www.scala-lang.org/news/2.11.0
Tooling yes that is needed and welcome, but for many software engineers, IDEs are the least exciting part of a language ecosystem. Plenty programmers, and that too very capable programmers manage just fine with an editor. Speaking strictly for myself I couldnt care less for an IDE, it is irrelevant for me, especially in the context of a language that is syntactically lot more terse than Java. I would rather have the Scala team focus on more pressing problems, code optimizers, compile times, run times, other backends (LLVM?, GCC?), more airtight abstractions around the JVM. These are damned hard problems and Scala has been making steady progress, so this is the part that I find exciting and promising. Then again many are incapable of programming without an IDE and throwing in an IDE often does not drastically improve the quality of their code either.
The IDE is much better than it was a year ago, in terms of stability, refactoring reliability, implicit highlighting.... It's still got a way to go, but it's there.
I don't know about SBT, I've always found Maven to be better.
Improvements to any language could be argued to threaten Scala, but most of Scala's advantages over Java still apply - better type inference, better concurrency support, case classes, pattern matching, higher-kinded types. Certainly I don't think anyone who's adopted Scala will be lured back by Java 8. If anything Java 8 seems to me to be more of a threat to C#, and especially to Kotlin, than to Scala.
I don't know about big companies, but in London startups it feels like Scala adoption is still going up.
If you frame your question in that way, it's hard to provide a good answer. IDE support and SBT are great for many people as was pointed out already, and they are improving. These complaints are, in my perception, quickly becoming a thing of the past.
You also don't mention why you find Java 8 such a nice comeback. If it's mainly about lambdas, I don't think you properly understand why people use Scala. In addition, of the companies you mention, only Twitter uses Scala significantly.
I'm going to opine that I think IDE support can (and probably eventually will) be a lot better. I don't think Scala support in IntelliJ or Eclipse is where Java support was in Eclipse 10 years ago. When I first used Eclipse with Java, it was like the IDE new the entire language it was masterful at providing me all the docs and refactorings I might want. Granted, Java then had a much bigger audience than Scala now, and Scala's a much bigger language for and IDE to fully understand. But I'm finding IntelliJ improves visibly on pretty much a biweekly basis.
SBT to me is damn near disastrous, though. It's like they created this obtuse DSL for describing transformations to the immutable project settings, when they could have just used mutable features Scala already has, and obviated the need to learn a gnarly layer on top, with its weird rules and quasi-Scala syntax. The other thing that annoys the hell out of me is how difficult it is understand what keys exists in what scopes, what order tasks run in, and all the arbitrary translations between identifiers in the SBT shell versus the files themselves.
I read the entire 40+ page Getting Started guide, and I still find it incredibly frustrating when I need to do nontrivial modifications to my build process. I really hope SBT is up for a complete overhaul. I don't think it is though, because I think it provides people who invest the time to learn all the magic incantations that warm fuzzy feeling of superiority, because only they can peer through the layers of inscrutable DSL-syntax and see the elegance far beneath.
...Now that I've got that out of my system, I agree that Scala is on the rise, and we'll see these aspects improve as more people start to realize just how powerful the language is building large systems -- the horizontal and vertical scaling benefits Odersky always talks about.
Have you tried SBT 0.13? I think it addresses some of your issues with SBT syntax. The DSL usage has been greatly simplified (many fewer operators to remember). Also, the identifiers are now the same in both the config files and the shell.
In older versions of SBT (SBT 0.7 and earlier) the configuration was done using immutable settings. In SBT 0.10 there was a switch to the current model. I believe this was done to make the configuration more declarative. A side of affect of this is that builds can now be easily parallelized without becoming nondeterministic. It also means that, in general, you shouldn't care what order tasks a
re run in.
As far determining what keys are available in which scopes, I have always found this pretty easy to
determine by using the 'inspect' command in the SBT repl. Although, perhaps I am not entirely under
standing your issue here.
SBT does have a problem for specific cases when you do need tasks to run in a particular order. However,
this is something that is currently intended to be addressed[1] in the SBT 1.0 release, which is scheduled for the end of this year.
I'm on 0.13.5, and I honestly don't notice much of a difference. Maybe once projects and docs fully bring themselves up to date, it will be better, but I still see some keys and scopes that have hyphens in their names in the console. Looking up cryptic operators is also a huge pain. Since `<<=` is no longer emphasized, I couldn't find it anywhere within the current manual, and yet many StackOverflow answers and readmes recommend using it to add a dependency to an existing task.
It's easy to inspect a single setting with `inspect`, but it's very unwieldy to try to examine the entire dependency tree this way.
The very un-Scala-esque syntactic rules in build.sbt files also are a huge pain for anybody who just needs to dabble. Clearly, writing Build.scala files was a major drag, but it really feels to me that the answer should have been a better pure Scala API, rather than a DSL. It's a really leaky abstraction.
The old versions of the keys and scopes with the hyphens are still present for backwards compatibility. However, you don't need to use those.
Yeah, I can certainly understand your frustration about finding info for the '<<=' operator. This is an unfortunate side effect of cleaning up the API. All of the old examples uses it but now the documentation doesn't explain it up front.
It sounds like your problem with examining the dependency tree might be solved by one of the 'inspect' sub commands (tree|uses|definitions). Running 'inspect tree <command>' will give you the entire dependency tree for '<command>'.
I'm not sure I understand your last point. What do you mean by 'better pure Scala API, rather than DSL'? Do you mean that the SBT API is not very good? Do you mean that the '.sbt' files are annoying to use? I'm a little confused since the DSL is a Scala API.
In general, I completely agree with you. SBT has been a disaster. It was a mistake from the very beginning and a time sink the entire community simply did not need.
That said, later versions are starting to at least move in the right direction with the removal/de-emphasizing of the cryptic symbols and the consolidation of the core concepts.
I was only able to get over my hatred of sbt when I came to realize that it isn't a build tool. It is a hugely powerful task system with declarative, type safe syntax. That you can use it to build scala projects is just a side effect. Whether anyone needs a hugely powerful task system is another question...
I admit that I have not taken SBT anywhere near its limits. For simple projects, later versions do present a very easy to read build system in my experience, though.
I wouldn't describe writing a build script for Gradle as "using" Groovy. Most Gradle build files out there seem to be fairly short, usually 20 to 100 lines long, and consist of nothing more than the standard DSL calls. The template was probably copied from some other project, and the blanks filled in. I've yet to see a Gradle build script that actually drops out of the DSL to use Groovy to do any programming. The build script "authors" would have no idea how to use any of the more advanced Gradle API calls or Groovy language features beyond collection literals and closures. Gradle even still ships with the ancient Groovy version 1.8.
In Germany there was a Groovy craziness in JUGs around 2010, mostly coupled to Grails projects. At JSF days in Austria Oracle guys were discussing adding first class support for writing JSF applications with Groovy.
Fast forward to 2014, in our consulting projects when Groovy skills are requested, they usually mean that developer needs to take care of Gradle scripts.
Outside some banks in London City and the typical Fortune 10 companies like Twitter, Facebook and friends, there is very little uptake if you expand the spectrum to the Fortune 500 boring corporate world.
Which is the world I live in. Customers always ask for plain Java[1], since it gives them the highest value[2] for developer rotation.
As others are going to say, I wish they spent some of the time optimizing the compile times, yes you can fix it with multi-projects and the incremental compiler but sometimes that doesn't work to well especially when you start to throw in advance scala features like macros. But maybe they are hoping the new backend helps, from the docs it looks like it may in some cases.
First off, macros are not just an advanced feature. They are experimental. Investments in tooling will always focus on official features first (not to say we ignore macros, but we have to prioritize).
We spend a lot of time optimizing the compiler. Several big users have reported nice speed ups on 2.11. I won't quote numbers (too hard to benchmark accurately), but encourage you to give it a go. The upgrade is easy.
Also, it's just generally a good idea to design your software so that it can be compiled incrementally in smaller modules.
I would be more excited to use Scala if the compile times didn't suck so much. Even on a tiny project, it just feels like it takes too long to do anything.
I guess I'm too used to Ruby and PHP where I save and things happen right away. My test suite runs in less than a second. Stuff like that.
Instead, everything takes 5-10 sec and that just breaks my flow constantly. I used Golang and while I don't love the language I love the compiler a lot. It's really fast and nice and feels like a dynamic language.
Anyhow, Scala compiler has been slow forever and probably will be slow forever. Oh well.
This is where IDEs and sbt's ~compile come in handy. I don't hit build very often, since the IDE tells me about my mistakes interactively, and, once it's happy, a build makes for a great micro-break :-)
I agree that for code changes, incremental compilation is usually reasonably painless. Compile times really kill me when I have to do a lot of clean builds, like when modifying my build scripts.
One of the things I was hoping for in improved scala tooling was better IDE agnostic solutions for developers.
For instance, I much prefer to use vim than an IDE, but due to the near requirement to have syntax checking in the editor to avoid long compile cycles I always end up giving up and using an IDE. That a common response to slow tooling complaints is that an IDE hides these issues is an indictment of the scala tool chain.
I agree that the compiler slowness is pretty annoying. A clean build for my project takes more than 30 seconds. That being said, I do appreciate how much the compiler does for me, with the type system and implicits.
I think a faster compiler is coming, but probably not right away. From another comment on this page, it sounds like Scala 3 is going to break source compatibility to remove or improve a lot of the stuff that's holding progress back.
Scala 3 will be a bigger undertaking, likely to have a major amount of breaking of core language contructs and other fundamental changes. Experimentation is already underway: it's called the Dotty compiler [1].
The changes as seen from the outside will be much more conservative than that. We're not ready to comment on specifics yet, but the main focus is to remove warts (deprecating them in 2.12/2.13 first where possible), make the type system and (thus) type inference more regular, and make safari^W scalac snappier.
The compiler internals and language concepts are being simplified substantially (we've learned a lot of interesting lessons in compiler engineering in the last decade, some of which require bigger changes).
The Scala 2.x series is for source compatible releases. The goal is that you can just recompile your Scala 2.11 code on Scala 2.12. (As was the case for the upgrade from 2.10 to 2.11 -- modulo some changes necessary for important bug fixes.)
We're reserving the bigger version bump for when source compatibility is not fully maintained. We take backwards compatibility very seriously, so the jump will be as small as we can make it (balanced with the need to keep evolving and improving the language). We'll share our thoughts on Scala's longer-term roadmap once they've crystallized (soon!).
