Add support for Scala 2.12.0 #258

piyushnarang · 2016-11-28T21:00:50Z

Building on #257. Adding 2.12.0 as a cross version to include. Had to disable building 2.12 for a few projects as they require some finagle 2.12 artifacts (which in turn requires bijection-core 2.12). Once the finagle artifacts are in, we can reenable those projects.

johnynek · 2016-11-28T21:19:55Z

.travis.yml

  include:
    - scala: 2.10.6
-      script: ./sbt ++$TRAVIS_SCALA_VERSION clean scalafmtTest test mimaReportBinaryIssues
+      script: ./sbt "+++$TRAVIS_SCALA_VERSION clean"  "+++$TRAVIS_SCALA_VERSION test" "+++$TRAVIS_SCALA_VERSION mimaReportBinaryIssues"


why the 3 +++? I thought it was just ++?

Had to hook up the sbt-doge plugin to ensure that the overridden crossScalaVersions is respected. Without this I still get the error in the 3 projects we want to skip. I think we've done something similar in: twitter/chill#253

codecov-io · 2016-11-28T22:08:29Z

Current coverage is 65.21% (diff: 100%)

Merging #258 into develop will decrease coverage by 0.93%

@@            develop       #258   diff @@
==========================================
  Files            50         43     -7   
  Lines          1397       1259   -138   
  Methods        1347       1214   -133   
  Messages          0          0          
  Branches         50         45     -5   
==========================================
- Hits            924        821   -103   
+ Misses          473        438    -35   
  Partials          0          0

Powered by Codecov. Last update 469154b...d2b7314

clhodapp · 2016-11-28T22:22:46Z

FWIW, scalafmtTest should work. The output is a bit janky (there seems to be some mixing between the output for different projects b/c of the way I hacked calling it into the build), but it should report accurate results.

piyushnarang · 2016-11-28T22:26:34Z

@clhodapp ah you're right. Just tried running it and it seems to work. I can update to turn it back on.

clhodapp

Nothing here stands out as crazy

johnynek · 2016-11-28T23:23:49Z

If we turn on coverage by default doesn't that dramatically slow down the local tests?

piyushnarang · 2016-11-28T23:34:42Z

@johnynek looking at the build time in travis - https://travis-ci.org/twitter/bijection/builds/179597059
Was around 5-6 mins. There seems to be a fair bit of variance in general in the builds there. Seems like it was turned on earlier for 2.11.8 right? Do we want to have it turned on only for that?

johnynek · 2016-11-28T23:49:57Z

you should not need to do coverage for more than one version of scala, since it is the same code, right?

I think just picking one version "the main one", whatever we consider that to be, should work.

piyushnarang · 2016-11-29T00:03:05Z

@johnynek yes, running coverage on one version should suffice. I wasn't able to figure out how to enable coverage via the command line using sbt. Was running into an error:

[info] Set current project to bijection (in build file:/home/travis/build/twitter/bijection/)
[error] Expected ':' (if selecting a configuration)
[error] Not a valid key: coverage (similar: coverageReport, coverageEnabled, coverageMinimum)
[error] bijection-core/coverage
[error]                        ^
The command "./sbt "+++$TRAVIS_SCALA_VERSION clean" "+++$TRAVIS_SCALA_VERSION coverage" "+++$TRAVIS_SCALA_VERSION test" "+++$TRAVIS_SCALA_VERSION coverageReport" "+++$TRAVIS_SCALA_VERSION mimaReportBinaryIssues"" exited with 1.

Can check if there's an issue running the sbt-coverage plugin with the sbt-doge +++ operator. Another option would be to enableCoverage in the build.sbt only for 2.11.8.

piyushnarang · 2016-11-29T02:31:17Z

@johnynek ok not 100% sure what the best way to proceed on this is. Here's what I see. When I set up coverageEnabled conditionally:

val buildLevelSettings = Seq(
...
coverageEnabled := (if (scalaVersion.value startsWith "2.11") true else false),
...
)

I still end up with coverage running in 2.12.0:

[tw-mbp-pnarang bijection (scala-2.12)]$ ./sbt "+++2.12.0 test"
[info] Loading global plugins from /Users/pnarang/.sbt/0.13/plugins
[info] Loading project definition from /Users/pnarang/workspace/bijection/project
[info] Resolving key references (10437 settings) ...
[info] Set current project to bijection (in build file:/Users/pnarang/workspace/bijection/)
[info] Setting version to 2.12.0
[info] Reapplying settings...
[info] Set current project to bijection (in build file:/Users/pnarang/workspace/bijection/)
[info] Compiling 27 Scala sources and 2 Java sources to /Users/pnarang/workspace/bijection/bijection-core/target/scala-2.11/classes...
[info] [info] Cleaning datadir [/Users/pnarang/workspace/bijection/bijection-core/target/scala-2.11/scoverage-data]
[info] [info] Beginning coverage instrumentation
...

Seems like our default setting of scalaVersion = 2.11.8 is ensuring that this can't be updated even if we try to pass it with ++ or +++ (it always is 2.11.8).

This shows up even in the sbt console:

$ ./sbt
> ++ 2.10.6
[info] Setting version to 2.10.6
[info] Reapplying settings...
[info] Set current project to bijection (in build file:/Users/pnarang/workspace/bijection/)
> show scalaVersion
[info] bijection-guava/*:scalaVersion
[info] 	2.11.8
...

Now if I comment out the line in build.sbt setting scalaVersion := "2.11.8", this seems to work ok - both the sbt console and the coverage builds behave as expected. We show the version passed in ++ or +++ and coverageEnabled = true only for 2.11.

Seems like this problem is present even currently. We are checking scalaVersion to set scalacOptions - https://github.com/twitter/bijection/blob/develop/build.sbt#L26. When I tested this out on the sbt command line, it seems like we're always setting the options for 2.11.

My proposal is to remove the line setting scalaVersion. There might be a better way to do this that I may be missing so if folks have other ideas / suggestions, I could try them out.

clhodapp · 2016-11-29T16:29:36Z

What if you move the setting for coverageEnabled down into the module method?

johnynek · 2016-11-29T17:37:39Z

build.sbt

  scalaVersion := "2.11.8",
  javacOptions ++= Seq("-source", "1.6", "-target", "1.6"),
  javacOptions in doc := Seq("-source", "1.6"),
+  coverageEnabled := true,


do we need to enable this in the build? Can't we do +++$TRAVIS_SCALA_VERSION coverage test or something?

What is it about this change that is requiring us to always do coverage?

That's a good point. As a dev just trying to see if you broke the tests, you probably wouldn't want coverage to run anyway.

Yeah so I originally tried to do that. The +++$TRAVIS_SCALA_VERSION coverage command doesn't work with the sbt-doge plugin for some reason (see above): #258 (comment)

I needed to pull in the sbt-doge plugin cause I wanted to ensure that bijection-scrooge, bijectionUtil and bijectionFinagleMySql can override the cross-scala versions to skip 2.12 as we don't have the finagle artifacts. Based on searching that seemed to be the best option to skip modules selectively for a given scala version. If you folks know of a better alternative, I could try that and drop the doge plugin.

piyushnarang · 2016-11-29T23:59:14Z

@clhodapp / @johnynek - made some updates, now code coverage is enabled only on 2.11. Some of the pain is related to sbt-doge which doesn't seem to have been working with the inThisBuild method for some reason. It was always using scalaVersion = 2.11.8. The sbt-doge changes are temporary till finagle is able to publish their 2.12 jars. At that point we can yank it out and enable the builds on the 3 projects that depend on finagle.

clhodapp · 2016-11-30T02:19:14Z

Not setting a build-level scala version breaks tools like ensime.

Could you try moving just scalaVersion out of the list as scalaVersion in ThisBuild := "2.11.8" and renaming the list back to commonSettings? I wasn't thinking about compatibility with sbt-doge when I did my build refactoring, but I believe that doing this should work.

piyushnarang · 2016-11-30T02:34:32Z

@clhodapp so I tried setting the scalaVersion like you specified and renaming. Seems like that doesn't end up compiling to 2.12. So the version ends up staying at 2.11.8 even though we specify +++2.12.0.
I don't think we were setting the build level scala version earlier right? I guess I'm also wondering if it is ok to break it for just the finagle release to proceed? Once the finagle jars are published, we don't need the sbt-doge stuff.

clhodapp · 2016-11-30T03:21:32Z

Bleh. OK. I give up then haha. With your PR as-is, bijection is no worse off in any way than it was a few days ago, so it seems reasonable. Once you remove doge, you can set a build-level scala version if desired.

johnynek · 2016-11-30T03:58:29Z

Yeah, I guess that's fine. It's a bummer to make the tests always slow, but I guess doge is incompatible with coverage somehow.

…

On Tue, Nov 29, 2016 at 17:21 Chris Hodapp ***@***.***> wrote: Bleh. OK. I give up then haha. With your PR as-is, bijection is no worse off in any way than it was a few days ago, so it seems reasonable. Once you remove doge — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#258 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEJdkE4pphERcNgZ1anOWTaIFsXv4iOks5rDOu9gaJpZM4K-QaC> .

sritchie · 2016-11-30T16:10:29Z

.travis.yml

  include:
    - scala: 2.10.6
-      script: ./sbt ++$TRAVIS_SCALA_VERSION clean scalafmtTest test mimaReportBinaryIssues
+      script: ./sbt "+++$TRAVIS_SCALA_VERSION clean" "+++$TRAVIS_SCALA_VERSION scalafmtTest" "+++$TRAVIS_SCALA_VERSION test" "+++$TRAVIS_SCALA_VERSION mimaReportBinaryIssues"


one last thing - do we want this binary check for 2.10 as well? In algebird we only check the 2.11 build.

sritchie · 2016-11-30T16:10:48Z

Other than my one comment, 👍 !

johnynek · 2016-11-30T16:28:03Z

Actually good point @sritchie but I do think we want to check binary compat in all versions. For instance, the rules in scala 2.12 are different than 2.11 (more changes should be binary compatible due to java 8 features being used). Scala 2.10 and 2.11 might be the same or close, but just to be sure it seems like a good idea to run on all supported versions.

piyushnarang · 2016-11-30T18:29:22Z

Thanks for the review and feedback @johnynek, @clhodapp, @mosesn, @sritchie! Will follow up once the finagle artifacts are out with cleaning this back up :-)

stuhood · 2016-11-30T21:56:12Z

Thanks everyone! Will watch for a release.

sritchie · 2016-12-04T00:35:22Z

@stuhood bijection is out! 0.9.4 should get you going. We're working on algebird now.

Piyush Narang added 2 commits November 28, 2016 12:57

Add support for Scala 2.12.0

5933f12

Update yml indent

8d8e1c2

johnynek reviewed Nov 28, 2016

View reviewed changes

Turn on coverage by default

931087a

piyushnarang mentioned this pull request Nov 28, 2016

2.12-ification #248

Closed

Turn on scalaFmtTest again

d2b7314

clhodapp approved these changes Nov 28, 2016

View reviewed changes

johnynek reviewed Nov 29, 2016

View reviewed changes

Remove call to inThisBuild, enable coverage only for 2.11

ede96f3

Piyush Narang added 2 commits November 29, 2016 16:42

Use oraclejdk8 for scala 2.12.0

cbe3dd3

Turn off mima for 2.12 as this is the first release

9f928db

mosesn approved these changes Nov 30, 2016

View reviewed changes

sritchie reviewed Nov 30, 2016

View reviewed changes

johnynek merged commit 886f42e into twitter:develop Nov 30, 2016

Add support for Scala 2.12.0 #258

Add support for Scala 2.12.0 #258

Uh oh!

Conversation

piyushnarang commented Nov 28, 2016

Uh oh!

johnynek Nov 28, 2016

Choose a reason for hiding this comment

Uh oh!

piyushnarang Nov 28, 2016

Choose a reason for hiding this comment

Uh oh!

codecov-io commented Nov 28, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current coverage is 65.21% (diff: 100%)

Uh oh!

clhodapp commented Nov 28, 2016

Uh oh!

piyushnarang commented Nov 28, 2016

Uh oh!

clhodapp left a comment

Choose a reason for hiding this comment

Uh oh!

johnynek commented Nov 28, 2016

Uh oh!

piyushnarang commented Nov 28, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johnynek commented Nov 28, 2016

Uh oh!

piyushnarang commented Nov 29, 2016

Uh oh!

piyushnarang commented Nov 29, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clhodapp commented Nov 29, 2016

Uh oh!

johnynek Nov 29, 2016

Choose a reason for hiding this comment

Uh oh!

clhodapp Nov 29, 2016

Choose a reason for hiding this comment

Uh oh!

piyushnarang Nov 29, 2016

Choose a reason for hiding this comment

Uh oh!

piyushnarang commented Nov 29, 2016

Uh oh!

clhodapp commented Nov 30, 2016

Uh oh!

piyushnarang commented Nov 30, 2016

Uh oh!

clhodapp commented Nov 30, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johnynek commented Nov 30, 2016 via email

Uh oh!

sritchie Nov 30, 2016

Choose a reason for hiding this comment

Uh oh!

sritchie commented Nov 30, 2016

Uh oh!

johnynek commented Nov 30, 2016

Uh oh!

piyushnarang commented Nov 30, 2016

Uh oh!

stuhood commented Nov 30, 2016

Uh oh!

sritchie commented Dec 4, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

codecov-io commented Nov 28, 2016 •

edited

Loading

piyushnarang commented Nov 28, 2016 •

edited

Loading

piyushnarang commented Nov 29, 2016 •

edited

Loading

clhodapp commented Nov 30, 2016 •

edited

Loading