scip-java v0.8 with Bazel support and "Find implementations"

We are excited to announce the release of scip-java v0.8. The highlights of this release include:

• New name: the project was previously called "lsif-java" and it has now been renamed into "scip-java" to reflect that it now emits a code indexing format called SCIP by default.
• Find implementations: it’s now possible to navigate from interfaces and abstract methods to the implementations to their concrete implementations.
• Bazel support: it’s now possible to use scip-java with Bazel builds.

New name: scip-java

We recently released SCIP, a new protocol that we are using at Sourcegraph to power precise code navigation. Previously, our Java indexer was called lsif-java because it emitted LSIF (Language Server Index Format) by default. We have renamed the indexer into scip-java to reflect that it now emits SCIP by default.

How to upgrade from lsif-java

Migrating from lsif-java to scip-java requires the following steps:

• Replace all usages of lsif-java with scip-java. The scip-java command-line interface supports the same sub-commands and flags as the old lsif-java interface. The only functional difference is that running scip-java index now produces an index.scip file instead of dump.lsif.
• Make sure you are using the latest version of the src command-line tool, which you can now optionally install via npm using the command yarn global add @sourcegraph/src. The latest version of src is needed to recognise the new index.scip file, which is generated by scip-java index. Running src precise-intel upload (previously, src lsif upload) accepts both index.scip and dump.lsif files. If you’re not uploading to Sourcegraph, run scip convert --from=index.scip --to=dump.lsif to convert any SCIP payload into LSIF. The scip command-line tool can be installed from here.

Find implementations

Previously, scip-java only supported "Go to definition" and "Find references". This release adds support to "Find implementations" such that you can navigate from an interface or abstract method to their concrete implementation. For example, consider the following type hierarchy

interface Animal {
  String sound();
}
class Dog implements Animal {
  @Override 
  String sound() { return "Woof!"; }
}

The "Find implementations" action navigates from the Animal interface and Animal.sound() method, to the Dog class and Dog.sound() method, respectively.

Bazel support

Previously, scip-java did not have great support for Bazel. It was possible to use scip-java with Bazel but the steps were undocumented and it required awkward workarounds. This release adds proper support for Bazel that is compatible with Bazel's build cache and allows incremental indexing.

It requires custom configuration to WORKSPACE and BUILD files to integrate scip-java with Bazel builds. It's simply not possible for scip-java index command to automatically configure everything like it does for Gradle and Maven while remaining compatible with Bazel's build model.

Check out the documentation on how to get started with configuring Bazel and scip-java.

Pull Requests

• Add LICENSE (#451) @jdorfman
• Rename LSIF into SCIP (#449) @olafurpg
• Fix brittle code related to launching a Bazel process (#448) @olafurpg
• Bump pr-auditor go version to 1.18 (#447) @sanderginn
• Add Bazel support, fixes #88 (#439) @olafurpg
• Fix a few bugs on Windows (#436) @olafurpg
• Clarify what files are needed for Gradle and Maven builds (#438) @olafurpg
• copy JDK_HOME bootstrap files to forwarding toolchain directory for Kotlin (#418) @Strum355

lsif-java v0.7.7

What's Changed

• workflows: add pr-auditor and test plans to PR templates by @bobheadxi in #402
• Promote synthetic definitions into normal definitions if they share names by @olafurpg in #404

New Contributors

• @bobheadxi made their first contribution in #402

Full Changelog: v0.7.6...v0.7.7

lsif-java v0.7.5

Improved code navigation for Scala

Previously, "goto defintion" did not work for synthetic symbols such as
case class companion objects or var setter methods.

class Main {
  case class User(name: String)

  var user = User("Susan") // go to definition on `User` did not work

  user = User("John") // go to definition on `user` did not work
}

Now, lsif-java emits occurrences for these synthetic symbols so that "goto definition" works as expected.

• Emit additional occurrences for synthetic Scala symbols (#398) @olafurpg

Pull Requests

• Add support for LSIF Typed (#394) @olafurpg
• fix MD5 checksum computation (#392) @Strum355
• Add minimized reproduction for Lombok Builder() annotation (#383) @olafurpg
• Document how to customize the default Java version for lsif-java (#381) @olafurpg
• Fix bug when indexing Scala package repo (#373) @olafurpg

lsif-java v0.7.2

Kotlin hover support

Previously, the hover message only displayed the name of Kotlin symbols. Now, the hover message shows the symbol signature and KDoc.

Improved Java 17 support

Running lsif-java index will now use the correct --add-exports flags in more cases when compiling code with Java 17. There are still many situations where users on Java 17 need to manually add the following JVM options

--add-exports jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED

Updated Docker container

The Docker container sourcegraph/lsif-java is now automatically published on every release alongside the Maven Central artifacts. The Docker container is fairly large because it includes pre-installed Java 8, Java 11 and Java 17. If the container size is a problem then we recommend creating your own Docker container with the Java launcher as documented here https://sourcegraph.github.io/lsif-java/docs/getting-started.html#java-launcher

Pull Requests

• Automate Docker container build and fix Java version detection (#372) @olafurpg
• Bump to the latest lsif-kotlin (#371) @olafurpg
• Update gradle:7.2.0-jdk8 Docker digest to 51a6950 (#369) @renovate
• Update dependency ch.epfl.scala:sbt-bloop to v1.4.11 (#364) @renovate
• Update dependency org.scalameta:sbt-native-image to v0.3.2 (#368) @renovate
• Update snapshot command to fallback to the .documentation field (#370) @olafurpg
• Simplify build configuration for Java 17 (#360) @olafurpg
• Remove PackageHub code (#359) @olafurpg
• Update gradle:7.2.0-jdk8 Docker digest to fa25782 (#343) @renovate
• Update dependency ch.epfl.scala:sbt-bloop to v1.4.9-33-c93326ba (#344) @renovate
• Update dependency com.google.cloud.sql:postgres-socket-factory to v1.3.4 (#348) @renovate
• Update dependency net.bytebuddy:byte-buddy-agent to v1.11.21 (#345) @renovate
• Update dependency org.scalameta:sbt-mdoc to v2.2.24 (#327) @renovate
• Update dependency org.flywaydb:flyway-core to v8 (#349) @renovate
• Make compiler plugin work with Java 17 (#358) @olafurpg
• Start testing with Java 17 (Zulu) (#357) @olafurpg
• Fix #319 - prevent IndexOutOfBoundsException (#356) @olafurpg
• Update documentation (#354) @olafurpg

lsif-java v0.6.11

Kotlin support for Gradle projects

Running lsif-java index in Gradle projects will now index Java, Scala and Kotlin sources. Note that Kotlin support is a few feature that's not as feature-complete as the suport for Java and Scala. Please give it a try and report feedback :)

What's Changed

• Add support for Kotlin Gradle projects by @olafurpg in #351
• Add support for Kotlin files in lsif-java.json builds by @olafurpg in #350

Full Changelog: v0.6.9...v0.6.11

lsif-java v0.6.9

SemanticDB synthetics are now indexed

Previously, "find references" wouldn't show results for Scala implicits or usages of apply methods. Now, these symbol usages are indexed by lsif-java.

Pull Requests

• Update dependency com.lightbend.sbt:sbt-java-formatter to v0.6.1 (#323) @renovate
• Update dependency org.scalameta:munit to v0.7.29 (#326) @renovate
• Update dependency org.rauschig:jarchivelib to v1.2.0 (#329) @renovate
• Update gradle Docker tag to v7.2.0 (#330) @renovate
• Update dependency org.flywaydb:flyway-core to v7.15.0 (#328) @renovate
• Update dependency com.google.cloud.sql:postgres-socket-factory to v1.3.3 (#334) @renovate
• Update dependency ch.epfl.scala:sbt-scalafix to v0.9.31 (#322) @renovate
• Update dependency net.bytebuddy:byte-buddy-agent to v1.11.16 (#325) @renovate
• Update dependency ch.epfl.scala:sbt-bloop to v1.4.9-14-67e8541e (#321) @renovate
• Return non-zero exit code when missing SemanticDB files (#332) @olafurpg
• Add index-dependency command to troubleshoot package repo issues. (#331) @olafurpg

lsif-java v0.6.8

Scala Gradle support

Previously, running lsif-java index in Gradle projects indexed only the Java parts of the codebase. Now, lsif-java index also indexes Scala code

https://sourcegraph.com/github.com/apache/samza@a7cb039d2d53cbcf207a12fce504066da8bc21cb/-/blob/samza-kv/src/main/scala/org/apache/samza/storage/kv/LoggedStore.scala?L38:69#tab=references

Fixed bug in Gradle projects using annotation processors

Sometimes, lsif-java index would report an error "plug-in not found: semanticdb" in Gradle projects using annotation processors. This bug should be fixed with this release.

Pull Requests

• Add support for Gradle Scala projects (#316) @olafurpg
• Properly fix issue with checkerframework Gradle plugin. (#315) @olafurpg
• Disable checkerframework Gradle plugin for lsif-java index (#314) @olafurpg
• Add docs for how to manually enable cross-repository navigation (#312) @olafurpg
• fix LsifBuildTool errors.nonEmpty checks (#310) @Strum355

lsif-java v0.6.6

Pull Requests

• Add docs for how to manually enable cross-repository navigation (#312) @olafurpg
• fix LsifBuildTool errors.nonEmpty checks (#310) @Strum355
• Handle -cp javac compiler option just like -classpath (#311) @olafurpg
• fix range emitted for member references to only include the reference (#309) @Strum355
• change manual gradle configuration to use compileOnly instead of implementation (#308) @Strum355
• Fix: message error '--build-tools=BUILD_TOOL_NAME' (#303) @marcel-dias
• Handle null/unit constant types. (#300) @olafurpg
• Trim the inferred metadata information POM files (#299) @olafurpg
• Make snapshot-lsif command more LSIF-compliant (#298) @olafurpg

lsif-java v0.6.4

Scala support

It's now possible to index Scala code with lsif-java. For example, this link shows you references to the java.lang.String.split() method from the Scala standard library, Spark and MUnit. The versions Scala 2.11.x, 2.12.x, 2.13.x and 3.x are supported.

sbt support

Previously, only the build tools Gradle and Maven were supported for the lsif-java index command. Now, you can also run lsif-java index in sbt projects. Both Java and Scala sources will be indexed.

For example, you can now use precise code intelligence on Sourcegraph.com to browse the akka/akka and haifengl/smile repositories.

• https://sourcegraph.com/github.com/akka/akka/-/blob/akka-actor/src/main/java/akka/actor/dungeon/AbstractActorCell.java?L10:13#tab=references
• https://sourcegraph.com/github.com/haifengl/smile/-/blob/core/src/main/java/smile/validation/metric/Specificity.java?L36:14#tab=references

Observe that "find references" shows results in both Java and Scala sources.

⚠️ Caveats

The lsif-java index command does not yet support Scala projects using the build tools Maven or Gradle. Subscribe to #301 to get notified when we add support for Scala Maven projects and #302 when we add support for Scala Gradle projects.

Pull Requests

• Handle null/unit constant types. (#300) @olafurpg
• Trim the inferred metadata information POM files (#299) @olafurpg
• Make snapshot-lsif command more LSIF-compliant (#298) @olafurpg
• Upgrade to the latest sbt-sourcegraph (#295) @olafurpg
• Add build tool support for sbt. (#292) @olafurpg
• Update dependency org.flywaydb:flyway-core to v7.12.1 (#289) @renovate
• Update dependency ch.epfl.scala:sbt-bloop to v1.4.8-98-e7d4e01e (#290) @renovate
• Update dependency net.bytebuddy:byte-buddy-agent to v1.11.12 (#287) @renovate
• Update dependency com.google.cloud.sql:postgres-socket-factory to v1.3.2 (#282) @renovate
• Swap order of signature and docstrings in hover messages (#285) @olafurpg
• Start emitting SymbolInformation for Scala package repos. (#280) @olafurpg
• Update dependency ch.epfl.scala:sbt-bloop to v1.4.8-94-1cfdf0bd (#269) @renovate
• Update dependency net.bytebuddy:byte-buddy to v1.11.9 (#273) @renovate
• Update dependency org.scalameta:sbt-scalafmt to v2.4.3 (#275) @renovate
• Update dependency com.zaxxer:HikariCP to v5 (#276) @renovate
• Update olafurpg/setup-scala action to v13 (#277) @renovate
• Add Scala support. (#278) @olafurpg

lsif-java v0.5.7

Rework emitted packageInformation vertex data

Previously, lsif-java included the library version in the name field of the packageInformation vertices. This releases changes this to only include the version info in the version field for consistency with other LSIF indexers.
The manager field is also changed to no longer have the value "packagehub". This was a temporary value during lifetime of the POC of PackageHub, which has been replaced by a new External Service type.

Pull Requests

• emit LSIF dump with rw-r--r-- permissions (#268) @Strum355
• Update packageInformation structure (#267) @olafurpg
• Update dependency net.bytebuddy:byte-buddy-agent to v1.11.8 (#256) @renovate
• Update dependency org.postgresql:postgresql to v42.2.23 (#257) @renovate
• Update dependency org.scalameta:munit to v0.7.27 (#262) @renovate
• Update dependency org.scalameta:sbt-native-image to v0.3.1 (#258) @renovate