Publishing Scala libraries to Sonatype

Summary

1. Install SBT PGP Plugin
2. Get an identity
3. Register yourself with Sonatype
4. Register your project with Sonatype
5. Configure SBT to publish
6. Publish SNAPSHOT(s)
7. (optional) Suppress PGP passphrase prompt
8. Stage a release
9. Publish release
10. Enable sync
11. (optional) Announce on ls

1. Install SBT PGP Plugin

Frequency: once

This plugin is needed for signing the artefacts, and can also be used to generate keys.

Install the SBT PGP Plugin using these instructions. Note that this documentation suggests creating the file ~/.sbt/plugins/gpg.sbt, however your SBT config may be in another location. If you are using v0.13, check for config in ~/.sbt/0.13/plugins/.

2. Get an identity

Frequency: once

For Sonatype to trust your artefacts are genuinely from you, they will need to be signed. You will need to have a PGP key pair and publish the public key to a public key server pool.

Actually, Sonatype require that the artefacts be signed, but they don't use it for trust. Instead they use user/password authentication with a checksum to protect against modification. For our purposes all we need to know is that our artefacts need signing.

Generating a key pair

If you already have a PGP key pair that you wish to use, you can skip this step.

There are normally three options available for generating the keys:

1. Use the GPG executable.
2. Use the SBT PGP Plugin, delegating to the GPG executable.
3. Use the SBT PGP Plugin, delegating to the BouncyCastle PGP implementation.

Usually I'd recommend option 3 as the simplest option, because the SBT PGP Plugin is required anyway in order to publish artefacts. However, at the time of writing there is a bug with the plugin that prevents it from generating keys. So, let's proceed with option 1.

Generating a key pair with GPG executable

At the terminal, enter gpg --gen-key and follow the prompts for information. If in doubt, accept the defaults.

If you do not have the gpg executable, then download it for windows or mac, or build it yourself. The Debian/Ubuntu package is gnupg and is probably already installed.

Generating a key pair with SBT

This method is currently broken. Proceed with caution!

By default the SBT PGP plugin will use the BouncyCastle implementation. If you wish it to delegate to a local GPG implementation, then follow these instructions: Configuration: GPG Command-Line Utility.

To generate a new key pair run sbt and then

set pgpReadOnly := false
pgp-cmd gen-key

Follow the prompts. See the SBT PGP plugin usage page for more details.

Publishing your key

If you have a key pair and have already published your public key you can skip this step. If you are unsure about whether you have published in the past you can check with the GPG executable: gpg --search-keys <name|id>. In fact, here's a cute way to pipe the id into your search directly:

gpg --search-keys gpg --list-keys|grep ^pub| cut -d'/' -f2|cut -d' ' -f1

You can send your key to a server in a public key server pool (from where it will then be synced) by using either the GPG executable or the PGP SBT Plugin.

By default both programs used the same keyring on my ubuntu 13.10 build. The GPG method is faster, uses less keystrokes/commands and lets you push more than one key at a time if you wish. But both ways are pretty quick and easy.

Publishing with GPG executable

First identify the ID of your public key

$ gpg --list-keys
/home/jem/.gnupg/pubring.gpg
----------------------------
pub 2048R/79060711 2014-01-18
uid Jeremy Mawson (Jem)

sub 2048R/3C00E0B0 2014-01-18

The public key is listed on the line starting with pub. The ID is the hexademical value after the forward slash (in my case 79060711).

With this information you are able to send the key.

$ gpg --keyserver hkp://pool.sks-keyservers.net --send-key 79060711
gpg: sending key 79060711 to hkp server pool.sks-keyservers.net

Publishing with SBT

First identify the ID of your public key

$ sbt
Loading /home/jem/tools/sbt-0.13.1/bin/sbt-launch-lib.bash
[info] Loading global plugins from /home/jem/.sbt/0.13/plugins
[info] Set current project to projects (in build file:/home/jem/projects/)
> pgp-cmd list-keys
/home/jem/.gnupg/pubring.gpg
----------------------------
pub RSA@2048/79060711 2014-01-18
uid Jeremy Mawson (Jem) 
sub RSA@2048/3c00e0b0 2014-01-18

and then publish

> pgp-cmd send-key 79060711 hkp://pool.sks-keyservers.net
[info] Sending PublicKeyRing(PublicKey(873c61fc79060711, Jeremy Mawson (Jem) \
, RSA@2048),PublicKey(166556b93c00e0b0, , RSA@2048)) to \
HkpServer(http://pool.sks-keyservers.net:11371)
INF: [console logger] dispatch: pool.sks-keyservers.net POST /pks/add HTTP/1.1
> exit

3. Register yourself with Sonatype

Frequency: once

You will need an account on Sonatype's JIRA. Register at issues.sontatype.org.

4. Register your project with Sonatype

Frequency: per-project

This step is one of a few that are seemingly not automated by Sonatype. For your part, you will need to create a JIRA ticket and then wait up to 2 business days for that ticket to be complete.

Make sure you choose an appropriate groupId and advise if this id already exists in the maven central repository for another project.

While waiting for your JIRA ticket to be completed, continue with the next steps.

5. Configure SBT to publish

Frequency: per-project

If you read further on the Sonatype wiki you may have noticed a lot maven-specific instructions involving copious amounts of XML. Thankfully SBT automates a great deal of this (although some XML is still required).

Take a look at the SBT docs for deploying to Sonatype. Firstly, make sure your exact version of SBT is selected in the drop-down list. These instructions are version-specific. For this article I am referencing the version for 0.13.1.

1. Skip the instructions for setting up a PGP key as we have already covered this. It's not required per project. Instead make directly for the maven publishing settings. This details the build.sbt entries required to publish to the correct location and in the correct format for Sonatype.
2. Add the POM metadata to build.sbt. The example given should be sufficient to explain what is required here. There is help available should you need to choose an appropriate open-source licence.
3. Next, add your Sonatype login credentials to the SBT configuration. Of course, don't add this to build.sbt or any file which is public. It belongs in SBT configuration. The documentation recommends ~/.sbt/0.13/sonatype.sbt.

6. Publish snapshot(s)

Frequency: as often as you like

Sonatype allows us to publish snapshot versions as often as needed, without much quality control on their part. However, to continue, your JIRA ticket from step 3 must be approved.

Check your build.sbt and make sure your version setting ends with the text SNAPSHOT. Then execute sbt publishSigned. This will prompt you for your PGP passphrase, sign the artefacts and publish them to the Sonatype snapshot repo.

Do this as often as you like. It's a good idea to test the published artefacts by defining them in another project and seeing if they can be resolved and used.

7. Suppress PGP passphrase prompt

Frequency: once, optional

If you find the process of entering your PGP passphrase onerous, you can configure SBT to automate it (scroll down to Configuration: Automating Passphrase Entry) by adding a pgpPassphrase setting to SBT. Again, always put credentials in SBT's local config and not to any file under version control. Perhaps put it in a new file called ~/.sbt/0.13/pgp.sbt.

8. Stage release

Frequency: per-release

The library is polished, the artefacts look good and the bugs are dead. Now it's time to publish the real thing. Sonatype has greater quality controls for release artefacts than for snapshots, so there is a two-stage process. This first stage is akin to publishing a snapshot, but merely stages the artefacts on sonatype for the quality control process.

Change your :version setting so that it no longer ends with the text SNAPSHOT. If you're not sure, I'd suggest 0.1 to start with. Execute sbt publishSigned again. The change to the version number will alter the location the publishSigned task pushes the artefacts to. Your signed artefacts are now staged.

The location published to is determined by the version number. The code that controls this resides in your build.sbt and was set up in step 5 under maven publishing settings.

9. Publish release

Frequency: per-release

Now you can attempt to release the library. At the time of writing this must still be done manually via the Sonatype site. The UI is weird and sometimes confusing, but don't worry about making mistakes. Persist and you'll get there.

Update: The newly released SBT Sonatype Plugin takes care of this step automatically. If you still wish to do this step manually, read on. Otherwise check out the plugin and come back here for step 10.

Follow the instructions to close and release a staging repository.

Closing a staging repository will trigger the quality control checks. If there are any errors reported you will need to drop the staging repository and stage the release again after fixing the errors.

Releasing a closed repository will publish your artefacts. The only step left is to enable sync with the central repository.

10. Enable sync

Frequency: per-project

A comment on your JIRA ticket from step 3 to say that you've published your first release is required in order to get central sync activated for your project.

11. Announce on ls

Frequency: per-release, optional

If you don't know, ls.implicit.ly is a catalog service for Scala libraries. It is useful to publish there. This is a separate process to publishing to a maven repository. It will only work if you publish your source on GitHub.

Keeping it simple, there are two steps:

1. Creating a JSON file to the project meta-data, commiting this to your project and pushing it; and
2. Poking the LS process with curl to ensure it reads this file.

Taking it further, you can enable an SBT plugin to automate these steps. See the ls guide to publishing for instructions.

That's it. I hope these instructions have made this process easier for you. Let me know in the comments how you went.