Skip to main content

Gem rebuild gotcha

Production machines should never have anything compiled from source, and they should not have the tools to do that. Keeping that in mind I was packaging ruby gems using the Effing Package Management.

Usually I write rpm spec files manually when no existing ones fit our purposes, making sure the updates won't affect existing installation, however packaging 100+ rubygems was not the thing I would like to spend a day working on.

Enter fpm

$ gem fetch berkshelf
Fetching: berkshelf-3.1.5.gem (100%)
Downloaded berkshelf-3.1.5
$ fpm -s gem -t rpm berkshelf-3.1.5.gem
no value for epoch is set, defaulting to nil {:level=>:warn}
no value for epoch is set, defaulting to nil {:level=>:warn}
Created package {:path=>"rubygem-berkshelf-3.1.5-1.noarch.rpm"}

Great! Except that the thing does not build the dependencies, but it references them in Requires field:

$ rpm -qpR rubygem-berkshelf-3.1.5-1.noarch.rpm
...
rubygem(octokit) >= 3.0
rubygem(octokit) < 4.0
rubygem(celluloid) >= 0.16.0.pre
rubygem(celluloid) < 0.16.1.0
rubygem(celluloid-io) >= 0.16.0.pre
rubygem(celluloid-io) < 0.16.1.0
...

See that 0.16.0.pre version?

The Version field in the spec is where the maintainer should put the current version of the software being packaged. If the version is non-numeric (contains tags that are not numbers), you may need to include the additional non-numeric characters in the release field. -- Fedora Packaging Naming Guidelines

To make the story short, our berkshelf RPM will not be installable, celluloid RPM with version 0.16.0 will not satisfy 0.16.0.pre requirements.

A quick and dirty way of handling this would be to build celluloid RPM as is, but update berkshelf's gemspec to reference the version we can use.

Rebuilding gem

Should be as easy as gem unpack and gem build:

$ gem unpack berkshelf-3.1.5.gem
Unpacked gem: '/tmp/vendor/cache/berkshelf-3.1.5'
$ sed -i 's/0\.pre/0/' berkshelf-3.1.5/berkshelf.gemspec
$ gem build berkshelf-3.1.5/berkshelf.gemspec
fatal: Not a git repository (or any of the parent directories): .git
WARNING:  description and summary are identical
  Successfully built RubyGem
  Name: berkshelf
  Version: 3.1.5
  File: berkshelf-3.1.5.gem

Notice fatal: Not a git repository and look at the resulting gem:

$ ls -l berkshelf-3.1.5.gem
-rw-r--r-- 1 rye rye 4608 Oct 25 16:56 berkshelf-3.1.5.gem

The resulting gem is almost 5kiB, down from original 103K. Our gem is empty now.

Note: gem unpack --spec would produce yaml-formatted gemspec file which will not be accepted by gem build. fpm --no-gem-prerelease does not affect dependencies.

Enter git

Look at berkshelf.gemspec and notice that it uses git to provide the file listing:

...
  s.homepage                  = 'http://berkshelf.com'
  s.license                   = 'Apache 2.0'
  s.files                     = `git ls-files`.split($\)
  s.executables               = s.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
  s.test_files                = s.files.grep(%r{^(test|spec|features)/})
...

That's where the 'fatal' message is coming from, and since that appears to be a recommended way of writing the gemfile, that's what makes our resulting gem file empty (error from git ls-files is silently ignored). It is expected that the gem will be always built from git repository, which is not true in our case.

Again, fixing it quick and dirty way - making unpackged gem folder a git repository:

$ git init berkshelf-3.1.5
Initialized empty Git repository in /tmp/vendor/cache/berkshelf-3.1.5/.git/
$ pushd berkshelf-3.1.5
$ git add .
$ git commit -m "Dummy commit"
$ gem build berkshelf.gemspec
WARNING:  description and summary are identical
  Successfully built RubyGem
  Name: berkshelf
  Version: 3.1.5
  File: berkshelf-3.1.5.gem
$ mv berkshelf-3.1.5.gem ../
$ popd
$ ls -l berkshelf-3.1.5.gem
-rw-r--r-- 1 rye rye 105472 Oct 25 17:10 berkshelf-3.1.5.gem

Much better.

Final build

$ fpm -s gem -t rpm berkshelf-3.1.5.gem
no value for epoch is set, defaulting to nil {:level=>:warn}
no value for epoch is set, defaulting to nil {:level=>:warn}
Created package {:path=>"rubygem-berkshelf-3.1.5-1.noarch.rpm"}
$ rpm -qpR rubygem-berkshelf-3.1.5-1.noarch.rpm
...
rubygem(celluloid) >= 0.16.0
rubygem(celluloid) < 0.17.0
rubygem(celluloid-io) >= 0.16.0
rubygem(celluloid-io) < 0.17.0
...

While the original issue may be seen as a bug in FPM (will update the post if/when GitHub issue is created for that), the dependency on git for file listing may cause a bit of confusion for an unsuspected developer/release engineer.