Integrating Travis CI with Dist::Zilla

Both Dist::Zilla and Travis CI can greatly improve the quality of your Perl distribtions. But using them in combination causes issues. Fortunately, they can be easily solved.

Dist::Zilla not only simplifies the development of CPAN modules. It is also a quality assurance tools as it either automates or tests error-prone steps in the release process. Travis CI continuously tests your Perl module after every push against a configurable list of Perl versions, more precisely against a version/feature matrix.

Making the Distribution Buildable and Testable by Travis CI

Dist::Zilla generates Build.PL and Makefile.PL for you. As generated files, they are normally excluded from version control and without additional measures, Travis CI will be unable to build, leave alone test your distribution.

One possible solution would be to add Dist::Zilla as a requirement to your .travis.yml. That has its own issues:

• Dist::Zilla has a very long dependency list. It will slow down testing of your module and it will put unnecessary load on the Travis CI servers.
• You also have to add all the Dist::Zilla plug-ins you are using. This is error-prone until somebody writes a Dist::Zilla plug-in that generates a travis.yml file.

Likewise, anybody else who wants to build your module from a git checkout has to install Dist::Zilla and all its plug-ins. And as a matter of fact, Dist::Zilla is quite controversial.

Fortunately, it is easy to keep out of the advocacy battles and accomodate the requirements of both Travis CI and the people that want to work with your sources from git. You have to write back the files that are necessary for building your package into the top-level directory.

The Dist::Zilla plug-in CopyFilesFromBuild is your friend here. Add the following section to dist.ini:

[CopyFilesFromBuild]
copy = Build.PL
copy = LICENSE
copy = MANIFEST
copy = Makefile.PL

The next time that you run dzil build, these files will be copied back from the directory YOUR-MODULE-VERSION to the top-level directory. You should, of course, add them to git, so that they are available to people that check them out.

That breaks the mantra "do not put generated files under version control". But you should lightheartedly break that rule, whenever somebody in your team lacks a tool not strictly needed for development. Dist::Zilla is such a tool.

The files MANIFEST and LICENSE are not strictly needed, but they make github and other people happy.

There are a couple of gotchas, though:

The files only get updated, after you run dzil build. You must not forget that step, when they would have to be updated.

Another implication is that MANIFEST gets dirty, whenever you add a file to the distribution, and the *.PL build makers get dirty, whenever a dependency changes or you change the version number. Consider it a friendly reminder, and not a caveat.

There is, however, a real problem. When working with git, you will normally replace the [GatherDir] plug-in with [GitGatherDir] so that only files under version control are copied into the output directory by dzil build. The four files will now cause a conflict because they are both a source and a target file.

The can be easily fixed by telling [Git::GatherDir] about them:

[Git::GatherDir]
exclude_filename = Build.PL
exclude_filename = LICENSE
exclude_filename = MANIFEST
exclude_filename = Makefile.PL

They are now ignored.

Running Author Tests On Travis CI

Another nice feature of Dist::Zilla are author tests. They are tests that are meant to be run only by the author because they do cosmetic checks that do not affect the run-time behavior of a module.

A typical configuration in dist.ini looks like this:

[Test::Perl::Critic]
[PodCoverageTests]
[PodSyntaxTests]

When building the distribution, Dist::Zilla will now auto-generate the tests t/author-critic.t, t/author-pod-coverage.t, and t/author-pod-syntax.t in such a way that hey are run with dzil test but skipped for people that download a tarball of your distribution.

But they are also not present in your top-level test directory t, and that implies that they are not run by Travis CI. I usually run tests with prove -l instead of dzil test, so that I would also skip the author tests on a regular basis because they are not present in the top-level directory t.

This may be exactly what you want, but I prefer to be notified about problems as early as possible.

The solution is the same as described above for Makefile.PL and friends. First you make sure that the generated files are copied back into your test directory. Change dist.in like this:

[Git::GatherDir]
exclude_filename = Build.PL
exclude_filename = LICENSE
exclude_filename = MANIFEST
exclude_filename = Makefile.PL
exclude_match = t/author-.*\.t
[Test::Perl::Critic]
[PodCoverageTests]
[PodSyntaxTests]
[CopyFilesFromBuild]
copy = Build.PL
copy = LICENSE
copy = MANIFEST
copy = Makefile.PL
copy = t/author-critic.t
copy = t/author-pod-coverage.t
copy = t/author-pod-syntax.t]

That instructs [Git::GatherDir] to also ignore the author tests, and it copies them back.

But this is not yet enough. Travis CI would now run the test scripts, but the tests contained would be skipped because the environment variable AUTHOR_TESTING is not set. This must be fixed in travis.yml:

env:
  - AUTHOR_TESTING=1

And, voilà, Travis CI will set the environment variable while testing the build, and the author tests will no longer be skipped.

But there is another small gotcha. You also have to tell Travis CI to install the dependencies for the author tests. The first version of this post recommended to put the following snippet into dist.ini:

[Prereqs / TestRequires]
Pod::Coverage::TrustPod = 0
Test::Perl::Critic = 0

This turned out to be a bad decision because most people run the tests, when installing software, and hence both Pod::Coverage::TestPod and Test::Perl::Critic will become build dependencies. This is undesirable because exactly those two modules have a long dependency chain. Instead you can add them as dependencies for Travis CI only by adding this snippet to .travis.yml:

before_install:
  - cpanm Pod::Coverage::TrustPod Test::Perl::Critic

Unfortunately, these modules somehow did not install correctly on Travis CI's Ubuntu and I had to bump the minimum Perl version to be tested against from 5.10 to 5.14.

The Casus Irreducibilis

In File-Globstar I use the plug-in [PkgVersion] which automatically inserts version information in all .pm files. But at the time of this writing, this conflicts with a test that checks whether all of your Perl files use strict, see https://github.com/rjbs/Dist-Zilla/issues/602.

I first solved that problem by adding the configuration option use_package to [PkgVersion]. The plug-in now generated lines like package File::Globstar 0.2 instead of adding something like $File::Globstar::VERSION = '0.2'. But I did not realize that the variant of the package directive that includes the version was added in Perl 5.12 or so. My local Perl is more recent and Travis CI tests with for example lib/File/Globstar.pm and not with the file File-Globstar-0.2/lib/File/Globstar.pm that is later distributed to CPAN.

I now paid the price for breaking the rule "never patch files during release". File-Globstar-0.2 was released to CPAN and it failed the tests on all older Perls because the released files used the new syntax of the package directive.

The only way to prevent this type of problem is to add Dist::Zilla and all its plug-ins as a dependency for Travis CI and change the test command accordingly. On the other hand, I doubt that it is possible to build Dist::Zilla on a system running Perl 5.10.

But still, you can do that, and a good compromise would probably be to change your travis.yml file only before a release.

Another solution is to push the releases built by Dist::Zilla into a second repository and let Travis CI test that one instead. This is in fact, the better approach because you let the distribution be tested as it will be built on your machine. Making Travis CI install Dist::Zilla instead means that you test distributions that a Dist::Zilla on Travis CI's servers would build.

Unfortunately, you are currently on your own, when you want to implement such a workflow. There is no Dist::Zilla plug-in that automatically updates another git checkout from your build directory.