Avoid Publishing SNAPSHOTs to Nexus

3. July, 2018

Once in a while I’m running into people who deploy SNAPSHOT versions of Maven dependencies to a company-wide Nexus server with a job on a CI server. This is usually a very bad idea, especially when using branches.

Scenario: Two developers, John and Mary, each working in their own branch. They push their branches, CI builds them and they end up on Nexus.

Problem: Nexus doesn’t know or care about branches. Whichever job finishes last wins.

Often, this is not a problem. Now let’s add another project B. B depends on A.

As long as B depends on a release of A, everything is fine.

Now, John needs to make some changes in A. So he updates the dependency in B to A-x.y.z-SNAPSHOT. Everything is still fine, since Mary still uses the latest release of A.

Then Mary also creates a feature branch in her clone of A. That still doesn’t break anything because Maven caches SNAPSHOTs for a day.

The next morning, John makes a change to B and builds it.

This build might break when Mary’s CI job finished last!

The problem here, which can go unnoticed for years, is that Maven silently downloaded Mary’s version of A onto John’s computer and used that to compile. John will see the source code from his branch of A but the binaries will be something else.

Eventually, one of them will make a small changes which affects the others project. They will see MethodNotFoundException or get strange compile errors while the source code (which isn’t affected by this) will look perfectly fine or unit tests will break in odd ways.

That is the main reason why you shouldn’t deploy SNAPSHOT branches to a shared Maven repository: It creates a small chance for subtle bugs which will take a long time to find since your mental model (“I see the source, this is what I get”) will be wrong.

You can get away with publishing the master branch to Nexus (i.e. only a single branch with SNAPSHOTs will ever be published to Nexus).

Note: If your CI server shares local Maven repositories between projects, your builds can fail on the CI server for the same reason. Configure your CI server for per-project local repositories and wipe them before the build to avoid such issues for sure.


Jazoon 2013 – Real World Git Workflow

25. October, 2013

Jazoon 2013 badgeWhen using git, you really have to define a workflow that you want to use. In his talk “Real World Git Workflow” (slides on slideshare), Stefan Saasen explains some kinds of workflows and when they are appropriate.

The first step in any workflow design is asking questions. Some examples that will sound familiar:

  • “Can we fix a bug for a specific release?”
  • “Can we do a fast hotfix for the current release?”
  • “Can we build the current code?”
  • “Is the code for that feature complete?”
  • “Has everybody reviewed the code for this feature?”

Your questions will be different but make sure you ask them. There is no perfect workflow but there is your workflow. Omit this step at your own peril.

Collaboration Models

First, which collaboration model do you use?

“Anarchy” (anyone can push anywhere, slide 17), “Gatekeeper” (one person reviews all changes, often used by OSS projects, slide 18), “Dictator and Lieutenants” (Linux, slide 19) or “Centralised” (slide 20). See also “Git Workflows” on atlassian.com/git.

In the enterprise, the centralized model is often used. This approach makes it most simple to integrate all the tools (CI servers, code quality tools, deployment, …). (slide 23)

Branching Models

The two most common branching models are “continuous delivery” and “product releases.” (slide 25)

From slide 28:

Significant branches map to a concept in the outside world. It may be a past release, an environment or a role. Those branches are long-running and stable whereas feature branches are short lived and volatile.”
– Stefan Saasen

Slide 27 shows the branches for “continuous delivery”. PR is a “pull request.”

As you can see, development is consolidated in the staging branch and pushed into production (master branch) from there. If you make a hotfix in the master branch, it is cherry-picked back into staging.

Slide 30 shows how to handle “product releases.” You have a single, central repository with a master branch which consolidates the development (no staging). Each feature and bugfix happens in a short-lived “feature branch.” When a release is made (slide 31), then a new long living release branch is created. Bug fixes still happen in short lived branches.

When the bug is fixed, the fix is first merged back into the oldest affected release. From there in the next release until you get to the latest release from which it is then merged back into master (slide 33).

But what about changes you don’t want to merge like the changes made by the Maven release plugin? Maybe you will want to replace it with a better release workflow.

Or we use the correct git merge strategy: ours. This creates a new changeset with the merge information without actually merging anything. For git, it will look as if everything has been done and it won’t bother us with merging those changes ever again (slide 39).

Merge Protocols

While the above sounds reasonable, the question is why? What are the rules and forces which make this better? Stefan introduces “the merge protocol” to answer this. See slide 42 for this.

In a nutshell, you always try to merge more stable branches into less stable ones: Bug fix branch into branches where the bug hasn’t been fixed, yet. Features into branches which don’t have the feature.

That’s why you never merge master back into a release branch: Releases are most stable. You merge them into master. If you have fixed in master that you really need in a release, you cherry-pick them (slide 43).

Pull Requests

A lot of people understand why code reviews would be a good thing, “but …” Sounds familiar? Then pull requests are for you.

Pull requests are an easy, low-overhead tool to have as much “code review” as you feel comfortable with. You can merge with by clicking a button or you can review the changes line-by-line. Your choice.

Topology

Most projects will use a single canonical repository but remote forks are useful, too. Imagine you have fixed an important (for you at least) bug in a OSS project. You send them a pull request but it’s rejected! What do you do?

You fork the project. git allows you to still track the changes made by the original project while isolating your life as much as you want (slide 52)

A fork is nice if you want to do an innovation spike – code that might never be included in the product. Fork instead of polluting the project history with dead experiments (53).

Some department needs big changes to some component? Fork it until the feature stabilizes. You can still merge them if you want, but you don’t have to (54).

Reduce the noise (55). A fork allows you to rewrite history.

Hooks

You can use pre and post hooks to make everyone’s life easier. Use a local pre-commit or pre-push hooks to make sure some important tests have been run. For example, you could run FindBugs or checkstyle.

An interesting post-checkout hook would be to check whether the branch is green (66), i.e. code builds and all tests pass. Stop wasting time to search for bugs that were already there before you started your work. You can get this gem from bitly.com/green-builds (69).

Continuous Integration

The explosion of branches can quickly bog down your build server if you don’t come up with a strategy to handle this (71). Usually, it’s enough to build stable and master but developers will love it when they can manually trigger feature branch builds (72).


Push Button Builds

8. September, 2011

A few days ago, kingargyle posted about “Push Button Builds.” The idea is that building a software should be as simple as triggering a light switch.

A good example are Maven (mvn install) or most automake builds (configure && make && make install). They usually just work and if they fail, they give you some context to work from.

A good negative example are Eclipse PDE builds: Often there is no build script. If there is a build script, it will fail with an obscure error message in an Ant script that was recursively called by about 100 other Ant scripts. Each Eclipse project uses PDE but each build has different prerequisites (which you must know). If the build uses p2 repositories, there is no way of telling whether two subsequent builds will produce the same result. Some projects try really hard to help newcomers to build their stuff with a recipe that needs 100+ easy steps. In four words: It’s a mess.

Or a surefire way to attract only the most dedicated committers because everyone else will be utterly demotivated very quickly (and you don’t want those, do you? 😉

So like Dave Carver, I would like to spread the idea. Make builds more simple. Here is an example and following Dave’s advice, I’m going to make the build script even more simple.

Why a script and not, say, a web site with instructions? Several reasons:

  1. A script will make you more dedicated. It’s easy to be sloppy with a README. With a script, at least the syntax will be right.
  2. When a README fails, you won’t really know what to do next. With a script, you have a very precise starting point for a bug report: “The setup script failed in line 15” vs. “Install with Maven … and which version?”
  3. Scripts can prepare the environment. Since you don’t want to make anyone angry, create a new directory and put everything in there. Download what you need, unpack it, configure it. If you can’t do it with a script, why should a newcomer fare better?
  4. Can’t download something? You can always print an error message with exact instructions: Go to website … download xxx, version yyy … put it here. In a script, you can say ‘echo “Put download into $build_dir”‘ Try that in a README.
  5. You can prove that a script works. Good luck trying that with a README.
  6. You can run scripts on a CI server to make sure they work.
  7. A script always needs an interpreter. Bash scripts run on Mac and Linux. What about Windows users? Well, 100% of the Windows users build OSS software from .msi files. The tiny rest has three options: a) Read the bash script, b) install Cygwin/MinGW, c) install Linux.