Patching Guide for Ruby on Rails Applications

If you run a Ruby on Rails application, be it for fun, for a client or for your own organization, it is critical that you have a regular patch schedule to keep the application current and secure. I'm fortunate to have spent several years of my career supporting Fortune 100 clients' production systems while at IBM (now spun off to Kyndryl) and I understand deeply that there are good and not so good ways to go about this.

First and foremost, make sure you have an explicit patch policy led by your engineering function. If it doesn't exist, it needs to, and quickly. This is relevant for every software project on the planet. In 2024 there is absolutely no excuse to run unpatched software for extended periods of time.

Patch Drivers

The first area to think through when developing a patching policy is with respect to the prioritization of the drivers for your application. That is, upside and downside evaluations for both security and development efficacy and which are more or less important for the domain your application operates within:

• Upside Security - If your application is inherently security focused (i.e. a password manager, a bank, defence app ..), then an aggressive patching policy is a positive value proposition for your work. Give your marketing team a freebie with an industry leading patching practice.
• Downside Security - If you're just a run-of-the-mill application, then you at least don't want to end up on the frontpage for a 2 year old exploit. It isn't a good look. In this case, you're managing the risk of ending up with your name in the paper in a bad light.
• Upside Development Efficacy - Running current software, with a culture of pulling in the latest and greatest is a fantastic way to attract top talent to your organization. Developers love shiny new toys, and if your application is running two versions behind you're likely going to be attracting developers running two versions behind.
• Downside Development Efficacy - Running old software, particularly in an open source space, subjects you to community risk. That is, you're at risk to a group of individuals that you have no direct relationship with. When your team goes to ask for help, the first thing they are going to be told to do is to upgrade to the latest version and report if it is still an issue. Getting behind in software leaves you vulnerable to find yourself on a risk island without a supportive community.

You can pick and choose from any of the above, but it is important to understand and communicate to your whole team the reason why you approach patching the way you do. By the way, patching doesn't have to be a terrible thing. But I've definitely seen many organizations choose to make it a terrible thing!

Patch Process

There really isn't a whole lot of variability in the patch process. I'll get into some of the complicating factors, but the general flow:

1. Need for patch identified.
2. Branch created.
3. Patch(es) applied.
4. Test suite run.
5. Code PR'd, merged.
6. Deployed through environments, eventually to production.

The real difference comes down to how often this process is executed, and who performs each of the steps. The initial trigger indicating a patch is needed might come from a security focal watching CVE's roll in, or it might come from a developer stuck on a feature because of a bug in the current version of software. Or my favourite, from a tech savvy Product Owner who knows a new version of a package offers a feature that is going to bring new value to end users.

There are other more automated options as well, like GitHub's Dependabot. This software will automate the identification, patch and PR aspects of the process. It is then just up to your team to manage the responsible deployment through environments.

The cadence of inflow dictates how well oiled your process will be. If you fall into the camp of running it every 90 days, security patch only with exceptions for critical CVEs ... well, you're in that camp.

Harled's Approach

The approach we love to take is to upgrade relentlessly. That is, apply everything as frequently as possible. This is driven primarily by the philosophy of creating many small problems to resolve, compared to a quarterly bundle update that leaves the system in complete chaos with everybody scratching their heads over which of the 50 packages is to blame.

We find this frequent approach keeps our applications malleable and greatly increases our ability to quickly resolve issues. This is contrary to many projects we've witnessed that have neglected patches for months or even years. The chore of upgrading them is an exponentially more complex task than if it had been done incrementally.

I'd like to think that our patch process can easily and successfully be executed by a junior developer in a way that doesn't put them in an unfair position. It should really be that straightforward with such little risk and complexity.

Patching Runbook

Here is the runbook (more or less) that we use to apply patches to ruby gems and node packages:

1. Capture current state:
   bundle outdated and yarn outdated
2. Capture known vulnerabilities:
   bundle audit and yarn audit
3. Update all patch and minor versions:
   bundle update and yarn upgrade
4. Run test suite:
   rails test or rspec

A couple of important comments on the above:

• Your application dependencies and their popularity will greatly influence the relevance of the findings from the audit commands. That is, if you're using a gem with very low usage and an inactive GitHub repository it is highly unlikely anyone will even be finding exploits, let alone reporting them.
• Your Gemfile and package.json version specifications will impact what the update commands do. If you've pinned things to specific versions, the update isn't going to do much. We try to avoid pinning unless we have some very good reason to do so.
• Major version upgrades will be excluded from this process. We typically treat major version upgrades as separate projects, however, if it is a smaller package we might just send it.
• Frequent patching does introduce the risk of introducing new, unknown vulnerabilities. We like this compared to the alternative, but there is some real associated risk.
• You will be trailblazing relative to much of your peer community and may find you are some of the first to find issues. We don't mind this as we find it a meaningful way to contribute to the open source community.

Looking Forward

I hope this post was helpful and that it has empowered you to have a conversation with our organization or execute if working solo. Patching doesn't have to be complex and the result of a pristine system not only feels great but means you get to spend your days working on the best possible foundation.

Want to talk more about patching? I'd be happy to chat with you to hear what you are doing! Let's talk.