Rabid Logo

Rabid is now Ackama

In September 2018, on the company’s 8th birthday, Rabid became Ackama - this was to reflect that fact that we work on large digital ecosystems, and given that nature is the largest ecosystem in the world we wanted a name that came from nature. Ackama, or Makamaka, is a small bushy tree with white flowers and red seeds. It’s found on the coast from Whangarei north.

Go to the Ackama blog

Managing multiple Chef repositories

Knifing the Chef

A sidenote: I’ve used the terms ‘Chef’ and ‘Knife’ here fairly interchangably here, because on a workstation they kind of are. Chef is the actual framework where you store your configurations, data, etc., and Knife is the tool you use to actually interact with Chef. If you’re a Berkshelf user, you can add berks to the list of interchangable terms - Berkshelf will go ahead and use the same configuration as Knife will when you run it.

If you’re a typical user of Chef, you’re probably going one of two ways: you use chef-solo to manage all of your server setup needs, or you connect to a single Chef server and use the knife utility to interact with nodes, users, data bags and all the goodness that you can access going this route. This blog post deals with the third scenario - you connect to multiple Chef servers from your workstation, and need these to play nicely with one another. I use a multiple Chef servers regularly, both as part of my professional work at RabidTech, and with my own side projects.

First of all, let’s go over some general concepts about how Chef expects to work. The core of Chef working on a workstation (as opposed to a server), is the Chef repository. This doesn’t have to be a actual SCM repository (i.e. managed with Git), but it’s expected that it is. Within this directory are stored your cookbooks, data bags, environments - in other words, all the bits and pieces you need to have stored locally to bootstrap and manage your servers.

Within this Chef repository is a secret little folder named .chef. This folder stores a couple of key files, and most importantly, the knife.rb file. The knife.rb file is the most important file you store on your workstation for Chef (well, knife) to use, because it contains configuration that points Chef to your Chef server, as well as telling it where to find cookbooks, data bags, and all the other bits and pieces. Without this knife.rb file, there’s really nothing Chef can do - it has no destination.

If you’re using Opscode’s hosted Chef server, you will have been prompted to download your Chef repository when you created a user for yourself - the reason that these repositories magically “just work” after you download them is because the hosted service has kindly pre-written a knife.rb file within your .chef directly already configured to connect to the hosted Chef server as you.

So, all this works with a single Chef repository great! The trick is to understand what the impacts of this are when you don’t just have one repository.

The key to understanding this is to appreciate how Chef looks up these knife.rb files. Basically, it’ll start in the folder you’re currently in, and will start traversing up through the folders until it finds one. This is why you can run something like knife data bag show users deploy anywhere in your Chef repository and have it work, or, if you loath the idea of a Chef repository, you can have your cookbooks, data bags etc scattered all over the place and referenced in a single knife.rb file stored in the .chef folder in your home folder - wherever you run knife from, it’ll just keep going up until it finds something.

What this means then, is that you can absolutely have multiple Chef repositories, but you do need to be careful about where they’re kept. Ideally, they should all live in a single folder - I call mine “Ops”, and be named something that identifies the Chef server to which they relate (for example, I have “chef-solo”, “chef.local”, “chef-rabid”, and “chef-rocket-pack”). Within each of these folders is the perfectly standard Chef repositories - the only key difference is that they each have a specific knife.rb file that contains details of the location of the Chef server and which key it should use to connect (since the key you use to connect to the Chef server will be different for each repository). This way, whichever Chef repository you’re in at the time, knife will just use the correct Chef server, with no fuss.

There are a couple of gotchas though:

The first is that you should make sure you don’t have a .chef/knife.rb file in your home directory. By ensuring that this file does not exist, you are adding a safety check - you cannot connect to a Chef server unless you’re located in the appropriate Chef repository (after all, the last thing you want to do is go ahead and delete all the users from your Chef server just because you’re in the wrong folder!).

The second is that you need to watch out for .chef folders inside cookbook repositories - FYI these should never be pushed to your source control system, but sometimes they are. Because of the way I’ve described Chef loads your configuration from knife.rb, it will go ahead and use the config in the cookbook before it even gets to your Chef repo - and that could do something you don’t expect. Basically, when you’re writing cookbooks, even if they’re for internal use only, you should always write them as if they’re going to be released as open-source: they should be well-documented, and should not have any configuration specific to your set-up within them. Treat them as standalone resources, and pull them into your Chef repository when you need them - your configuration lives in your Chef repo (synced with your Chef server of course!), and the cookbooks just provide the instructions with where to put your configuration.

So, while having more than one Chef repository isn’t an incredibly common scenario, it does happen. It’s not terribly difficult to set up and work with this arrangement, but it’s important to be aware of the tricks to make sure you’re always interacting with the Chef you think you are!

Josh McArthur