Set Up Chef Infra Server, Workstation, Bootstrap Node and Create Cookbook

Introduction

Chef is an infrastructure as code (IAC) automation tool used to configure downstream clients known as nodes. This is one of the many configuration management tools used in DevOps among Ansible and Puppet. There are currently more jobs requiring Ansible than Chef, and Ansible is easier to set up with lesser steps as well as commands to execute automation on the infra management. However, it is still advantageous to understand how Chef works and add it into our DevOps tool arsenal.

Prerequisites

A Chef infra server sits in between the Workstation and Node(s), therefore, a minimum of three virtual machines is required for this tutorial, and they are based on different popular Linux distros.

Chef Infra Server (AlmaLinux 8) – Store cookbooks and communicate with nodes.
Workstation (Debian 10) – Create recipes in cookbooks and upload them to Infra Server.
Node (CentOS Stream 8) – Pull recipes from a cookbook from Infra Server to run automation.

Step 1 – Chef Infra Server Setup

1a) Select OS and download the package from https://www.chef.io/downloads/tools/infra-server

# For Red Hat Linux, CentOS, AlmaLinux and Rocky
$ sudo rpm -Uvh /tmp/chef-server-core-<version>.rpm

# For Ubuntu and Debian
$ sudo dpkg -i /tmp/chef-server-core-<version>.deb

1b) Start all the services: sudo chef-server-ctl reconfigure

1c) Create a chef infra administrator.

USER_NAME – john
FIRST_NAME – John
LAST_NAME – Smith
EMAIL – john.smith@techsch.com
PASSWORD – (self-explanatory)
ORG_NAME – techsch (must begin with a lower-case letter or digit, may only contain lower-case letters, digits, hyphens, and underscores)
ORG_FULL_NAME – Tech Sch

# Create a hidden directory to store chef keys (*.pem)
$ mkdir ~/.chef

# Create Chef administrator
$ sudo chef-server-ctl user-create USER_NAME FIRST_NAME LAST_NAME EMAIL 'PASSWORD' --filename ~/.chef/USER_NAME.pem

# Create organization
$ sudo chef-server-ctl org-create ORG_NAME "ORG_FULL_NAME" --association_user USER_NAME --filename ~/.chef/ORG_NAME.pem

# View all users on Chef Server
$ sudo chef-server-ctl user-list

# View all organizations on Chef Server
$ sudo chef-server-ctl org-list

1d) Create chef-server.rb if not using standard ports: sudo vi /etc/opscode/chef-server.rb
Chef’s in-built nginx service runs 80, 443 and 9683 as mentioned on https://docs.chef.io/server/server_firewalls_and_ports/, but this might crash with the nginx service on the web server.

$ sudo cat /etc/opscode/chef-server.rb
nginx['non_ssl_port'] = 8080
nginx['ssl_port'] = 8443

$ sudo chef-server-ctl reconfigure
$ sudo chef-server-ctl restart

Step 2 – Workstation Setup

2a) Select OS and download the package from https://www.chef.io/downloads/tools/workstation

# For Red Hat Linux, CentOS 7 and below
$ yum localinstall chef-workstation-<version>.rpm
# Rocky and AlmaLinux 8.x
$ yum install chef-workstation-<version>.rpm

# For Ubuntu and Debian
$ dpkg -i chef-workstation_<version>.deb

# Verify Workstation installation
$ chef -v

2b) Create a local Chef repository to store cookbooks, which in turn store recipes. The directory structure can be found at https://docs.chef.io/chef_repo/#directory-structure. Thereafter, create a hidden directory (.chef) to store the *.pem files created in Step 2d.

$ chef generate repo chef-repo
$ mkdir ~/chef-repo/.chef

2c) Add hostnames of workstation, infra server and node to Workstation’s /etc/hosts because a hostname may be included in Chef commands. Thus, they need to resolve to an IP address.

$ cat /etc/hosts
# Generated by SolusVM
127.0.0.1       localhost localhost.localdomain
::1     localhost localhost.localdomain
96.9.xx.xxx workstation
195.85.xx.xx infraserver
107.155.xx.xx clientnode # required for 3b) --node-name

2d) Generate RSA private keys between the Workstation and Infra server.

# Generate RSA key-apir
$  ssh-keygen -b 4096

# Copy workstation public key to infra (exclude -p 10222 if using standard port 22)
$ ssh-copy-id -p 10222 user@195.85.xx.xx

# Copy the *.pem files from infraserver (Step 1c) to workstation's ~/chef-repo/.chef directory
# (exclude -P10222 if using standard port 22)
$ scp -P10222 user@195.85.xx.xx:~/.chef/*.pem ~/chef-repo/.chef/

# Check both admin and org *.pem files are copied over successfully
$ ls ~/chef-repo/.chef

Step 3 – Bootstrap (ready and validate) a Node

3a) SSH into the node machine and update /etc/hosts to identify Chef infra-server hostname.

$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
107.155.xx.xx clientnode
195.85.xx.xx   infraserver

3b) Bootstrap the node from the workstation and confirm that the bootstrapping is successful. Replace user, password and <node name>, also to ensure user is in sudo group. Invoke validatorless bootstraps with –node-name <clientnode> starting with Chef 12.2.0 client – https://www.chef.io/blog/validatorless-bootstraps.

$ cd ~/chef-repo

# Bootstrap client node (exclude -p 10222 if using standard port 22)
$ knife bootstrap 107.155.xx.xx -p 10222 -U user -P password --sudo --use-sudo-password --node-name <node name>

# Confirm bootstrap was successful
$ knife node list
clientnode

Step 4 – Create and Deploy a Cookbook for Testing (optional)

4a) Generate a ‘Nginx’ cookbook in subdirectory ‘cookbooks‘ on the workstation. Several template files will be auto-generated in new directory ~/chef-repo/cookbooks/nginx.

$ cd ~/chef-repo/cookbooks
$ chef generate cookbook nginx

4b) Update the configuration management codes in ~/chef-repo/cookbooks/nginx/recipes/default.rb to install the nginx package and start the service on the node.

$ knife node edit <node-hostname>
ERROR: You must set your EDITOR environment variable or configure your editor via knife.rb
$ export "EDITOR=vim"

# https://docs.chef.io/run_lists/#run_list-add
$ knife node run_list add <node-hostname> 'recipe[nginx]'

$ cat recipes/default.rb
#
# Cookbook:: nginx
# Recipe:: default
#
# Copyright:: 2022, The Authors, All Rights Reserved.
package 'nginx' do
  action :install
end

service 'nginx' do
  action [ :enable, :start ]
end

Finally, upload the cookbook to the Chef Infra server: knife cookbook upload nginx

It is optional to update the maintainer information in ~/chef-repo/cookbooks/nginx/metadata.rb, but the ‘Supermarket’ feature will not be used in this tutorial. Supermarket is Chef’s open-source community platform for sharing cookbooks, tools, and plugins.

4d) SSH into client node and download (pull) the cookbook from Chef Infra server: sudo chef-client

$ sudo chef-client
Chef Infra Client, version 17.10.3
Patents: https://www.chef.io/patents
Infra Phase starting
[2022-08-25T16:21:41+08:00] ERROR: shard_seed: Failed to get dmi property serial_number: is dmidecode installed?
Resolving cookbooks for run list: ["nginx"]
Synchronizing cookbooks:
  - nginx (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Chef InSpec profile files:
Loading Chef InSpec input files:
Loading Chef InSpec waiver files:
Converging 2 resources
Recipe: nginx::default
  * dnf_package[nginx] action install (up to date)
  * service[nginx] action enable (up to date)
  * service[nginx] action start
    - start service service[nginx]

Running handlers:
Running handlers complete
Infra Phase complete, 1/3 resources updated in 01 minutes 20 seconds

# Confirm nginx processes running on Linux system
$ ps aux|grep nginx
root       36977  0.0  0.2 103376  2116 ?        Ss   16:22   0:00 nginx: master process /usr/sbin/nginx
nginx      36980  0.1  0.2 122628  2556 ?        S    16:22   0:00 nginx: worker process
user    38404  0.0  0.1   9200  1108 pts/0    R+   16:23   0:00 grep --color=auto nginx

4e) Open the node public IP (e.g., http://107.155.xx.xx/) in a browser and the ‘Welcome to nginx on Red Hat Enterprise Linux!’ webpage is loaded.

Chef had successfully installed Nginx on the node.

Conclusion

Docker, Ansible and Terraform are popular alternative infrastructure as code tools, and they are commonly mentioned in DevOps or even site reliability engineer (SRE) roles. Chef has a steep learning curve and high resources perquisites (e.g., Infra server and Workstation). However, if you are keen on Chef, there is an open-source community repository, https://supermarket.chef.io/, where one can download ready-made cookbooks. Otherwise, do try out infrastructure provisioning tool Terraform, followed by configuration management Ansible, might be a better approach for learning DevOps.