Migrating from Chef to Cinc

Posted on by Carlos Aya 7 mins read

Intro

Many users already in the Chef ecosystem may have reason to migrate from Chef Infra Server to Cinc server. Thankfully, this is a relatively painless process and all cookbooks, users, data bags, roles, environments, etc. can be retained during the migration process. This blog post documents the migration process from Chef Infra Server to Cinc Server in Red Hat Enterprise Linux 8.

Cinc Server Setup

In order to create your new Cinc Server, stand up a new server on your OS of choice by following the steps detailed at Server Installation.

Once the Cinc server has been configured, you must configure the admin user:

cinc-server-ctl user-create #{username} #{first_name} #{last_name} #{email} #{password} --filename #{output_file_name}

for example…

cinc-server-ctl user-create cincadmin cinc admin cincadmin@dev.local thisisasecret --filename /tmp/cincadmin.pem

Workstation Setup

Backup / Restore operations are performed via a 3rd machine rather than on the source or destination servers. This machine will be used as a workstation on which Chef object-based backups can be created. I deployed an RHEL8 server for this purpose with enough disk space to store the backup. For reference purposes, our server with 17 users, 63 environments, 481 roles, and 539 cookbooks (each with many versions) consumed 9.2GB for the backup. This is a relatively small environment - others may require far more storage to create the backup.

After deployment of the workstation, the following tasks were completed:

Install Cinc Workstation, knife-ec-backup, knife-tidy, and pre-reqs

Use the commands below to install the required tools:

curl -L https://omnitruck.cinc.sh/install.sh | sudo bash -s -- -P cinc-workstation -v 22
sudo yum -y install gcc postgresql-devel
chef gem install knife-ec-backup
chef gem install knife-tidy

The following files were then created on the server:

# /chef_backups/conf/knife_src_server.rb (Source Server Knife Config)
log_level       :info
log_location    STDOUT
client_key      '/chef_backups/conf/knife_src.pem'
chef_server_url 'https://chef.domain.local'
node_name       'berks'

# /chef_backups/conf/knife_dst_server.rb (Destination Server Knife Config)
log_level       :info
log_location    STDOUT
client_key      '/chef_backups/conf/knife_dst.pem'
chef_server_url 'https://cinc.domain.local'
node_name       'cincadmin'
ssl_verify_mode :verify_none

Copy the Destination Server client_key

Copy /tmp/cincadmin.pem from the Cinc Server to /chef_backups/conf/knife_dst.pem on the Workstation.

Copy the Source Server WebUI Private Key

Copy /etc/opscode/webui_priv.pem from the source Chef Infra Server to /chef_backups/conf/webui_priv_src.pem on the Workstation.

Destination Server WebUI Private Key

Copy /etc/cinc-project/webui_priv.pem from the destination Chef Infra Server to /chef_backups/conf/webui_priv_dst.pem on the Workstation.

Source Certificate Trust

Run the following command to trust SSL certificates for the source Chef server. 

knife ssl -c knife_src_server.rb fetch

I was not able to do this for the destination server in my environment since it was configured to use the same FQDN as my source server and DNS was still pointing at the source server. If you are in the same position, then SSL checks for the destination server can be disabled in /chef_backups/conf/knife_dst_server.rb by setting ssl_verify_mode to :verify_none.

Backing Up Chef Infra Server

OK, assuming everything above has gone well, it is now time to backup your Chef server! From your Workstation, run the command below to start the backup:

knife ec -c /chef_backups/conf/knife_src_server.rb backup /chef_backups/ --webui-key /chef_backups/conf/webui_priv_src.pem --concurrency 10

Set concurrency to whatever your source server can handle. I have seen some people recommend 5, but we had enough overhead to go with 10 and honestly, I could have probably gone higher. This took several hours the first time. Subsequent backup runs seem to capture the deltas and took 30-45 mins each time.

Knife Tidy

Theoretically, you should be able to use knife tidy at this point to remove unused cookbook versions, clients, etc, however this didn’t work in my testing and frankly, I didn’t feel comfortable doing this at the same time as the server migration. See the knife-tidy git repo for more information on this utility.

Data Restoration

Once the backup is done, run the following command to perform a dry-run of the backup to the destination server.  The dry-run will still create the Chef Organization on the destination server, but will not copy nodes, clients, roles, environments, acls, cookbooks, etc.

knife ec -c /chef_backups/conf/knife_dst_server.rb restore /chef_backups/ --webui-key /chef_backups/conf/webui_priv_dst.pem --dry-run

Once that has been done, run the following command on the destination Cinc server to associate the cincadmin user with the newly created organization:

cinc-server-ctl org-user-add myOrgName cincadmin

Now you can restore the data to the destination server:

knife ec -c /chef_backups/conf/knife_dst_server.rb restore /chef_backups/ --webui-key /chef_backups/conf/webui_priv_dst.pem

Local Testing

Cinc offers a build in method for running automated tests against the server.  The following was done on the Cinc server:

  1. If DNS does not yet resolve the intended FQDN to the IP address of the Cinc server, then update /etc/hosts so that your desired FQDN resolves to the IP address of the Cinc server.
  2. Run cinc-server-ctl test

The following output was observed:

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) opscode-account user association user not in org can be invited to the org by an admin when the inviting admin is removed from the org, invites issued by that admin cannot be accepted
     # Known failure: passes w/ 200 b/c no USAG cleanup performed for deleted user
     # ./spec/api/account/account_association_spec.rb:662

  2) opscode-account user association user not in org can be invited to the org by an admin when the inviting admin is removed from the system, invites issued by that admin can't by accepted
     # Known failure: passes w/ 200 b/c no USAG or other group cleanup performed for deleted user
     # ./spec/api/account/account_association_spec.rb:671


Finished in 54.85 seconds (files took 3.85 seconds to load)
173 examples, 0 failures, 2 pending

In our case, I built our new Cinc server using a cookbook on our Chef server, so I was able to run chef-client on the Cinc server to ensure that the server was able to execute a Chef client run against itself (remember, we previously updated /etc/hosts to make this possible).

Remote Testing

The following remote tests were performed to ensure other existing clients were able to connect and execute their Chef runs against the Cinc server…

  1. If necessary, update /etc/hosts on an existing machine so that the FQDN of the Cinc server resolves to the IP address of the Cinc server.
  2. If the FQDN of your Cinc server is different than that of the existing Chef server, then update client.rb and knife.rb accordingly.
  3. Run chef-client
  4. Run the following to ensure the Cinc server saw the test client run:  knife status --run-list

To ensure our pipelines for Roles, Environments, Cookbooks, etc were still working, I temporarily modified the jobs to update /etc/hosts so that chef.domain.local resolved to the Cinc server IP, and then executed each job to ensure they were successfully able to create and update their respective objects.

I then executed knife commands with the option -c /chef_backups/conf/knife_dst_server.rb to ensure I could see the updates on the destination Cinc server.

For example…

# knife role show myUpdatedRole -c /chef_backups/conf/knife_dst_server.rb
# knife cookbook show myUpdatedCookbook -c /chef_backups/conf/knife_dst_server.rb
# knife environment show myUpdatedEnvironment -c /chef_backups/conf/knife_dst_server.rb

Examining the New Environment

The following commands can be used to examine the new environment and ensure it matches the source environment. If you receive errors regarding the cincadmin user not having access to the org, just re-run the following command on the destination server:

cinc-server-ctl org-user-add orgName cincadmin

List all cookbooks (and versions)

knife cookbook list -c /chef_backups/conf/knife_dst_server.rb

List all nodes

knife node list -c /chef_backups/conf/knife_dst_server.rb

Show details for a single node

knife node show myNodeName -c /chef_backups/conf/knife_dst_server.rb

List all nodes and their last checkin date

knife status -c /chef_backups/conf/knife_dst_server.rb

Get last checkin date for a specific node

knife status "hostname:anyExistingServerName" -c /chef_backups/conf/knife_dst_server.rb

Node Migration

Migrating nodes is a simple operation since the newly built server has retained client keys for each node.  If the new Cinc server has the same FQDN as the Chef server that it is replacing then all that needs to be done is a DNS update so that the FQDN now resolved to the IP address of the Cinc server. If the Cinc server has a new FQDN, then you must update client.rb and knife.rb on each client machine to perform the migration.

References

See the following links for more information on the above process: