C2B2 logo icon

Chef Analytics

In this post, c2b2 Principal Consultant and Chef specialist, Alan Fryer explains how Chef Analytics can be utilised as a fantastic system audit tool - especially for security audits

Taking you step-by-step through the installation and configuration, he then describes how to create custom cookbooks for auditing.

I’m currently working with one of our financial services clients to help them implement Chef to provision their IT infrastructure, and was in the kitchen making myself a cup of tea after a rather lengthy meeting, when I overheard the IT Security Manager talking about getting the resources together for an impending security audit.

This got me thinking - Could Chef provide the solution? Step in Chef Analytics.

The Chef Analytics platform is a new feature that provides real-time visibility into what’s happening on the Chef Server and the nodes that they're managing. It includes a rule processing system that allows notifications to be triggered based on events such as: Cookbook uploads, changes to environments, resources updated by Chef client runs, and Chef client run failures - exactly the sort of thing that would prove useful for carrying out system audits.

Installing Chef Analytics

Chef Analytics needs to be installed on a standalone machine and needs to communicate with a Chef server. 

I already had a Chef server configured on an Ubuntu vm on mylaptop - hostname chefserver01.c2b2.co.uk, so I created a second vm with thehostname chefanalytics.c2b2.co.uk.

I ssh’d onto Chef Analytics vm chefanalytics.c2b2.co.uk as root and downloaded the Ubuntu package installer from https://downloads.chef.io/analytics/ into the /tmp folder.

When the installer finished downloading, I ran the following command to install it:

dpkg -i /tmp/opscode-analytics_1.1.6-1_amd64.deb

Now it’s time to configure the software.

Configure the Chef Server

The first thing to configure is the Chef server to communicate with the Chef Analytics server, so ssh onto the vm chefserver01.c2b2.co.uk and performed the following steps:

Step 1) Add the following content to /etc/opscode/chef-server.rb

oc_id['applications'] = {

        'analytics' => {

        'redirect_uri' => 'https://<analytics_fqdn>/'

        }

      }

In this case analytics_fqdn refers tothe hostname of the Chef Analytics vm chefanalytics.c2b2.co.uk

Step 2) Stop the Chef Server

$ chef-server-ctl stop

Step 3) Enable remote access to RabbitMQ on the Chef server by adding the following configuration to /etc/opscode/chef-server.rb

rabbitmq['vip'] = 'BACKEND_VIP'
rabbitmq['node_ip_address'] = '0.0.0.0'  

(where BACKEND_VIP is the external IP address of the Chef Server)

Step 4) Reconfigure the Chef Server

$ chef-server-ctl reconfigure

This updates the Chef Server and creates the actions-source.json file in the folder /etc/opscode-analytics, together with the certificate webui_priv.pem used to authenticate the connection between Chef Analytics and the Chef Server.

Step 5)  Restart the Chef Server

$ chef-server-ctl restart

Step 6) Reconfigure the Chef management console

$ opscode-manage-ctl reconfigure


Configure the Chef Analytics Server

Step 1) Copy the files in /etc/opscode-analyics from the Chef Server to the Chef Analytics Server

analytics_fqdn "chefanalytics.c2b2.co.uk"
topology "standalone

Step 2) Configure the Chef Analytics server by setting the analytics_fqdn in /etc/opscode-analytics/ opscode-analytics.rb. This will be the hostname of the Chef Analytics server and be used for the Chef Analytics web application.

Step 3) Verify the configuration using the pre-flight check command

$ opscode-analytics-ctl preflight-check

Step 4) Reconfigure the Chef Analytics server

$ opscode-analytics-ctl reconfigure

Step 5) Verify the Chef Analytics server configuration

opscode-analytics-ctl test

Now that’s Chef Analytics installed and configured, let’s get started and create some rules using the Webui.

Creating Rules

The first task is to create a Notification to send the outcome of the rule to, this can be to a Webhook, Hipchat, Email or Splunk.

Click on the Notification tab and click on ‘+’ and selectE-mail from the list. Enter send_email for the name of the Notification and complete the appropriate details for thesending the Email notification over smpt.

Next create a rule by clicking the ‘+’ on the Rules tab,this opens the Rules editor screen with the following template rule definition:

Rename the Rule to audit_ssh_config and cut and paste thefollowing code into the editor replacing the content and save it.

rules 'force key based auth'

   rule onrun_control_group

  when

     name = 'ssh' andstatus != 'success'

  then

      alert:error('Thenode {{ message.run.node_name }} Failed ssh config audit, PermitEmptyPasswordsand PermitRootLogin should equal no '),

      notify("send_email","The node {{ message.run.node_name }} Failed ssh config audit,PermitEmptyPasswords and PermitRootLogin should equal no")

  end

 end

So what’s going on here? Each rule is associated with a specific message type, the following message types are supported:

 

Message Type Description


action

Builds rules for messages about actions that occur on the Chef Server


run_control


Builds rules for a single audit to be evaluated


run_control_group


Builds rules for a group of audit to be evaluated


run_converge


Builds rules for messages that are sent at the end of a chef-client run.


run_resource


 
run_start Builds rules for messages that are sent at the start of a chef-client run.

This example uses a run_control_group and run_control message type which acts on a Chef control resource  which is a specific resource create for the Chef Analytics platform. The control resource is used to define audit rules within a recipe, which are processed by the chef-client when run in audit-mode.

The when statement defines an expression for a specific test in a rule and uses fields contained within the expression that are specific to the message type in the rule.

The then statement specifies what action to perform when the rule in the when statement is matched.

In this example we define a rule that checks for arun_control_group named ‘ssh’.  Thecontrol group is defined in the ‘default’ recipe in the cookbook ‘analytics’,the code below for the recipe is listed below: 

# Cookbook Name:: analytics

# Recipe:: default

#

# Copyright (c) 2015 The Authors, All RightsReserved.

control_group 'ssh' do

  control'sshd config file' do

   let(:config_file) { file('/etc/ssh/sshd_config') }



    it 'PermitRoot Login' do

     expect(config_file.content).to match(/PermitRootLogin no/)

    end



    it 'PermitEmpty Passwords' do

     expect(config_file.content).to match(/PermitEmptyPasswords no/)

    end

  end

end

The control resource contains 2 it statements to define the checks to audit the ssh configuration check that it is configured correctly, in this case that the PermitEmptyPasswords and PermitRootLogin are set to no.

The rule checks for a control group named ‘ssh’ and sends an alert and email via the send_email Notification we created earlier if any of the test fail.

The analytics cookbook needs to be uploaded to the Chef Server and added to any nodes run list for which the audit check needs to be run on.

To test the example in this post,edit the file /etc/ssh/sshd_config and set the properties the PermitEmptyPasswords and PermitRootLogin toyes and run the chef client in audit mode, this executes the audit rules and produces the following output:

[2015-10-12T12:19:40-07:00] INFO: Forking chefinstance to converge...

[2015-10-12T12:19:40-07:00] INFO: *** Chef 12.4.3***

[2015-10-12T12:19:40-07:00] INFO: Chef-client pid:5095

[2015-10-12T12:19:41-07:00] INFO: Run List is[recipe[analytics]]

[2015-10-12T12:19:41-07:00] INFO: Run List expandsto [analytics]

[2015-10-12T12:19:41-07:00] INFO: Starting Chef Runfor jboss1.ubuntu.c2b2.co.uk

[2015-10-12T12:19:41-07:00] INFO: Running starthandlers

[2015-10-12T12:19:41-07:00] INFO: Start handlerscomplete.

[2015-10-12T12:19:41-07:00] INFO: Loading cookbooks[analytics@0.1.0]

[2015-10-12T12:19:41-07:00] INFO: Starting auditphase

[2015-10-12T12:19:42-07:00] INFO:

[2015-10-12T12:19:42-07:00] INFO: ssh

[2015-10-12T12:19:42-07:00] INFO:   sshd config file

[2015-10-12T12:19:42-07:00] INFO:     Permit Root Login (FAILED - 1)

[2015-10-12T12:19:42-07:00] INFO:     Permit Empty Passwords (FAILED - 2)

[2015-10-12T12:19:42-07:00] INFO: Successfullyexecuted all `control_group` blocks and contained examples

[2015-10-12T12:19:42-07:00] INFO:

Failures:

  1) ssh sshdconfig file Permit Root Login

    Failure/Error: expect(config_file.content).to match(/PermitRootLoginno/)

  2) ssh sshdconfig file Permit Empty Passwords

    Failure/Error: expect(config_file.content).tomatch(/PermitEmptyPasswords no/)

[2015-10-12T12:19:42-07:00] INFO:

Finished in 0.34221 seconds (files took 0.69314seconds to load)

2 examples, 2 failures

Failed examples:

rspec/etc/chef/cache/cookbooks/analytics/recipes/default.rb[1:1:1] # ssh sshd configfile Permit Root Login

rspec/etc/chef/cache/cookbooks/analytics/recipes/default.rb[1:1:2] # ssh sshd configfile Permit Empty Passwords

[2015-10-12T12:19:42-07:00] INFO: Chef Run completein 1.281711128 seconds

[2015-10-12T12:19:42-07:00] INFO: Running reporthandlers

[2015-10-12T12:19:42-07:00] INFO: Report handlerscomplete

[2015-10-12T12:19:42-07:00] INFO: Sending resource update report(run-id: b3f48f5c-9b09-4f4b-9e0e-0b6753309330)

The command alert:error(‘Message to be sent') in the rule sends a message to the Chef Analytics console, see the screen shot below 

and the command notify("send_email", "Message to be sent") sends an email using the send_email Notification.

There you have it, a way to create custom cookbooks to automatically perform audit checks on your infrastructure.

...Must catch up with the Security Manager next time I see him in the kitchen!