chef-shell (executable)

[edit on GitHub]

chef-shell is a recipe debugging tool that allows the use of breakpoints within recipes. chef-shell runs as an Interactive Ruby (IRb) session. chef-shell supports both recipe and attribute file syntax, as well as interactive debugging features.

The chef-shell executable is run as a command-line tool.

Modes

chef-shell is tool that is run using an Interactive Ruby (IRb) session. chef-shell currently supports recipe and attribute file syntax, as well as interactive debugging features. chef-shell has three run modes:

Mode	Description
Standalone	Default. No cookbooks are loaded, and the run-list is empty.
Solo	chef-shell acts as a Chef Solo Client. It attempts to load the chef-solo configuration file at `~/.chef/config.rb` and any JSON attributes passed. If the JSON attributes set a run-list, it will be honored. Cookbooks will be loaded in the same way that chef-solo loads them. chef-solo mode is activated with the `-s` or `--solo` command line option, and JSON attributes are specified in the same way as for chef-solo, with `-j /path/to/chef-solo.json`.
Client	chef-shell acts as a Chef Infra Client. During startup, it reads the Chef Infra Client configuration file from `~/.chef/client.rb` and contacts the Chef Infra Server to get the node's run_list, attributes, and cookbooks. Chef Infra Client mode is activated with the `-z` or `--client` options. You can also specify the configuration file with `-c CONFIG` and the server URL with `-S SERVER_URL`.

Options

This command has the following syntax:

chef-shell OPTION VALUE OPTION VALUE ...

This command has the following options:

-a, --standalone

Run chef-shell in standalone mode.

-c CONFIG, --config CONFIG

The configuration file to use.

-h, --help

Show help for the command.

-j PATH, --json-attributes PATH

The path to a file that contains JSON data. Use this option to define a run_list object. For example, a JSON file similar to:

"run_list": [
  "recipe[base]",
  "recipe[foo]",
  "recipe[bar]",
  "role[webserver]"
],

may be used by running chef-shell -j path/to/file.json.

In certain situations this option may be used to update normal attributes.

Warning

Any other attribute type that is contained in this JSON file will be treated as a normal attribute. Setting attributes at other precedence levels is not possible. For example, attempting to update override attributes using the -j option:

{
  "name": "dev-99",
  "description": "Install some stuff",
  "override_attributes": {
    "apptastic": {
      "enable_apptastic": "false",
      "apptastic_tier_name": "dev-99.bomb.com"
    }
  }
}

will result in a node object similar to:

{
  "name": "maybe-dev-99",
  "normal": {
    "name": "dev-99",
    "description": "Install some stuff",
    "override_attributes": {
      "apptastic": {
        "enable_apptastic": "false",
        "apptastic_tier_name": "dev-99.bomb.com"
      }
    }
  }
}

-l LEVEL, --log-level LEVEL

The level of logging to be stored in a log file.

-o RUN_LIST_ITEM, --override-runlist RUN_LIST_ITEM

Replace the current run-list with the specified items. Only applicable when also using solo or server modes.

-s, --solo

Run chef-shell in chef-solo mode.

-S CHEF_SERVER_URL, --server CHEF_SERVER_URL

The URL of the Chef Infra Server.

-v, --version

The Chef Infra Client version.

-z, --client

Run chef-shell in Chef Infra Client mode.

Configure

chef-shell determines which configuration file to load based on the following:

If a configuration file is specified using the -c option, chef-shell will use the specified configuration file
If a NAMED_CONF is given, chef-shell will load ~/.chef/NAMED_CONF/chef_shell.rb
If no NAMED_CONF is given chef-shell will load ~/.chef/chef_shell.rb if it exists
If no chef_shell.rb can be found, chef-shell falls back to load:
- /etc/chef/client.rb if -z option is given.
- /etc/chef/solo.rb if –solo-legacy-mode option is given.
- .chef/config.rb if -s option is given.
- .chef/knife.rb if -s option is given.

chef-shell.rb

The chef-shell.rb file can be used to configure chef-shell in the same way as the client.rb file is used to configure Chef Infra Client. For example, to configure chef-shell to authenticate to the Chef Infra Server, copy the node_name, client_key, and chef_server_url settings from the config.rb file:

node_name                'your-knife-clientname'
client_key               File.expand_path('~/.chef/my-client.pem')
chef_server_url          'https://api.opscode.com/organizations/myorg'

and then add them to the chef-shell.rb file. Other configuration possibilities include disabling Ohai plugins (which will speed up the chef-shell boot process) or including arbitrary Ruby code in the chef-shell.rb file.

Run as a Chef Infra Client

By default, chef-shell loads in standalone mode and does not connect to the Chef Infra Server. The chef-shell can be run as a Chef Infra Client to verify functionality that is only available when Chef Infra Client connects to the Chef Infra Server, such as search functionality or accessing data stored in data bags.

chef-shell can use the same credentials as knife when connecting to a Chef Infra Server. Make sure that the settings in chef-shell.rb are the same as those in config.rb, and then use the -z option as part of the command. For example:

chef-shell -z

Debugging Cookbooks

chef-shell allows the current position in a run-list to be manipulated during a Chef Infra Client run. Add breakpoints to a recipe to take advantage of this functionality.

Step Through Run-list

To explore how using the breakpoint to manually step through a Chef Infra Client run, create a simple recipe in chef-shell:

chef > recipe_mode
  chef:recipe > echo off
  chef:recipe > file "/tmp/before-breakpoint"
  chef:recipe > breakpoint "foo"
  chef:recipe > file "/tmp/after-breakpoint"

and then run Chef Infra Client:

chef:recipe > run_chef
  [Fri, 15 Jan 2020 14:17:49 -0800] DEBUG: Processing file[/tmp/before-breakpoint]
  [Fri, 15 Jan 2020 14:17:49 -0800] DEBUG: file[/tmp/before-breakpoint] using Chef::Provider::File
  [Fri, 15 Jan 2020 14:17:49 -0800] INFO: Creating file[/tmp/before-breakpoint] at /tmp/before-breakpoint
  [Fri, 15 Jan 2020 14:17:49 -0800] DEBUG: Processing [./bin/../lib/chef/mixin/recipe_definition_dsl_core.rb:56:in 'new']
  [Fri, 15 Jan 2020 14:17:49 -0800] DEBUG: [./bin/../lib/chef/mixin/recipe_definition_dsl_core.rb:56:in 'new'] using Chef::Provider::Breakpoint

Chef Infra Client ran the first resource before the breakpoint (file[/tmp/before-breakpoint]), but then stopped after execution. Chef Infra Client attempted to name the breakpoint after its position in the source file, but Chef Infra Client was confused because the resource was entered interactively. From here, chef-shell can resume the interrupted Chef Infra Client run:

chef:recipe > chef_run.resume
  [Fri, 15 Jan 2020 14:27:08 -0800] INFO: Creating file[/tmp/after-breakpoint] at /tmp/after-breakpoint

A quick view of the /tmp directory shows that the following files were created:

after-breakpoint
before-breakpoint

You can rewind and step through a Chef Infra Client run:

chef:recipe > Chef::Log.level = :debug # debug logging will not turn on automatically in this case
    => :debug
  chef:recipe > chef_run.rewind
    => 0
  chef:recipe > chef_run.step
  [Fri, 15 Jan 2020 14:40:52 -0800] DEBUG: Processing file[/tmp/before-breakpoint]
  [Fri, 15 Jan 2020 14:40:52 -0800] DEBUG: file[/tmp/before-breakpoint] using Chef::Provider::File
    => 1
  chef:recipe > chef_run.step
  [Fri, 15 Jan 2020 14:40:54 -0800] DEBUG: Processing [./bin/../lib/chef/mixin/recipe_definition_dsl_core.rb:56:in 'new']
  [Fri, 15 Jan 2020 14:40:54 -0800] DEBUG: [./bin/../lib/chef/mixin/recipe_definition_dsl_core.rb:56:in 'new'] using Chef::Provider::Breakpoint
    => 2
  chef:recipe > chef_run.step
  [Fri, 15 Jan 2020 14:40:56 -0800] DEBUG: Processing file[/tmp/after-breakpoint]
  [Fri, 15 Jan 2020 14:40:56 -0800] DEBUG: file[/tmp/after-breakpoint] using Chef::Provider::File
    => 3

From the output, the rewound run-list is shown, but when the resources are executed again, they will repeat their checks for the existence of files. If they exist, Chef Infra Client will skip creating them. If the files are deleted, then:

chef:recipe > ls("/tmp").grep(/breakpoint/).each {|f| rm "/tmp/#{f}" }
    => ["after-breakpoint", "before-breakpoint"]

Rewind, and then resume your Chef Infra Client run to get the expected results:

chef:recipe > chef_run.rewind
  chef:recipe > chef_run.resume
  [Fri, 15 Jan 2020 14:48:56 -0800] DEBUG: Processing file[/tmp/before-breakpoint]
  [Fri, 15 Jan 2020 14:48:56 -0800] DEBUG: file[/tmp/before-breakpoint] using Chef::Provider::File
  [Fri, 15 Jan 2020 14:48:56 -0800] INFO: Creating file[/tmp/before-breakpoint] at /tmp/before-breakpoint
  [Fri, 15 Jan 2020 14:48:56 -0800] DEBUG: Processing [./bin/../lib/chef/mixin/recipe_definition_dsl_core.rb:56:in 'new']
  [Fri, 15 Jan 2020 14:48:56 -0800] DEBUG: [./bin/../lib/chef/mixin/recipe_definition_dsl_core.rb:56:in 'new'] using Chef::Provider::Breakpoint
  chef:recipe > chef_run.resume
  [Fri, 15 Jan 2020 14:49:20 -0800] DEBUG: Processing file[/tmp/after-breakpoint]
  [Fri, 15 Jan 2020 14:49:20 -0800] DEBUG: file[/tmp/after-breakpoint] using Chef::Provider::File
  [Fri, 15 Jan 2020 14:49:20 -0800] INFO: Creating file[/tmp/after-breakpoint] at /tmp/after-breakpoint

Debug Existing Recipe

chef-shell can be used to debug existing recipes. The recipe first needs to be added to a run-list for the node, so that it is cached when starting chef-shell and then used for debugging. chef-shell will report which recipes are being cached when it is started:

loading configuration: none (standalone session)
Session type: standalone
Loading.............done.

Welcome to the chef-shell 15.8.23
For usage see https://docs.chef.io/chef_shell.html

run `help' for help, `exit' or ^D to quit.

chef (15.8.23)>

To just load one recipe from the run-list, go into the recipe and use the include_recipe command. For example:

chef > recipe_mode
  chef:recipe > include_recipe "getting-started"
    => [#< Chef::Recipe:0x10256f9e8 @cookbook_name="getting-started",
  ... output truncated ...

To load all of the recipes from a run-list, use code similar to the following:

node.run_list.expand(node.chef_environment).recipes.each do |r|
  include_recipe r
end

After the recipes that are to be debugged have been loaded, use the run_chef command to run them.

Advanced Debugging

In chef-shell, it is possible to get verbose debugging using the tracing feature in Interactive Ruby (IRb). chef-shell provides a shortcut for turning tracing on and off. For example:

chef > tracing on
tracing is on
=> nil
chef >

and:

chef > tracing off
#0:(irb):2:Object:-: tracing off
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:109:Shell::Extensions::ObjectCoreExtensions:>:       def off
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:110:Shell::Extensions::ObjectCoreExtensions:-:         :off
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:111:Shell::Extensions::ObjectCoreExtensions:<:       end
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:272:main:>:       def tracing(on_or_off)
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:273:main:-:         conf.use_tracer = on_or_off.on_off_to_bool
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:162:Shell::Extensions::Symbol:>:       def on_off_to_bool
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:163:Shell::Extensions::Symbol:-:         to_s.on_off_to_bool
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:149:Shell::Extensions::String:>:       def on_off_to_bool
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:150:Shell::Extensions::String:-:         case self
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:154:Shell::Extensions::String:-:           false
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:158:Shell::Extensions::String:<:       end
#0:/opt/chef-workstation/embedded/lib/ruby/gems/2.6.0/gems/chef-15.8.23/lib/chef/shell/ext.rb:164:Shell::Extensions::Symbol:<:       end
tracing is off
=> nil
chef >

Manipulating Chef Infra Server Data

When chef-shell is configured to access a Chef Infra Server, chef-shell can list, show, search for and edit cookbooks, clients, nodes, roles, environments, policyfiles, and data bags.

The syntax for managing objects on the Chef Infra Server is as follows:

chef-shell -z named_configuration

Where:

named_configuration is an existing configuration file in ~/.chef/named_configuration/chef_shell.rb, such as production, staging, or test.

Once in chef-shell, commands can be run against objects as follows:

chef (preprod) > items.command

Where:

items is the type of item to search for: cookbooks, clients, nodes, roles, environments or a data bag.
command is the command: list, show, find, or edit.

For example, to list all of the nodes in a configuration named “preprod”, enter:

chef (preprod) > nodes.list

Which will return something similar to:

=> [node[i-f09a939b], node[i-049a936f], node[i-eaaaa581], node[i-9154b1fb],
    node[i-6a213101], node[i-c2687aa9], node[i-7abeaa11], node[i-4eb8ac25],
    node[i-9a2030f1], node[i-a06875cb], node[i-145f457f], node[i-e032398b],
    node[i-dc8c98b7], node[i-6afdf401], node[i-f49b119c], node[i-5abfab31],
    node[i-78b8ac13], node[i-d99678b3], node[i-02322269], node[i-feb4a695],
    node[i-9e2232f5], node[i-6e213105], node[i-cdde3ba7], node[i-e8bfb083],
    node[i-743c2c1f], node[i-2eaca345], node[i-aa7f74c1], node[i-72fdf419],
    node[i-140e1e7f], node[i-f9d43193], node[i-bd2dc8d7], node[i-8e7f70e5],
    node[i-78f2e213], node[i-962232fd], node[i-4c322227], node[i-922232f9],
    node[i-c02728ab], node[i-f06c7b9b]]

The list command can take a code block, which will applied (but not saved), to each object that is returned from the server. For example:

chef (preprod) > nodes.list {|n| puts "#{n.name}: #{n.run_list}" }