clk

A very opinionated framework to ease the creation of command line interfaces

View on GitHub

I use AWS a lot at work. The AWS CLI is powerful but verbose. Every command needs --profile to specify which account I’m targeting, and often --region too. I find myself typing things like:

aws --profile company-prod --region eu-west-1 s3 ls s3://some-bucket
aws --profile company-prod --region eu-west-1 ec2 describe-instances
aws --profile company-staging --region eu-west-1 s3 cp file.txt s3://staging-bucket/

This gets old fast. I wanted something where I could set my usual profile once and forget about it, but still be able to switch when needed.

wrapping AWS with clk

I created a clk wrapper around AWS. The idea is simple: a group command that handles the --profile and --region options, then passes them to subcommands via environment variables.

Here’s a simplified version (in real life, the subcommands would call the actual aws CLI).

First, create the group:

clk command create python aws --group --description "AWS CLI wrapper"

Then edit it to add the options. The code looks like this:

from clk.config import config
from clk.decorators import group, option


@group()
@option("--profile", "-p", default="default", help="The AWS profile to use")
@option("--region", "-r", default="us-east-1", help="The AWS region")
def aws(profile, region):
    "AWS CLI wrapper with persistent configuration"
    config.override_env["AWS_PROFILE"] = profile
    config.override_env["AWS_REGION"] = region
    config.init()


@aws.group()
def s3():
    "S3 operations"

The subcommands are bash scripts that use the environment variables set by the parent group. We can declare arguments directly on the command line:

clk command create bash aws.s3.ls --description "List S3 buckets or objects" \
    --argument 'path:str:S3 path to list:{"required": false}' \
    --body 'echo "[${AWS_PROFILE}/${AWS_REGION}] aws s3 ls ${CLK___PATH:-}"'

clk command create bash aws.s3.cp --description "Copy files to/from S3" \
    --argument 'source:str:Source path' \
    --argument 'destination:str:Destination path' \
    --body 'echo "[${AWS_PROFILE}/${AWS_REGION}] aws s3 cp ${CLK___SOURCE} ${CLK___DESTINATION}"'

clk command create bash aws.ec2 --description "EC2 operations" \
    --argument 'args:str:EC2 command arguments:{"nargs": -1}' \
    --body 'echo "[${AWS_PROFILE}/${AWS_REGION}] aws ec2 ${CLK___ARGS}"'

Now I can use it like this:

clk aws --profile company-prod --region eu-west-1 s3 ls s3://prod-bucket

[company-prod/eu-west-1] aws s3 ls s3://prod-bucket

That’s better, but I still have to type the profile every time.

setting default parameters

Here’s where clk shines. I can persist options so they become the default.

clk parameter set aws --profile company-prod --region eu-west-1

New global parameters for aws: --profile company-prod --region eu-west-1

Now I don’t need to specify them anymore:

clk aws s3 ls s3://prod-bucket
clk aws s3 cp backup.sql s3://prod-bucket/backups/
clk aws ec2 describe-instances

[company-prod/eu-west-1] aws s3 ls s3://prod-bucket
[company-prod/eu-west-1] aws s3 cp backup.sql s3://prod-bucket/backups/
[company-prod/eu-west-1] aws ec2 describe-instances

Much cleaner! And when I need to work with staging, I just override:

clk aws --profile company-staging s3 ls s3://staging-bucket

[company-staging/eu-west-1] aws s3 ls s3://staging-bucket

The region is still eu-west-1 from my persisted parameters, only the profile changed.

managing parameters

I can check what’s currently set:

clk parameter show aws

aws --profile company-prod --region eu-west-1

And remove options I no longer want persisted:

clk parameter remove aws --region eu-west-1
clk parameter show aws

Erasing aws parameters --region eu-west-1 from global settings
aws --profile company-prod

Or clear everything:

clk parameter unset aws
clk aws s3 ls

Erasing global parameters of aws (was: --profile company-prod)
[default/us-east-1] aws s3 ls

per-project configuration

Different projects often use different AWS accounts. I can set parameters at the project level so they only apply when I’m in that directory.

mkdir -p webapp-project
cd webapp-project
mkdir .clk

Note that simply creating the .clk dir make webapp-project a project in clk point of view.

clk --project . parameter set aws --profile webapp-prod --region ap-southeast-1
clk parameter show aws

New local parameters for aws: --profile webapp-prod --region ap-southeast-1
aws --profile webapp-prod --region ap-southeast-1

clk aws s3 ls s3://webapp-assets

[webapp-prod/ap-southeast-1] aws s3 ls s3://webapp-assets

When I leave the project, my global defaults (or lack thereof) take over again:

cd ..
clk aws s3 ls

[default/us-east-1] aws s3 ls

This way, I never accidentally run a command against the wrong account just because I forgot to switch profiles.

environment variable parameters

Sometimes you don’t want to persist parameters in a configuration file. For example, in a CI/CD pipeline or when quickly testing with different settings. You can set parameters through environment variables using the CLK_P_ prefix.

The naming convention is simple: CLK_P_ followed by the command name in uppercase. Dots in command names become underscores, hyphens become double underscores.

Let’s go back to the root directory and try it:

export CLK_P_AWS="--profile env-prod --region us-west-2"
clk aws s3 ls s3://env-bucket

[env-prod/us-west-2] aws s3 ls s3://env-bucket

These parameters are also visible in the command help:

clk aws --help 2>&1 | grep "current parameters"

The current parameters set for this command are: --profile env-prod --region us-west-2

And they show up in parameter show as well:

clk parameter show aws

aws --profile env-prod --region us-west-2

When you unset the environment variable, the parameters are gone:

unset CLK_P_AWS
clk aws s3 ls

[default/us-east-1] aws s3 ls

preserving environment variables when no option is given

In some setups, AWS_PROFILE and AWS_REGION are already set in the environment (e.g. by a CI pipeline or a .envrc file). I want my wrapper to use those values when I don’t pass the corresponding options, instead of overwriting them with defaults.

To do this, I simply remove the defaults from both options:

from clk.config import config
from clk.decorators import group, option


class AwsProfile:
    pass


@group()
@option("--profile", "-p", expose_class=AwsProfile, help="The AWS profile to use")
@option("--region", "-r", help="The AWS region")
def aws(region):
    "AWS CLI wrapper with persistent configuration"
    config.override_env["AWS_PROFILE"] = config.awsprofile.profile
    config.override_env["AWS_REGION"] = region
    config.init()


@aws.group()
def s3():
    "S3 operations"

Notice that the code still unconditionally assigns both config.override_env["AWS_PROFILE"] and config.override_env["AWS_REGION"]. Yet when those options are not provided, clk knows the values were not given and won’t set the corresponding environment variables. This means the code stays simple — no need for guards — and the existing environment is preserved:

export AWS_PROFILE=from-ci
export AWS_REGION=eu-west-1
clk aws s3 ls

[from-ci/eu-west-1] aws s3 ls

When the options are explicitly passed, they override the environment as expected:

clk aws --profile company-prod --region ap-southeast-1 s3 ls

[company-prod/ap-southeast-1] aws s3 ls