GopherCon 2019 - Design Command-Line Tools People Love
Presenter: Carolyn Van Slyck
Liveblogger: Jennifer Davis
This post was a contribution to the liveblog for GopherCon 2019.
Overview
It is a joy to build command-line tools that are not only easy to learn, but that other developers are willing to maintain. Often a team’s engineering efforts are spent on the backend, while the cli doesn’t receive the same level of attention. This can result in hard-to-test tools and dumping maintenance of them to whoever most recently joined the team. Learn how to take full advantage of popular Go libraries, structure your Go code to improve reuse and testability, publish binaries, and of course design your commands to be user friendly.
Summary
This talk pulls together Carolyn’s experience with command-line development and runs through an example of creating an awesome CLI with emoticons (the pre-cursor of emojis). The problem she’s trying to address is that often CLIs are just thrown together creating technical debt that haunts us and the humans who use our CLIs.
Design Goals
- Predictable
- Task Oriented
- Friendly to both people and scripts
- High quality
At the high-level we want CLIs that are guessable and intuitive, somewhat predictable in use. They should be task-oriented solving a specific issue that might be composed of multiple steps but not require the individual to know how to do those steps. The commands and outputs should be friendly to the humans who have to use them and talk about them, and the systems that can be automated. Quality needs to be baked in from the start rather than handing off all the problems to junior developers to resolve after the CLIs are in use.
Carolyn’s Experience
Carolyn has had a number of experiences that have informed and influenced this talk including:
- Docker Version Manager (dvm)
- dep
- Kubernetes Service Catalog
- porter
Porter was super informative as it was a brand new environment to apply all of the learning she had obtained from years of working on CLIs.
Command Design
Carolyn provides a number of design parameters when thinking about CLIs:
- Grammar aka Rules
Start with grammar. Speak the command out loud, does it sound like something you could explain to someone? When folks start using the command line, you are explaining to another person and you need to be able to talk through what they need to do. It has a subject, noun, and verbs. If we start from that perspective, we can set up the rules of how the command works. Early on noone is watching the development of the CLI so the preferences can be set. Once people start using, harder to change the grammar.
- Understand Precedent
Except you can’t just start from scratch without thinking about the precedent of how other commands. Within the eco-system folks have patterns. If you use different verbs, then it can be problematic to users (remember empathy to users!). Follow patterns that are set in eco-system, build on top of what they have learned and can adopt CLI with some understanding of what the verbs will do.
These followed the precedent:
- kubectl -> service catalog
- node version manager -> docker version manager
Sometimes it makes sense to intentionally not follow the precedent though:
- dep didn’t follow glide’s precedent
Intentional breaking from precedent if the commands don’t work the same way. This helps alleviate user confusion when the outcomes are not the same. With dep they intentionally changed the verbs in use.
Sometimes there is no precedent to follow, this is a lot of work!
- porter setting precedent
When there is no precedent, you’re setting the precedent and tools will copy the patterns in use.
Design
Before emojis, we had emoticons. We want a command-line that helps us quickly get emoticons pasted into slack.
Sentence Readability
emote add emoticon gopher --value ʕ •ᴥ•ʔ
We want sentences that are easy to say.
Antipattern: positional arguments with order matters
emote add repo funk https://x.com/funk.json
versus
emote add repo https://x.com/funk.json funk
We may have multiple arguments to pass in to our command for example the name of the emoticon and then URL or URL and then name, kind of like symlink on Unix and remembering which thing comes first. This is a cognitive cost to our user! We can make this better.
Instead:
emote add repo funk --url https://x.com/funk.json
If we use flags, we don’t have to worry about order anymore.
Support Automation
With natural sounding commands, we handle human element. Next we need to be able to support system automation. Giving an –output command allows us to specify json to have machine readable content.
Default to human first output
Once you have the option to have multiple outputs, human output should be the primary. Sometimes this also means thinking about the presentation of the values of the content, for example dates. “2019-07-15T14:32:22Z” is not very readable. “10 days ago” is better.
https://github.com/dustin/go-humanize go-humanize has functions to help make human readable output.
Implicit Resources in the Domain
Sometimes the binary name is a verb or a resource all on its own.
$emote list
list is an alias!
Aliases provide compromise between brevity and discoverability
Customize help output
Once you create aliases, helpful to customize the output of help.
Instead of all the options and all the flags (the most common help output), give some guidance of what the user SHOULD be doing with this command.
https://carolynvanslyck.com/talk/go/cli/#/alias-helptext
We can group the set of resources that folks can work with, and then group the aliases or shortcuts for the commands.
Task-oriented commands
We’re not done, because we need to do more than just list the emote list. We need to print out the emoji for example. We want to get something done. We want to be able to print out the emoticon.
Domain versus Grammar
Combine commands to make tasks easier for people to use.
Example of quality task command:
travis encrypt MY_SECRET_ENV=super_secret --add env
- download
- encrypt
- insert
If they didn’t do this, would require piping a bunch of commands together. Some people might figure this out but it’s not easy to discover, and it will be copy pasted around rather than something that folks love to use.
Build a CLI
High quality code is critical. Quality code embraces:
- great user experience
- robust
- maintainable
- testable
- error messages.
spf13/cobra
Out of the box you get :
- CLI Framework / Main Entrypoint
- Command Routing
- Error Handling
- Help Text (Don’t forget to customize!)
- Flag Parsing and Validation
If we want to create a shrug command, cobra will handle a lot of the setup. We know we need a destination to send the output, in this case we are going to send it to the clipboard so we can paste it in to wherever we want.
We aren’t done though.
If it’s in main, folks can’t build on top of it. It’s an anonymous fuction, and embedded in cobra so it’s hard to test. Shrug is hard coded, so if we want to add new emoticons we have to add another cobra command.
Recommended Practice: Create an application package
Make a package outside of main, write a struct , single command that you support write a function. if i had a command called shrug add a function shurg. Take the CLI out of your CLI.
https://carolynvanslyck.com/talk/go/cli/#/extract-app
Now we have more testable code.
https://carolynvanslyck.com/talk/go/cli/#/buildshrug2
Configuration
Start with a TOML file, we can add shrug and new emoticon tableflip. https://carolynvanslyck.com/talk/go/cli/#/config
With open source, folks want different configurations:
https://carolynvanslyck.com/talk/go/cli/#/config
- YAML
- JSON
People are going to want different things based on how their tools. So meet folks where they are at. Instead of demanding they use TOML, provide the capability. They’ll be more likely to use the tool if we support all the configuration mechanisms through viper.
spf13/viper
- Single combined configuration from multiple sources
- Reads from flags, config files, remote key/value stores, environment variables
- Smart defaulting: can tell if it was defaulted or set by the user
- Supports config files of multiple formats: json, yaml, toml, and more
Viper will look for specific configuration files, and use the appropriate encoder and save it in the viper field and load it for use.
Next change, remove the shrug emotion, but we now have a generic that can pull out specifics from the configuration file so we can do shrug and tableflip (or whatever!).
https://carolynvanslyck.com/talk/go/cli/#/app-with-viper
By refactoring the configuration mechanism, we can refactor the cobra code so that we have positional arguments and simplified code that allows for dynamic commands.
https://carolynvanslyck.com/talk/go/cli/#/boilerplate2
Recommended Practice: Keep viper isolated
This code is great, but something will haunt me. Anyone who is working on my application now will need to understand viper. This code is simple but in more complex examples will cause problems with folks not understanding how to work on the code.
Create a configuration package and load all the viper into that. Then configuration is loaded once and read over and over again.
Example: https://carolynvanslyck.com/talk/go/cli/#/sting-of-the-viper
Refactored application: https://carolynvanslyck.com/talk/go/cli/#/app-with-config-pkg
https://carolynvanslyck.com/talk/go/cli/#/bind-viper-flag
There are a lot more features to viper that isn’t covered in this code, so don’t invent your own. One feature that’s great is having flags. For destination, BindPFlags.
Instead of one destination, this allow us to bind the destination to the configuration.
Testing
By isolating viper, can mock out viper and test the code!