Command line interface
Command line interface#
The showyourwork! package implements a single command-line utility:
showyourwork, which allows users to set up, configure, and build their
open source article. Below we describe this command and discuss its various
Usage: showyourwork [OPTIONS] COMMAND [ARGS]... Easily build open-source, reproducible scientific articles. Options: --help Show this message and exit. Commands: build Build an article in the current working directory. cache Caching-related operations. clean Clean the article build in the current working directory. setup Set up a new article repository in the current working directory. tarball Generate a tarball of the build in the current working directory.
Running showyourwork! (without any arguments) is a shortcut for
(see showyourwork build below).
showyourwork setup --help
Usage: showyourwork setup [OPTIONS] <user/repo> Set up a new article repository in the current working directory. This command expects a single positional argument, `SLUG`, of the form `user/repo`, where `user` is the user's GitHub handle and `repo` is the name of the repository (and local directory) to create. Options: -y, --yes Accept all `Press any key to continue` prompts automatically. -q, --quiet Don't prompt the user, and don't display informational output. -c, --cache Set up intermediate result caching on Zenodo Sandbox. Requires a Zenodo Sandbox API token provided as the environment variable and GitHub repository secret `SANDBOX_TOKEN`. -o, --overleaf PROJECT_ID Overleaf project id to sync with (optional). Requires Overleaf credentials, provided as the environment variables and GitHub repository secrets `OVERLEAF_EMAIL` and `OVERLEAF_PASSWORD`. -s, --ssh Use ssh to authenticate with GitHub? Default is to use https. -a, --action-spec SPEC Version spec of showyourwork to use in the GitHub action. Default is the latest version on PyPI. --help Show this message and exit.
setup subcommand sets up an open source article repository from scratch
in the current working directory. This is an interactive command (unless you
--quiet options; see below).
Let’s step through what it does here.
To set up a new open source article repository, run
showyourwork setup $USER/$REPO
where you should replace
$USER with your GitHub user name and
$REPO with the name of your new article repository. For definiteness,
here we’ll use my user name (
rodluger) and we’ll call our repository
setup command as above should bring up the following prompt:
Let's get you set up with a new repository. I'm going to create a folder called article in the current working directory. If you haven't done this yet, please visit https://github.com/new at this time and create an empty repository called rodluger/article
As requested, if you haven’t yet created the remote repository, go to github.com/new in your browser to create an empty repository of the same name. There’s no need to create a README, gitignore file, or LICENSE at this time, as showyourwork! will set those up for you.
Press any key to bring up the next prompt. What you see next depends on whether
or not you provided the
If you didn’t request Zenodo caching functionality (see below), you’ll see the following the message:
You didn't provide a caching service (via the --cache command-line option), so I'm not going to set up remote caching for this repository.
If instead you passed the
--cache flag, you’ll see the following message:
You requested remote caching on Zenodo, so I'm going to create a deposit draft where intermediate results will be cached. Please make sure at this time that you have defined the ZENODO_TOKEN environment variable containing your API key for Zenodo. If you don't have one, visit https://zenodo.org/account/settings/applications/tokens/new to create a new personal access token with deposit:actions and deposit:write scopes and store it in the environment variable ZENODO_TOKEN. In order for this to work on GitHub Actions, you'll also have to visit https://github.com/tmp/tmp/settings/secrets/actions/new at this time to create a ZENODO_TOKEN secret with your API access token.
Note that, in addition to the
--cache flag, which enables caching on Zenodo,
users may also provide the
--sandbox flag, which switches the host to Zenodo
Sandbox. Zenodo Sandbox behaves in exactly the same way as Zenodo, but it is
explicitly meant as a test bed for dataset archiving. While deposits on Sandbox get assigned
DOIs, they are no actual registered DOIs and have a limited lifespan.
Sandbox is therefore a great choice for debugging and development; read more about
it at Zenodo integration. Note that if you choose the
--sandbox option, you’ll need
a Zenodo Sandbox API token stored int the
SANDBOX_TOKEN environment variable
and GitHub Actions secret.
Never commit your Zenodo API token (or any API token) directly to your repository!
You can read more about GitHub secrets (and the security measures in place to prevent them from getting exposed to the outside world) at the GitHub documentation.
Press any key to bring up the next prompt. What you see next depends on whether
or not you specified the
If you didn’t pass the
--overleaf option, you’ll see the following:
You didn't provide an Overleaf project id (via the --overleaf command-line option), so I'm not going to set up Overleaf integration for this repository.
If you would like to set up integration with an Overleaf project (see Overleaf integration),
Ctrl+C and run
showyourwork setup --overleaf=62150dd16134ef045f81d1c8
where you should replace
62150dd16134ef045f81d1c8 with the 24-character id
of a new (blank) Overleaf project. Once you create a new Overleaf project, you
can grab the id from the last bit of the project’s URL. Note that showyourwork!
requires the Overleaf project to be empty, otherwise it will refuse to set up
the integration. For more information on how this integration works, and what
to do if you have an existing Overleaf project you’d like to integrate with
showyourwork!, please see Overleaf integration.
If you specified the
--overleaf option (see Step 3A),
you’ll get the following message:
You provided an Overleaf project id, so I'm going to set up Overleaf integration for this repository. Please make sure at this time that you have defined the OVERLEAF_EMAIL and OVERLEAF_PASSWORD environment variables. In order for this to work on GitHub Actions, please go to https://github.com/tmp/tmp/settings/secrets/actions/new at this time and create OVERLEAF_EMAIL and OVERLEAF_PASSWORD secrets with your Overleaf credentials.
To allow showyourwork! to push to/pull from your Overleaf project, create
the environment variables
populate them with your Overleaf email address and password, respectively;
then re-run the setup command.
Again, take care to never actually commit this information to your repository!
Finally, press any key to generate the repository. This will create a new folder
in the current working directory with the same name as your repo (
the example above) and set up
git tracking for it. Note that the first time
you commit and push your changes to the GitHub repository, you’ll have to specify
the upstream branch as follows:
git push -u origin main
showyourwork build --help
Usage: showyourwork build [OPTIONS] [SNAKEMAKE_ARGS]... Build an article in the current working directory. Options: --help Show this message and exit.
Run this command to build the article in the current working directory. Note that
you must run this command from the top level of the repository (an error will
be thrown otherwise). The command accepts any number of arguments, all of which
are forwarded to
showyourwork passes the following arguments to
-c1 --use-conda --reason --cache
Some of these, like the number of cores, can be overridden. For example, you may run
showyourwork build -c2
to run the workflow using two cores (see the snakemake docs
for details). Additional arguments can also be provided, like
--verbose to increase
the verbosity of the Snakemake logs (see Logging), or
force the re-execution of the rule that builds the manuscript or all of the rules
in the workflow, respectively (regardless of whether the outputs are up to date
or not). Positional arguments are also allowed; for instance, to only build a specific
figure, you may run, e.g.,
showyourwork build --force src/tex/figures/figure.pdf
You can check out the complete list of Snakemake arguments and options at the snakemake documentation.
Not all Snakemake options are compatible with showyourwork!. If you run into issues when specifying custom options, please let us know.
Note that the build process in showyourwork! happens in two steps, each of
which executes a separate Snakemake workflow. The first
step is a preprocessing step that parses the user config file and does a quick
first-pass compiling of the TeX manuscript to look for
\script calls, which it uses to build the graph of dependencies for
your article. The second step is the main step, in which all of the dependencies
are built (if needed) and the final article PDF is generated. Arguments
showyourwork build are ingested only during the second step.
Finally, showyourwork! takes full advantage
of the dependency tracking and caching functionality of Snakemake. When
showyourwork build, only files whose upstream dependencies have
changed (since the last build) will be re-generated. This is true even when
running on GitHub Actions; the
showyourwork-action caches results across
runs to minimize compute time for the build. We even go a step further, and
extend the Snakemake functionality to allow caching of intermediate
dependencies on Zenodo; read about it at Zenodo integration.
showyourwork clean --help
Usage: showyourwork clean [OPTIONS] Clean the article build in the current working directory. Options: -f, --force Forcefully remove everything in the `src/tex/figures` and `src/data` directories. -d, --deep Forcefully remove the `.snakemake` and `~/.showyourwork` directories. --help Show this message and exit.
This command removes all of the output from previous
steps. Depending on the state of your repository, and if there are errors in
your config file or missing dependencies, this command may fail silently, in
which case some of the output may remain after running it.
showyourwork clean didn’t remove all of the output, you can force
the deletion of all the programmatically-generated figures and datasets by
--force option, which will remove
everything in the
src/data, and temporary
(assuming you’re respecting the showyourwork! conventions; see Repository layout).
If you want to start over from scratch, you can also pass the
for a deep clean. This removes
.snakemake folder, which houses the
for your build (among other things), so
deleting it will force a re-install of all packages used in your workflow.
This option also removes the
folder located in your
$HOME path, which also houses
used at different stages of the build step. You can safely remove it at any time
(at the cost of a longer runtime the next time you execute
showyourwork tarball --help
Usage: showyourwork tarball [OPTIONS] Generate a tarball of the build in the current working directory. Options: --help Show this message and exit.
showyourwork tarball command builds your article, but
also gathers all of the relevant files needed to build it using a standard
TeX engine into a tarball called
arxiv.tar.gz. It’s named that because
you should be able to directly upload this tarball when submitting a paper
to the arXiv article service.
showyourwork cache --help
Usage: showyourwork cache [OPTIONS] COMMAND [ARGS]... Caching-related operations. Options: --help Show this message and exit. Commands: create Create a Zenodo Sandbox deposit draft for the given branch. delete Delete the latest draft of a Zenodo Sandbox deposit. freeze Publishes the current Zenodo Sandbox deposit draft for the... publish Publishes the current Zenodo Sandbox deposit draft for the...
Usage: showyourwork cache create [OPTIONS] Create a Zenodo Sandbox deposit draft for the given branch. Requires a Zenodo Sandbox API token provided as the environment variable and Github repository secret `SANDBOX_TOKEN`. Options: -b, --branch TEXT Which branch to create the deposit for. Default is current branch. --help Show this message and exit.
Usage: showyourwork cache delete [OPTIONS] Delete the latest draft of a Zenodo Sandbox deposit. Requires a Zenodo Sandbox API token provided as the environment variable and Github repository secret `SANDBOX_TOKEN`. Options: -b, --branch TEXT Branch whose deposit is to be deleted. Default is current branch. --help Show this message and exit.
Usage: showyourwork cache freeze [OPTIONS] Publishes the current Zenodo Sandbox deposit draft for the given branch to Zenodo Sandbox. Useful for preserving the current cache state even when changes are made to the rule inputs. Note that Zenodo Sandbox storage is temporary, so if you'd like to publish your cache on Zenodo, you should call ``publish`` instead. Requires a Zenodo Sandbox API token provided as the environment variable and Github repository secret `SANDBOX_TOKEN`. Options: -b, --branch TEXT Branch whose deposit is to be frozen. Default is current branch. --help Show this message and exit.
Usage: showyourwork cache publish [OPTIONS] Publishes the current Zenodo Sandbox deposit draft for the given branch to Zenodo for permanent, public storage. This action assigns a permanent, static DOI to the deposit, which cannot be deleted or undone. Requires a Zenodo API token provided as the environment variable `ZENODO_TOKEN`. Options: -b, --branch TEXT Branch whose deposit is to be published. Default is current branch. --help Show this message and exit.
Utilities for creating, deleting, and publishing the Zenodo deposit drafts used to cache intermediate results from your workflow; see Zenodo integration.