Command line interface#

The showyourwork! package implements a single command-line utility: showyourwork, which allows users to set up, configure, and build their open source article. Below we describe this command and discuss its various subcommands.

showyourwork#

showyourwork --help

Usage: showyourwork [OPTIONS] COMMAND [ARGS]...

  Easily build open-source, reproducible scientific articles.

Options:
  -v, --version  Show the program version and exit.
  --help         Show this message and exit.

Commands:
  build    Build an article in the current working directory.
  cache    Caching-related operations.
  clean    Clean the article build in the current working directory.
  setup    Set up a new article repository in the current working directory.
  tarball  Generate a tarball of the build in the current working directory.

Running showyourwork! (without any arguments) is a shortcut for showyourwork build (see showyourwork build below).

showyourwork setup#

showyourwork setup --help

Usage: showyourwork setup [OPTIONS] <user/repo>

  Set up a new article repository in the current working directory.

  This command expects a single positional argument, `SLUG`, of the form
  `user/repo`, where `user` is the user's GitHub handle and `repo` is the name
  of the repository (and local directory) to create.

Options:
  -y, --yes                  Accept all `Press any key to continue` prompts
                             automatically.
  -q, --quiet                Don't prompt the user, and don't display
                             informational output.
  -c, --cache                Set up intermediate result caching on Zenodo
                             Sandbox. Requires a Zenodo Sandbox API token
                             provided as the environment variable and GitHub
                             repository secret `SANDBOX_TOKEN`.
  -o, --overleaf PROJECT_ID  Overleaf project id to sync with (optional).
                             Requires Overleaf credentials, provided as the
                             environment variables and GitHub repository
                             secrets `OVERLEAF_EMAIL` and `OVERLEAF_PASSWORD`.
  -s, --ssh                  Use ssh to authenticate with GitHub? Default is
                             to use https.
  -v, --version VERSION      Version of showyourwork to use. Default is the
                             current version.
  --help                     Show this message and exit.

The setup subcommand sets up an open source article repository from scratch in the current working directory. This is an interactive command (unless you provide the --yes or --quiet options; see below). Let’s step through what it does here.

To set up a new open source article repository, run

showyourwork setup $USER/$REPO

where you should replace $USER with your GitHub user name and $REPO with the name of your new article repository. For definiteness, here we’ll use my user name (rodluger) and we’ll call our repository article.

Step 1#

Running the setup command as above should bring up the following prompt:

Let's get you set up with a new repository. I'm going to create a folder called

    article

in the current working directory. If you haven't done this yet, please visit

    https://github.com/new

at this time and create an empty repository called

    rodluger/article

As requested, if you haven’t yet created the remote repository, go to github.com/new in your browser to create an empty repository of the same name. There’s no need to create a README, gitignore file, or LICENSE at this time, as showyourwork! will set those up for you.

Press any key to bring up the next prompt. What you see next depends on whether or not you provided the --cache option.

Step 2A#

If you didn’t request Zenodo caching functionality (see below), you’ll see the following the message:

You didn't provide a caching service (via the --cache
command-line option), so I'm not going to set up remote caching for this repository.

Step 2B#

If instead you passed the --cache flag, you’ll see the following message:

You requested remote caching on Zenodo, so I'm going to create a deposit draft where
intermediate results will be cached. Please make sure at this time that you have
defined the ZENODO_TOKEN environment variable containing your API key for Zenodo.
If you don't have one, visit

    https://zenodo.org/account/settings/applications/tokens/new

to create a new personal access token with deposit:actions and deposit:write
scopes and store it in the environment variable ZENODO_TOKEN. In order for
this to work on GitHub Actions, you'll also have to visit

    https://github.com/tmp/tmp/settings/secrets/actions/new

at this time to create a ZENODO_TOKEN secret with your API access token.

Note that, in addition to the --cache flag, which enables caching on Zenodo, users may also provide the --sandbox flag, which switches the host to Zenodo Sandbox. Zenodo Sandbox behaves in exactly the same way as Zenodo, but it is explicitly meant as a test bed for dataset archiving. While deposits on Sandbox get assigned DOIs, they are no actual registered DOIs and have a limited lifespan. Sandbox is therefore a great choice for debugging and development; read more about it at Zenodo integration. Note that if you choose the --sandbox option, you’ll need a Zenodo Sandbox API token stored int the SANDBOX_TOKEN environment variable and GitHub Actions secret.

Warning

Never commit your Zenodo API token (or any API token) directly to your repository!

You can read more about GitHub secrets (and the security measures in place to prevent them from getting exposed to the outside world) at the GitHub documentation.

Press any key to bring up the next prompt. What you see next depends on whether or not you specified the --overleaf option.

Step 3A#

If you didn’t pass the --overleaf option, you’ll see the following:

You didn't provide an Overleaf project id (via the --overleaf command-line
option), so I'm not going to set up Overleaf integration for this repository.

If you would like to set up integration with an Overleaf project (see Overleaf integration), hit Ctrl+C and run

showyourwork setup --overleaf=62150dd16134ef045f81d1c8

where you should replace 62150dd16134ef045f81d1c8 with the 24-character id of a new (blank) Overleaf project. Once you create a new Overleaf project, you can grab the id from the last bit of the project’s URL. Note that showyourwork! requires the Overleaf project to be empty, otherwise it will refuse to set up the integration. For more information on how this integration works, and what to do if you have an existing Overleaf project you’d like to integrate with showyourwork!, please see Overleaf integration.

Step 3B#

If you specified the --overleaf option (see Step 3A), you’ll get the following message:

You provided an Overleaf project id, so I'm going to set up Overleaf integration
for this repository. Please make sure at this time that you have defined the
OVERLEAF_EMAIL and OVERLEAF_PASSWORD environment variables. In order for this to
work on GitHub Actions, please go to

    https://github.com/tmp/tmp/settings/secrets/actions/new

at this time and create OVERLEAF_EMAIL and OVERLEAF_PASSWORD secrets with your
Overleaf credentials.

To allow showyourwork! to push to/pull from your Overleaf project, create the environment variables $OVERLEAF_EMAIL and $OVERLEAF_PASSWORD and populate them with your Overleaf email address and password, respectively; then re-run the setup command. Again, take care to never actually commit this information to your repository!

Step 4#

Finally, press any key to generate the repository. This will create a new folder in the current working directory with the same name as your repo (article, in the example above) and set up git tracking for it. Note that the first time you commit and push your changes to the GitHub repository, you’ll have to specify the upstream branch as follows:

git push -u origin main

showyourwork build#

showyourwork build --help

Usage: showyourwork build [OPTIONS] [SNAKEMAKE_ARGS]...

  Build an article in the current working directory.

Options:
  --help  Show this message and exit.

Run this command to build the article in the current working directory. Note that you must run this command from the top level of the repository (an error will be thrown otherwise). The command accepts any number of arguments, all of which are forwarded to snakemake. By default, showyourwork passes the following arguments to snakemake:

-c1 --use-conda --reason --cache

Some of these, like the number of cores, can be overridden. For example, you may run

showyourwork build -c2

to run the workflow using two cores (see the snakemake docs for details). Additional arguments can also be provided, like --verbose to increase the verbosity of the Snakemake logs (see Logging), or --force and --forceall to force the re-execution of the rule that builds the manuscript or all of the rules in the workflow, respectively (regardless of whether the outputs are up to date or not). Positional arguments are also allowed; for instance, to only build a specific figure, you may run, e.g.,

showyourwork build --force src/tex/figures/figure.pdf

You can check out the complete list of Snakemake arguments and options at the snakemake documentation.

Warning

Not all Snakemake options are compatible with showyourwork!. If you run into issues when specifying custom options, please let us know.

Note that the build process in showyourwork! happens in two steps, each of which executes a separate Snakemake workflow. The first step is a preprocessing step that parses the user config file and does a quick first-pass compiling of the TeX manuscript to look for \includegraphics and \script calls, which it uses to build the graph of dependencies for your article. The second step is the main step, in which all of the dependencies are built (if needed) and the final article PDF is generated. Arguments passed to showyourwork build are ingested only during the second step.

Finally, showyourwork! takes full advantage of the dependency tracking and caching functionality of Snakemake. When running showyourwork build, only files whose upstream dependencies have changed (since the last build) will be re-generated. This is true even when running on GitHub Actions; the showyourwork-action caches results across runs to minimize compute time for the build. We even go a step further, and extend the Snakemake functionality to allow caching of intermediate dependencies on Zenodo; read about it at Zenodo integration.

showyourwork clean#

showyourwork clean --help

Usage: showyourwork clean [OPTIONS]

  Clean the article build in the current working directory.

Options:
  -f, --force  Forcefully remove everything in the `src/tex/figures` and
               `src/data` directories.
  -d, --deep   Forcefully remove the `.snakemake` and `~/.showyourwork`
               directories.
  --help       Show this message and exit.

This command removes all of the output from previous showyourwork build steps. Depending on the state of your repository, and if there are errors in your config file or missing dependencies, this command may fail silently, in which case some of the output may remain after running it.

Force clean#

If showyourwork clean didn’t remove all of the output, you can force the deletion of all the programmatically-generated figures and datasets by passing the --force option, which will remove everything in the src/tex/figures, src/data, and temporary .showyourwork folders (assuming you’re respecting the showyourwork! conventions; see Repository layout).

Deep clean#

If you want to start over from scratch, you can also pass the --deep option for a deep clean. This removes the hidden .snakemake folder, which houses the conda environments for your build (among other things), so deleting it will force a re-install of all packages used in your workflow. This option also removes the .showyourwork folder located in your $HOME path, which also houses conda environments used at different stages of the build step. You can safely remove it at any time (at the cost of a longer runtime the next time you execute showyourwork).

showyourwork tarball#

showyourwork tarball --help

Usage: showyourwork tarball [OPTIONS]

  Generate a tarball of the build in the current working directory.

Options:
  --help  Show this message and exit.

Like build, the showyourwork tarball command builds your article, but also gathers all of the relevant files needed to build it using a standard TeX engine into a tarball called arxiv.tar.gz. It’s named that because you should be able to directly upload this tarball when submitting a paper to the arXiv article service.

showyourwork cache#

showyourwork cache --help

Usage: showyourwork cache [OPTIONS] COMMAND [ARGS]...

  Caching-related operations.

Options:
  --help  Show this message and exit.

Commands:
  create   Create a Zenodo Sandbox deposit draft for the given branch.
  delete   Delete the latest draft of a Zenodo Sandbox deposit.
  freeze   Publishes the current Zenodo Sandbox deposit draft for the...
  publish  Publishes the current Zenodo Sandbox deposit draft for the...

Subcommand documentation:

Usage: showyourwork cache create [OPTIONS]

  Create a Zenodo Sandbox deposit draft for the given branch.

  Requires a Zenodo Sandbox API token provided as the environment variable and
  Github repository secret `SANDBOX_TOKEN`.

Options:
  -b, --branch TEXT  Which branch to create the deposit for. Default is
                     current branch.
  --help             Show this message and exit.
Usage: showyourwork cache delete [OPTIONS]

  Delete the latest draft of a Zenodo Sandbox deposit.

  Requires a Zenodo Sandbox API token provided as the environment variable and
  Github repository secret `SANDBOX_TOKEN`.

Options:
  -b, --branch TEXT  Branch whose deposit is to be deleted. Default is current
                     branch.
  --help             Show this message and exit.
Usage: showyourwork cache freeze [OPTIONS]

  Publishes the current Zenodo Sandbox deposit draft for the given branch to
  Zenodo Sandbox.

  Useful for preserving the current cache state even when changes are made to
  the rule inputs. Note that Zenodo Sandbox storage is temporary, so if you'd
  like to publish your cache on Zenodo, you should call ``publish`` instead.

  Requires a Zenodo Sandbox API token provided as the environment variable and
  Github repository secret `SANDBOX_TOKEN`.

Options:
  -b, --branch TEXT  Branch whose deposit is to be frozen. Default is current
                     branch.
  --help             Show this message and exit.
Usage: showyourwork cache publish [OPTIONS]

  Publishes the current Zenodo Sandbox deposit draft for the given branch to
  Zenodo for permanent, public storage. This action assigns a permanent,
  static DOI to the deposit, which cannot be deleted or undone.

  Requires a Zenodo API token provided as the environment variable
  `ZENODO_TOKEN`.

Options:
  -b, --branch TEXT  Branch whose deposit is to be published. Default is
                     current branch.
  --help             Show this message and exit.

Utilities for creating, deleting, and publishing the Zenodo deposit drafts used to cache intermediate results from your workflow; see Zenodo integration.