Developer API

Note

The API documentation is still under development. Please check back soon for more details.

Modules

These modules are located in showyourwork/worfklow/rules and their contents are imported into the global namespace when executing the workflow. The functions and variables defined in these modules are used in several of the rules.

config.py

Processes the user config settings, setting default values if none are provided. Current config options are

  • verbose (bool): Enable verbose output? Default False.

  • figexts (list): List of recognized figure extensions. Default is ["pdf", "png", "eps", "jpg", "jpeg", "gif", "svg", "tiff"]

  • arxiv_tarball_exclude (list): List of files/paths to exclude from the tarball.

  • tectonic_latest (bool): Use the latest version of tectonic (built from source)? Default False.

  • tectonic_os (str): Operating system (used for choosing which tectonic binary to install). This is usually determined automatically, but can be overridden. Options are x86_64-unknown-linux-gnu, x86_64-apple-darwin, or x86_64-pc-windows-msvc.

  • dependencies (dict): List of dependencies for each figure.

  • ms (str): Path to the main TeX manuscript. Default src/ms.tex

  • scripts (dict): List of script extensions and instructions on how to execute them.

  • zenodo (dict): Rules for how to upload/download dependencies.

errors.py

Defines figure “error” (status) codes. These codes are used to determine the color of the GitHub icon next to each figure caption, ranging from blue (all is well!) to yellow (the script has uncommitted changes or is not version controlled) to red (the script can’t be located).

rules.errors.ScriptDoesNotExist = <rules.errors._ScriptDoesNotExist object>

The figure script does not exist.

rules.errors.ScriptHasUncommittedChanges = <rules.errors._ScriptHasUncommittedChanges object>

The figure script has uncommitted changes.

rules.errors.ScriptNotVersionControlled = <rules.errors._ScriptNotVersionControlled object>

The figure script is not version controlled.

rules.errors.ScriptUpToDate = <rules.errors._ScriptUpToDate object>

The figure script is version controlled and up to date.

exceptions.py

Defines a custom exception for worfklow errors that tries to print informative error messages at the very end of the build log.

exception rules.exceptions.ShowyourworkException(message, exception_file=None, script=None, rule_name=None, context=None, delayed=True, brief='An error occurred while executing your workflow.', display_full_message=False, *args, **kwargs)

Custom exception that stores error messages in a temporary file to be displayed at the end of the build process.

files.py

Miscellaneous files used throughout the workflow. The static members of the class files are populated automatically as the workflow reads the user config.

class rules.files.files

Houses lists of support files used throughout the workflow.

aux = []

Auxiliary files we copy over to the user’s src/ directory.

cls = {}

Class-specific auxiliary files.

dot_zenodo = []

The .zenodo files in which we store the URL for the datasets We make the PDF generation depend on these so we always have access to the URLs for the icon generation. These are updated in zenodo.smk.

exception = PosixPath('.showyourwork/exception.log')

Store temporary exception messages

script_extensions = ['py']

Recognized figure script extensions

special_figures = ['orcid-id.png', 'orcid-ID.png', 'showyourwork.pdf']

Figures that are allowed directly in the src/ directory

tectonic = []

Tectonic command

tmp_syw = '.showyourwork-ms'

Temporary tex file.

tmp_xml = '.showyourwork-xml-ms'

Temporary tex file.

unknown = 'unknown-script'

Dummy file dependency for figures w/ unknown parent scripts.

zenodo_files_auto = []

The showyourwork-managed datasets for this article

zenodo_files_manual = []

The manually-uploaded datasets for this article that must be downloaded

functions.py

Functions used as inputs / params in various rules.

rules.functions.check_figure_format(figure)

Check that all figures are declared correctly in tex/ms.tex so we can parse them corresponding XML tree.

rules.functions.class_files(wildcards)

Return a list of class files needed to compile the PDF.

rules.functions.figure_script(wildcards)

Return the figure script that produces wildcards.figure.

rules.functions.figure_script_dependencies(wildcards)

Return user-specified dependencies of the current figure script.

rules.functions.figures(wildcards)

Return all the figure files required by the manuscript.

rules.functions.input_class_file(wildcards)

Return the path to the LaTeX document class file.

rules.functions.script_cmd(wildcards, input)

Returns the shell command to produce a figure output from its script.

rules.functions.script_dependencies(wildcards)

Return user-specified dependencies of the current script.

rules.functions.script_name(wildcards, input)

Returns the name of the figure script for the figure rule.

Assumes the first python file in the input is the one that generates the script.

paths.py

Sets global constants for the workflow.

class rules.paths.abspaths

Absolute paths used throughout the workflow.

temp = PosixPath('/path/to/repo/.showyourwork')

Path to the temp folder

user = PosixPath('/path/to/repo')

Path to the user’s repository root

rules.paths.posix(path)

Convert a path object to a string containing the posix path.

class rules.paths.relpaths

Relative paths used throughout the workflow.

dot_github = PosixPath('.github')

Path to the showyourwork workflow

figures = PosixPath('src/figures')

Path to the showyourwork workflow

temp = PosixPath('.showyourwork')

Path to the temp folder

tex = PosixPath('src')

Path to the showyourwork workflow

zenodo.py

Collect information about all the datasets we’ll need to upload to/download from Zenodo.

class rules.zenodo.zenodo

Class containing metadata for all Zenodo deposits.

deposit_contents = {}

Contents of each dataset (if it’s a tarball)

deposit_creators = {}

Zenodo authors for each dataset

deposit_description = {}

Zenodo deposit description for each dataset

deposit_title = {}

Zenodo deposit title for each dataset

file_name = {}

File name for each dataset

file_path = {}

Path to each dataset

sandbox = {}

Use Zenodo sandbox for a particular dataset?

script = {}

Script for generating each dataset

token_name = {}

Zenodo token name (name of an env. var.) for each dataset

zenodo_id = {}

Id for each Zenodo download

zenodo_url = {}

Url for each Zenodo download

Rules

These rules are located in files of the same name in the showyourwork/workflow/rules directory with the extension .smk. They are the building blocks of the Snakemake workflow, and can even be called externally as make <rule_name>.

rule repo

Generates repository metadata (git url, branch name, commit sha). Runs every time the article is generated.

rule aux_file

Copies auxiliary tex files to the src directory.

rule tectonic

Installs the latest version of tectonic. This rule isn’t used unless the user explicitly asks for tectonic_latest in the config file. We originally implemented this rule when issues with archive.org caused the released versions of tectonic to fail, but an unreleased patch was functioning. This rule is probably of very limited use in general.

rule class_name

Infers the name of the LaTeX document class for the article by doing a regex on the TeX file. This helps showyourwork decide which class files to include in the build.

rule class_file

Copies the LaTeX class file to the src directory.

rule tmptexfile

Copies the manuscript to a temporary TeX file for the XML build step.

rule xml

Generates the article XML tree. Specifically, builds the article using tectonic, but re-defines figure, caption, and label commands to print XML tags to a special log file. This way, we can use LaTeX to construct a full XML tree of the document for us, without any need for parsing the TeX file ourselves. This XML tree is then used to determine relationships between the figure scripts and the figure files.

rule script_info

Builds the figure dependency tree from the XML file generated in the xml rule. Stores it in a JSON file in the temporary showyourwork directory.

rule metadata

Generates article metadata from the output of the repo and script_info rules. Saves it as a JSON in the temporary showyourwork directory.

rule figure

Generates a figure given a figure script and optional dependencies. This is the workhorse of showyourwork, which generates all the figures in the article. Users can subclass this rule or override it entirely to customize the build process.

rule texfile

Copies the manuscript to a temporary TeX file for the final build.

rule stylesheet

Generates the showyourwork.sty LaTeX style sheet from a Jinja template and copies it to the src directory.

rule pdf

Builds the final article PDF. This is the final step in the article build process.

rule arxiv

Builds a tarball of the article PDF and all output for posting to the arXiv.

rule docstrings

Generates simple documentation for the rules in this workflow. This rule is called when building the API documentation, but only when running it locally (since we do not install Snakemake, or conda for that matter, on ReadTheDocs). Therefore, we should build the documentation locally from time to time and push changes to the rules.rst file containing these docstrings so that our online docs are up to date.

Scripts

These scripts are located in showyourwork/workflow/scripts and are called from some of the rules defined above.

zenodo.py

Uploads or downlods a dataset to/from Zenodo. This script is called from the zenodo rule.

arxiv.py

Creates the output tarball for easy arXiv upload. This script is called from the arxiv rule.

pdf.py

Builds the article PDF. This script is called from the pdf rule.

stylesheet.py

Creates the showyourwork stylesheet. This script is called from the stylesheet rule.

tectonic.py

Downloads and installs tectonic from source. This script is called from the tectonic rule.

figure.py

Generates a figure output. This script is called from the figure rule.

xml.py

Builds the article to get the XML tree. This script is called from the xml rule.