Overleaf integration
Contents
Overleaf integration#
Warning
Overleaf integration is still an experimental feature. The user interface may change in future releases. If you run into problems or have any suggestions to improve this functionality, please open an issue on GitHub.
Overview#
showyourwork! now allows users to integrate their projects with Overleaf, which can greatly facilitate collaborative article writing in LaTeX. While Overleaf supports integration with git and GitHub in a few different ways, none of these are quite right for an Overleaf-showyourwork! bridge (you can read all about why here).
Instead, whenever showyourwork
is executed (either locally or on GitHub Actions),
a specific set of files is synced between the git repository and the Overleaf project.
The user can choose which files get synced, but the default is to push figure output
to Overleaf and pull TeX files from Overleaf.
In order to prevent merge conflicts and data loss, syncing only happens in one direction:
by default,
figures only ever get pushed to Overleaf, while TeX files only ever get pulled from
Overleaf to the git repository.
This means that if users wish to integrate their project with Overleaf, they must only
make changes to the TeX file(s) on Overleaf. Similarly, users should not manually change/edit/upload
figures to Overleaf, and instead let showyourwork! do the grunt work.
In the event of a merge conflict—for example, if the user edited ms.tex
locally—the
showyourwork! build will acknowledge that and fail, refusing to overwrite
the local changes. See Managing conflicts below for details.
Below, we’ll discuss how to set up the integration with Overleaf and take a look at how to configure the syncing.
Setup#
Let’s set up Overleaf integration for a new repository. It is much, much, much easier to set up Overleaf integration for new projects, so here we’ll create both a new GitHub repository and a new Overleaf project for our article. If you want to set up Overleaf integration for an existing project, see Integrating existing projects.
To start, create a new repository on GitHub by visiting github.com/new.
For definiteness,
in the example below we’ll create the repository article
under my user
name (rodluger
).
Next, create a new project on Overleaf from the Blank Project template. You’ll be directed to the new project, whose URL will look something like
https://www.overleaf.com/project/6272c02ffe09ce2c9a5f0ff6
That last bit of the URL (6272c02ffe09ce2c9a5f0ff6
) is the 24-character
project ID, which uniquely identifies your project; we’ll use that in just
a moment. On your project page, you should see a single file called main.tex
containing some minimal TeX stuff. Don’t edit anything in it, as it will get
overwritten momentarily by showyourwork!.
Next, on your machine, define the environment variables $OVERLEAF_EMAIL
and $OVERLEAF_PASSWORD
with your Overleaf credentials. Unfortunately,
Overleaf does not yet support token-based authentication, so the only way
to grant showyourwork! access to your project is by providing the email
and password for your Overleaf account. As with the other credentials required
by showyourwork!, you’ll also need to create corresponding
GitHub Actions secrets at the following URL
https://github.com/rodluger/article/settings/secrets/actions/new
where you should replace rodluger/article
with your GitHub username
and the name of the repository.
Warning
Never commit your Overleaf credentials (or any credentials) directly to your repository!
Finally, on your local machine, set up a new showyourwork! repository by running
showyourwork setup rodluger/article --overleaf=6272c02ffe09ce2c9a5f0ff6
where, again, you should replace rodluger/article
with your repository slug
and 6272c02ffe09ce2c9a5f0ff6
with your Overleaf project ID. You should see
a couple messages saying the Overleaf repository is being set up. Once that’s
done, if you navigate to your Overleaf project in the browser, you’ll see that
the single TeX file main.tex
is gone and has been replaced by the following
files:
- figures
- .gitignore
- output
- .gitignore
- .gitignore
- bib.bib
- ms.tex
- showyourwork.sty
These, in fact, are the same files as in the src/tex
folder of your repository
(see Repository layout); showyourwork! will keep your Overleaf project up to date
with the contents of that folder (more on this below).
Note that the TeX manuscript is now called
ms.tex
(the default name in showyourwork!).
Returning to our local showyourwork! repository, if you open the config file
showyourwork.yml
, you’ll see that the setup
command populated the
overleaf
field with some stuff:
overleaf:
id: 6272c02ffe09ce2c9a5f0ff6
push:
- src/tex/figures
- src/tex/output
pull:
- src/tex/ms.tex
- src/tex/bib.bib
In addition to your Overleaf project ID, it has also defined some files and folders
in the push
and pull
fields. To understand what these mean, read on!
Pushing and pulling#
As we mentioned above, syncing between your git repository and your Overleaf project
only happens in one direction for any given file. Files listed under push:
are only ever pushed to Overleaf, while files listed under pull:
are only
ever pulled from overleaf. Pulling happens automatically in the pre-processing
step of every build performed in the main
branch of your repository (both locally
and when running in GitHub Actions), and automatically commits changes when
running locally.
Pushing happens automatically at the end of every
build on the main
branch (also both locally and on the remote). Note that a given file
may only be specified under push
or pull
, but not both, as that could lead
to merge conflicts.
It is highly recommended that you limit the pull
section to your main TeX
files (e.g., the manuscript and the bibliography) and the push
section to
programmatically-generated files (e.g., figure outputs or programmatically-generated
text files that are included in your manuscript using the \variable
command).
Note
You can disable Overleaf syncing at any time by commenting out the
Overleaf project id
in your showyourwork.yml
config file.
Once you’re done making changes to the LaTeX files in your article,
it’s a good idea to delete the entire Overleaf section in the config
file to prevent future changes to the your repository.
If you build frequently, you may occasionally run into a Rate limit exceeded
error
on the Overleaf side. Simply wait a minute and try again.
Finally, it is important to note that your showyourwork! repository
and your Overleaf project are completely
separate git repositories with unrelated commit histories. Under the hood, push
and pull
events are implemented as simple file copies from the head commit of one
repository to the head of the other. In order to minimize the chances that changes
to either repository will get lost or overwritten on a sync event, showyourwork!
will fail when attempting to pull
from Overleaf if it detects that any of the
relevant files have been modified since the last pull
. Read more about this—
and how to resolve these kinds of conflicts—in the next section.
Warning
There are no merge conflict checks when doing a push
to Overleaf.
If, for example, you manually upload a new version of a figure to the
Overleaf project, it will get overwritten the next time you build your article.
Managing conflicts#
If you’ve accidentally made a local change to a file listed under pull
, the
next time you build your article you might see the following error message:
Uncommitted changes to local file: <filename>.
Refusing to overwrite with Overleaf version.
If you’ve made a change and committed the change to git, you’ll see the following message instead:
Local file changed since the last Overleaf sync: <filename>.
Refusing to overwrite with Overleaf version.
Please see the docs for details on how to resolve this.
In these cases, showyourwork! fails because it wants to avoid overwriting your local changes with the contents of the Overleaf versions of the file(s). You’ll encounter this error even if you haven’t made changes to the Overleaf project, as showyourwork! simply copies files over from Overleaf each time you build (see above).
If you want to keep your local changes to these files, manually copy them over to Overleaf to ensure both versions are in sync with each other. Then, if you ran into the first error (uncommitted changes), simply reset the local changes to that file:
git checkout -- <filename>
and re-build your article. If you ran into the second error (i.e., you changed
the file and committed it), you’ll have to do a bit of extra work. If
you haven’t yet pushed your changes to GitHub, copy the changes to Overleaf
as above (if desired) and then undo that commit by running
git reset
(see here) followed by
git checkout -- <filename>
to discard your changes to that specific file. You should then be able to re-build your article. Note that at this point you make have uncommitted changes (from the commit you just undid), so make sure to add any relevant files and commit your changes back.
Finally, if you made local changes, committed them, and pushed them to GitHub,
you shouldn’t do a git reset
. Instead, once you copy your changes to Overleaf
(if desired) you should trick showyourwork! into thinking everything is OK and
proceeding with the sync process. To achieve this, make a dummy change to each
of the problematic files (e.g., add a space to any line in the file) so you have
something to commit, then commit with a message containing the string [showyourwork]
.
This flag is used internally whenever showyourwork! commits changes originated
in the Overleaf project, so this will trick the workflow into thinking everything
is in sync. The next time you build your article, showyourwork! will overwrite
those files with the versions on Overleaf.
Integrating existing projects#
Existing git repository#
If you have an existing repository for your project, create a new Overleaf
project and manually copy over your TeX files (e.g., ms.tex
and bib.bib
)
to Overleaf. Then, grab the project ID from the Overleaf URL, and add the following to your
showyourwork.yml
config file:
overleaf:
id: <ID>
push:
- src/tex/figures
- src/tex/output
pull:
- src/tex/ms.tex
- src/tex/bib.bib
The next time you build your article using showyourwork
, all files will get
synced between both projects.
Existing Overleaf project#
If, instead, you have an Overleaf project but no git repository, things can get much trickier. We recommend you hold off on using showyourwork! until your next project, as it’s much, much easier to set up Overleaf integration for brand new projects!
But if you really want to set up integration for an existing Overleaf project,
we strongly recommend you create a new showyourwork! repository and a new
Overleaf project (see the Setup section above). Then, copy over your TeX files
to the new Overleaf project (making sure to change the name of your main TeX
file to ms.tex
) and populate your local repository with the scripts
needed to build all your figures. You’ll likely run into lots of issues, such
as missing files, missing dependencies, etc., so this might take a lot of
debugging to get right!