This web page describes how to piece together various patches, plugins, configurations, and scripts to support relaying the local machine's mailbox through gmail with OAUTH2. Then you can use any mail client that uses your local machine's UNIX/Linux mailbox to read and send email. (I personally prefer mutt, but any email client, gui-based or not, should work.)
It might also work to serve out the local mailbox(es) to other machines using basic POP/IMAP/SMTP services to bypass gmail OAUTH2, although I haven't tested or documented that.
It is kind of complex to get all the pieces set up, but on the plus side, you can leave out (incoming only?), replace (prefer something other than postfix?), or tweak the details of any of these pieces as desired.
Table of Contents
OAUTH2 is a fancy mechanism for apps/websites/etc to delegate arbitrarily complex multi-factor login capabilities to a central authentication management web site. It is primarily driven by Google for use in mobile-phone-based apps, but can be used in other ways as well.
It is usually optional on the part of the individual user, but in some cases can be required. One such case I've encountered is when my $DAYJOB offloaded company email to Google's business "G Suite" service, and configured it at the organization level to require the use of OAUTH2. It is really just gmail, even though the hostname belongs to $DAYJOB instead of using "gmail.com".
Basically OAUTH2 works as follows:
For more details about how it actually works under the hood, see Google documentation such as Using OAuth 2.0 to Access Google APIs, A Run-Through of semi-manually going through the various steps with the aid of small-ish "oauth2.py" script, etc. (My "fetchmail-oauth2.py" script (below) was derived from this script, and should still support the arguments used by the run-through.)
It is doubtful that oauth2 is really more secure than just picking a strong password and protecting it carefully. Especially for "application specific passwords" generated by the server. The extra complexity significantly expands the vulnerability footprint, even if in some vague theoretical ways it might have some slight advantages.
These instructions are mostly geared towards "gmail" and/or "G Suite", but can probably be adapted to other providers that use OAUTH2. For others, you may need to find an equivalent to the "Google API Console" and research various optional settings for fetchmail-oauth2.py's config file (some settings not mentioned below default to Google).
"Makefile" and "lockedMake" probably belong under /etc/postfix, some files are patches only needed when rebuilding packages, and others should go somewhere under your home directory. Perhaps re-use existing directories where you've put cron jobs or other mail-related scripts and configurations. I'm intentionally avoiding specific recommendations to make it slightly less likely worms and rootkits will try to look for sensitive information in variable locations, although that is a pretty weak form of so-called "security".
You might want to comment out the
"sudo /etc/postfix/lockedMake" line until you
get to the postfix setup section below.
fetchmail-oauth2.py --help",
but the following example may be adequate:
client_id=FROM_GOOGLE_CONSOLE_ABOVE client_secret=FROM_GOOGLE_CONSOLE_ABOVE refresh_token_file=/FULL/PATH/TO/REFRESH_FILE access_token_file=/FULL/PATH/TO/ACCESS_TOKEN_FILE max_age_sec=1900
Some other settings (not in example) default to Google, so if you are using some other provider, you might need to research them.
chmod 600 THE_CONFIG_FILE"
so that no one else can read it.
fetchmail-oauth2.py -c /PATH/TO/CONFIG_FILE --obtain_refresh_token_file".
It will print a URL that you need to log into using a
browser. After you do, the browser will print a code you
need to cut and paste into the still-running script.
Depending on how your account is setup, you might be able to make use of the extra "googleAuthenticator" script while logging in. When you first set up the server to use Google Authenticator, click on the "I can't scan it" link to get the secret authenticator string. Strip out the spaces and save it somewhere (probably your password manager). Then when you need to generate an authentication code, run the "googleAuthenticator" script and paste the secret into it. It will then print the current 6 digit code. Your system clock needs to be reasonably accurate. (It doesn't use command line arguments, to try to keep the secret out of .bash_history.)
UPDATE: In May 2022, Cajus Hahn (cajus_hahn AT arcor dot de) reported that the renewal token only seems to work for one week, but I haven't seen that myself.
UPDATE (2022-10-16): The simplest way to get a non-OOB option
working seems to be to edit the REDIRECT_URI line in the script to
read something like
"REDIRECT_URI = 'http://127.0.0.1:8018/fetchmail'"
Pick any unused port number. After logging in, your browser will
fail to load the redirect web page, but you can carefully select
and copy/paste the token part from the middle of the URL out of
your browser's address bar and into the waiting script. FUTURE:
The URI can't currently be overridden with a config file. It also
would be more convenient if the script actually ran a small
localhost-only web server to capture the token, and you didn't
need to copy/paste anything. Maybe someday I'll get around to
such enhancements, or maybe someone else will, but in the
meantime, this technique works.
UPDATE (2023-01-01): Alexander Zangerl (az at breathe-safe dot com) has reported success using Thunderbird's client_id and client_secret for O365 (which I hesitate to post here, to avoid getting into a three-way war over pretending to be a different client, despite the fact that the secret probably isn't well protectected...), along with the following values (some of which probably require modifying scripts, not just the config file):
imap_server=outlook.office365.com,
smtp_server=outlook.office365.com,
scope="https://outlook.office365.com/IMAP.AccessAsUser.All
https://outlook.office365.com/POP.AccessAsUser.All
https://outlook.office365.com/SMTP.Send offline_access"
auth_url=https://login.microsoftonline.com/common/oauth2/v2.0/authorize
token_url=https://login.microsoftonline.com/common/oauth2/v2.0/token
redirect_uri=http://localhost
max_age_sec=1800
ALTERNATIVE: Some people have reported success with a different sasl plugin: https://github.com/tarickb/sasl-xoauth2 Jamen Lang (jamenl at gillettewy dot gov) mentioned it it 2020, and Freek de Kruijf (f.de.kruijf at gmail dot com) mentioned it in mid 2022. He has also indicated an intent to document his procedure at https://en.opensuse.org/Configure_Postfix_to_send_email_with_xoauth2_HOWTO, although it is currently still a stub as I write this. From its github readme, this plugin looks like it is much more actively developed than my cobbled together instructions, and it comes with a oauth script that is presumably similar to my own. For postfix it can apparently use the renewal token to get new access tokens on demand instead of requiring a cron job. It might help with the DIFFICULTY and/or Microsoft cases mentioned above, although I haven't dug deep enough to be sure. The readme instructions are focused on outgoing email (postfix), but given that the same tokens can be used by both, maybe it could also be used with incoming email and/or fetchmail. Currently I intend to look at this more carefully when I have some time.
The "fetchmail-oauth2-passwordfile-*.patch" files are patches that can be applied to the 6.3 or 6.4 branches of fetchmail. Fetchmail 6.X development has continued for far longer than I would have expected (with only an unofficial alpha releases of 7), and that development keeps (slightly) breaking patches. "...v2.patch" applies to 6.3.* and early 6.4 development, "...br64-v3.patch" applies to 6.4.1, "...br64-v4.patch" applies to some intermediate 6.4.x versions, "...br64-v6.patch" applies to 6.4.8 through at least 6.4.27 on gentoo, and "...br64-v8.patch" applies to 6.4.34 and later (and includes an increased PASSWORDLEN to hopefully allow its use with Microsoft's long O365 tokens).
Andrew C. Aitchison (andrew at aitchison dot me dot uk) has supplied two more versions that I haven't tried: "b64-v7.patch" appears to be the same as v6 except it corrects line numbers and nearby context lines to apply more cleanly (but identically), and 650b7.patch is intended for the 6.5beta7 release. Francois Manchon (Francois.Manchon at sita dot aero) supplied fetchmail6.4.34-xoauth2.patch, which I haven't tried but apparently adds support for POP3 (like the unreleased version 7.0).
Once you build and install the patched fetchmail, configure ".fetchmailrc" using something similar to the following:
poll imap.gmail.com protocol imap auth oauthbearer username "USER@gmail.com" passwordfile "/PATH/TO/ACCESS_TOKEN_FILE" is LOCALUSER here ssl sslcertck sslproto tls1.2+The way these older, rough patches are constructed, it actually implements xoauth2 only, but "oauthbearer" is a synonym for future compatibility with version 7 below. This version of the patch only reads the token file once, and so fetchmail's daemon mode is unlikely to work. Run fetchmail using cron instead. It also only supports IMAP (not POP).
I have a report about some trouble trying to apply the patches with "git apply" rather than "patch". Most of the time my testing is mostly automated and consists of the gentoo mechanism described below, which is basically based on "patch"ing the "stable" version of fetchmail, so I might be slow to notice problems that this approach doesn't test. "git apply" is pickier than "patch" about failing if nearby lines have changed, so if you have trouble applying the patches you might try "patch" and/or basing on a slightly older "gentoo stable" version of fetchmail. (See https://packages.gentoo.org/packages/net-mail/fetchmail for information about which versions are currently testing vs stable.)
If you are using Gentoo Linux, then it is possible to use the /etc/portage/patches (documentation) feature to automatically apply the patch when re-emerging fetchmail:
mkdir -p /etc/portage/patches/net-mail/fetchmail cp fetchmail-oauth2-passwordfile-br64-v6.patch /etc/portage/patches/net-mail/fetchmail/.You could optionally try to tweak names to restrict the patch to specific versions; see documentation link above.
The development branch for fetchmail 7 has included an enhanced version of this patch for years. Additional improvements include: daemon mode will re-read the passwordfile regularly, it actually implements (and prefers) standardized "oauthbearer" in addition to "xoauth2", and adds POP support. See merge requests.
Fetchmail 7 is apparently going improve how to configure ssl, so the "ssl" keyword in the configuration file will probably need to change to "sslmode wrapped" to avoid warnings from fetchmail.
Unfortunately, it isn't clear when 7 will go "live". There was an alpha release over a year ago, but 6 has had various releases since then, including a new "6.5" branch...
ebuild cyrus-sasl-xoauth2-0.2.ebuild digest"
on the copy, and then "emerge cyrus-sasl-xoauth2".
See also the plugin's own instructions, although they are a bit terse.
There may be other tools that can use this plugin (both
as a client and as a server), although I haven't tried any.
LOCALUSER ALL=(root) NOPASSWD: /etc/postfix/lockedMake".
Replace "LOCALUSER" with your username.
[smtp.gmail.com]:587 encrypt", since xoauth2
generally requires SSL in order to work.
USER@HOST [smtp.gmail.com]:587".
# single or default/fallback relay: relayhost = [smtp.gmail.com]:587 # optional multiple relays: sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost_map smtp_sasl_password_maps = hash:/etc/postfix/saslpass-autogen smtp_sasl_auth_enable = yes smtp_sasl_security_options = # Filter out other mechanisms. It might work without this, # but not reliably. If you want multiple relays and only # use xoauth2 for some of them, then it is probably best to # setup multiple transports in master.cf with different filters, # and use sender_dependent_default_transport_maps to select which # transport. smtp_sasl_mechanism_filter = xoauth2 #smtp_sasl_mechanism_filter = plain, login smtp_tls_security_level = may # The "may" above should work most of the time, but xoauth2 won't # work without ssl: smtp_tls_policy_maps = hash:/etc/postfix/tls_policy #### Optional related: # Potentially useful if the machine wants to relay multiple different # local users to gmail: smtp_sender_dependent_authentication = yes
sudo /etc/postfix/lockedMake"
line in the cron.oauth2 script above, you should uncomment it now.
Older versions of this web page outlined future ideas for making the configuration of multiple relays easier, but the above multiple-transport option and/or the ALTERNATIVE plugin (further above) apparently cover those ideas fairly well.
14,39 * * * * /path/to/cron.fetchmail". You might
also configure it to avoid nights and weekends, as appropriate,
since you can always run "fetchmail" manually (as long as the
access token isn't expired; see below). You might need to be
a member of the "cron" group (in /etc/groups) to be allowed to
use crontab and cron.
Standard trick: avoid "even" minutes to avoid overloading the machine on the hour if it has a lot of cron jobs set to start at the same time...
A few years ago one of my gmail accounts spontaneously stopped working in fetchmail with just my normal password. Other gmail accounts were still working. I was never able to figure out why it just stopped working, but here are various details: