FrostyX's blog - Fedora articles

Emacs - Evil :set option

Sun, 07 Aug 2022 00:00:00 +0000

Do you miss all the easy-to-use :set commands from Vim? You know, :set wrap, :set number, :set colorcolumn, :set expandtab, etc.

Narcissistic rant

Even though the Evil project itself doesn’t claim any such title, urban legends say that Evil is the best Vim emulator ever implemented. Naturally, my expectations were high. And yet, I was blown away. Evil does everything perfectly (or at least well enough to fool me) - modes, macros, window management, plugins, colon commands (although not all of them are implemented - I am looking at you :retab).

For me, the only major letdown was missing support for :set options such as :set wrap, :set number, and so on. Of course, Emacs has its own commands for achieving the same behavior. My issue is simply the non-existing:set command and, therefore, inability to use the options that I already know.

Introducing evil-set-option

For this reason, I created evil-set-option.

Click, it's a gif!

Installation

Use Quelpa (or any package manager that you prefer) to install the package.

(use-package evil-set-option
  :ensure t
  :quelpa (evil-set-option
           :fetcher github
           :repo "FrostyX/evil-set-option"
           :branch "main")
  :config
  (evil-set-option-mode))

And now you can run your favorite :set commands.

:set wrap
:set number
:set colorcolumn=80

Supported options

Currently, only a dozen of options that seemed to be the most popular, are supported. Vim comes with hundreds of options, and for understandable reasons, I wasn’t going to implement all of them within the first release. Please let me know what are your favorite options that we simply must have.

The list of all currently supported options can be found here.

Custom option behavior

Sometimes the default implementation of an option might not be applicable. You might want to use a different mode that suits your build more, you might run an older version of Emacs, that doesn’t support some modes, etc. In such cases, you can override the default implementation.

;; The default implementation for `:set number' uses
;; `display-line-numbers-mode' which is available since Emacs 26.
;; See https://www.emacswiki.org/emacs/LineNumbers
;; You can override it to use `linum-mode' instead

(defun evil-set-option-number (value)
  (linum-mode (if value 1 0)))

Future plans

I have a bigger picture in mind. The evil-set-option package won’t get bloated with another functionality apart from supporting more and more options. My plan is to create a new package called evil-vimrc that will provide the ability to parse a .vimrc file - although very naively. VimL is a full-featured scripting language and it would require executing the config file code to parse the options accurately. I don’t think that would be time worth spending. A dummy implementation that parses all set statements is still going to be useful in many cases.

The hypothetical usage will look like this

(evil-vimrc-load "~/.vimrc")

And the ~/.vimrc as expected.

set nocompatible
set nowrap
set title
set expandtab

Or alternativelly, defining the .vimrc configuration within Emacs Lisp.

(evil-vimrc
 (:set nocompatible)
 (:set nowrap)
 (:set title)
 (:set expandtab))

Stay tuned.

A year with Emacs

Wed, 12 May 2021 00:00:00 +0000

After a decade of exclusively using Vim for all my text editing, I decided to explore the other side of the barricade and see what the eternal adversary is doing. Ever since that day, I live in Emacs.

Motivation

Why would I commit such a heinous crime? After ten years, I came close to hitting my peak knowledge about Vim. Of course, nobody can claim to truly know Vim in its full depth but I got to the point where the learning curve slowed to a deadly pace and new tricks just slightly adjust your approach when solving obscure scenarios. The only real game-changer yet to be conquered was plugins development. I tried but I dislike VimL as a language for writing code and thus I have no intention of doing so.

On the other hand, I am fascinated by the functional programming paradigm and Lisp in particular. This alone, was a strong enough impulse for migrating to Emacs. I won’t hit the ceiling regarding custom package development.

Also, I came to realize, that my Vim workflow is different than for most other users (at least in my social circle). Everyone seems to have a shell-centric approach, find-ing, grep-ing, cat-ing their way through a project, and then opening the intended file in Vim, editing, and closing. I do the same thing when it comes to searching within a project but I do it in a separate terminal. When it comes to Vim, I start it once, do everything from within, and then close it after six months when a new Fedora is released and I need to upgrade. If it reminds you of something, it’s Emacs.

There is also a couple of tempting Emacs features but about them later.

Preview

On my triple-monitor setup, two of the screens are dedicated to maximized Emacs frames. This is how my PIM screen looks like.

The main monitor is designated for development.

The goal was to make the development setup identical to my previous Vim configuration.

Misconceptions

Due to my heavy addiction to Vim keybindings and modal editing, a non-negotiable requisite was a decent Vim emulator inside of Emacs. As it turned out, Evil is not only a great name but also a great piece of software. However, it is often marketed in a way, that you don’t have to learn Emacs key bindings at all. This was a big selling point for me, that however, turned out not to be true.

Yes, Evil is probably the best Vim emulator out there and provides support for every Vim feature that I know of. The thing we need to realize is that Emacs has a much broader scope than just text editing. It is more of an application platform rather than an editor. Naturally, there cannot be any Vim keybindings for things that are beyond its abilities. Don’t worry though. You can get far enough by learning M-x for running commands, M-: for Elisp evaluation, and by defining custom key bindings.

Another unpleasant surprise was that my .vimrc was completely useless for the migration. For some reason I expected Evil to parse and apply my current Vim configuration, and even use the Vim plugins I had installed. You can laugh at me now. That, of course, is not possible but in the end, it cost me just a couple extra days of fieldwork. Most of the popular Vim plugins have authentic Emacs clones.

Here is a migration table based on the plugins that I use.

Vim plugin	Emacs alternative	Note
Vundle.vim	use-package	Emacs has a built-in package manager
vim-fugitive	magit
vim-rhubarb	browse-at-remote
vim-sleuth	dtrt-indent
vim-fugitive-pagure	browse-at-remote
nerdtree	neotree	Use dired instead, trust me
fzf.vim	helm	Much more then just fzf
tcomment_vim	evil-commentary
matchit	evil-matchit
delimitMate	smartparens
Vim-Jinja2-Syntax	jinja2-mode
vim-markdown	markdown-mode
syntastic	company-mode
vimwiki	org-mode
goyo.vim	writeroom
base16-vim	base16-theme
vim-css-color	rainbow-mode

For migrating the Vim settings themselves, you will need to read the manual (or my Emacs config) because it is not possible to cover here. However, I still think it would be pretty amazing if we could write some package that would parse ~/.vimrc and print Elisp code for doing the same thing in Evil. Anyone wants to collaborate on that?

So far, this whole thing may seem like a chore with no benefits. Well, buckle up.

IDE on steroids

During the last year, I was basically living in Emacs. At first, it replaced only my IDE but then Emacs quickly became my RSS aggregator, task manager, IRC client, Email client and kept consuming all standalone programs. All of them now share the same configuration language, keybindings, theme, etc. Thanks to Evil, those keybindings happen to be Vim-like.

I love this great paradox that thanks to Emacs, I have more Vim in my life.

As a consequence, all of these applications are integrated together and it is possible to seamlessly create tasks based on emails, paste short pieces of code into IRC, viewing a git history and jumping to the changed files, sending a SQL query to a coworker via email (the possibilities are endless) without ever touching a mouse or even needing the system clipboard. It’s all in the Vim/Evil registers, so only yanking and pasting like a true gentleman.

For the past ten years, I was trying to achieve such a setup using a combination of Tmux, Vim, Mutt, Weechat, and other tools but the experience was vastly different. No matter what, it felt like using several single-purpose programs, each of them isolated in its own bubble and with no way of cooperating with each other.

Literate config

Since the operating system becomes just a bootloader for Emacs (and maybe a web browser), it means that every application is used and configured within Emacs. Luckily for us, it has the best configuration apparatus ever invented. Remember the literate programming paradigm, which was only briefly noted in a school curriculum as a slow, retarded cousin of procedural and object-oriented paradigms? It turns out to be pretty fucking awesome after all.

Emacs configuration can be written in an Org-mode document and can look like a book explaining various features, settings, and customizations, with thorough code sample coverage. Quite literally because the document can be exported as a web page or stylized PDF. See my config on GitHub to get the idea.

The Org configuration file renders beautifully on GitHub and possibly other Git forges.

Email client, finally

Accessing email in the terminal was a recurring nerd-fantasy of mine ever since I started using GNU/Linux in 2008. Up until very recently, all my attempts of achieving this holy grail rendered futile.

Finally, I could cross this off my list thanks to a combination of mbsync and mu4e. There is nothing much to say. It is an email client. Inside of Emacs. But for some undefinable reason, it is the greatest thing ever invented.

The hardest part isn’t the email client configuration itself but rather synchronization with IMAP (in the age of two-factor authentication). If you are interested, I wrote a whole article on this topic - Synchronize your 2FA Gmail with mbsync

IRC with Vim keybindings

Weechat is an exceptionally good piece of software but to be honest I was aching to replace it for quite some time. There just wasn’t a better alternative. My frustrations were caused solely by two pitfalls.

Copy-pasting from Tmux + Weechat combo is god-awful and borderline psychotic. Imagine Weechat window vertically split into multiple panes. Since it is an application running in a single terminal window, the terminal has no perception of any separation, splits, or panes that are displayed within. Selecting a multi-line text with a mouse, therefore, isn’t limited to the desired pane but rather to everything that stands in the way. The copied text then contains mixed messages from multiple chat windows, ASCII symbols that were used as separators, timestamps, and all. Copy-pasting from Tmux + Weechat is quite alright if you are aware of rectangle selection.

Apart from this, I was always uncomfortable with Weechat’s approach to generated configuration files. Even though it is generally looked at as a feature allowing to configure Weechat from within itself, saving its current state, and eliminating the need to edit the configuration file in its written form. I never grew accustomed to this paradigm and prefer to edit configuration files by hand.

Circe is in many aspects inspired by Weechat and inherits some of its traits while eradicating the pitfalls beyond perfectly. Circe chat windows are standard Emacs buffers with full Vim emulation and shared registers with the whole ecosystem. Lisp configuration also surpasses anything else we could wish for.

Magit

Magit is one of the tools that I would refuse to believe I might enjoy using. What’s the point with git diff, git add, and git commit commands already hard-wired in my muscle memory in such a manner, that executing them resembles more of an involuntary spasm rather than typing words.

Yet, I am forever hooked to Magit now. One shall never stage a whole file before checking its git diff (because of possible unwanted changes). That’s two commands per file, all the time. Whereas Magit provides a compact view of files and their changes at once. Staging a whole file or its chunks is then a matter of pressing one key.

Committing, pushing, iterating over a series of commits, blaming, rebasing, everything is a bit more convenient. I still use the git command for more complicated operations though.

Emacs in browser

As a software engineer, I frequently engage in technical discussions on various git forge sites (such as GitHub, Pagure.io), and Reddit. Consequently, I write comments containing pieces of code on daily basis (either being code reviews and suggestions, bug reproducers, or proposed solutions).

My workflow usually involved firing up the good ol’ text editor, pre-formatting the code, and copy-pasting it into the comment. Omitting this step and editing the code directly in the web browser is doable but especially adjusting whitespace sucks so much, that you eventually end up spawning a text editor and regretting you didn’t do it in the first place.

Edit with Emacs is a handy convenience tool that provides a key binding to make a temporary Emacs frame with the contents of your textarea. After you are done, it pipes the output back to the browser.

There is nothing special about this functionality in regards to Emacs and there surely is a plugin compatible with any text editor. On the contrary, I think Emacs lags behind Neovim in this scenario, which is able to transform a textarea into a Neovim instance directly in the web browser thanks to Firenvim.

Synchronize your 2FA Gmail with mbsync

Fri, 12 Feb 2021 00:00:00 +0000

In comparison to graphical email applications, configuring the command-line clients can be a needlessly painful experience. Not because of the client configuration itself but rather finding out the proper IMAP settings for your account. Personally, I spent more hours on moving my mail into Emacs (and previously into Mutt), than I care to admit. And in the end, it turned out that the only real obstacle was figuring out, how to get the freaking synchronization with Gmail working. Let me share my findings to potentially ease the pain for you.

TL;DR: Jump to the last section Gmail with 2FA

The obligatory intro on email clients

If you use, or ever tried to use some of the mainstream email clients such as Thunderbird, Evolution, Geary, or some mobile phone client (I am not a mobile phone nerd, so I don’t know any of them. I just have some Gmail thingy there), you might have formed an idea what an email client is supposed to do - download your messages, index them, filter them, and let you interactively work with them. And of course, allowing you to send messages as well. There is nothing groundbreaking about this, that’s just how we use email, and therefore what we expect email clients to do.

Command-line (or rather text-based) email clients do much less and delegate a lot of work to a series of other small tools (aka the Unix philosophy). Typically, we have a separate program for simply downloading the mail from a server. There are many options, such as mbsync from isync package, OfflineIMAP, getmail, etc. Their only job is to download the mail and save it (one message per file) to your disk. This can be useful on its own, e.g. for backing-up your messages in case the email provider shuts down its business.

Optionally, it’s up to us to configure a spam filter either on the server-side (e.g. in Gmail settings) or on the client by using procmail, Bogofilter, or SpamAssasin. The next building block is indexing. This is generally not true for everybody but after years of countless mailing list subscriptions, many of us have our inboxes flooded with thousands or hundreds of thousands of messages. Directory-based storages just ain’t gonna cut it. We need some kind of a real database with indexes. For this, we can use tools like Mu, or Notmuch.

Finally, we are getting to the fun part, which is displaying, reading, and interacting with email. This is the most discussed and tutorial-covered link in the chain. And while exciting, and hacker-ish looking on screenshots, the configuration revolves mainly around color schemes and key bindings, which is quite intuitive and also not that big of a deal when using some default settings. Anyway, we can pick from the following frontends - Mu4e, Notmuch, Mutt, NeoMutt, Gnus, Alpine, and probably from a bunch of less-known clients as well. For sending email messages, one would expect to also utilize some specialized tool (and it certainly is possible) but usually, it is handled by the frontend program itself.

As you noticed, there is a lot of small, interchangeable utilities in play. Some of them optional, some of them even more optional. The categories I presented are not strictly defined and some tools overlap across multiple categories. It’s up to you to choose and put those lego blocks together and build your own email setup.

Today we are going to focus on the very first category, which is downloading email messages to your computer. We are going to utilize mbsync command, and we are going to set it up for your enterprise 2FA Gmail account. Since the result is a bunch of downloaded files (which is not very impressive), and every cool blog post should have at least one screenshot, I am going to jump a few steps forward and show you how the end-game might look like once you manage to successfully download your mail and open it in Emacs.

This beautiful configuration is not mine. The screenshot is taken from the rougier/mu4e-dashboard repository.

A less hostile provider than Gmail

First, please install the isync package, so we can get this out of our way. Use the package manager provided by your GNU/Linux distribution, on Fedora I would do

dnf install isync

Configuring multiple accounts in mbsync is trivial, and Gmail IMAP is a mess. Therefore, I would recommend setting up an account from a different email provider first (that is if you have the option) and learn how mbsync works. Let’s create the ~/.mbsyncrc configuration file and insert the following settings.

IMAPStore foo-remote
Host imap.foo.com
SSLType IMAPS
User frostyx@foo.com
Pass supercomplicatedpassword

MaildirStore foo-local
Path ~/Mail/foo/
Inbox ~/Mail/foo/INBOX
Subfolders Verbatim

Channel foo
Master :foo-remote:
Slave :foo-local:
Create Both
Expunge Both
Patterns *
SyncState *

A more complicated version of this snippet can be found on ArchWiki and countless blogs (1, 2, 3, 4, …), so I am not going to thoroughly describe the settings here. I would by paraphrasing the manpage anyway. Let’s just vaguely say, that the first section describes how to access your email account provided by some third-party. The second section describes how to store the downloaded messages on our computer. Finally, the last section configures what messages should be synchronized between those two, he said while pretending to understand it.

Replace all foo symbols in the snippet with some short label for your email account (e.g. personal, work, test, …) and properly set the Host, User, and Pass values in the first section. Then you should be able to successfully run the following command

# Use the short email label you chose before
mbsync -V foo

There is not going to be any successful message in the output. Don’t panic, seeing a bunch of Opening and Synchronizing... lines means that it works fine. To make sure, list the mail directory

ls -1 ~/Mail

Because we don’t want to store our super-secret email passwords in plaintext, we should stash them into some keychain and let mbsync how to get them. Don’t even think about using the gpg command, that’s more complicated than rocket science. Use pass instead!

# Initialize the keychain, you will run this just once in your life
pass init <gpg-id or email>

# Insert the password for your email account
pass insert email/frostyx@foo.com

# And print it to the terminal to make sure it was stored properly
pass email/frostyx@foo.com

This is all you need to know about pass (although it has some amazing features such as storing passwords to git, which is worth checking out). Now, remove the Pass line from your ~/.mbsyncrc config and use this one instead.

PassCmd "pass email/frostyx@foo.com"

Make sure that mbsync -V foo still works, even when using a password from the keychain.

Gmail IMAP sucks

Before moving to the Gmail configuration, I would like to make a disclaimer - Gmail IMAP support has limitations, idiosyncrasies, and to be scientifically precise, it just sucks. Please have patience.

Each tag is stored as a folder, therefore messages with multiple tags will be downloaded multiple times (into each folder)
The daily download limit is 2500 MB, so it may take several days for you to initially download your mail
Your Gmail password will not work, an App Password is required
Emails are visible in the web UI for a couple of minutes before they appear in IMAP. This is either an issue on my side or there is some delay

All of these are minor inconveniences, that can be workarounded, solved, or lived with. It’s just something to keep in mind.

Gmail with a plain password

Armored with bulletproof patience and unending determination, we shall continue to mbsync configuration for Gmail. Now, if you think “perhaps I should try a personal Gmail account without 2FA, it is going to be easier”, please smother the idea in this instant. It is not possible. Please repeat after me, it is not possible. I refused to believe, tried over and over, failed miserably, over and over, and ended up configuring 2FA anyway. Please save yourself the time and the pain.

Log-in to your Gmail account and click to your profile picture in the top-right. Then continue to “Manage your Google Account”. In the left menu, click on “Security”, and turn on the 2-Step Verification.

Now we need to generate an App Password. Click on “2-Step Verification”, then at the bottom of the page, you will find “App passwords” section. When opening it, you will see this.

Use whatever app name you want and click “Create”.

After closing this page, you won’t be able to display it again, so please write down the generated password. We will put it to the keychain in a minute.

Isn’t 2FA like a 2FA

Just a small digression in case you skipped the previous section because you already have 2FA configured. The Google documentation about App Passwords explicitly says:

App Passwords can only be used with accounts that have 2-Step Verification turned on.

The statement is correct but it can be a bit misleading. In fact, this happened to be one of the biggest pain-points for me. My corporate email uses Single-Sign-On authentication through Kerberos. The login page looks like this.

As you can see, my password consists of a PIN, which is a persistent passphrase, followed by a TOKEN - a randomly generated short string from a small device, that I wear in my wallet. In my view, this counts as a 2-Step verified account and should pass the constraint for using App Passwords. This is absolutely not true though. Please follow the same exact steps that we did in the Gmail with a plain password section.

Gmail with 2FA

We finally managed to jump through all the necessary hoops and tweaked our Google account. Now we can get back to the mbsync configuration. Let’s take the snippet from the Less hostile provider than Gmail section and make just a couple of changes.

IMAPStore gmail-remote
Host imap.gmail.com
SSLType IMAPS
AuthMechs LOGIN
User foo@gmail.com
PassCmd "pass email/foo@gmail.com"

MaildirStore gmail-local
Path ~/Mail/gmail/
Inbox ~/Mail/gmail/INBOX
Subfolders Verbatim

Channel gmail
Master :gmail-remote:
Slave :gmail-local:
Create Both
Expunge Both
Patterns * !"[Gmail]/All Mail" !"[Gmail]/Important" !"[Gmail]/Starred" !"[Gmail]/Bin"
SyncState *

The only fields that you need to modify are User and PassCmd. Your password is the generated App Password that we transcribed from the yellow stripe in the previous section - Gmail with a plain password. Don’t forget to stash it to the pass keychain.

pass insert email/foo@gmail.com

My first Emacs contribution

Sun, 15 Mar 2020 00:00:00 +0000

You may already notice, that I like to customize and improve the tools that I use daily, share the configuration with others and write blog posts about what I learned while doing it. On today’s menu, Emacs and its packages.

This year I finally dared to migrate from Vim to Emacs - utilizing Evil of course. I wasn’t capable of magically growing new extremities from my body to constantly press all the Emacs chords. Neither was I capable of doubling my brain capacity to even remember them.

Thank you National Geographic and Pinterest for this rare depiction of a vanilla Emacs user in his native environment.

Even though I was using Vim for almost ten years and have a highly customized and personalized configuration with a lot of enabled plugins, the migration was quite smooth. A general rule of thumb is, for every possible Vim plugin, there is a better or worse (still sufficient) Emacs alternative. The only hard thing is to find them. Maybe let’s discuss this topic in another article.

There is an exception to this rule though - one-man show plugins, particularly those you’ve written and maintain yourself. In my case, it is a FrostX/vim-fugitive-pagure plugin, which provides a pagure.io support for :Gbrowse command. I collaborate with my coworkers remotely and make frequent use of it. There is was no Emacs alternative so I missed the feature since migration. I finally decided to implement a pagure.io support for browse-at-remote and here follows what I learned from my first Emacs contribution.

Disclaimer: As I repeatedly mentioned, this was my first experience with writing packages for Emacs, therefore this is not an authoritative manual of how things work and should be done.

Still ranting

Because of the very nature of Lisp language (its ability to modify programs while they run) and Emacs philosophy to consume everything, things get very weird very fast. It is a vast underestimation to say, that workflows when using Emacs are unconventional, compared to the rest of the world. Although it is a positive thing, the problem is that there seems to be very little material showing the real-life usages of Emacs. In this case, explaining how to proceed when writing one’s own package. Don’t get me wrong, there is powerful documentation within Emacs but in 2020 I would expect a lot of youtube videos, blogs, reddit posts, etc giving an insight to custom methods of individual developers.

Assimilate everything

A dream of advanced Emacs users (Emacsites) is to start their computer, run Emacs, and never ever leave it. If God forbids there is a need to launch and interact with any other application, we need to unite and implement the functionality within an Emacs package. Using other applications is against the beliefs of the Church Of Emacs.

For new users, this concept is initially very hard to grasp, so it makes sense to progress gradually. As a newbie myself, I tried to utilize as many command-line tools outside of Emacs and ended up with the following workflow for contributing to an existing package.

My workflow

First, we will need Cask, which is a management tool for Emacs projects (packages). Please see the installation manual. It is a bit disappointing that there are no official packages for Linux distributions, we definitely need to repent from our sins and create a cask package for Fedora. But until then, we need to go with:

curl -fsSL https://raw.githubusercontent.com/cask/cask/master/go | python
export PATH="$HOME/.cask/bin:$PATH"
cask upgrade-cask

Clone a package that you are interested in and change your working directory:

git clone https://github.com/rmuslimov/browse-at-remote
cd browse-at-remote

Install all the package dependencies:

cask install

Hopefully, the project has at least some unit tests. In my case, the author used a library called ERT. I don’t know how many alternatives are there but it seems like a go-to option. You can simply run all the tests with:

cask exec ert-runner

Now, let’s proceed with some test-driven development until the desired feature is finished. I spend several hours trying to figure out how to have an interactive debugger using edebug but unsuccessfully. Therefore, I needed to get by using just (print foo) to evaluate expressions.

Once the feature is done, let’s see it in action. Up until this point you probably included the package in your init.el like this:

(use-package browse-at-remote
  :ensure t)

Temporarily remove it and load the enhanced package from your local directory:

(add-to-list 'load-path (expand-file-name "~/git/browse-at-remote"))
(require 'browse-at-remote)

And reload your Emacs configuration by pressing:

M-x load-file RET

or restarting the Emacs entirely. You should be done now. If not, repeat a necessary subset of previous steps.

C-x C-c :wq

Emacs - Browse At Pagure

Sat, 07 Mar 2020 00:00:00 +0000

TL;DR: Work in Emacs, select some lines, press M-x browse-at-remote to open it on pagure.io, and share the URL.

Recently I have published an article about :Gbrowse command in Vim, that allows you to open a current file or line selection on a remote hosting platform, such as GitHub, GitLab, etc. The command is provided by tpope/vim-fugitive plugin, you should definitely check it out. Other contributors already implemented support for various hosting sites, so I decided to go ahead and create vim-fugitive-pagure extension to support also pagure.io.

One of the annual goals that I set for myself was to finally migrate to Emacs. Everything is going well, the only thing that I am missing since then is the :Gbrowse feature. To be precise, there is a browse-at-remote package providing more than sufficient alternative, but Pagure support is missing. Well, not anymore!

Usage

While working with your code in Emacs, select one or more lines and run M-x browse-at-remote.

Installation

The package is available on MELPA.

(use-package browse-at-remote
  :ensure t)