Delete Python’s __pycache__ directories

Intro

First things first, actually I don’t want to remove them at all, I just want to get rid of them forever from my workflow. We will see that later.

This is a compherensive guide to deal with Python’s __pycache__ directories and .pyc files.

__pycache__ is a directory which is containing Python 3 bytecode compiled and ready to be executed.

From the official python tutorial Modules:

To speed up loading modules, Python caches the compiled version of each module in the pycache directory under the name module.version.pyc, where the version encodes the format of the compiled file; it generally contains the Python version number. For example, in CPython release 3.6 the compiled version of spam.py would be cached as pycache/spam.cpython-36.pyc.

From Python doc Programming FAQs:

When a module is imported for the first time (or when the source file has changed since the current compiled file was created) a .pyc file containing the compiled code should be created in a pycache subdirectory of the directory containing the .py file. The .pyc file will have a filename that starts with the same name as the .py file, and ends with .pyc, with a middle component that depends on the particular python binary that created it.

So they are necessary and should not be deleted usually.

Solution - Default

There are a lot elements to “remove” __pycache__ directories and .pyc files.

Git

The first and foremost thing is that we need to do ignoring them in our project’s git trees. Add them into our .gitignore file(s), locally and/or globally.

# .gitignore

# Pyhon byte-compiled / optimized files
__pycache__/
*.py[cod]
*$py.class

Editor

However this is not enough, even though we already added them into .gitignore file(s), we will still see them in our IDEs and explorers. In order to prevent that I use below configurations per editor.

VSCode

// settings.json

"files.exclude": {
     "**/__pycache__": true
}

Neovim

For nvim-tree plugin:

-- init.vim or init.lua or nvimtree.lua

local nt = lvim.builtin.nvimtree

-- Ingore/Exclude directories/files patterns
nt.setup.filters.custom = {
    "__pycache__",
}

Vim

For NERDTree plugin:

" .vimrc or nerdtree.vim

" NERDTree Plugin"

" Ingore/Exclude directories/files patterns
let NERDTreeIgnore = [
            \ '\.pyc$',
            \ '^__pycache__$',
            \]

Docker

There is a environment variable which is responsible to disable to write .pyc files named PYTHONDONTWRITEBYTECODE.

# Dockerfile

ENV PYTHONDONTWRITEBYTECODE 1

From Python’s official doc bytecode:

PYTHONDONTWRITEBYTECODE

If this is set to a non-empty string, Python won’t try to write .pyc files on the import of source modules. This is equivalent to specifying the -B option.

It’s not usually an optimum solution to set this variable in other than containers. Since you run a single python process in containers, which does not spawn other python processes during its lifetime, then there is no disadvantages in doing that.

Delete

After all of the above, we may still encounter some cases to delete all of __pycache__ directories. Here is the shell command to remove them:

In the root of your project run:

$ find . | grep -E "(/__pycache__$|\.pyc$|\.pyo$)" | xargs rm -rf

$ find . | grep -E "(/__pycache__$|\.pyc$|\.pyo$)"

You can add this command as an alias for easy access:

alias rm-pycache='find . | grep -E "(/__pycache__$|\.pyc$|\.pyo$)" | xargs rm -rf'

Also you would like to add it as a make targets in your project’s Makefile for ci/cd pipelines.

rm-pycache:
    find . | grep -E "(/__pycache__$|\.pyc$|\.pyo$)" | xargs rm -rf

Lastly there is a python library just for this delete cause: pyclean. In my opinion it’s an overkill but you may take a look.

Solution - New

This is my current solution about the issue.

Beginning from Python 3.8 you can use PYTHONPYCACHEPREFIX environment variable to use a global cache directory to avoid them created under your projects:

$ export PYTHONPYCACHEPREFIX="$HOME/.cache/cpython/"

By this environment variable Python won’t create any __pycache__ directory in your projects, instead it will put all of them under ~/.cache/cpython/ directory.

From Python’s official doc prefix:

PYTHONPYCACHEPREFIX

If this is set, Python will write .pyc files in a mirror directory tree at this path, instead of in pycache directories within the source tree. This is equivalent to specifying the -X pycache_prefix=PATH option.

New in version 3.8.

You can add this environment variable in one of the configuration file of yours: .zshrc, .zshenv, .bashrc, .bash_profile, .profile, .zprofile etc.

At last they are out of my sight.

All done!