Python interpreter doesn't seem to support readline error in Emacs on MacOS

I have historically used Python in Emacs via Conda. I don't especially enjoy using Conda and much prefer a virtualenv workflow. So, for a recent project, I used virtualenv (actually uv venv, and I recommend checking out uv for your package management and virtual environment needs!).

When I tried to use Python in Emacs, however, I got the warning above. This wasn't catastrophic, but it was annoying. The Python interpreter would return this before the results of whatever code I was trying to run:

__PYTHON_EL_eval("try:\n    with open('/var/folders/m_/nbhhpg550yl539yhlgch8qqr0000gp/T/babel-FML4fd/python-qrbxMD') as f:\n        exec(compile(f.read(), f.name, 'exec'))\nexcept:\n    raise\nfinally:\n    print('org_babel_python_eoe')", "<string>")

and would echo all of my inputs in the interactive shell. Not catastrophic, but very annoying.

I am not the first person to encounter these issues. Here's a sampling of others…

• Python in org babel will not work
• Your ‘python-shell-interpreter’ doesn’t seem to support readline
• Emacs 26, Mojave, elpy, readline
• emacs: remove __PYTHONELeval message
• __{PYTHONELevalfile} prints in interpreter when sending python file from buffer using C-x C-l
• __PYTHONELeval printouts in ipython REPL

And that list is by no means comprehensive.

After trying to troubleshoot that issue for a while and finding no solution, I recalled that I had no issues when using Python installed via Conda. So what was the difference between the two?

Conda:

>>> import readline
>>> readline.__doc__
'Importing this module enables command line editing using GNU readline.'

virtualenv:

>>> import readline
>>> readline.__doc__
'Importing this module enables command line editing using libedit readline.'

Alright, now we're getting somewhere. Given that the original warning message talked about readline, and that the (working) Conda Python installation uses GNU readline while the (non-working) virtualenv version uses libedit, it seems like we have to find a way to get the virtualenv version to use GNU readline instead.

I tried a bunch of different things to try to force the virtualenv to use GNU readline, to no avail. Then I found the gnureadline package on PyPi, which seemed promising.

The gnureadline Python package explains the issue:

If you install Python on macOS via a popular open-source package manager such as Homebrew or MacPorts, you'll get a readline extension module that calls libedit internally (even though it's confusingly still called "readline"!).

While a lot of effort has gone into making GNU Readline and Editline interchangeable within Python, they are not fully equivalent. If you want proper Readline support, this module provides it by bundling the standard Python readline module with the GNU Readline source code, which is compiled and statically linked to it.

And it provides the very straightforward solution:

1. Install gnureadline with pip install gnureadline.
2. Run the included override script with python -m override_readline.

The gnureadline docs suggest the following usage pattern:

try:
    import gnureadline as readline
except ImportError:
    import readline

However, when it comes to setting up the shell for use with Emacs, running this within the shell would be too late. We actually need to run this when the site module imports customization modules when the Python interpreter is starting.

How do we do this? Again from the gnureadline docs:

The script [the above override_readline script] first tries to add the workaround to usercustomize and then falls back to sitecustomize if the user site is not enabled (for example in virtualenvs). If you want to go straight to sitecustomize, add the standard -s option.