Problem

I was reacquainting myself with gRPC by creating a basic “Hello, world!” project when I ran into a problem: I wanted to put the generated code files in their own generated_grpc directory, but then imports within those files failed:

ModuleNotFoundError: No module named 'helloworld_pb2'

It turns out that protobuf simply does not generate correct imports in this situation.

Solution

LLMs suggested adding an option python_package line to my .protos file, but that option is not supported. Actual solutions include putting the generated files in the project root directory and wrapping the protobuf generation command in a script that fixes the imports.

The approach I landed on is to add a few lines to the __init__.py file within the generated_grpc/ package:

from importlib import import_module
import sys

_messages_module = import_module(".helloworld_pb2", package=__name__)

sys.modules.setdefault("helloworld_pb2", _messages_module)

I prefer this approach to the alternatives because it is only annoying once. It allows me to use standard commands and a sensible directory structure. The only cost is a bit of ugliness in a file that I will probably never look at again. I can live with that!