Adding to the great suggestions above, here’s another option for those times when external libraries or io.open() aren’t feasible.
Manual Encoding Handling for Python 2
If you cannot rely on io.open() or compatibility libraries, you can manage encoding yourself. While slightly less elegant, this approach is straightforward and works reliably.
import sys
if sys.version_info[0] < 3:
# Python 2.x
with open(fname, "rb") as f:
content = f.read().decode("utf-8")
else:
# Python 3.x
with open(fname, "rt", encoding="utf-8") as f:
content = f.read()
# Use the 'content' variable for further processing
In this method, you open the file in binary mode (rb) in Python 2, then manually decode the content to utf-8. For Python 3, you use the native open() with the encoding parameter. While this introduces a bit more boilerplate, it ensures that your python open encoding logic is explicit and adaptable.