Yeah, this issue is all too common when dealing with non-ASCII characters in Python. The best fix? Explicitly specify utf-8 encoding while using pandas to CSV. This ensures that all Unicode characters are handled properly:
import pandas as pd
df = pd.DataFrame({'col1': ['α', 'β', 'γ'], 'col2': [1, 2, 3]})
df.to_csv('out.csv', encoding='utf-8', index=False) # Specify UTF-8 encoding
This should take care of most encoding issues. But if you’re dealing with an environment where you can’t use UTF-8 for some reason, there are other workarounds too.