Issue #8622: Add PYTHONFSENCODING environment variable to override the · python/cpython@94908bb · GitHub
Skip to content

Commit 94908bb

Browse files
author
Victor Stinner
committed
Issue #8622: Add PYTHONFSENCODING environment variable to override the
filesystem encoding. initfsencoding() displays also a better error message if get_codeset() failed.
1 parent 56ab01b commit 94908bb

7 files changed

Lines changed: 93 additions & 34 deletions

File tree

Doc/using/cmdline.rst

Lines changed: 12 additions & 3 deletions

Doc/whatsnew/3.2.rst

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,15 @@ Major performance enhancements have been added:
232232

233233
* Stub
234234

235+
236+
Unicode
237+
=======
238+
239+
The filesystem encoding can be specified by setting the
240+
:envvar:`PYTHONFSENCODING` environment variable before running the intepreter.
241+
The value should be a string in the form ``<encoding>``, e.g. ``utf-8``.
242+
243+
235244
IDLE
236245
====
237246

Lib/test/test_pep277.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@
4343

4444
# Is it Unicode-friendly?
4545
if not os.path.supports_unicode_filenames:
46-
fsencoding = sys.getfilesystemencoding() or sys.getdefaultencoding()
46+
fsencoding = sys.getfilesystemencoding()
4747
try:
4848
for name in filenames:
4949
name.encode(fsencoding)

Lib/test/test_sys.py

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -863,31 +863,47 @@ def test_pythontypes(self):
863863
def test_getfilesystemencoding(self):
864864
import codecs
865865

866-
def check_fsencoding(fs_encoding):
866+
def check_fsencoding(fs_encoding, expected=None):
867867
self.assertIsNotNone(fs_encoding)
868868
if sys.platform == 'darwin':
869869
self.assertEqual(fs_encoding, 'utf-8')
870870
codecs.lookup(fs_encoding)
871+
if expected:
872+
self.assertEqual(fs_encoding, expected)
871873

872874
fs_encoding = sys.getfilesystemencoding()
873875
check_fsencoding(fs_encoding)
874876

875-
# Even in C locale
877+
def get_fsencoding(env):
878+
output = subprocess.check_output(
879+
[sys.executable, "-c",
880+
"import sys; print(sys.getfilesystemencoding())"],
881+
env=env)
882+
return output.rstrip().decode('ascii')
883+
876884
try:
877885
sys.executable.encode('ascii')
878886
except UnicodeEncodeError:
879887
# Python doesn't start with ASCII locale if its path is not ASCII,
880888
# see issue #8611
881889
pass
882890
else:
891+
# Even in C locale
883892
env = os.environ.copy()
884893
env['LANG'] = 'C'
885-
output = subprocess.check_output(
886-
[sys.executable, "-c",
887-
"import sys; print(sys.getfilesystemencoding())"],
888-
env=env)
889-
fs_encoding = output.rstrip().decode('ascii')
890-
check_fsencoding(fs_encoding)
894+
try:
895+
del env['PYTHONFSENCODING']
896+
except KeyError:
897+
pass
898+
check_fsencoding(get_fsencoding(env), 'ascii')
899+
900+
# Filesystem encoding is hardcoded on Windows and Mac OS X
901+
if sys.platform not in ('win32', 'darwin'):
902+
for encoding in ('ascii', 'cp850', 'iso8859-1', 'utf-8'):
903+
env = os.environ.copy()
904+
env['PYTHONFSENCODING'] = encoding
905+
check_fsencoding(get_fsencoding(env), encoding)
906+
891907

892908
def test_setfilesystemencoding(self):
893909
old = sys.getfilesystemencoding()

Misc/NEWS

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,9 @@ What's New in Python 3.2 Alpha 2?
1212
Core and Builtins
1313
-----------------
1414

15+
- Issue #8622: Add PYTHONFSENCODING environment variable to override the
16+
filesystem encoding.
17+
1518
- Issue #5127: The C functions that access the Unicode Database now accept and
1619
return characters from the full Unicode range, even on narrow unicode builds
1720
(Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and others). A visible difference

Modules/main.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ PYTHONHOME : alternate <prefix> directory (or <prefix>%c<exec_prefix>).\n\
9999
The default module search path uses %s.\n\
100100
PYTHONCASEOK : ignore case in 'import' statements (Windows).\n\
101101
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.\n\
102+
PYTHONFSENCODING: Encoding used for the filesystem.\n\
102103
";
103104

104105
FILE *

Python/pythonrun.c

Lines changed: 43 additions & 22 deletions

0 commit comments

Comments
 (0)