cat <<EOF >> .pre-commit-config.yaml
- repo: local
hooks:
- id: utf8_error_test
name: UTF8 cmd output Error test
language: script
entry: echo_problematic_char.sh
files: ''
EOF
cat <<EOF >echo_problematic_char.sh
#!/bin/bash
echo 'Problematic character: �'
exit 1
EOF
chmod u+x echo_problematic_char.sh
git add .pre-commit-config.yaml
pre-commit run utf8_error_test
UTF8 cmd output Error test.........................................................................................................................................An unexpected error has occurred: UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in position 23: invalid start byte
Check the log at ~/.pre-commit/pre-commit.log
An unexpected error has occurred: UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in position 23: invalid start byte
Traceback (most recent call last):
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/error_handler.py", line 34, in error_handler
yield
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/main.py", line 144, in main
return run(runner, args)
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/commands/run.py", line 185, in run
return _run_hooks(repo_hooks, args, write, environ)
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/commands/run.py", line 117, in _run_hooks
retval |= _run_single_hook(hook, repo, args, write, skips)
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/commands/run.py", line 88, in _run_single_hook
retcode, stdout, stderr = repo.run_hook(hook, filenames)
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/repository.py", line 123, in run_hook
self.cmd_runner, hook, file_args,
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/languages/script.py", line 19, in run_hook
retcode=None,
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/prefixed_command_runner.py", line 41, in run
return cmd_output(*replaced_cmd, __popen=self.__popen, **kwargs)
File "/home/lucas/.local/lib/python2.7/site-packages/pre_commit/util.py", line 153, in cmd_output
stdout = stdout.decode(encoding)
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in position 23: invalid start byte
The problematic UTF8 character is '»' (U+00BB), and it originally came from a maven command stderr output.
I won't have time to try to solve this issue right now, but I'll try to get a look at it soon.
I initially mentioned this issue in #207, but only found an easy-to-reproduce test this week.
I confirm this bug is still present in pre-commit v0.5.3 :
The command output:
And the error log:
The problematic UTF8 character is '»' (U+00BB), and it originally came from a maven command stderr output.
I won't have time to try to solve this issue right now, but I'll try to get a look at it soon.