{{ message }}
Repository navigation
Conversation
With --script py the command templates read the helpers' JSON from a captured stdout. On Windows that stream uses the ANSI code page, and ensure_ascii=False made setup_plan, check_prerequisites, setup_tasks and resolve_template --json raise UnicodeEncodeError and print nothing for the stock constitution and tasks templates and for non-Latin feature names. github#3890 and github#3892 fixed the text output of two of them; this covers the JSON paths. Callers parse the JSON, so the decoded values are the same. Assisted-by: Codex CLI (model: gpt-6.1-sol, autonomous) Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The new setup-plan assertion expects LF-only bytes, but its fixture produces CRLF on Windows.
1 open issue
What changed in this PR
Makes Python helper JSON safe for legacy Windows stdout encodings while preserving decoded Unicode values.
Changes:
- Enables ASCII escaping in four JSON writers.
- Adds regression tests for Unicode paths and template content.
Validation used isolated standard-library checks; project tests were not run.
AI disclosure: GitHub Copilot (model unknown, autonomous), reviewing for the requester; review and suggested fix are AI-generated.
| File | Description |
|---|---|
| tests/test_setup_tasks_python_parity.py | Tests stock template JSON under cp1252. |
| tests/test_setup_plan_python_parity.py | Tests Unicode feature paths under cp1252. |
| tests/test_resolve_template_python_parity.py | Tests Unicode template JSON under cp1252. |
| tests/test_check_prerequisites_python_parity.py | Tests Unicode prerequisite paths under cp1252. |
| scripts/python/setup_tasks.py | Escapes non-ASCII JSON characters. |
| scripts/python/setup_plan.py | Escapes non-ASCII JSON characters. |
| scripts/python/resolve_template.py | Escapes non-ASCII JSON characters. |
| scripts/python/check_prerequisites.py | Escapes non-ASCII JSON characters. |
Review effort: Max
| assert Path(payload["FEATURE_DIR"]) == feature | ||
| assert Path(payload["FEATURE_SPEC"]) == feature / "spec.md" | ||
| assert Path(payload["IMPL_PLAN"]) == feature / "plan.md" | ||
| assert (feature / "plan.md").read_bytes() == TEMPLATE_BODY.encode("utf-8") |
…late The fixture writes plan-template.md with write_text(), which gives CRLF on Windows, and setup_plan copies those bytes. Comparing against the LF constant failed on Windows even with the JSON fix in place. Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Contributor
Author
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
With
--script py, the command templates run the Python helpers with--jsonand read their stdout. On Windows a captured stdout uses the ANSI code page (cp1252 on most Western setups), and_json_line/json.dumpswrite withensure_ascii=False. Any character cp1252 can't encode then raisesUnicodeEncodeError, the script exits 1, and the agent gets no JSON:resolve_template.py constitution-template --json(/speckit.constitution) fails with the stock template, which contains→.setup_tasks.py --json(/speckit.tasks) fails with the stock tasks template, which contains emoji.setup_plan.py --json,check_prerequisites.py --jsonandsetup_tasks.py --jsonfail for any feature whose name isn't Latin. Since [bug-fix] Fix non-latin-feature-names: preserve Unicode feature names #4780,create_new_feature.pykeeps names like001-Додати-експорт-звітів, and these scripts print that path.#3890 and #3892 fixed the same crash in the text output of
check_prerequisites.pyandsetup_tasks.py: the check marks fall back to[OK]/[FAIL]when stdout can't encode them. The--jsonpaths, which the command templates use, still crash. This PR writes their JSON withensure_ascii=True. Callers parse the JSON, so they get the same values; only the bytes on the wire are escaped. A UTF-8 stdout is unaffected.I didn't change
create_new_feature.py, which already writes UTF-8 since #4780. Forcing UTF-8 the same way in these four scripts would bypass the text-mode fallback from #3890/#3892, so I kept to the JSON encoding. I left outresolve_template.pywithout--json(raw template text): no shipped command calls it that way.I haven't seen a user report of this. It came up while reading the Python helpers.
Repro on Linux, with
PYTHONIOENCODING=cp1252standing in for a captured Windows stdout. The project is fromspecify init <dir> --integration codex --script py, thencreate_new_feature.py --json "Додати експорт звітів":main(505781d)setup_plan.py --jsonUnicodeEncodeError: 'charmap' codec can't encode characters in position 123-128check_prerequisites.py --json --paths-onlyUnicodeEncodeErrorsetup_tasks.py --jsonUnicodeEncodeErrorresolve_template.py constitution-template --jsonUnicodeEncodeError: 'charmap' codec can't encode character '→' in position 644Testing
uv run specify --helpuv sync && uv run pytestNew tests, each running the installed script in a subprocess with a cp1252 stdout and parsing the JSON:
tests/test_setup_tasks_python_parity.py:test_installed_python_json_preserves_stock_tasks_template(realspecify init --script py, then feature and plan setup, thensetup_tasks.py --jsonwith the stock template).tests/test_setup_plan_python_parity.pyandtests/test_check_prerequisites_python_parity.py:test_python_json_preserves_non_ascii_feature_on_legacy_stdout(feature directoryspecs/001-功能).tests/test_resolve_template_python_parity.py:test_python_json_preserves_constitution_on_legacy_stdout(an override template with Cyrillic and→).With only the four scripts reverted to
main, all four new tests fail; with the fix they pass. The existing text-mode tests from #3890/#3892 pass either way.Full suite on Linux, Python 3.13: 12531 passed, 377 skipped (
uv sync --extra test, thenuv run pytest, 14m).Ruff 0.15.0 (
ruff check src tests) passes. I haven't run it on Windows itself.AI Disclosure
AI disclosure: Two agents, both autonomous. Codex CLI (model: gpt-6.1-sol, reasoning effort xhigh, autonomous, sandboxed and offline) found the
setup_tasks.pycrash while reading the Python helpers, wrote the reproduction, and generated the tests. Claude Code (model: claude-opus-5-5, autonomous) extended the fix to the other three scripts, chose theensure_asciiapproach after comparing #3890/#3892 with #4780, edited the fix and two of the tests by hand, reran the tests againstmainand the fix, reran the repro with the realspecifyCLI, ran the full suite and lint, and drafted this description. Commits carryAssisted-by:trailers.