bpo-29546: Improve from-import error message with location#103
bpo-29546: Improve from-import error message with location#103warsaw merged 3 commits intopython:masterfrom
Conversation
7124949 to
ff44351
Compare
There was a problem hiding this comment.
Missing a space between ) and {.
warsaw
left a comment
There was a problem hiding this comment.
Yep, that's exactly the format I was thinking! Great work.
|
Good, and thanks. I'll cleanup and add tests then ! |
ff44351 to
36c8950
Compare
|
Added tests, Misc/NEWS, What's new. The only thing I'm unsure is what the module's It's also a bit ugly if a module has neither name not a file:
But that seem to be rare from how it's currently tested: class AlwaysAttributeError:
def __getattr__(self, _):
raise AttributeError
module_name = 'test_from_import_AttributeError'
self.addCleanup(unload, module_name)
sys.modules[module_name] = AlwaysAttributeError() |
|
On Feb 15, 2017, at 07:56 PM, Matthias Bussonnier wrote:
It's also a bit ugly if a module has neither name not a file:
> "cannot import name 'does_not_exist' from '<unknown module name>' (unknown location)"
But that seem to be rare from how it's currently tested:
Agreed it's ugly, but if it won't happen commonly in practice, it's probably
not worth dealing with specially. I'm also not sure how you'd print that any
better - it is worth knowing what's unknowable. :)
|
warsaw
left a comment
There was a problem hiding this comment.
Please review the potential reference counting bugs.
There was a problem hiding this comment.
I think there are some reference counting bugs lurking here.
- It's possible the code jumps to
error:afterpkgnameis decref'd. I think that decref should be deferred until just before the good-path return. pkgname_or_unknownwill be a new reference ifpkgnameis NULL, or it will steal thepkgnamereference if it's not. In either case, theerrorstanza should decrefpkgname_or_unknown, which will take care of ensuring the object gets decref for anygoto errorpath.pkgpathdoesn't get decref'd. I think you shouldPy_XDECREFit before thereturn NULLjust in casepkgpathis itself NULL.
There was a problem hiding this comment.
Thanks for the detailed explanation, I still need to get used to do manual ref counting.
I've push changes. Let me know if I got things rights.
There was a problem hiding this comment.
@Carreau Yes, I think the refcounting is right now. It can sometimes be difficult to trace all the various exit paths to make sure that everything gets decref'd and incref'd properly, but as best I can tell, you've got it right now. Thanks!
The only other thing is that it's possible for the PyUnicode_FromString("<unknown module name>") to fail and return NULL. Probably the only realistic cause of that would be a memory error (not sure if an encoding error could do it given that this is an ascii C string). If you want to be pedanditc, it would be useful to check for that error condition and do something more reasonable in that case.
There was a problem hiding this comment.
If you want to be pedanditc, it would be useful to check for that error condition and do something more reasonable in that case
I'm unsure what "more reasonable" is, I check and failed with the original error message in that case, but I don't see how if allocating PyUnicode_FromString("<unknown module name>") fails we can do something sensible.
There was a problem hiding this comment.
Probably the best you can do is to just decref any objects that may still have a reference count, and then immediately return NULL. The PyUnicode_FromString failure would have already set an exception, so that's the one you'll see. Something like:
1 file changed, 4 insertions(+)
Python/ceval.c | 4 ++++
modified Python/ceval.c
@@ -5024,6 +5024,10 @@ import_from(PyObject *v, PyObject *name)
pkgpath = PyModule_GetFilenameObject(v);
if (pkgname == NULL) {
pkgname_or_unknown = PyUnicode_FromString("<unknown module name>");
+ if (pkgname_or_unknown == NULL) {
+ Py_XDECREF(pkgpath);
+ return NULL;
+ }
} else {
pkgname_or_unknown = pkgname;
}
warsaw
left a comment
There was a problem hiding this comment.
Semicolons to end lines.
Please check for PyUnicode_FromString() returning NULL.
There was a problem hiding this comment.
Oh whoops, these lines need to end with semicolons.
0e29d44 to
e757eec
Compare
|
semicolon added |
warsaw
left a comment
There was a problem hiding this comment.
Two more small comments, but I think this is really close!
There was a problem hiding this comment.
Probably the best you can do is to just decref any objects that may still have a reference count, and then immediately return NULL. The PyUnicode_FromString failure would have already set an exception, so that's the one you'll see. Something like:
1 file changed, 4 insertions(+)
Python/ceval.c | 4 ++++
modified Python/ceval.c
@@ -5024,6 +5024,10 @@ import_from(PyObject *v, PyObject *name)
pkgpath = PyModule_GetFilenameObject(v);
if (pkgname == NULL) {
pkgname_or_unknown = PyUnicode_FromString("<unknown module name>");
+ if (pkgname_or_unknown == NULL) {
+ Py_XDECREF(pkgpath);
+ return NULL;
+ }
} else {
pkgname_or_unknown = pkgname;
}
There was a problem hiding this comment.
You need a space between the paren and open brace.
Add location information like canonical module name where identifier cannot be found and file location if available. First iteration of this was pythongh-91
from Barry Warsaw: - It's possible the code jumps to error: after pkgname is decref'd. I think that decref should be deferred until just before the good-path return. - pkgname_or_unknown will be a new reference if pkgname is NULL, or it will steal the pkgname reference if it's not. In either case, the error stanza should decref pkgname_or_unknown, which will take care of ensuring the object gets decref for any goto error path. - pkgpath doesn't get decref'd. I think you should Py_XDECREF it before the return NULL just in case pkgpath is itself NULL.
dfa3512 to
1b6e462
Compare
|
Comments should be addressed. Thanks ! |
|
@Carreau Thanks, this LGTM now. I'm pushing the big green button. |
Fix an invalid assertion during interpreter shutdown and add a test case for this problem. https://bitbucket.org/stackless-dev/stackless/issues/103 (grafted from a2f660260c93e911fde4969dac79c9eb6cc03e72)
The reworked C-function slp_kill_tasks_with_stacks() didn't work correctly for deeply nested tasklets. Therefore I had to rework it again. This version first kills a tasklet and only then clears the tstate of its cstacks. I added several new test cases and reorganised them a bit. https://bitbucket.org/stackless-dev/stackless/issues/103 (grafted from 130a60b8d3719e06807acd7e9b63c07bd5435c32)
Documents Phase 3 completion per supervisor 01:02:46Z + theologian 01:01:50Z Tier 8 spec adoption. All 6 batches landed + pushed: B1 c905827 +140L PhxHirBuilderState foundation B2 1343895 +80L exception_table_ Class B-kept closure B3 b92d85e -13L pending_b2_blocks_ DEAD-STATE delete B4 b44a514 +27L block_map_ Class B-kept closure B5 782d56d +9L static_method_stack_ rename B6 7971941 +14L temps_ rename + §5 forcing-decision COMPLETE NET cumulative: +257L (foundation cost; transitional per pythia python#103 + Tier 8 spec). §5 forcing-decision endpoint: 4 Class B-kept + 1 dead-deleted. Honest framing: keep-bias is DESIGN CHOICE (architectural decision, not deferral); Tier 8 pilot validates migrate-arm by porting one Class B field to pure C. Document covers: batch ledger + §5 endpoint + 9-bridge foundation surface + falsifier infrastructure landed in-flight (W44 + W45 §1-2 + W45 §3.5) + per-bench floor 4-criterion + Tier 8 pilot scheduling + cross-links to specs + pythia checkpoints + MEMORY.md ZERO-C++ terminal goal. Authorization: supervisor 01:02:46Z post-Batch-6 closure summary directive. Theologian §5 spec amendment ("keep-bias as design choice + Tier 8 migrate-arm pilot scheduled") pending in parallel.
Phase 3 closure summary at fa8dfef cross-linked 4 spec docs that were never committed/pushed (untracked working files only). Per supervisor 01:08:22Z DISPOSITION (D-track-specs-only): track the 4 referenced specs to make cross-links valid on remote. Specs tracked: docs/tier7-phase3-hirbuilder-state-extraction-spec.md (319L, incl theologian §5 closure amendment 'TRANSITIONAL DESIGN CHOICE' replacing withdrawn 'FINAL' framing per pythia python#103) docs/tier8-class-b-cport-migrate-arm-spec.md (213L, theologian 01:01:50Z + supervisor 01:02:46Z ADOPTED; exception_table_ pilot, 10 acceptance criteria, 9-step strategy) docs/w44-do-not-use-callers-gate.md (124L) docs/w45-bridge-signature-drift-falsifier.md (295L, incl §1-§2 + §2.5 retro fixtures + §2.6 §2.7 §3.5 amendments) Closure summary line 51 numeric clarification: '7 _cpp bridges' → '9 extern surfaces (6 _cpp + 2 _c algorithmic + 1 init)' per theologian 01:07:07Z (2) clarification. (D-track-all-docs) rejected — benchmarks/gates/golden are working data not canonical spec. (D-leave-untracked) rejected — dishonest dangling cross-links per supervisor 01:08:22Z framing. Doc-only bundle, no builder*.{cpp,h,c} touched. §3.5 BUILD MODE exempt per supervisor 00:39:20Z trivial-push rule. NET +953L (4 NEW specs + 1L closure-summary correction). Authorization: supervisor 01:08:22Z (D-track-specs-only) + theologian 01:07:07Z (cross-check + §5 amendment incorporated).
…DE.md Bundle of two theologian-authored doc amendments per supervisor dispositions: CLAUDE.md (+2L) — Pre-Edit WT Integrity 'undiagnosed-recurring' honest framing per pythia python#105 (3) + supervisor 02:13:12Z + theologian 02:13:54Z. Acknowledges 5+ prior incidents closed at SYMPTOM layer without root-cause attribution; this 4-step discipline codifies reactive HALT, not automated detection. Detection still depends on agent noticing mid-edit. Recurrence-prevention claim does not exceed what text mechanizes; automated detection feasibility (file-watcher / git-hook) is fixup PIR territory and remains open. docs/tier8-class-b-cport-migrate-arm-spec.md (+93/-20) — pythia python#104 amendments per theologian 01:38:08Z + supervisor 01:38:26Z + 01:37:05Z: §1.1 container-shape transferability caveat (vector-pilot ≠ hash/ stack/allocator-shape pattern transferability) §5 python#11 NEW: bridge-count-delta acceptance (each pilot must NET- SUBTRACT bridges OR honestly add with W45 fixture coverage) §5 python#12 NEW: Phase B FORCING-FUNCTION (Tier 8 SECOND-PILOT block_map_ Step A BLOCKED until exception_table_ Phase B commit lands) Bundle scope: doc-only, no §3.5 BUILD MODE (no builder*.{cpp,h,c} touched). Atomic per supervisor 02:13:12Z + 01:38:26Z dispositions. Push 28 advances the Tier 8 Phase A resume deadline tracker per theologian 02:13:54Z + supervisor 02:13:12Z (push 28 OR session-end whichever first). If Alex disposition + fixup PIR not received by deadline, theologian amends Phase 3 closure summary fa8dfef to remove Tier 8 pilot citation + re-opens pythia python#103 escape question.
…hon#7) Per supervisor 02:45:37Z fast-path TRIGGER post Tier 8 Phase A incident python#7 (R-retry under enhanced mtime-checkpoint discipline also ROLLED BACK at HEAD 48339e2, same pattern as 01:23Z incident python#6). mtime-checkpoint discipline DETECTED proactive (Write tool safety check + post-write system-reminders) but did NOT prevent recurrence. (R-retry) reasoning ('discipline addresses risk class') empirically INVALIDATED — same external force re-reverts at every Tier 8 attempt. Pythia python#105 'fever has name infection still spreads' validated twice. Theologian-pre-drafted amendment 02:26:02Z replaces 'Tier 8 pilot scheduled' section with 'Tier 8 pilot BLOCKED (push 28+, 2026-04-24)': - Pilot field exception_table_ POD-equivalent - Phase A attempted 01:18:35Z + 02:39:00Z (R-retry); HALTED 01:23:55Z + 02:45:12Z by external file-state revert (incidents python#6 + python#7; observed-but-not-attributed per generalist 01:24:43Z + 02:45:12Z) - Resume gated on Alex disposition + fixup PIR; both unmet by push 28 deadline (supervisor 02:13:12Z + theologian 02:13:54Z) - DEADLINE reached push 28 at 02:15:47Z; second incident at 02:45:12Z re-confirms permanent-block-this-session **Pythia python#103 escape question RE-OPENED.** Phase 3 keep-bias is the END STATE pending future migrate-arm validation. 'Transitional foundation cost paid back' framing is HONEST-ASPIRATIONAL not RESOLVED. ZERO-C++ terminal goal (MEMORY.md L70 + L104) remains gated on Tier 8 pilot landing in a future session. Phase 3 cumulative +257L is END STATE this session. Future session must re-attempt Tier 8 Phase A under same spec (post external-revert root-cause attribution, since detection alone insufficient) OR honestly amend Phase 3 closure framing to 'permanent scaffolding'. Cross-link section also amended: §5 amendment line updated to 'keep-bias as design choice; Tier 8 migrate-arm pilot FILED but BLOCKED — see Tier 8 pilot BLOCKED section above'. Doc-only +24L. No §3.5 BUILD MODE per touched-files rule. Authorization: theologian 02:26:02Z STAGED-ON-DISK + supervisor 02:45:37Z fast-path TRIGGER post HALT python#7.
Per docs/tier8-class-b-cport-migrate-arm-spec.md theologian 01:01:50Z + supervisor 01:02:46Z ADOPTED + supervisor 01:18:35Z + 03:44:19Z + 04:14:27Z (8-incident root-cause attribution to §3.5 restore-trap + b83f084 fix LIVE). Migrates HIRBuilder std::vector<ExceptionTableEntry> exception_table_ field to PhxExceptionTable (purpose-built typed-inline pure-C container in PhxHirBuilderState.exception_table_phx). Validates Pythia python#103 + python#94 (3) §5 forcing-decision MIGRATE-ARM via 1-pilot port (vs Phase 3's 4-Class-B-kept disposition). CONTAINER: PhxExceptionTable (builder_state_c.h): typed-inline data/count/capacity with 6 inline funcs (init/destroy/push/size/at/clear). Lazy-init, doubling realloc, free at HIRBuilder dtor. ExceptionTableEntry (builder_state_c.h): POD mirror of deleted C++ struct, fields flattened BCOffset → int + bool → unsigned char. C BODY PORTS (builder_state_c.c): hir_builder_state_init: also calls phx_exception_table_init hir_builder_state_destroy: NEW (calls phx_exception_table_destroy) parse_exception_table_c: pushes ExceptionTableEntry via phx_exception_table_push (replaces deleted push_cpp bridge) find_exception_handler_c: linear scan via phx_exception_table_size + at (replaces deleted size_cpp/entry_cpp bridges) C++ SHIM (transient compatibility per Phase A; Phase B deletes): HIRBuilder::parseExceptionTable → 1-line delegate to C body HIRBuilder::findExceptionHandler → C body returns index, shim converts via phx_exception_table_at preserving caller-contract HIRBuilder::buildHIRImpl translate-loop iterates PhxExceptionTable via size+at; .clear() goes to phx_exception_table_clear HIRBuilder::getSimpleExceptInfo wraps handler.target in BCOffset{} (now plain int post-C struct migration) emit_call_method_exception_handler_inline_c at builder.cpp:2883 still calls self->findExceptionHandler (KEPT shim — Phase B will rewire) DELETED: 3 _cpp bridge impls in builder.cpp (push/size/entry, ~37L) 3 friend decls in builder.h C++ struct ExceptionTableEntry in builder.h (5L) std::vector<ExceptionTableEntry> exception_table_ field in builder.h W45 §1-§2 fixture removals (3): the deleted bridges no longer have signatures to fuzz. Cumulative bridge-count delta: -3 (per Tier 8 spec §5 python#11 acceptance). Numstat (vs HEAD b83f084): Python/jit/hir/builder.cpp +21 -44 (-23 NET) Python/jit/hir/builder.h +14 -19 (-5 NET) Python/jit/hir/builder_state_c.c +30 -17 (+13 NET) Python/jit/hir/builder_state_c.h +103 -48 (+55 NET) scripts/w45_bridge_drift_falsifier.sh +0 -3 (-3 NET) TOTAL: NET +37L, bridge-count delta -3. Per Tier 8 spec §5 python#10 amendment (theologian 01:14:29Z): full Tier 8 endpoint ≤+0L cumulative is across all 4 Class B containers, not single pilot. exception_table_ pilot subtracts ~19% of Phase 3 +257L foundation cost; Phase 3 + Tier 8 Phase A cumulative now +257 + 37 = +294L. Apply mechanism: python single-process write (8-incident root cause was §3.5 restore-trap, NOT Write-tool-burst — but python single-process remains best practice per Pythia python#107 (4) isolation principle). Apply script: /tmp/apply_phase_a.py. EXPANDED PRE-COMMIT GATE per supervisor 01:17:23Z (testkeeper 04:24:49Z): Stage 1 compile-check: BUILD_EXIT=0 Stage 2 §3.5 BUILD MODE: 4/4 PASS, PhxExceptionTable INTACT post (§3.5 fix b83f084 VINDICATED) Stage 3 per-bench gate: GEO 1.27x, all 4 floor criteria PASS §5 forcing-decision MIGRATE-ARM EMPIRICALLY VALIDATED via this pilot: exception_table_ migrated to PhxArray-equivalent pure-C container without C++-side-keep dependency in C-side reads. Pattern propagatable to remaining 3 Class B containers (block_map_, temps_, static_method_stack_) in future Tier 8 batches per spec §1.1 container-shape transferability caveat. Phase B follow-up commit (NEXT) deletes C++ shims + rewires the remaining caller at builder.cpp:2883 per Tier 8 spec §5 python#5. Authorization: supervisor 03:44:19Z + 04:14:27Z; theologian 03:33:50Z patch-apply hybrid + 03:54:15Z content-trigger refinement (later attributed to §3.5 trap, not content) + 04:14:27Z fix directive. Cross-link: 8 incidents resolved via b83f084 §3.5 restore-trap fix. Pythia python#105 'fever has name infection still spreads' RESOLVED — fever was self-inflicted instrumentation. W48 spec marked CANCELLED post- this-commit per supervisor 04:14:27Z cascade re-retract.
Per supervisor 04:26:14Z cascade triggered ON Tier 8 Pilot Phase A STRICT VERIFY clean (testkeeper 04:38:26Z + push 33 6945b96). Theologian-authored amendments to 3 docs: (1) docs/2026-04-24-pre-edit-revert-pir.md (+14/-1): Status header changed UNDIAGNOSED-RECURRING → RESOLVED 04:13:58Z. Root cause attributed to scripts/w45_section_3_5_derivation_drift.sh restore_files() git-checkout-HEAD-- destroying pre-existing unstaged user modifications. Fix landed b83f084 (push 31). Tier 8 Phase A re-attempted clean at 6945b96 (push 32/33). 8-incident class CLOSED with explicit attribution. Sister-script audit clean per librarian 04:28:11Z (only §3.5 had unsafe pattern). (2) docs/tier7-phase3-closure-summary.md (+32/-21): RETRACT 'Tier 8 BLOCKED (push 28+)' section. REPLACE with 'Tier 8 Pilot Phase A LANDED at 6945b96' citing root-cause attribution + §5 forcing-decision MIGRATE-ARM EMPIRICALLY VALIDATED. Pythia python#103 escape question RESOLVED via successful pilot. (3) docs/w48-nbs-inotify-file-watcher-spec.md (+15/-1): Status: ACTIVE → CANCELLED-RESOLVED. Rationale: 8-incident class root cause attributed to OUR OWN §3.5 restore_files() trap, not external actor. inotify daemon NOT NEEDED. Sister-script audit confirms no other unsafe patterns. W48 cancelled, scope superseded by §3.5 fix b83f084. Doc-only bundle, no §3.5 BUILD MODE per touched-files rule. Atomic per supervisor cascade-by-reference 04:30:59Z. Authorization: supervisor 04:26:14Z cascade trigger post-STRICT-VERIFY + theologian cascade execution + librarian sister-script audit clean + shepard 04:30:46Z echo-loop verbosity discipline (this commit references prior posts, doesn't re-elaborate).

Add location information like canonical module name where identifier
cannot be found and file location if available.
First iteration of this was gh-91
Testless for now, I can add tests once the exact format has been agreed upon.