[FIX] Skip un-clonable connectors gracefully; cascade to dependent workflows by chandrasekharan-zipstack · Pull Request #22 · Zipstack/unstract-python-client · GitHub
Skip to content

[FIX] Skip un-clonable connectors gracefully; cascade to dependent workflows - #22

Merged
chandrasekharan-zipstack merged 2 commits into
mainfrom
clone-skip-oauth-connectors
Jun 19, 2026
Merged

[FIX] Skip un-clonable connectors gracefully; cascade to dependent workflows#22
chandrasekharan-zipstack merged 2 commits into
mainfrom
clone-skip-oauth-connectors

Conversation

@chandrasekharan-zipstack

Copy link
Copy Markdown
Contributor

Problem

OAuth-backed connectors (e.g. Google Drive) and redacted-metadata connectors (auto-provisioned, e.g. Unstract Cloud Storage) cannot be cloned — the Platform API never exposes OAuth refresh tokens, and a valid token can only be minted by completing the OAuth flow as the target user (UI-only). The clone already skipped them, but still cloned their workflows and pipelines connector-less → every scheduled ETL/TASK run failed (the "failing every half hour" symptom).

Changes

  1. Adopt-before-recreate (connector.py): a same-name target connector is adopted before any recreate attempt. This is the recovery path — the operator provisions the connector on the target (where OAuth completes), re-runs, and the clone adopts it and wires dependent endpoints. (Backend match is exact connector_name, org-scoped.)
  2. Record skipped connector ids + surface a report warning for each genuine skip (OAuth / redacted), instead of a log line only.
  3. Cascade-skip (workflow.py): a workflow whose SOURCE/DEST endpoint references a skipped connector is skipped; pipeline / api_deployment / tool_instance cascade off the missing workflow remap (same mechanism as the frictionless-adapter cascade). No more guaranteed-failing pipelines.
  4. Surface frictionless skips (tool + workflow) in the report's Warnings block, not just the log.

The wf → connectors map is built from a single bulk endpoint listing, and only when connectors were actually skipped (zero cost otherwise).

Tests

tests/clone/184 passed. New/strengthened: OAuth/redacted skip records id + warning; OAuth connector adopted when a same-name target exists; workflow cascade-skip on skipped endpoint connector. Live dry-run against a dev org: no regression (identical counts, Completed successfully).

🤖 Generated with Claude Code

…rkflows

OAuth-backed and redacted-metadata connectors can't be recreated via the
Platform API (credentials can't be minted server-side). Previously they were
skipped but their workflows/pipelines were still cloned connector-less, so
every scheduled run failed.

- connector: adopt a same-name target connector before recreating, so an
  operator who provisions one on the target (where OAuth completes) gets it
  adopted on re-run, wiring dependent endpoints.
- connector: record genuinely-skipped connector ids + surface a report warning.
- workflow: cascade-skip workflows whose endpoints use a skipped connector;
  pipeline / api_deployment / tool_instance cascade off the missing remap.
- report: surface frictionless tool/workflow skips as warnings, not just logs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CsGrHbs5SWmQkKqiimg6CF
@greptile-apps

greptile-apps Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

- workflow: a workflow blocked by both a skipped tool and a skipped connector
  now surfaces both reasons instead of only the first (no extra re-run to learn
  the second blocker).
- tests: assert the tool-skip cascade emits its report warning; cover the
  dual-skip (tool + connector) case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CsGrHbs5SWmQkKqiimg6CF
@chandrasekharan-zipstack
chandrasekharan-zipstack merged commit 92738b5 into main Jun 19, 2026
3 checks passed
@chandrasekharan-zipstack
chandrasekharan-zipstack deleted the clone-skip-oauth-connectors branch June 19, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants