source-pylon: issue_messages and knowledge_base_articles records carry no parent ID · Issue #87521 · airbytehq/airbyte · GitHub
Skip to content

source-pylon: issue_messages and knowledge_base_articles records carry no parent ID #87521

Open

Description

What

Two of the three child streams in source-pylon emit records with no reference to their parent, which makes them unjoinable in a destination.

  • issue_messages is requested at issues/{{ stream_partition.issue_id }}/messages
  • knowledge_base_articles is requested at knowledge-bases/{{ stream_partition.knowledge_base_id }}/articles

In both cases the connector holds the parent ID at request time via SubstreamPartitionRouter, and then discards it. Neither the record payload nor the emitted schema carries it.

The thread_id field on issue_messages cannot substitute: it is Pylon's Slack/email thread identifier, not the issue ID.

issue_threads is unaffected, because the Pylon API returns issue_id in that payload already. That is part of why the gap is easy to miss.

Impact

Message bodies become unattributable once they land. In a Postgres destination holding 8,910 synced issue_messages rows:

  • 0 rows join to issues on thread_id
  • 282 rows can recover a parent indirectly through the issue_threads stream

So 96.8% of message content cannot be tied back to a ticket, an account, or a customer. Only a ticket's opening message (issues.body_html) is reliably usable, and every reply is lost to analysis.

The connector's own CONTRIBUTING.md already lists these three streams as deferred_child.

Expected behaviour

Each child record carries its parent ID, as issue_id on issue_messages and knowledge_base_id on knowledge_base_articles.

Environment

  • Connector: source-pylon 0.0.24 (present since 0.0.1)
  • Airbyte Cloud

I have a manifest-only fix ready and will open a PR referencing this issue.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions