Comparing main...v2 · mevdschee/github-export · GitHub
Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: mevdschee/github-export
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: mevdschee/github-export
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v2
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 10 commits
  • 47 files changed
  • 2 contributors

Commits on May 31, 2026

  1. Add v2 plan: SQLite-primary store, OpenAPI, MCP, gh/GitHub-MCP parity

    Plan to make SQLite the source of truth with on-demand one-way markdown
    export, a GitHub-API-compatible local server (REST + GraphQL mirror, git
    backend for repo content, write proxy), an OpenAPI 3.1 spec, and an MCP
    server matching GitHub's official toolset. Includes a phased roadmap
    (Phases 0-6) and settled design decisions.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    e99fc76 View commit details
    Browse the repository at this point in the history
  2. Phase 1: SQLite as primary store, markdown export on demand

    Make SQLite (github.sqlite) the source of truth. `sync` fetches from
    GitHub and upserts into the store; the new `export` subcommand renders
    markdown from the store on demand (one-way — markdown is never read back).
    Incremental state (synced_at) and change detection now live in the store
    instead of being recovered from on-disk markdown.
    
    New packages:
    - internal/store   – SQLite schema/migrations, upserts, prior-state reads
                         for change detection, and export reads (modernc.org/sqlite,
                         cgo-free; Go 1.22 floor preserved by pinning v1.34.4)
    - internal/ghmodel – shared domain types (discussion nodes) and pure helpers
    - internal/render  – markdown writers moved out of sync, reusing document
    - internal/exporter– renders the store to the github-data/ layout on demand
    
    sync refactor: each entity fetch now detects events by diffing against the
    stored row, then upserts; events are recorded in an events table and rendered
    to events/*.md (stamped once) on export.
    
    CLI: `sync` and `export` subcommands (bare invocation = sync). owner/repo is
    resolved from arg, then the database, then the origin git remote (closes the
    TODO). Default db github.sqlite.
    
    Tests: store gets standalone unit tests (no token); e2e reframed to
    sync→SQLite (asserting store contents) then export (format-tolerant parse
    assertions). Project assertions skip when the token lacks read:project.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    e3b502b View commit details
    Browse the repository at this point in the history
  3. Phases 2-6: query layer, HTTP API, OpenAPI, git backend, write proxy,…

    … MCP, native CLI
    
    Build the serving layers on top of the SQLite store from Phase 1.
    
    - internal/query: backend-agnostic read layer returning GitHub-shaped JSON
      with filtering, pagination, and qualifier-aware issue search.
    - internal/httpapi: `serve` GitHub-compatible REST mirror (issues, pulls,
      comments, timeline, reviews, labels, milestones, releases, discussions,
      search, status), Link-header pagination, freshness headers, /docs + OpenAPI.
    - api/openapi.yaml: embedded OpenAPI 3.1 spec served at /openapi.yaml.
    - internal/writeproxy: forward unmirrored reads + all writes to api.github.com,
      with synchronous targeted re-sync after writes; --proxy=off for offline.
    - internal/gitbackend: branches/tags/commits/contents/code-search from a local
      git clone, mapped to GitHub REST shapes (cgo-free shell-out).
    - internal/mcp: MCP server (official Go SDK) with GitHub-MCP-matching tool
      names; read tools over query/git, write tools over the proxy; stdio + HTTP.
    - internal/shadow: --debug-compare parity harness (local vs remote diff,
      deduped, optional auto-filed gap issues).
    - Native issue/pr/search/release subcommands and a `gh api`-style passthrough.
    
    POST /graphql currently proxies to GitHub (local GraphQL mirror is next).
    CGO-free static build verified; go bumped to 1.25 for the MCP SDK.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    95b982a View commit details
    Browse the repository at this point in the history
  4. GraphQL mirror: serve common repository reads from SQLite, proxy the …

    …rest
    
    Add internal/graphqlmirror, a SQLite-backed GraphQL executor wired into
    POST /graphql with strict proxy fallthrough: a query is served locally only
    if every requested field resolves correctly, otherwise the whole request is
    forwarded to api.github.com (never a half-correct response).
    
    Covered: repository(owner,name) with issue/pullRequest/discussion, the
    issues/pullRequests/discussions connections (first/after cursor pagination,
    states, totalCount, nodes/edges/pageInfo), nested author/labels/assignees/
    comments, fragments, inline fragments, and variables. Discussions/projects
    resolve directly from their stored GraphQL-shaped raw_json; issues/PRs are
    translated from the stored REST payload. Mutations, unmapped fields, and
    un-synced repos fall through.
    
    Wired into serve and the api passthrough. Uses github.com/vektah/gqlparser/v2
    for parsing.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    4577a0c View commit details
    Browse the repository at this point in the history
  5. writeproxy: re-sync before responding (read-after-write fix) + tests;…

    … GraphQL projectsV2
    
    writeproxy:
    - Fix the re-sync ordering: buffer the upstream response and run the targeted
      re-sync BEFORE writing the response to the caller, so the local store is
      consistent the moment a write call returns (read-after-write). Previously the
      re-sync ran after WriteHeader, so it could not affect the response and the
      X-GitHub-Export-Resync header could never be sent.
    - Make the upstream base URL overridable for testing.
    - Add a full test suite against a fake upstream: response/auth/query streaming,
      disabled→501, write→resync, GET no-resync, failed-write no-resync,
      resync-failure header, programmatic Request, and classifyWrite mapping.
    
    graphqlmirror:
    - Add repository scalar fields (description, url, isPrivate, timestamps,
      star/fork counts, defaultBranchRef, owner) backed by the stored repo payload.
    - Add projectV2(number) and the projectsV2 connection (Projects v2 are
      GraphQL-only and stored in GraphQL shape, so they resolve directly).
    - Extract clampRange helper shared by the discussion/project connections.
    - Tests for projectV2, projectsV2, and repository scalars.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    d90f37a View commit details
    Browse the repository at this point in the history
  6. mcp: expand tool set toward GitHub MCP parity

    Add read tools backed by the local store/git clone: get_issue_comments,
    get_pull_request_comments, list_discussions, get_discussion,
    get_discussion_comments, get_commit, list_branches, list_tags. Add write tools
    update_pull_request and merge_pull_request (proxy + re-sync). New query helper
    DiscussionComments extracts comment nodes from the stored discussion payload.
    
    Update the tool-name parity test and README tool list.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    a5ae9f7 View commit details
    Browse the repository at this point in the history
  7. store: FTS5 full-text search for issues/PRs with incremental migration

    Replace the substring scan in SearchIssues with a relevance-ranked FTS5 index.
    
    - Add a proper incremental migration runner (migrations[]) and bump the schema
      to v2. v1→v2 creates fts_issues (fts5 over title+body); existing databases are
      upgraded and backfilled from the issues table on open.
    - Keep the index in sync inside UpsertIssue (delete-then-insert by rowid=number).
    - store.SearchFTS runs the MATCH query ordered by rank; a malformed expression
      reports ok=false so query.SearchIssues can degrade to a substring scan.
    - query.SearchIssues now intersects the FTS-ranked numbers with the
      qualifier-filtered set, preserving relevance order.
    - Tests: FTS search + index update on re-upsert, and the v1→v2 backfill path.
    
    Verified FTS5 is available in the pure-Go modernc driver (build stays cgo-free).
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    f47edb1 View commit details
    Browse the repository at this point in the history
  8. ci: add GitHub Actions workflow and OpenAPI spec validation tests

    - .github/workflows/ci.yml: vet, test, cgo-free static build, and gofmt check
      on push/PR to main and v2 (matches the plan's CGO_ENABLED=0 CI requirement).
    - api/spec_test.go: validate the embedded OpenAPI 3.1 document with the existing
      yaml.v3 dependency (no new deps) — asserts version/info, that every path has an
      operation with responses, and that the documented mirror paths are present so
      the spec cannot silently drift from the handlers.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    e3929e5 View commit details
    Browse the repository at this point in the history
  9. api: add --debug-compare for one-shot local-vs-GitHub parity checks

    The `api` passthrough can now compare its locally-served answer against
    api.github.com synchronously and log any divergence (normalized, deduped via
    internal/shadow). Completes the shadow-compare surface on serve + api; the
    one-shot form is handy for capturing the exact queries gh/MCP emit.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed May 31, 2026
    Configuration menu
    Copy the full SHA
    faa1040 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2026

  1. sync: check token scopes up front, skip projects when read:project is…

    … missing
    
    A token with repo but not read:project made the projects v2 GraphQL query
    fail field-by-field ("the 'createdAt' field requires ['read:project']"),
    emitting one error per scope-gated field on every run.
    
    Add Client.Scopes() (reads X-OAuth-Scopes; reports unknown for fine-grained /
    app tokens) and a featureScopes table checked before sync. Projects is now
    skipped with a single clear message when its scope is absent.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    mevdschee and claude committed Jun 1, 2026
    Configuration menu
    Copy the full SHA
    964d243 View commit details
    Browse the repository at this point in the history
Loading