Navigation Menu
-
Notifications
You must be signed in to change notification settings - Fork 0
Comparing changes
Open a pull request
base repository: mevdschee/github-export
base: main
head repository: mevdschee/github-export
compare: v2
- 10 commits
- 47 files changed
- 2 contributors
Commits on May 31, 2026
-
Add v2 plan: SQLite-primary store, OpenAPI, MCP, gh/GitHub-MCP parity
Plan to make SQLite the source of truth with on-demand one-way markdown export, a GitHub-API-compatible local server (REST + GraphQL mirror, git backend for repo content, write proxy), an OpenAPI 3.1 spec, and an MCP server matching GitHub's official toolset. Includes a phased roadmap (Phases 0-6) and settled design decisions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for e99fc76 - Browse repository at this point
Copy the full SHA e99fc76View commit details -
Phase 1: SQLite as primary store, markdown export on demand
Make SQLite (github.sqlite) the source of truth. `sync` fetches from GitHub and upserts into the store; the new `export` subcommand renders markdown from the store on demand (one-way — markdown is never read back). Incremental state (synced_at) and change detection now live in the store instead of being recovered from on-disk markdown. New packages: - internal/store – SQLite schema/migrations, upserts, prior-state reads for change detection, and export reads (modernc.org/sqlite, cgo-free; Go 1.22 floor preserved by pinning v1.34.4) - internal/ghmodel – shared domain types (discussion nodes) and pure helpers - internal/render – markdown writers moved out of sync, reusing document - internal/exporter– renders the store to the github-data/ layout on demand sync refactor: each entity fetch now detects events by diffing against the stored row, then upserts; events are recorded in an events table and rendered to events/*.md (stamped once) on export. CLI: `sync` and `export` subcommands (bare invocation = sync). owner/repo is resolved from arg, then the database, then the origin git remote (closes the TODO). Default db github.sqlite. Tests: store gets standalone unit tests (no token); e2e reframed to sync→SQLite (asserting store contents) then export (format-tolerant parse assertions). Project assertions skip when the token lacks read:project. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Configuration menu - View commit details
-
Copy full SHA for e3b502b - Browse repository at this point
Copy the full SHA e3b502bView commit details -
Phases 2-6: query layer, HTTP API, OpenAPI, git backend, write proxy,…
… MCP, native CLI Build the serving layers on top of the SQLite store from Phase 1. - internal/query: backend-agnostic read layer returning GitHub-shaped JSON with filtering, pagination, and qualifier-aware issue search. - internal/httpapi: `serve` GitHub-compatible REST mirror (issues, pulls, comments, timeline, reviews, labels, milestones, releases, discussions, search, status), Link-header pagination, freshness headers, /docs + OpenAPI. - api/openapi.yaml: embedded OpenAPI 3.1 spec served at /openapi.yaml. - internal/writeproxy: forward unmirrored reads + all writes to api.github.com, with synchronous targeted re-sync after writes; --proxy=off for offline. - internal/gitbackend: branches/tags/commits/contents/code-search from a local git clone, mapped to GitHub REST shapes (cgo-free shell-out). - internal/mcp: MCP server (official Go SDK) with GitHub-MCP-matching tool names; read tools over query/git, write tools over the proxy; stdio + HTTP. - internal/shadow: --debug-compare parity harness (local vs remote diff, deduped, optional auto-filed gap issues). - Native issue/pr/search/release subcommands and a `gh api`-style passthrough. POST /graphql currently proxies to GitHub (local GraphQL mirror is next). CGO-free static build verified; go bumped to 1.25 for the MCP SDK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for 95b982a - Browse repository at this point
Copy the full SHA 95b982aView commit details -
GraphQL mirror: serve common repository reads from SQLite, proxy the …
…rest Add internal/graphqlmirror, a SQLite-backed GraphQL executor wired into POST /graphql with strict proxy fallthrough: a query is served locally only if every requested field resolves correctly, otherwise the whole request is forwarded to api.github.com (never a half-correct response). Covered: repository(owner,name) with issue/pullRequest/discussion, the issues/pullRequests/discussions connections (first/after cursor pagination, states, totalCount, nodes/edges/pageInfo), nested author/labels/assignees/ comments, fragments, inline fragments, and variables. Discussions/projects resolve directly from their stored GraphQL-shaped raw_json; issues/PRs are translated from the stored REST payload. Mutations, unmapped fields, and un-synced repos fall through. Wired into serve and the api passthrough. Uses github.com/vektah/gqlparser/v2 for parsing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for 4577a0c - Browse repository at this point
Copy the full SHA 4577a0cView commit details -
writeproxy: re-sync before responding (read-after-write fix) + tests;…
… GraphQL projectsV2 writeproxy: - Fix the re-sync ordering: buffer the upstream response and run the targeted re-sync BEFORE writing the response to the caller, so the local store is consistent the moment a write call returns (read-after-write). Previously the re-sync ran after WriteHeader, so it could not affect the response and the X-GitHub-Export-Resync header could never be sent. - Make the upstream base URL overridable for testing. - Add a full test suite against a fake upstream: response/auth/query streaming, disabled→501, write→resync, GET no-resync, failed-write no-resync, resync-failure header, programmatic Request, and classifyWrite mapping. graphqlmirror: - Add repository scalar fields (description, url, isPrivate, timestamps, star/fork counts, defaultBranchRef, owner) backed by the stored repo payload. - Add projectV2(number) and the projectsV2 connection (Projects v2 are GraphQL-only and stored in GraphQL shape, so they resolve directly). - Extract clampRange helper shared by the discussion/project connections. - Tests for projectV2, projectsV2, and repository scalars. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for d90f37a - Browse repository at this point
Copy the full SHA d90f37aView commit details -
mcp: expand tool set toward GitHub MCP parity
Add read tools backed by the local store/git clone: get_issue_comments, get_pull_request_comments, list_discussions, get_discussion, get_discussion_comments, get_commit, list_branches, list_tags. Add write tools update_pull_request and merge_pull_request (proxy + re-sync). New query helper DiscussionComments extracts comment nodes from the stored discussion payload. Update the tool-name parity test and README tool list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for a5ae9f7 - Browse repository at this point
Copy the full SHA a5ae9f7View commit details -
store: FTS5 full-text search for issues/PRs with incremental migration
Replace the substring scan in SearchIssues with a relevance-ranked FTS5 index. - Add a proper incremental migration runner (migrations[]) and bump the schema to v2. v1→v2 creates fts_issues (fts5 over title+body); existing databases are upgraded and backfilled from the issues table on open. - Keep the index in sync inside UpsertIssue (delete-then-insert by rowid=number). - store.SearchFTS runs the MATCH query ordered by rank; a malformed expression reports ok=false so query.SearchIssues can degrade to a substring scan. - query.SearchIssues now intersects the FTS-ranked numbers with the qualifier-filtered set, preserving relevance order. - Tests: FTS search + index update on re-upsert, and the v1→v2 backfill path. Verified FTS5 is available in the pure-Go modernc driver (build stays cgo-free). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for f47edb1 - Browse repository at this point
Copy the full SHA f47edb1View commit details -
ci: add GitHub Actions workflow and OpenAPI spec validation tests
- .github/workflows/ci.yml: vet, test, cgo-free static build, and gofmt check on push/PR to main and v2 (matches the plan's CGO_ENABLED=0 CI requirement). - api/spec_test.go: validate the embedded OpenAPI 3.1 document with the existing yaml.v3 dependency (no new deps) — asserts version/info, that every path has an operation with responses, and that the documented mirror paths are present so the spec cannot silently drift from the handlers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for e3929e5 - Browse repository at this point
Copy the full SHA e3929e5View commit details -
api: add --debug-compare for one-shot local-vs-GitHub parity checks
The `api` passthrough can now compare its locally-served answer against api.github.com synchronously and log any divergence (normalized, deduped via internal/shadow). Completes the shadow-compare surface on serve + api; the one-shot form is handy for capturing the exact queries gh/MCP emit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for faa1040 - Browse repository at this point
Copy the full SHA faa1040View commit details
Commits on Jun 1, 2026
-
sync: check token scopes up front, skip projects when read:project is…
… missing A token with repo but not read:project made the projects v2 GraphQL query fail field-by-field ("the 'createdAt' field requires ['read:project']"), emitting one error per scope-gated field on every run. Add Client.Scopes() (reads X-OAuth-Scopes; reports unknown for fine-grained / app tokens) and a featureScopes table checked before sync. Projects is now skipped with a single clear message when its scope is absent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Configuration menu - View commit details
-
Copy full SHA for 964d243 - Browse repository at this point
Copy the full SHA 964d243View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff main...v2
