docs(cookbook): tag documents on upload and ask by tag by BukeLy · Pull Request #617 · VectifyAI/PageIndex · GitHub
Skip to content

docs(cookbook): tag documents on upload and ask by tag - #617

Draft
BukeLy wants to merge 3 commits into
mainfrom
cookbook-pageindex-metadata
Draft

BukeLy wants to merge 3 commits into
mainfrom
cookbook-pageindex-metadata

Conversation

@BukeLy

@BukeLy BukeLy commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Adds cookbook/pageindex-metadata.ipynb, following Metadata in the SDK docs. It declares a folder's metadata fields (type, team, year) with create_metadata_schema(), uploads four PDFs from examples/documents/ into the folder with matching tags, reads them back, and then asks two questions only the tags can answer:

  • "How many documents does each team own?" The agent calls get_metadata_schema, then search_documents(mode="metadata") with group_by: ["team"] and $count.
  • "Which documents are from 2019 or later?" The agent calls search_documents(mode="metadata") with metadata_filter: {"year": {"$gte": 2019}}.

The layout follows the other cloud chat notebooks and ends with the optional usage and cost estimate. A rerun reuses the folder and the documents.

Needs a PageIndex API key and an OpenAI key. A run creates one folder, uploads four small PDFs and takes about 5 minutes.

Merge after the SDK release that ships #618 (create_metadata_schema). Until then, pip install -U pageindex does not have it.

@BukeLy
BukeLy marked this pull request as ready for review October 21, 2026 10:21
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af95d9e34f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread cookbook/pageindex-metadata.ipynb Outdated
"source": [
"### Metadata rules\n",
"\n",
"Metadata is a flat dict of `str`, `int`, `float`, or `bool` values, with keys that start with a letter and contain only letters, digits, and underscores. Its serialized size is at most 8 KB, and PageIndex Cloud rejects metadata that breaks a rule with `PageIndexAPIError`. See the [metadata documentation](https://docs.pageindex.ai/sdk/documents#metadata) for details.\n"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require finite numeric metadata values

When a reader supplies float("nan"), float("inf"), or float("-inf"), the value satisfies the stated float rule but PageIndex Cloud rejects the submission. Add the finite-number restriction to this rules summary so users do not receive an unexpected PageIndexAPIError after following the notebook's guidance.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ab24ee2: the rules now say numbers must be finite, so no NaN or Infinity.

Comment thread cookbook/pageindex-metadata.ipynb Outdated
"source": [
"messages = [{\"role\": \"user\", \"content\": \"What revenue did the finance team's 2025 earnings release report?\"}]\n",
"\n",
"for event in client.chat(messages, stream=True, protocol=\"responses\"):\n",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope tag queries to the uploaded documents

Both chat calls search the user's entire cloud library because neither doc_id nor a dedicated folder_id is passed. If the account already contains documents with matching tags—or if the notebook is rerun and creates suffixed duplicates—the answers can include unrelated files or become ambiguous instead of reproducing the displayed two-document result; scope the calls to doc_ids.values() or upload into an isolated folder.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly fixed in ab24ee2. Step 02 now reuses a document already in the library (get_document_id()), so a rerun no longer uploads suffixed duplicates with the same tags. The chat calls stay unscoped on purpose: the point of the notebook is that the agent finds documents across the whole library by their tags. If the library holds other documents with matching tags, the agent finding them is the intended behavior.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ab24ee27b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +131 to +133

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Verify metadata before reusing an existing document

If a reader previously uploaded any of these example PDFs without tags, get_document_id() still finds it and this branch skips the only call that attaches metadata. The read-back step then returns missing or different tags, and the tag-based questions cannot reproduce the demonstrated results, especially the research-team query whose ownership exists only in metadata. Check that the existing document's metadata matches DOCS before reusing it; otherwise explain how to replace it with a tagged upload, since the notebook states metadata cannot be updated.

Useful? React with 👍 / 👎.

@BukeLy
BukeLy marked this pull request as draft October 21, 2026 11:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant