Draft PyArrow Dataset reader impl by wjones127 · Pull Request #21 · datafusion-contrib/datafusion-python · GitHub
Skip to content
This repository was archived by the owner on Jul 25, 2022. It is now read-only.

Draft PyArrow Dataset reader impl#21

Closed
wjones127 wants to merge 3 commits into
datafusion-contrib:mainfrom
wjones127:10-pyarrow-datasets
Closed

Draft PyArrow Dataset reader impl#21
wjones127 wants to merge 3 commits into
datafusion-contrib:mainfrom
wjones127:10-pyarrow-datasets

Conversation

@wjones127

@wjones127 wjones127 commented Jan 30, 2022

Copy link
Copy Markdown

Work in progress. Working toward being able to stream record batches from a PyArrow dataset.

Fixes #10.

@wjones127

Copy link
Copy Markdown
Author

@kdbrooks

Copy link
Copy Markdown

I believe I have a working solution for this that I developed for the company I work for. I will get a PR out there soon. Is there still a need for this?

@wjones127

Copy link
Copy Markdown
Author

I believe I have a working solution for this that I developed for the company I work for. I will get a PR out there soon. Is there still a need for this?

This was mostly an experimental curiosity, but a PR would be cool if you are willing :)

I probably won't get around to finishing this for a while.

@kdbrooks

Copy link
Copy Markdown

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support reading from PyArrow datasets

2 participants