Description
When extending a DocList with itself, the code seems to add the list recursively, causing a huge increase in the length of the list. For example, the following code produces a DocList of length 9,052,000 (I had to interrupt the execution):
from docarray import DocVec, BaseDoc, DocList
from docarray.documents.image import ImageDoc
docs = DocList[ImageDoc]([ImageDoc(url=f'http://url.com/{i}.png') for i in range(10)])
docs.extend(docs)
However, a similar operation works fine with Python lists:
l = list(range(10))
l.extend(l)
print(l) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Questions
- Why does
DocList behave differently than Python lists in this situation?
- Are there plans to support the
extend method for DocVec in the future?
Steps to reproduce
- Run the following code snippet:
from docarray import DocVec, BaseDoc, DocList
from docarray.documents.image import ImageDoc
docs = DocList[ImageDoc]([ImageDoc(url=f'http://url.com/{i}.png') for i in range(10)])
docs.extend(docs)
-
Interrupt the execution because it hangs.
-
Observe that the resulting DocList has an enourmous length, rather than the expected 20.
Description
When extending a
DocListwith itself, the code seems to add the list recursively, causing a huge increase in the length of the list. For example, the following code produces aDocListof length 9,052,000 (I had to interrupt the execution):However, a similar operation works fine with Python lists:
Questions
DocListbehave differently than Python lists in this situation?extendmethod forDocVecin the future?Steps to reproduce
Interrupt the execution because it hangs.
Observe that the resulting
DocListhas an enourmous length, rather than the expected 20.