FilesExpand file tree

main

/

fetch_node.py

Copy path

Blame

More file actions

Blame

More file actions

Latest commit

 

History

History

History

30 lines (22 loc) · 565 Bytes

main

/

fetch_node.py

Copy path

Top

File metadata and controls

• Code
• Blame

30 lines (22 loc) · 565 Bytes

Raw

Copy raw file

Download raw file

Open symbols panel

Edit and raw actions

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

"""

Example of custom graph using existing nodes

"""

from scrapegraphai.nodes import FetchNode

# ************************************************

# Define the node

# ************************************************

fetch_node = FetchNode(

input="url | local_dir",

output=["doc"],

node_config={

"headless": False

}

)

# ************************************************

# Test the node

# ************************************************

state = {

"url": "https://twitter.com/home"

}

result = fetch_node.execute(state)

print(result)