FilesExpand file tree

main

/

smart_scraper_lite_groq.py

Copy path

Blame

More file actions

Blame

More file actions

Latest commit

 

History

History

History

31 lines (26 loc) · 744 Bytes

main

/

smart_scraper_lite_groq.py

Copy path

Top

File metadata and controls

• Code
• Blame

31 lines (26 loc) · 744 Bytes

Raw

Copy raw file

Download raw file

Open symbols panel

Edit and raw actions

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

"""

Basic example of scraping pipeline using SmartScraper

"""

import os

import json

from dotenv import load_dotenv

from scrapegraphai.graphs import SmartScraperLiteGraph

from scrapegraphai.utils import prettify_exec_info

load_dotenv()

graph_config = {

"llm": {

"api_key": os.getenv("GROQ_API_KEY"),

"model": "mixtral-8x7b-32768",

},

"verbose": True,

"headless": False,

}

smart_scraper_lite_graph = SmartScraperLiteGraph(

prompt="Who is Marco Perini?",

source="https://perinim.github.io/",

config=graph_config

)

result = smart_scraper_lite_graph.run()

print(json.dumps(result, indent=4))

graph_exec_info = smart_scraper_lite_graph.get_execution_info()

print(prettify_exec_info(graph_exec_info))