torrent-search-engine

Python library to search torrents from multiple websites.

About

This Python library allows you to search torrents by scraping torrent websites.

You can add websites to scrape by providing a configuration file (see ... on how to create a configuration file).

Usage

Requirements

This library requires python 3.5 or higher.

Installation

You can install this library from Github using this command:

$ pip install git+https://github.com/AlexCovizzi/torrent-search-engine

Example

from torrentsearchengine import TorrentSearchEngine

search_engine = TorrentSearchEngine()
# add torrent provider from a file or url
search_engine.add_provider(path)
# perform the query and find max 50 results
results = search_engine.search(query, limit=50)
# retrieve the details of the first result
details = results[0].fetch_details()
# get the magnet from the details
magnet = details.link

Provider File

You can add torrent providers from json files or an urls to a json file.

You can find an explained example of provider file in this example

Structure

The torrent provider json file has the following structure:

• name (required) - The name (and identifier) of the torrent provider.

• fullname (optional) - The full name of the torrent provider, this is the diplayed name. Defaults to the name.

• url (required) - The base url of the torrent provider.

• headers (optional) - HTTP headers to use in the requests to this website.

• search (required) - The search path relative to the base url.

• whitespace (optional) - The whitespaces in the query are replaced with this character (by default they are replace with %20).

  The path needs to include the search query as a variable using the keyword: {query}.

• list (required) - This property defines how to scrape the data from the torrent provider.

  It includes the following properties:

  • items (required) - The CSS selector that selects all the torrent html elements (e.g the rows of a table).

  • next (optional) - The CSS selector that selects the link to the next page.

  • item (required) - This property defines the selectors to scrape the data from a the torrent html element.

    The attributes currently supported are:

    • name (required) - Name of the torrent.
    • url (optional) - Link to the page of the torrent.
    • size (optional) - Size of the torrent.
    • seeds (optional) - Number of seeders of this torrent (the selected value must be an integer).
    • leechers (optional) - Number of leechers of this torrent (the selected value must be an integer).

• item (optional) - This property defines how to scrape the data from the torrent page (the page linked by the url torrent attribute).

Torrent's properties

The torrent's properties currently supported are:

• name (required) - Name of the torrent. This selector has to be defined in list.item.
• url (optional) - Link to the page of the torrent. This selector has to be defined in list.item.
• link (required) - Link to magnet or .torrent file. This need to be defined either in list.item or in item.
• size (optional) - Size of the torrent.
• seeds (optional) - Number of seeders of this torrent (the selected value must be an integer).
• leechers (optional) - Number of leechers of this torrent (the selected value must be an integer).
• time (optional) - Time of the torrent.
• infohash (optional) - INFOHASH of te torrent.
• files (optional) - List of files in the torrent.
• trackers (optional) - List of trackers in the torrent.
• description (optional) - Description of the torrent.
• uploader (optional) - The name of the user that uploaded the torrent.
• uploader_url (optional) - The link to the page of the user that uploaded the torrent.

Selector

The selector used to select the torrent data has the following structure:

<CSS selector> @ <html attribute> | re: <regex matcher> | fmt: <regex formatter>

The default html attribute is text (the inner text of the html element selected).

The options re and fmt are optional, their job is to select only part of the text or format the text, you can use regex capturing groups in re and fmt.