Documentation Index
Fetch the complete documentation index at: /docs/llms.txt
Use this file to discover all available pages before exploring further.
Skip to main contentBasic
No need to run
You make HTTP requests by passing a URL and optional request parameters to one of Requests’ 7 HTTP request methods.
Here’s how to make a basic HTTP request on Pipedream:
The Response object
Requests automatically decodes the content of the response based on its encoding,
If JSON decoding fails, Making a
GET requests typically are for retrieving data from an API. Below is an example.
Making a
When you make a
Passing query string parameters to a
Retrieve fake comment data on a specific post using JSONPlaceholder, a free mock API service. Here, you fetch data from the
You should pass query string parameters as a dictionary using the
Downloading a file to the
This example shows you how to download a file to a file in the
Uploading a file from the
This example shows you how to make a
Alternatively, you can use Environment Variables as well for simple API key based GraphQL APIs.
Making HTTP Requests with Python
Workflows and String are shutting down on March 31, 2027. Learn more
requests HTTP client package available in Python to send HTTP requests, but you can use any PyPI package you’d like on Pipedream.
Basic requests usage notes
No need to run pip install, just import requests at the top of your step’s code and it’s available for your code to use.
To use requests on Pipedream, you’ll just need to import the requests PyPI package:
import requests
r = requests.get('https://swapi.dev/api/films/')
r contains a lot of information about the response: its content, headers, and more. Typically, you just care about the content, which you can access in the text property of the response:
r = requests.get('https://swapi.dev/api/films/')
# HTTP response content is in the text property
r.text
r.encoding, which is determined based on the HTTP headers.
If you’re dealing with JSON data, you can call r.json() to decode the content as JSON:
r = requests.get('https://swapi.dev/api/films/')
# The json-encoded content of a response, if any
r.json()
r.json() raises an exception.
Making a GET request
GET requests typically are for retrieving data from an API. Below is an example.
import requests
def handler(pd: "pipedream"):
url = "https://swapi.dev/api/people/1"
r = requests.get(url)
# The response is logged in your Pipedream step results:
print(r.text)
# The response status code is logged in your Pipedream step results:
print(r.status_code)
Making a POST request
import requests
def handler(pd: "pipedream"):
# This a POST request to this URL will echo back whatever data we send to it
url = "https://postman-echo.com/post"
data = {"name": "Bulbasaur"}
r = requests.post(url, data=data)
# The response is logged in your Pipedream step results:
print(r.text)
# The response status code is logged in your Pipedream step results:
print(r.status_code)
POST request, pass a dictionary with the data you’d like to send to the data argument. Requests automatically form-encodes the data when the request is made.
The code example above will NOT set the
Content-Type header, meaning it will NOT be set to application/json.If you want the header set to application/json and don’t want to encode the dict yourself, you can pass it using the json parameter and it will be encoded automatically: url = "https://postman-echo.com/post"
data = {"name": "Bulbasaur"}
r = requests.post(url, json=data)
Passing query string parameters to a GET request
Retrieve fake comment data on a specific post using JSONPlaceholder, a free mock API service. Here, you fetch data from the /comments resource, retrieving data for a specific post by query string parameter: /comments?postId=1.
import requests
def handler(pd: "pipedream"):
url = "https://jsonplaceholder.typicode.com/comments"
params = {"postId": 1}
# Make an HTTP GET request using requests
r = requests.get(url, params=params)
# Retrieve the content of the response
data = r.text
params keyword argument, like above. When you do, requests automatically URL-encodes the parameters for you, which you’d otherwise have to do manually.
Sending a request with HTTP headers
To add HTTP headers to a request, pass a dictionary to theheaders parameter:
import requests
import json
def handler(pd: "pipedream"):
url = "https://jsonplaceholder.typicode.com/posts"
headers = {"Content-Type": "application/json"}
data = {"name": "Luke"}
# Make an HTTP POST request using requests
r = requests.post(url, headers=headers, data=json.dumps(data))
Sending a request with a secret or API key
Most APIs require you authenticate HTTP requests with an API key or other token. Please review the docs for your service to understand how they accept this data. Here’s an example showing an API key passed in an HTTP header:import requests
def handler(pd: "pipedream"):
url = "https://jsonplaceholder.typicode.com/posts"
headers = {"X-API-KEY": "123"} # API KEY
data = {"name": "Luke"}
# Make an HTTP POST request using requests
r = requests.post(url, headers=headers, json=data)
Sending files
An example of sending a previously stored file in the workflow’s/tmp directory:
import requests
def handler(pd: "pipedream"):
# Retrieving a previously saved file from workflow storage
files = {"image": open("/tmp/python-logo.png", "rb")}
r = requests.post(url="https://api.imgur.com/3/image", files=files)
Downloading a file to the /tmp directory
This example shows you how to download a file to a file in the /tmp directory. This can be especially helpful for downloading large files: it streams the file to disk, minimizing the memory the workflow uses when downloading the file.
import requests
def handler(pd: "pipedream"):
# Download the webpage HTML file to /tmp
with requests.get("https://example.com", stream=True) as response:
# Check if the request was successful
response.raise_for_status()
# Open the new file /tmp/file.html in binary write mode
with open("/tmp/file.html", "wb") as file:
for chunk in response.iter_content(chunk_size=8192):
# Write the chunk to file
file.write(chunk)
Uploading a file from the /tmp directory
This example shows you how to make a multipart/form-data request with a file as a form part. You can store and read any files from the /tmp directory.
This can be especially helpful for uploading large files: it streams the file from disk, minimizing the memory the workflow uses when uploading the file.
import requests
from requests_toolbelt.multipart.encoder import MultipartEncoder
def handler(pd: "pipedream"):
m = MultipartEncoder(fields={
'file': ('filename', open('/tmp/file.pdf', 'rb'), 'application/pdf')
})
r = requests.post("https://example.com", data=m,
headers={'Content-Type': m.content_type})
IP addresses for HTTP requests made from Pipedream workflows
By default, HTTP requests made from Pipedream can come from a large range of IP addresses. If you need to restrict the IP addresses HTTP requests come from, you can Use a Pipedream VPC to route all outbound HTTP requests through a single IP address.Using an HTTP proxy to proxy requests through another host
By default, HTTP requests made from Pipedream can come from a range of IP addresses. If you need to make requests from a single IP address, you can route traffic through an HTTP proxy:import requests
def handler(pd: "pipedream"):
user = "USERNAME" # Replace with your HTTP proxy username
password = "PASSWORD" # Replace with your HTTP proxy password
host = "10.10.1.10" # Replace with the HTTP proxy URL
port = 1080 # Replace with the port of the HTTP proxy
proxies = {
"https": f"http://{user}:{password}@{host}:{port}",
}
r = requests.request("GET", "https://example.com", proxies=proxies)
Paginating API requests
When you fetch data from an API, the API may return records in “pages”. For example, if you’re trying to fetch a list of 1,000 records, the API might return those in groups of 100 items. Different APIs paginate data in different ways. You’ll need to consult the docs of your API provider to see how they suggest you paginate through records.Sending a GraphQL request
Construct a GraphQL query as a string and then using the requests library to send it to the GraphQL server:import requests
def handler(pd: "pipedream"):
url = "https://beta.pokeapi.co/graphql/v1beta"
query = """
query samplePokeAPIquery {
generations: pokemon_v2_generation {
name
pokemon_species: pokemon_v2_pokemonspecies_aggregate {
aggregate {
count
}
}
}
}
"""
r = requests.post(url, json={"query": query})
return r.json()
Sending an authenticated GraphQL request
Authenticate your connected accounts in Pipedream with GraphQL requests usingpd.inputs[appName]["$auth"]:
import requests
def handler(pd: "pipedream"):
url = "https://api.github.com/graphql"
query = """
query {
viewer {
login
}
}
"""
token = pd.inputs["github"]["$auth"]["oauth_access_token"]
headers = {"authorization": f"Bearer {token}"}
r = requests.post(url, json={"query": query}, headers=headers)
return r.json()
Was this page helpful?
Assistant
Responses are generated using AI and may contain mistakes.
Suggestions