Use Reddit Subreddit Post & Comment Harvester with Python
Pull posts from any subreddit with score, media, flairs, awards and moderation flags, plus optional nested comment threads. Filter by date, score, keyword, author, post type and more. No API key or login.
1. Choose the data you need
Review the published README for supported collection modes, examples, and limits. The recipe below uses the current published input definition.
2. Prepare a source-based input
This JSON contains published prefills and required-field defaults. Review every value for your task before running it; it is not a promise of successful collection.
{
"commentSort": "confidence",
"excludeCrossposts": false,
"excludeDeletedAuthor": false,
"excludeSpoilers": false,
"excludeStickied": false,
"fullSubreddit": false,
"includeComments": false,
"includeNSFW": false,
"maxCommentsPerPost": 0,
"maxItems": 10000,
"maxPosts": 25,
"onlyWithFlair": false,
"sort": "new",
"subreddits": [
"python"
],
"timeFilter": "day"
}
Download input.example.json and save your reviewed copy as input.json.
| Field | Type | Required | Description |
|---|---|---|---|
subreddits | array | required | List of subreddits to scrape. Accepts bare names (python), r/ prefixed (r/python) or full URLs (https://www.reddit.com/r/python/). |
maxPosts | integer | optional | Maximum number of posts to keep from each subreddit after filters. Reddit serves roughly 1,000 posts per hot/new listing and about 500 for top/controversial. |
maxItems | integer | optional | Global safety cap on the total number of records pushed for the whole run (posts and comments combined). |
sort | string | optional | How to sort posts in the subreddit. Reddit serves roughly 1,000 posts per hot/new listing and about 500 for top/controversial. |
timeFilter | string | optional | Time filter when sorting by 'top' or 'controversial'. |
includeComments | boolean | optional | When enabled, the full comment thread (including nested replies) is collected for every scraped post. Significantly increases run time and output. |
maxCommentsPerPost | integer | optional | Maximum number of comments to scrape per post (only used when 'Include comments' is enabled). |
commentSort | string | optional | How to sort the comment threads. |
postedAfter | string | optional | Only keep posts created on or after this date (UTC). When set, collection runs newest-first (sort=new) and the sort option above is ignored. Reddit caps each listing at roughly 1,000 posts, so a distant date window on a very active subreddit may not be fully reachable. |
postedBefore | string | optional | Only keep posts created on or before this date (interpreted as the end of that day, UTC). |
includeNSFW | boolean | optional | Include posts Reddit explicitly flags as NSFW. Posts in 18+ communities are served by the actor's adult-content cookie and are not distinguishable at community level. |
onlyWithFlair | boolean | optional | When enabled, posts without a flair tag are dropped from the results. |
postType | string | optional | Keep only posts of the selected content type (based on Reddit's listing type). Leave unset to keep all types. Legacy 'self' is treated as 'text'. |
minScore | integer | optional | Only keep posts with a score at or above this value. Leave unset to keep all scores. |
maxScore | integer | optional | Only keep posts with a score at or below this value. Useful for finding overlooked/low-visibility posts. Leave unset to keep all scores. |
minComments | integer | optional | Only keep posts with at least this many comments. Leave unset to keep all posts. |
maxComments | integer | optional | Only keep posts with at most this many comments. Useful for finding under-discussed posts. Leave unset to keep all posts. |
excludeStickied | boolean | optional | When enabled, posts pinned/stickied by moderators (e.g. rules, weekly threads) are dropped from the results. |
minUpvoteRatio | number | optional | Only keep posts whose upvote ratio (fraction of votes that are upvotes, 0-1) is at or above this value. Useful for filtering out controversial/divisive posts. Leave unset to keep all ratios. |
excludeCrossposts | boolean | optional | When enabled, posts that are crossposts from another subreddit are dropped from the results, keeping only original submissions to the scraped subreddit. |
excludeSpoilers | boolean | optional | When enabled, posts marked as a spoiler by their author are dropped from the results. |
titleContains | string | optional | Only keep posts whose title contains this text (case-insensitive substring match). Leave empty to keep all titles. |
contentContains | string | optional | Only keep self (text) posts whose body contains this text (case-insensitive substring match). Link/image/video posts have no body text and are excluded when this is set. Leave empty to keep all posts. |
flairContains | string | optional | Only keep posts whose flair text contains this text (case-insensitive substring match). Leave empty to keep all flairs. |
authors | array | optional | Only keep posts submitted by one of these usernames (bare name or u/-prefixed, case-insensitive). Leave empty to keep posts from all authors. |
domain | string | optional | Only keep posts whose link domain contains this text (case-insensitive, e.g. 'youtube.com' or 'self.python'). Leave empty to keep all domains. |
minAwards | integer | optional | Only keep posts with at least this many total awards. Leave unset to keep all posts. |
excludeDeletedAuthor | boolean | optional | When enabled, posts whose author account was deleted (shown as [deleted]/[removed]) are dropped from the results. |
excludeAuthors | array | optional | Drop posts submitted by any of these usernames (bare name or u/-prefixed, case-insensitive) — useful for filtering out bots (e.g. AutoModerator) or known spam accounts. Leave empty to keep posts from all authors. |
excludeKeywords | array | optional | Drop posts whose title, self-text body or flair contains any of these keywords or phrases (case-insensitive substring match) — useful for filtering out megathreads, giveaways or spam boilerplate. Leave empty to keep all posts. |
minContentLength | integer | optional | Only keep self (text) posts whose body is at least this many characters long. Link/image/video posts have no body and are excluded when this is set. Useful for dropping one-line/low-effort text posts when building NLP datasets. Leave unset to keep all lengths. |
maxContentLength | integer | optional | Only keep self (text) posts whose body is at most this many characters long. Leave unset to keep all lengths. |
fullSubreddit | boolean | optional | When enabled, walk Reddit's 'new' listing as deep as the source allows (roughly 1,000 posts per listing) instead of stopping at the first page. Combine with a higher 'Maximum posts per subreddit'. |
3. Start an Actor run
Set APIFY_TOKEN in your environment. Python 3.10 or newer is sufficient; this recipe uses no third-party packages.
Execution may incur charges. Review current pricing on Apify. The request asks for a 120-second timeout and a charge limit; these settings are subject to Apify’s pricing and do not guarantee an all-inclusive spending cap.
import json
import os
import urllib.request
# Save the schema-derived example as input.json; review and edit it first.
with open("input.json", encoding="utf-8") as stream:
actor_input = json.load(stream)
token = os.environ["APIFY_TOKEN"]
request = urllib.request.Request(
"https://api.apify.com/v2/acts/BKePs1sWHACaSpzyh/runs?timeout=120&maxTotalChargeUsd=0.50",
data=json.dumps(actor_input).encode("utf-8"),
headers={"Authorization": "Bearer " + token, "Content-Type": "application/json"},
method="POST",
)
# This starts a potentially billable run. Do not automatically retry this POST.
with urllib.request.urlopen(request, timeout=40) as response:
run = json.load(response)["data"]
print("Run ID:", run["id"])
print("Status:", run["status"])
# Inspect and download results in Apify Console once the run succeeds.
4. Inspect the actual results
Open the run in Apify Console and wait for a successful completion. Use the published output schema to locate results and the dataset schema to understand declared views. Missing or optional record fields must not be treated as guaranteed values.
Reddit subreddit posts with score, flair, selftext, images, galleries, video, polls and crosspost context, plus optional nested comment threads.
Questions before you run
Which inputs are required?
The published required fields are subreddits
What does this guide verify?
The fields, prefills, and schema links are rendered from the published build. This is a usage recipe, not a paid live extraction test. Source availability and collection results can vary.
What if the run request times out?
Check Apify Console before retrying. The first request may already have created a run. Do not automatically repeat a run-creation POST.
Generated from source build qdSuwLoNAqkhDc3y4 and its README and schemas. Source changes regenerate this guide; no third-party editorial claims are inferred.