Use Yandex News Story Clusters Across Russia & the CIS with Python

Search or browse Yandex News and get story clusters with every linked article, topic tags, source names and publication dates across Russian and CIS editions.

1. Choose the data you need

Review the published README for supported collection modes, examples, and limits. The recipe below uses the current published input definition.

2. Prepare a source-based input

This JSON contains published prefills and required-field defaults. Review every value for your task before running it; it is not a promise of successful collection.

{
  "category": "politics",
  "excludeDomains": [],
  "excludeKeywords": [],
  "excludePublishers": [],
  "includeArticles": true,
  "includeDetails": true,
  "includeDomains": [],
  "includeKeywords": [],
  "includePublishers": [],
  "maxArticlesPerStory": 3,
  "maxStories": 3,
  "mode": "searchNews",
  "searchQueries": [
    "Яндекс"
  ],
  "sortOrder": "newest",
  "storyUrls": [],
  "timeRange": "any"
}

Download input.example.json and save your reviewed copy as input.json.

Field	Type	Required	Description
`mode`	string	required	What to scrape. `searchNews` searches keywords, `browseCategory` reads a rubric feed, `topStories` reads the homepage sections, and `byStoryUrls` resolves specific story clusters.
`searchQueries`	array	optional	One or more keywords or phrases (mode=searchNews). Russian and English queries are supported. The daily-test prefill uses one query.
`category`	string	optional	Rubric to browse (mode=browseCategory). `all` reads all 16 rubrics in one run.
`storyUrls`	array	optional	Dzen News story or instory URLs, or bare cluster UUIDs (mode=byStoryUrls). A bare numeric persistent ID is not a valid cluster URL.
`maxStories`	integer	optional	Maximum number of story clusters to return across the whole run. Yandex search depth varies by query.
`sortOrder`	string	optional	Order of the returned search clusters (mode=searchNews). Yandex search itself is relevance-ranked; these options sort the returned set.
`timeRange`	string	optional	Keep clusters published in this window (mode=searchNews). Applied to the results Yandex returns. A custom date range overrides this field.
`publishedAfter`	string	optional	Inclusive start date in YYYY-MM-DD format (mode=searchNews). Leave blank for no lower bound.
`publishedBefore`	string	optional	Inclusive end date in YYYY-MM-DD format (mode=searchNews). Leave blank for no upper bound.
`includeDomains`	array	optional	Keep a cluster when at least one linked article is from one of these domains (for example `rbc.ru`).
`excludeDomains`	array	optional	Drop a cluster when any linked article is from one of these domains.
`includePublishers`	array	optional	Keep a cluster when any linked source name contains one of these values (case-insensitive).
`excludePublishers`	array	optional	Drop a cluster when any linked source name contains one of these values (case-insensitive).
`includeKeywords`	array	optional	Keep a cluster when its headline, summary or an article headline/summary contains at least one of these values.
`excludeKeywords`	array	optional	Drop a cluster when its headline, summary or an article headline/summary contains any of these values.
`includeDetails`	boolean	optional	Fetch the cluster page for topic labels, source-reported counts, images, country/region, comments and likes.
`includeArticles`	boolean	optional	Emit one record per linked article in every cluster. Turn this off for a story-only dataset.
`maxArticlesPerStory`	integer	optional	Maximum article records and article pages fetched per cluster. Yandex returns 100 articles per page.

3. Start an Actor run

Set APIFY_TOKEN in your environment. Python 3.10 or newer is sufficient; this recipe uses no third-party packages.

Execution may incur charges. Review current pricing on Apify. The request asks for a 120-second timeout and a charge limit; these settings are subject to Apify’s pricing and do not guarantee an all-inclusive spending cap.

import json
import os
import urllib.request

# Save the schema-derived example as input.json; review and edit it first.
with open("input.json", encoding="utf-8") as stream:
    actor_input = json.load(stream)

token = os.environ["APIFY_TOKEN"]
request = urllib.request.Request(
    "https://api.apify.com/v2/acts/ZB27QppaE9g40P9UW/runs?timeout=120&maxTotalChargeUsd=0.50",
    data=json.dumps(actor_input).encode("utf-8"),
    headers={"Authorization": "Bearer " + token, "Content-Type": "application/json"},
    method="POST",
)
# This starts a potentially billable run. Do not automatically retry this POST.
with urllib.request.urlopen(request, timeout=40) as response:
    run = json.load(response)["data"]
print("Run ID:", run["id"])
print("Status:", run["status"])
# Inspect and download results in Apify Console once the run succeeds.

4. Inspect the actual results

Open the run in Apify Console and wait for a successful completion. Use the published output schema to locate results and the dataset schema to understand declared views. Missing or optional record fields must not be treated as guaranteed values.

Story clusters and their linked articles from Yandex News / Dzen News, including source counts, topics, images and publication times.

Questions before you run

Which inputs are required?

The published required fields are mode

What does this guide verify?

The fields, prefills, and schema links are rendered from the published build. This is a usage recipe, not a paid live extraction test. Source availability and collection results can vary.

What if the run request times out?

Check Apify Console before retrying. The first request may already have created a run. Do not automatically repeat a run-creation POST.

Open on Apify Full Actor reference

Generated from source build bC913XMWUHe0Emsxm and its README and schemas. Source changes regenerate this guide; no third-party editorial claims are inferred.