Skip to content

Latest commit

 

History

History
461 lines (325 loc) · 30.2 KB

File metadata and controls

461 lines (325 loc) · 30.2 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.45.0] - 2026-09-10

Added

  • facebook.get_post_detail(post_id, url=...) and facebook.get_post_comments(post_id, url=...) — a Facebook post URL comes in several shapes (/reel/, permalink.php, {page}/posts/{pfbid}), and url overrides post_id for all of them. The parameter has always worked on the API; it was missing from the SDK. Reported in SB-001096.
  • facebook.search_marketplace(..., radius=...) — search radius around the location, in km (miles in the US). Honoured on search; ignored on category browse.
  • zillow.get_multifamily_building(url=...) — the untyped facade twin of zillow.properties.get_building added in 0.44.0.
  • TikTok Shoptiktok.tiktok_shop_root_categories, tiktok_shop_category_subcategories_top_products, tiktok_shop_product_detail, tiktok_shop_product_reviews, search_tiktok_shop_products and tiktok_shop_store_products.

Fixed

  • Nothing in the SDK, but worth knowing if you use the Facebook endpoints: the page/profile feed served one post per call with an empty page 2, post detail 404'd every URL shape, and comment threads always came back empty. All four are fixed on the API side (SB-001096) — no SDK change was needed.

[0.44.0] - 2026-09-09

Added

  • zillow.properties.get_building(url) — Zillow's for-rent inventory is mostly apartment communities served on /apartments/... and /b/... pages, which get_property cannot read (it now raises a clear 400 pointing here). Returns floor plans and every available unit with rent, base rent, required monthly fees, sqft, beds/baths and move-in date, plus amenities, unit features, policies, special offers, office hours, pet policy, schools, photos and walk/transit/bike scores. Requested in SB-001087.
  • Building, BuildingFloorPlan, BuildingUnit, BuildingResponse, UnitSummary models.
  • Building fields on Listingis_building, building_name, lot_id, units_available, unit_summaries[]. Building search cards now also carry a price/price_raw: only paid cards expose a base-rent range, so the rest fall back to the per-bedroom rollup (they were previously priceless).

[0.43.0] - 2026-09-04

Added

  • Image input on chatgpt.ask.ask and gemini.ask.ask — pass image_url and the model looks at the picture and answers about it (JPEG, PNG, GIF, WEBP, BMP; up to 5 MB). An image ask is slower than a text one — allow 90-150 s. Requested in SB-001078.
  • images on both ask responses (MediaItem[]) — the pictures the answer itself displayed, with favicon-sized assets excluded.

Note

Neither endpoint will GENERATE an image. Anonymous chatgpt.com and gemini.google.com both gate image generation behind a login, so images is only ever what the answer showed, never something the model drew.

[0.42.0] - 2026-09-04

Added

  • ebay.search.search_by_image() — eBay's own visual search, the one behind the camera icon in eBay's search bar. Give it a picture as image_url (a public http(s) URL) or image_base64 (JPEG or PNG, up to 10 MB decoded; a data:image/jpeg;base64,... URL is accepted too) and get back the same SearchResponse that search() returns, with query set to None. Takes the usual SRP filters — domain, category_id, page, per_page, condition, buying_format, min_price, max_price, free_shipping, location, language. Costs 10 credits (two eBay fetches: the upload, then the results page). Passing neither or both images raises ValueError locally rather than costing a round trip.

    There is deliberately no sort_by: eBay ignores sorting on a visual results page, and a parameter that silently does nothing is worse than no parameter.

[0.41.0] - 2026-09-02

Added

  • wait_for_found and js_scenario_report on ScrapeResult — whether the wait_for selector appeared, and a per-step trace of the js_scenario (step, action, selector, ok, error, url) so a flow that does not end where you expected can be read step by step.

Changed

  • web.scrape(): a js_scenario now runs before wait_for, and a missed wait_for or a failed scenario step is a free 422 (wait_for_timeout / js_scenario_failed) instead of a billed copy of the page you started on. Scenario steps use action (click, fill, scroll, wait, evaluate).

[0.40.1] - 2026-09-02

Changed

  • Instagram: seven more methods flagged temporarily unavailableusers.related, media.oembed, media.comment_likers, hashtags.get, hashtags.recent, search.hashtags and search.top need the authenticated Instagram tier, which is offline; they return 503 temporarily_unavailable (not billed) until it is back. Their docstrings now say so, matching the rest of the gated set. No signature changes. (SCR-42, support SB-001065)

[0.40.0] - 2026-09-02

Added

  • catalog_id on google.shopping.offers — look up multi-seller Google Shopping offers by Google's own Shopping catalog id (the catalog_id on every shopping.search tile, or prds=catalogid:<id> in a Google Shopping URL) as an alternative to barcode. The response carries catalog_id and total_offers; barcode is now optional (exactly one of the two is required).

[0.39.0] - 2026-08-23

Added

  • gallery_images on RedditPost — multi-image gallery posts now expose every image at full source resolution, in the order the poster arranged them. Previously a gallery returned only is_gallery: true, a ~140px thumbnail, and a url pointing at reddit.com/gallery/<id> — the images themselves were unreachable. Populated on every post-returning endpoint: posts.get, search.posts, subreddit/user/domain listings, trending and duplicates.
  • New RedditGalleryImage model — media_id, url, width, height, mime_type, and mp4_url for animated items (whose url is the .gif). Exported from both scrapebadger and scrapebadger.reddit.

[0.38.0] - 2026-08-19

Added

  • New booking namespace — the Booking.com API. Five operations on client.booking: search_properties, search_destinations, get_property_detail, get_property_reviews and get_room_types_and_live_rates. Global coverage, any Booking-supported currency and locale, no upstream Booking account or key required.
  • get_room_types_and_live_rates returns the full per-room rate table for a property and dates — every room type with every rate bookable on it (price, price before discount, price per night, discounts, badges, occupancy, meal plan), plus per-room facilities, alternative bed layouts, photos and a priced child/crib/extra-bed policy. search_properties returns only the single cheapest rate per property.

[0.37.0] - 2026-08-14

Added

  • Spec-generated additions on the v1 surface (zero breaking changes): Facebook Ad Library advertiser page search (facebook.search_advertiser_pages) and page info/transparency (facebook.get_advertiser_page_info), the app_store and google_play namespaces, Google ads-transparency operations, TikTok Ad Library detail/advertiser search, and other newly added API operations. Generated methods return raw dicts until typed models land.

[0.36.0] - 2026-08-11

Added

  • Spec-generated additions on the v1 surface (zero breaking changes): new facebook, idealista, perplexity and account namespaces, plus previously missing operations (Twitter stream/filter-rule management, ChatGPT/Gemini POST variants, apartments/depop/redfin/zillow detail routes and more). Generated methods return raw dicts/objects until typed models land.
  • SDK is now regenerated automatically from the ScrapeBadger OpenAPI spec on every API change; a surface-lock CI check guarantees existing methods never change.

[0.24.2] - 2026-08-03

Fixed

  • A transient 500 no longer kills a long-running job. 500 was missing from the default retry_on_status, so while a 502 was retried, a 500 raised ServerError on the spot and burned the remaining 9 retries. Real-world sequences look like 502 → 500 → 200; the middle hop ended runs that were thousands of pages deep. The default is now (500, 502, 503, 504). Override retry_on_status on ClientConfig if you want the old behaviour.
  • Retry every transient transport failure, not just three of them. The retry path caught only ConnectError, ReadTimeout and WriteTimeout, so a ConnectTimeout, PoolTimeout, ProxyError, or a server dropping a pooled connection (RemoteProtocolError) failed immediately — the last of which is common on long-lived connection pools. UnsupportedProtocol and LocalProtocolError are still raised straight away: they are caller/config mistakes that no retry can fix.

[0.24.1] - 2026-08-02

Added

  • Google Lens price — each visual match now carries price as {value, currency, extracted} alongside the raw tag chip it is parsed from, so a shoppable match gives you a number to compute on instead of a "$115" string. currency is the symbol Google printed, not an ISO code ($ is ambiguous between USD/CAD/AUD). tag is unchanged. (SCR-150)

[0.24.0] - 2026-08-02

Added

  • Vinted catalog_ids search filterclient.vinted.search.search("nike", catalog_ids="221") restricts a search to one or more Vinted categories. Vinted applies it before the search runs (sub-categories included), so you get whole pages of in-category results instead of filtering them out client-side. A catalog ID is the catalog[] value in a Vinted category URL (vinted.fr/catalog?catalog[]=221); IDs are per market. (SCR-159)
  • Vinted item fields the SDK was droppingVintedItemSummary and VintedItemDetail now model path, service_fee, total_item_price, is_visible and promoted (plus content_source on the summary, and color2 / can_bundle / can_reserve / is_favourite on the detail). The API had always returned these; the models used extra="ignore", so they were silently discarded. (SCR-159)

Changed

  • Vinted service_fee / total_item_price are plain decimal strings — on search results they used to come back as a stringified Python dict ("{'amount': '0.75', 'currency_code': 'EUR'}") that no client could parse; the item endpoint already returned "0.75". Both now return the amount alone, in price.currency_code. (SCR-159)
  • Vinted VintedItemDetail.category documented as a breadcrumb list — it is the category path, root first, localized to the market; upload_date is Vinted's relative "listed" label, not an ISO timestamp. (SCR-159)
  • Corrected Vinted search billing docs — a search is a flat 5 credits whatever filters you pass. seller_country seller lookups were never billed on top, despite the old "1 credit + 1 per uncached seller" note. (SCR-159)

[0.23.0] - 2026-07-25

Added

  • eBay pagination.has_more — the stop signal for paging through completed/sold listings. total_pages/total_results are None on eBay's sold grid, and past the last page eBay re-serves that page instead of returning empty, so looping "until the results are empty" never terminated. Loop while r.pagination.has_more instead; the API now also returns an empty page once you page past the end. The page ceiling is raised to 1000 (a broad sold search runs ~133 pages deep at per_page=240, ~26k listings). (SCR-124)

Changed

  • eBay is_sponsored is now bool | None and always None — previously typed bool and reported True for 100% of results. eBay renders its "Sponsored" badge into every result card as anti-scraping bait, so promoted placements cannot be distinguished from organic ones in the response. The field reports None (unknown) rather than a value that was wrong on every row. (SCR-124)

[0.22.0] - 2026-07-23

Added

  • Google Flights sort_byclient.google.flights.search(..., sort_by="price") now returns the FULL price-sorted inventory (every carrier, the cheap long-layover fares) plus Google's own price floor, typical range, and price history, instead of only the ~6-8 "best" preview. Default sort_by="top" keeps the fast best-picks behaviour. (SCR-123)

[0.21.0] - 2026-07-17

Added

  • eBay completed-search sold datesSearchResult now exposes sold_date (sale date text as rendered by eBay, e.g. "2 Jul 2026"; localized on non-English markets) and sold_date_at (best-effort ISO date "2026-07-02", None when the market's format isn't English) on client.ebay completed/sold result cards. (SCR-122)
  • eBay item is_endedItem now exposes is_ended (True when the listing has closed — sold or ended — any buying format, default False). end_time_utc/end_time_at broaden from auction-only to the listing end: auction close or sold time for ended listings of any format; still None for active fixed-price listings. (SCR-122)

[0.20.0] - 2026-07-13

Added

  • LinkedIn client (client.linkedin) — LinkedIn's public, no-auth (logged-out) surface. Endpoints: jobs_search(), get_job(), company_jobs(), get_company(), get_school(), get_profile(), get_post(), get_article(), get_course(), geo_suggest(), health() under /v1/linkedin. Fully-typed Pydantic models (LinkedInJobsSearchResponse, LinkedInJobDetail, LinkedInProfile incl. nested experience/education, LinkedInCompany, LinkedInSchool, LinkedInPost, LinkedInLearningCourse, LinkedInGeoSuggestResponse). Sourced from the guest Jobs API and public SSR JSON-LD pages; deep logged-in data is auth-gated and out of scope. (SCR-119)

[0.18.0] - 2026-07-12

Added

  • Redfin client (client.redfin) — Redfin (redfin.com, US) for-sale search, property detail, agent profiles, autocomplete, and markets. Endpoints: search(), get_property(), get_agent(), autocomplete(), list_markets(), mirroring the Immobiliare flat-client pattern, with fully-typed maximal-coverage Pydantic models (RedfinSearchResponse, RedfinProperty incl. nested address/price_history/tax_history/schools/amenities, RedfinAgent, etc.). Single market: redfin.com (US, USD, en-US). (SCR-116)

[0.17.0] - 2026-07-08

Added

  • LoopNet sub-client (client.loopnet) — commercial-real-estate (CoStar) listings, brokers, and reference data across loopnet.com/.ca/.co.uk/.fr/.es (US/CA/UK/FR/ES). Endpoints: search.search() (for-lease / for-sale / auctions, all property types, filters, pagination), listings.get(), brokers.get(), reference.markets(), reference.property_types(), with fully-typed maximal-coverage Pydantic models (LoopnetSearchResponse, LoopnetListingDetail incl. offers/facts/brokers/media, LoopnetBrokerProfile, LoopnetListingCard, etc.). LoopNet is behind Akamai Bot Manager (browser-farm-only) — served via the ScrapeBadger farm. (SCR-102)

[0.15.7] - 2026-07-07

Added

  • Zillow sub-client (client.zillow) — real-estate listings, property detail, and agent profiles from zillow.com (US + Canadian inventory). Five endpoints: search.search(), search.autocomplete(), properties.get_property(), agents.get_agent(), reference.list_markets(), with fully-typed maximal-coverage Pydantic models (ZillowSearchResponse, ZillowProperty incl. nested home_facts/price_history/tax_history/schools/zestimate_history, ZillowAgent, etc.). (SCR-99)

[0.15.6] - 2026-07-07

Added

  • Add Leboncoin Scraper API client (France) — 10 endpoints: search, ad detail, similar ads, seller profile + listings, markets/regions/departments/categories/location search.

[0.15.5] - 2026-07-02

Added

  • Realtor sub-client (client.realtor) — real-estate listings across realtor.com (US) and realtor.ca (Canada) behind a single market parameter. Four endpoints: search.search(), search.autocomplete(), properties.get_property(), reference.list_markets(), with fully-typed Pydantic models (RealtorSearchResponse, RealtorPropertyDetail, etc.). (SCR-98)

[0.15.4] - 2026-06-30

Added

  • client.twitter.tweets.advanced_search() / advanced_search_all() — aliases for search() / search_all() that match the /advanced_search REST endpoint name, so the endpoint-named call no longer raises AttributeError. (SCR-52)

Fixed

  • Auto-pagination no longer repeats the first page (*_all iterators, e.g. client.twitter.tweets.search_all). The shared paginate() helper now stops when the backend returns the same cursor it was given, instead of re-fetching the page it just yielded — previously a non-advancing cursor from the API could cause a repeat-page loop. (SCR-52)

[0.15.3] - 2026-06-30

Added

  • Google Shopping offers by barcodeclient.google.shopping.offers(barcode, *, gl=None, hl="en") calls GET /v1/google/shopping/offers. Resolves a product barcode (GTIN-8/UPC-A/EAN-13/GTIN-14) to a product via Google web search, then returns its multi-seller Google Shopping prices (barcode, resolved_query, product_title, offers). Costs 14 credits; returns 422 for an invalid barcode and 404 if unresolvable.

[0.15.2] - 2026-06-22

Added

  • eBay auction data on the eBay client models:
    • SearchResult.current_bid and Item.current_bid — an auction's current high bid (mirrors price).
    • Item.end_time_utc / Item.end_time_at — the absolute auction end time (Unix float / ISO-8601 Z).
    • Item.buy_it_now_price — the Buy It Now price for fixed-price listings, or an auction that also offers Buy It Now (None for pure auctions).
    • bids (bid count) and time_left (relative remaining, e.g. "12h 16m") are now reliably populated for auction listings.

[0.15.0] - 2026-06-21

Added

  • eBay API client (client.ebay) covering all 12 endpoints across 18 marketplaces:
    • ebay.search.search() — active-listing keyword search (category, condition, buying_format, price, free-shipping filters)
    • ebay.search.completed() — completed/sold listings (sold-price history)
    • ebay.search.autocomplete() — keyword suggestions
    • ebay.items.get_item() — full listing detail (images, shipping, item specifics, seller, returns)
    • ebay.items.get_item_reviews() — catalog product reviews + rating histogram (optional product_id)
    • ebay.sellers.get_seller() / get_seller_items() / get_seller_feedback()
    • ebay.categories.browse_category() — browse listings within a category
    • ebay.reference.list_categories() / list_markets()
  • Frozen, forward-compatible Pydantic models for every eBay response field (exported with an Ebay-prefix at the top level, e.g. EbayItem, EbaySearchResult, EbaySeller, EbayReview, EbayPrice).

[0.9.0] - 2026-05-29

Changed (Breaking)

  • Reddit response models trimmed to fields available via old.reddit.com HTML/RSS — after Reddit deprecated the unauthenticated .json API, the ScrapeBadger backend switched to scraping old.reddit.com HTML and RSS feeds. Fields that Reddit no longer exposes through that source have been removed from all models to keep the SDK types in sync with what the API actually returns. Consuming code that accessed removed fields will receive AttributeError at runtime.

    RedditPost — removed: ups, downs, upvote_ratio, view_count, num_duplicates, edited, edited_at, is_video, is_locked, is_archived, is_pinned, is_robot_indexable, is_meta, is_crosspostable, send_replies, author_flair_text, author_flair_type, author_flair_template_id, link_flair_background_color, link_flair_text_color, link_flair_template_id, link_flair_type, link_flair_css_class, distinguished, thumbnail, thumbnail_width, thumbnail_height, post_hint, preview_images, media, gallery_data, crosspost_parent, suggested_sort, total_awards, awards, content_categories, removed_by_category, treatment_tags, subreddit_subscribers.

    RedditComment — removed: ups, downs, controversiality, edited, edited_at, gilded, is_locked, is_score_hidden, is_submitter, parent_id, post_title, send_replies, subreddit_type, total_awards, distinguished, author_flair_text, author_flair_type.

    RedditSubreddit — removed: subscribers, active_users, description_html, public_description_html, submit_text, submit_text_html, header_title, type, submission_type, is_quarantined, is_advertiser_friendly, advertiser_category, language, icon_url, header_url, banner_url, banner_background_color, primary_color, key_color, wiki_enabled, allow_images, allow_videos, allow_galleries, allow_polls, allow_discovery, spoilers_enabled, emojis_enabled, free_form_reports, accept_followers, restrict_posting, link_flair_enabled, link_flair_position, user_flair_enabled, user_flair_position, comment_score_hide_mins, should_archive_posts, allowed_media_in_comments.

    RedditUser — removed: id, fullname, description, icon_url, snoovatar_url, banner_url, profile_title, profile_url, awardee_karma, awarder_karma, has_verified_email, verified, accepts_followers, has_subscribed, is_employee, is_mod, is_suspended, is_nsfw, pref_show_snoovatar.

    RedditRule — removed: description_html, kind, violation_reason.

  • Helper models deletedRedditPreviewImage, RedditMedia, RedditAward, and RedditUserSummary are no longer exported (they were only used by removed fields). Remove any imports of these names from consuming code.

Added

  • RedditPagination — new standalone pagination model (after, before, count, limit) exported from scrapebadger.reddit.

[0.8.3] - 2026-05-29

Fixed

  • Reddit search & listing methods sent the wrong query-parameter names, causing a 422 'q' field required against the live API. The API expects q (search query) and t (time filter); the SDK was sending query and time_filter. Fixed across search.posts, search.subreddits, search.users, search.domain_posts, subreddits.posts, users.posts, users.comments. The Python keyword arguments are unchanged (query=, time_filter=) — only the wire params are corrected.
  • search.domain_posts() hit the wrong URL/v1/reddit/search/domain with domain as a query param. The real route is /v1/reddit/domains/{domain}/posts (domain is a path segment). Fixed.

[0.8.2] - 2026-05-29

Fixed

  • client.reddit.posts.get() and client.reddit.posts.comments() were calling the wrong URL/v1/reddit/posts/{subreddit}/{post_id} (two path segments) instead of the actual API route /v1/reddit/posts/{post_id}. Both methods 404'd against the live API in 0.8.0/0.8.1. The subreddit positional argument has been removed from both methods — call posts.get(post_id) and posts.comments(post_id, ...). This is a breaking change for anyone who worked around the bug, but the previous signature never worked against production.

[0.8.1] - 2026-05-28

Removed

  • client.reddit.subreddits.moderators(...) — Reddit gated the moderator listing behind authentication in 2024. There is no public path that yields the data. Removed the method along with RedditModerator and SubredditModeratorResponse types rather than ship a permanently-broken endpoint.

Fixed

  • client.reddit.search.subreddits(...) no longer crashes when Reddit returns banned/quarantined subreddits with null values for required-typed fields (previously raised a backend 500 / ValidationError).

[0.8.0] - 2026-05-28

Added

  • Reddit Scraper API — new client.reddit.* namespace covering 22 endpoints across search, posts, subreddits, users, and wiki:
    • client.reddit.search.posts(q, ...) — global or subreddit-scoped post search with full Reddit syntax (title:, author:, subreddit:, flair:, AND/OR/NOT)
    • client.reddit.search.subreddits(q, ...), client.reddit.search.users(q, ...)
    • client.reddit.search.domain_posts(domain, ...) — posts linking to an external domain
    • client.reddit.posts.trending(...), client.reddit.posts.get(post_id)
    • client.reddit.posts.comments(post_id, depth=...) — full nested comment trees with configurable depth (0–10)
    • client.reddit.posts.duplicates(post_id, ...) — cross-post detection
    • client.reddit.subreddits.get(name), .posts(...), .rules(...), .moderators(...), .wiki_pages(...), .wiki_page(...), .popular(...), .new(...)
    • client.reddit.users.get(name), .posts(...), .comments(...), .moderated(...), .trophies(...)
  • Comprehensive Reddit response modelsRedditPost (66 fields), RedditComment (34), RedditSubreddit (48), RedditUser (27)
  • Datetime parity — every datetime field ships both Unix timestamp (*_utc) and ISO 8601 UTC string (*_at)
  • Frozen Pydantic models with extra="ignore" for forward compatibility

[0.7.0] - 2026-04-21

Added

  • client.google.shopping.product(product_id, ...) — Shopping product detail page fetch by product_id.
  • client.google.shopping.click(title, source, q, product_id?) — Resolve the direct merchant URL for a Shopping product tile via Google's "I'm Feeling Lucky" redirect.
  • client.google.search.light(q, ...) — Lightweight mode=fast SERP — organic results + related searches only, ~40% faster than the full SERP.
  • client.google.search.search(..., mode="full" | "fast") — New mode parameter on the main search method. mode="fast" hits Google's lite gbv=1 endpoint.
  • client.google.maps.posts(..., next_page_token=...) — Pagination support on business posts.

Changed

  • Removed hardcoded per-endpoint credit numbers from docstrings and docs. Credit costs are configured per-endpoint by ScrapeBadger admins and returned live from GET /public/pricing — no more stale "costs 2 credits" comments that go out of sync when pricing changes.
  • client.google.products.detail(product_id, q=None, ...)q is now optional. The backend accepts lookups by product_id alone; pass q only when you want the richer /async/oapv context blob.
  • CHANGELOG section heading for 0.6.0 renamed from "Google Scrapingdog parity" to "Google parity".

Removed

  • client.google.local — removed. The Local Pack is exposed via the SERP /v1/google/search response's local_results field rather than a dedicated endpoint. Breaking for anyone using the dedicated local client; migrate to reading local_results from client.google.search.search(...).

[0.6.0] - 2026-04-11

Added — Google parity (refs scrape-badger/scrapebadger#135)

Three new Google product sub-clients and deeper Scholar / Trends / Search surface:

  • client.google.local — Local Pack business listings ranked for a SERP query (tbm=lcl). Driven by q + location/uule, returns ratings, reviews, addresses, phone numbers, and GPS coordinates. Complementary to the Maps API.
  • client.google.shorts — Short-form vertical video results (YouTube Shorts, TikTok, Facebook Reels) via Google's Shorts SERP mode (udm=39).
  • client.google.flights — One-way, round-trip, and multi-city flight search with passenger config, cabin class, stops filter, and max-price. Returns best_flights, other_flights, price_insights, and per-offer carbon emissions.
  • client.google.scholar.profiles(mauthors, ...) — Author profile search by name with after_author / before_author pagination.
  • client.google.scholar.author(author_id, ...) — Full author profile (articles, citation stats, co-authors).
  • client.google.scholar.author_citation(author_id, ...) — Citations-per-year chart for a Scholar author.
  • client.google.scholar.cite(q, ...) — MLA / APA / Chicago / Harvard / Vancouver citation formats plus export links (BibTeX / RIS / EndNote / RefWorks).
  • client.google.trends.autocomplete(q, ...) — Categorized Knowledge Graph topic entities (mid, type, link) for a query prefix. Distinct from Google Search autocomplete.
  • client.google.search.search(..., ai_overview=True) — Optional flag that chases Google's deferred AI Overview page_token with a follow-up fetch and merges the result into ai_overview. Adds ~1s and 1 credit only when the SERP actually defers the overview.

The Google product roster is now 19 (was 16).

[0.2.0] - 2026-03-05

Added

  • Twitter Streams: Real-time tweet monitoring via WebSocket and webhooks
    • StreamClient with full monitor CRUD: create_monitor, list_monitors, get_monitor, update_monitor, pause_monitor, resume_monitor, delete_monitor
    • WebSocket streaming via connect() async context manager with auto-reconnect support
    • verify_webhook_signature() utility for HMAC-SHA256 webhook verification
    • Delivery log and billing log listing: list_delivery_logs, list_billing_logs
    • Full type-safe models: StreamMonitor, StreamMonitorList, TweetEvent, ConnectedEvent, PingEvent, ErrorEvent, DeliveryLog, BillingLog
    • WebSocketStreamError exception for stream connection failures

0.1.1 - 2024-12-27

Fixed

  • Fixed Python 3.10 compatibility for StrEnum
  • Fixed all ruff linting issues
  • Fixed mypy type checking errors
  • Fixed GitHub badge URLs to correct organization

0.1.0 - 2024-12-27

Added

  • Initial release of the ScrapeBadger Python SDK
  • Full async support with httpx
  • Strongly-typed responses using Pydantic v2
  • Twitter API client with 37+ endpoints:
    • Tweets: get by ID, search, get replies, retweeters, favoriters
    • Users: get by username/ID, followers, following, search
    • Lists: get details, members, tweets, search
    • Communities: get details, members, moderators, tweets
    • Trends: get trends, place-specific trends, available locations
    • Geo: search places, get place details
  • Automatic pagination with async iterators
  • Built-in retry logic with exponential backoff
  • Comprehensive exception handling
  • Full type hints for IDE support

Security

  • API key authentication
  • No sensitive data logged