You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/WebApi.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -412,7 +412,7 @@ catch (ApiException e)
412
412
413
413
Scrape a URL
414
414
415
-
Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
415
+
Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
Copy file name to clipboardExpand all lines: src/ScrapeBadger/Api/WebApi.cs
+8-8Lines changed: 8 additions & 8 deletions
Original file line number
Diff line number
Diff line change
@@ -111,7 +111,7 @@ public interface IWebApiSync : IApiAccessor
111
111
/// Scrape a URL
112
112
/// </summary>
113
113
/// <remarks>
114
-
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
114
+
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
115
115
/// </remarks>
116
116
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
117
117
/// <returns>Object</returns>
@@ -121,7 +121,7 @@ public interface IWebApiSync : IApiAccessor
121
121
/// Scrape a URL
122
122
/// </summary>
123
123
/// <remarks>
124
-
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
124
+
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
125
125
/// </remarks>
126
126
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
127
127
/// <returns>ApiResponse of Object</returns>
@@ -303,7 +303,7 @@ public interface IWebApiAsync : IApiAccessor
303
303
/// Scrape a URL
304
304
/// </summary>
305
305
/// <remarks>
306
-
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
306
+
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
307
307
/// </remarks>
308
308
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
309
309
/// <param name="cancellationToken">Cancellation Token to cancel the request.</param>
@@ -314,7 +314,7 @@ public interface IWebApiAsync : IApiAccessor
314
314
/// Scrape a URL
315
315
/// </summary>
316
316
/// <remarks>
317
-
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
317
+
/// Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
318
318
/// </remarks>
319
319
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
320
320
/// <param name="cancellationToken">Cancellation Token to cancel the request.</param>
@@ -1090,7 +1090,7 @@ public ScrapeBadger.Client.ApiResponse<Object> WebPollAnAutoUnblockDiscoveryJobW
1090
1090
}
1091
1091
1092
1092
/// <summary>
1093
-
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
1093
+
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
1094
1094
/// </summary>
1095
1095
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
1096
1096
/// <returns>Object</returns>
@@ -1101,7 +1101,7 @@ public Object WebScrapeAUrl()
1101
1101
}
1102
1102
1103
1103
/// <summary>
1104
-
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
1104
+
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
1105
1105
/// </summary>
1106
1106
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
1107
1107
/// <returns>ApiResponse of Object</returns>
@@ -1143,7 +1143,7 @@ public ScrapeBadger.Client.ApiResponse<Object> WebScrapeAUrlWithHttpInfo()
1143
1143
}
1144
1144
1145
1145
/// <summary>
1146
-
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
1146
+
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
1147
1147
/// </summary>
1148
1148
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
1149
1149
/// <param name="cancellationToken">Cancellation Token to cancel the request.</param>
@@ -1155,7 +1155,7 @@ public ScrapeBadger.Client.ApiResponse<Object> WebScrapeAUrlWithHttpInfo()
1155
1155
}
1156
1156
1157
1157
/// <summary>
1158
-
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply.
1158
+
/// Scrape a URL Scrape a URL and return its content. The Generic Web Scraping API is fully user-driven: callers pick their own request parameters (engine, proxy tier, country, JS rendering, …). A blocked target surfaces the raw 422 ``blocking_page_detected`` so the caller can tune parameters themselves — we do NOT auto-trigger host discovery. Curated per-origin overrides (which the dedicated scraper APIs depend on) still apply. One exception, added for SB-001083/SCR-64: a plain ``google.com/search`` web-SERP fetch is served by the dedicated Google scraper, whose rotation loop is built for SearchGuard. Same request shape, same response shape, billed at what it actually cost — the caller sees only a better hit rate. ``serp_adapter`` fails closed, so any request carrying a parameter it does not recognise as route-or-cost takes the generic path unchanged, and the user-driven contract above still holds.
1159
1159
/// </summary>
1160
1160
/// <exception cref="ScrapeBadger.Client.ApiException">Thrown when fails to make API call</exception>
1161
1161
/// <param name="cancellationToken">Cancellation Token to cancel the request.</param>
0 commit comments