Skip to content

Commit a093d3c

Browse files
lanbytescursoragent
lanbytes
and
cursoragent
committed
Add a TwexAPI Prefect collection for public X reads.
Co-authored-by: Cursor <cursoragent@cursor.com>
0 parents  commit a093d3c

22 files changed

Lines changed: 4258 additions & 0 deletions

‎.github/workflows/ci.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: CI
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches:
7+
- main
8+
9+
permissions:
10+
contents: read
11+
12+
jobs:
13+
test:
14+
name: Python ${{ matrix.python-version }}
15+
runs-on: ubuntu-24.04
16+
timeout-minutes: 20
17+
strategy:
18+
fail-fast: false
19+
matrix:
20+
python-version:
21+
- "3.10"
22+
- "3.11"
23+
- "3.12"
24+
- "3.13"
25+
steps:
26+
- uses: actions/checkout@v4
27+
with:
28+
persist-credentials: false
29+
- uses: astral-sh/setup-uv@v6
30+
- uses: actions/setup-python@v5
31+
with:
32+
python-version: ${{ matrix.python-version }}
33+
- run: uv sync --dev
34+
- run: uv run ruff format --check .
35+
- run: uv run ruff check .
36+
- run: uv run pytest

‎.github/workflows/publish.yml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
name: Publish to PyPI
2+
3+
on:
4+
release:
5+
types:
6+
- published
7+
8+
permissions: {}
9+
10+
jobs:
11+
publish:
12+
if: github.event.release.prerelease == false
13+
runs-on: ubuntu-24.04
14+
environment: pypi
15+
permissions:
16+
contents: read
17+
id-token: write
18+
steps:
19+
- uses: actions/checkout@v4
20+
with:
21+
ref: ${{ github.event.release.tag_name }}
22+
persist-credentials: false
23+
- uses: astral-sh/setup-uv@v6
24+
- uses: actions/setup-python@v5
25+
with:
26+
python-version: "3.13"
27+
- run: uv sync --dev
28+
- run: uv run pytest
29+
- run: uv build
30+
- uses: pypa/gh-action-pypi-publish@release/v1
31+
with:
32+
packages-dir: dist/

‎.gitignore‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
.venv/
2+
__pycache__/
3+
.pytest_cache/
4+
.ruff_cache/
5+
dist/
6+
build/
7+
*.egg-info/
8+
.coverage
9+
htmlcov/
10+
.DS_Store
11+
.env

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Changelog
2+
3+
## 0.1.0
4+
5+
- First release with six read-only TwexAPI Prefect tasks.

‎CODE_OF_CONDUCT.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Code of Conduct
2+
3+
Be respectful. Do not harass maintainers, contributors, or users.
4+
5+
Unacceptable behavior includes harassment, hate speech, and posting secrets or personal data.
6+
7+
Report issues on GitHub. Maintainers may remove comments, close issues, or block accounts that violate this policy.
8+
9+
This project is not affiliated with X Corp.

‎CONTRIBUTING.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Contributing
2+
3+
- Open an issue for bugs or missing tasks.
4+
- Keep PRs small. Do not add endpoints that are not in the TwexAPI OpenAPI.
5+
- Do not commit API keys or `.env` files.
6+
- Run `uv run ruff check .` and `uv run pytest` before opening a PR.

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 TwexAPI
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Prefect tasks for Twitter search and X API workflows
2+
3+
TwexAPI is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.
4+
5+
Schedule 6 read-only Twitter search, profile, timeline, and trend tasks in Prefect 3. Store API keys in Prefect blocks.
6+
7+
## Tasks
8+
9+
| Workflow step | Prefect task | TwexAPI route |
10+
| --- | --- | --- |
11+
| Search tweets | `search_tweets` | `POST /twitter/advanced_search/page` |
12+
| Read tweet detail | `get_tweet` | `POST /v2/tweet/detail` |
13+
| Search users | `search_users` | `GET /twitter/search-user/{keyword}/{target_count}` |
14+
| Read a profile | `get_user` | `GET /twitter/{screen_name}/about` |
15+
| Read a timeline page | `get_user_tweets` | `POST /twitter/{screen_name}/timeline/page` |
16+
| Read trending tweets | `get_trends` | `GET /twitter/global-trending/tweets` |
17+
18+
Use REST or an SDK for follower pagination or approved posting.
19+
20+
## Install
21+
22+
```bash
23+
pip install prefect-x-api-scraper
24+
```
25+
26+
## Store credentials
27+
28+
```bash
29+
prefect block register -m prefect_x_api_scraper
30+
```
31+
32+
```python
33+
from prefect_x_api_scraper import TwexApiCredentials
34+
35+
credentials = TwexApiCredentials(api_key="your_twexapi_key")
36+
credentials.save("twexapi", overwrite=True)
37+
```
38+
39+
Keep API keys in Prefect blocks. Never put them in flow source files.
40+
41+
## Create a flow
42+
43+
```python
44+
from prefect import flow
45+
from prefect_x_api_scraper import TwexApiCredentials, get_trends, search_tweets
46+
47+
48+
@flow
49+
async def twitter_signal_flow() -> dict:
50+
credentials = TwexApiCredentials.load("twexapi")
51+
52+
tweets = await search_tweets(
53+
credentials,
54+
'"prefect" OR "workflow orchestration"',
55+
sort_by="Latest",
56+
)
57+
trends = await get_trends(credentials, country="United States", count=10)
58+
59+
return {"tweets": tweets, "trends": trends}
60+
```
61+
62+
## Import tasks
63+
64+
```python
65+
from prefect_x_api_scraper import (
66+
get_trends,
67+
get_tweet,
68+
get_user,
69+
get_user_tweets,
70+
search_tweets,
71+
search_users,
72+
)
73+
```
74+
75+
Tasks return the raw TwexAPI JSON payload and support Prefect `with_options`:
76+
77+
```python
78+
from prefect_x_api_scraper import search_tweets
79+
80+
search_recent_tweets = search_tweets.with_options(
81+
name="Search recent tweets",
82+
retries=2,
83+
retry_delay_seconds=10,
84+
)
85+
```
86+
87+
The credentials block sends `Authorization: Bearer`.
88+
89+
## Documentation
90+
91+
- [TwexAPI docs](https://docs.twexapi.io)
92+
- [REST overview](https://docs.twexapi.io/api-reference/overview)
93+
- [Prefect tasks](https://docs.prefect.io/v3/develop/write-tasks)
94+
95+
TwexAPI is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.

‎SECURITY.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Security
2+
3+
Do not file public issues that include API keys, cookies, `auth_token` values, or personal data.
4+
5+
Report suspected vulnerabilities privately through GitHub Security Advisories on this repository.

‎examples/social_signal_flow.py‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
from __future__ import annotations
2+
3+
from prefect import flow
4+
5+
from prefect_x_api_scraper import TwexApiCredentials, get_trends, search_tweets
6+
7+
8+
@flow
9+
async def twitter_signal_flow() -> dict[str, object]:
10+
credentials = TwexApiCredentials.load("twexapi")
11+
12+
tweets = await search_tweets(
13+
credentials,
14+
'"prefect" OR "workflow orchestration"',
15+
sort_by="Latest",
16+
)
17+
trends = await get_trends(credentials, country="United States", count=10)
18+
19+
return {"tweets": tweets, "trends": trends}

0 commit comments

Comments
 (0)