Skip to content

Problems with requesting youtube channel page #32

Description

@rumca-js

Example code

    import httpmorph
    response =  httpmorph.get("https://www.youtube.com/channel/UCXuqSBlHAE6Xw-yeJA0Tunw", timeout=30)
    print(reponse.status_code)
Traceback (most recent call last):
  File "/home/hunter/WorkDir/git/crawler-buddy/manual_test.py", line 207, in <module>
    main()
  File "/home/hunter/WorkDir/git/crawler-buddy/manual_test.py", line 205, in main
    test_urls()
  File "/home/hunter/WorkDir/git/crawler-buddy/manual_test.py", line 198, in test_urls
    response =  httpmorph.get("https://www.youtube.com/channel/UCXuqSBlHAE6Xw-yeJA0Tunw", timeout=30)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunter/.cache/pypoetry/virtualenvs/crawler-buddy-sKUXuEBu-py3.12/lib/python3.12/site-packages/httpmorph/_client_c.py", line 959, in get
    return get_default_session().get(url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunter/.cache/pypoetry/virtualenvs/crawler-buddy-sKUXuEBu-py3.12/lib/python3.12/site-packages/httpmorph/_client_c.py", line 910, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunter/.cache/pypoetry/virtualenvs/crawler-buddy-sKUXuEBu-py3.12/lib/python3.12/site-packages/httpmorph/_client_c.py", line 825, in request
    raise Timeout(error_msg)
httpmorph._client_c.Timeout: Request timed out

Proposition

  • introduce manual tests to your project
  • test pages like youtube.com, x.com, reddit.com, etc

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions