lingxue

lingxue

向着遥不可及的梦想,进发!
steam
youtube
keybase
twitter

(Expired) How to crawl Twitter content after the free API is disabled.

: p [2023/09/16 The plugin has been removed] {.center}

: p [2023/08/27 The old version API should be completely invalid and will not return any content] {.center}

: p [On July 9, 2023, the daily limit was lifted and the old version API was restored, but I don't know how long it can be used] {.center}

: p [Due to Twitter's update on restrictions, the API in the plugin has become invalid and the plugin cannot be used] {.center}

I don't know when Twitter stopped using the free API. Previously, it was possible to apply for it to crawl content. There are many open-source crawlers on GitHub, but they require an API key. However, the new version API is expensive. Here is a browser crawler plugin that I have been using.

Disclaimer

The content of this article is time-sensitive, and it is not guaranteed that this tutorial will still be available in the future.
: p [This article only provides links and usage methods for crawler plugins found online. It is strongly discouraged to use them for excessive crawling and resource occupation on servers.] {.center}
: p [Using crawlers without the official API violates Twitter's terms of use. We are not responsible for any consequences of using crawlers that result in account suspension or restrictions.] {.center}
: p [They really do suspend accounts, and they won't be unblocked even if you appeal.] {.center}
Screenshot 2023-06-24 204602

Chrome version

Firefox version

Usage

After downloading, the default configuration does not need to be modified.

Screenshot 2023-06-24 210511

The above options save data by default. Selecting "disabled" allows you to download the original image.

The option below is for specifying the size at which the files should be zipped and saved.

Open any page on Twitter that contains tweets.

Here is an example using a personal profile page.

Screenshot 2023-06-24 210740
"media" downloads the author's Twitter content.
"like" downloads liked content.
The default interface looks like this.

Screenshot 2023-06-24 210905

Selecting "no media" downloads only the text content, and the content in parentheses below is for downloading retweets.

You don't need to worry about "dry run."

Conclusion

: p [The tutorial is over, let me remind you again.] {.center}

: p [This article only provides links and usage methods for crawlers found online. It is strongly discouraged to use them for excessive crawling and resource occupation on servers.] {.center}
: p [Using crawlers without the official API violates Twitter's terms of use. We are not responsible for any consequences of using crawlers that result in account suspension or restrictions.] {.center}
: p [They really do suspend accounts, and they won't be unblocked even if you appeal.] {.center}

Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.