lingxue

lingxue

向着遥不可及的梦想,进发!
steam
youtube
keybase
twitter

(Expired) How to crawl Twitter content after the free API is disabled

2023/09/16 Plugin has been taken down

2023/08/27 The old version API seems to be completely invalidated and will not return any content

On July 9, 2023, the daily limit was lifted and the old version API was restored, but I don't know how long it will last

Due to Twitter's update on restrictions, the API in the plugin has become invalid and the plugin cannot be used anymore

I don't know when Twitter stopped using the free API. Previously, it was possible to apply for it to crawl content. There are many open-source crawlers on GitHub, but they all require an API key. However, the new version API is expensive. Here is a browser crawler plugin that I have been using.

Disclaimer#

The content of this article is time-sensitive and there is no guarantee that this tutorial will still be available in the future.

This article only provides links to crawler plugins found online and their usage methods. It is strongly discouraged to use them for excessive crawling that consumes server resources.


Using crawlers without the official API violates Twitter's terms of use. We are not responsible for any consequences resulting from the use of crawlers, such as account suspension or restrictions.


They really do suspend accounts, and they won't unban them even if you appeal.


Screenshot 2023-06-24 204602

Chrome version

Firefox version

Usage#

After downloading, the default configuration does not need to be modified.

Screenshot 2023-06-24 210511

The above options save data by default. Select "disabled" to download the original image.

The option below is for specifying the size at which to package and save as a zip file.

Open any Twitter page that contains tweets.

Here is an example using a personal profile page.

Screenshot 2023-06-24 210740
"media" downloads the author's Twitter content.
"like" downloads liked content.
The default interface looks like this.

Screenshot 2023-06-24 210905

Selecting "no media" downloads only the text content, and the content in parentheses below is for downloading retweeted content.

You don't need to worry about "dry run".

Conclusion#

That's the end of the tutorial, let me remind you again

This article only provides links to crawler plugins found online and their usage methods. It is strongly discouraged to use them for excessive crawling that consumes server resources.


Using crawlers without the official API violates Twitter's terms of use. We are not responsible for any consequences resulting from the use of crawlers, such as account suspension or restrictions.


They really do suspend accounts, and they won't unban them even if you appeal.

Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.