<![CDATA[Tony Wang, A Software Engineer | RSS Feed]]> https://tonywang.io https://tonywang.io/logo.png Tony Wang, A Software Engineer | RSS Feed https://tonywang.io RSS for Node Mon, 24 Nov 2025 12:31:26 GMT Mon, 24 Nov 2025 12:31:26 GMT <![CDATA[After 2025, Can Your Google SERP Crawler Still Survive?]]> https://tonywang.io/blog/google-serp-crawler-after-2025 https://tonywang.io/blog/google-serp-crawler-after-2025 Mon, 24 Nov 2025 00:00:00 GMT <![CDATA[The Infra to handle 10M Requests in 10 Minutes for $0.0116]]> https://tonywang.io/blog/infra-10m-requests-10-minutes-0.0116 https://tonywang.io/blog/infra-10m-requests-10-minutes-0.0116 Mon, 04 Nov 2024 00:00:00 GMT <![CDATA[27.6% of the Top 10 Million Sites are Dead]]> https://tonywang.io/blog/top-10-million-sites-27-percent-dead https://tonywang.io/blog/top-10-million-sites-27-percent-dead Wed, 30 Oct 2024 00:00:00 GMT <![CDATA[Web Crawling at Scale: Navigating Billions of URLs with Efficiency]]> https://tonywang.io/blog/web-crawling-at-scale https://tonywang.io/blog/web-crawling-at-scale Sat, 14 Oct 2023 00:00:00 GMT <![CDATA[The Architecture of a Web Crawler: Building a Google-Inspired Distributed Web Crawler. Part 1]]> https://tonywang.io/blog/architecture-of-google-alike-web-crawler https://tonywang.io/blog/architecture-of-google-alike-web-crawler Fri, 13 Oct 2023 00:00:00 GMT <![CDATA[How to efficiently scrape millions of Google Businesses on a large scale using a distributed crawler]]> https://tonywang.io/blog/google-business-crawler https://tonywang.io/blog/google-business-crawler Mon, 31 Jul 2023 00:00:00 GMT <![CDATA[A Step-by-Step Guide to Building a Scalable Distributed Crawler for Scraping Millions of Top TikTok Profiles]]> https://tonywang.io/blog/step-by-step-guide-to-scalable-distributed-crawler-for-crawling-millions-of-top-tiktok-profiles https://tonywang.io/blog/step-by-step-guide-to-scalable-distributed-crawler-for-crawling-millions-of-top-tiktok-profiles Sun, 11 Jun 2023 00:00:00 GMT <![CDATA[Deploy your distributed system efficiently with fabric]]> https://tonywang.io/blog/fabric https://tonywang.io/blog/fabric Sun, 19 Mar 2017 00:00:00 GMT <![CDATA[How to build a scalable crawler to crawl million pages with a single machine in just 2 hours]]> https://tonywang.io/blog/build-scalable-crawler-to-crawl-million-pages https://tonywang.io/blog/build-scalable-crawler-to-crawl-million-pages Tue, 28 Feb 2017 00:00:00 GMT <![CDATA[How to build docker cluster with celery and RabbitMQ in 10 minutes]]> https://tonywang.io/blog/docker-cluster-with-celery-and-rabbitmq-in-10-minutes https://tonywang.io/blog/docker-cluster-with-celery-and-rabbitmq-in-10-minutes Sat, 25 Feb 2017 00:00:00 GMT