Monitor a website for changes

November 28, 20223 min read438 words

Introduction

This article will show you how to monitor a website for changes. This is useful if you want to be notified when a website changes, for example, when a new blog post is published, a black friday sale starts, or an item is back in stock.

Prerequisites

Steps

1. Create a Telegram bot

  1. Open Telegram and search for @BotFather

  2. Send /newbot to @BotFather

  3. Enter a name for your bot

  4. Enter a username for your bot

  5. Copy the token that @BotFather gives you

2. Create a Telegram chat

  1. Open Telegram and search for @userinfobot

  2. Send /start to @userinfobot

  3. Send /my_id to @userinfobot

  4. Copy the chat id that @userinfobot gives you

3. Create a Docker compose file

  1. On your ubuntu server, create a file called docker-compose.yml with the following content:
version: "3.9"

volumes:
  data:

services:
  changedetection:
    image: dgtlmoon/changedetection.io:dev
    container_name: changedetection
    hostname: changedetection
    volumes:
      - data:/datastore
    environment:
      - PORT=5000
      - PUID=1000
      - PGID=1000
      - WEBDRIVER_URL=http://browser-chrome:4444/wd/hub
      - PLAYWRIGHT_DRIVER_URL=ws://playwright-chrome:3000/?stealth=1&--disable-web-security=true
      - BASE_URL=http://localhost:5000
    restart: unless-stopped
    ports:
      - 5000:5000

  browser-chrome:
    hostname: browser-chrome
    image: selenium/standalone-chrome-debug:3.141.59
    environment:
      - VNC_NO_PASSWORD=1
      - SCREEN_WIDTH=1920
      - SCREEN_HEIGHT=1080
      - SCREEN_DEPTH=24
    volumes:
      - /dev/shm:/dev/shm
    restart: unless-stopped

  playwright-chrome:
    hostname: playwright-chrome
    image: browserless/chrome
    restart: unless-stopped
    environment:
      - SCREEN_WIDTH=1920
      - SCREEN_HEIGHT=1024
      - SCREEN_DEPTH=16
      - ENABLE_DEBUGGER=false
      - PREBOOT_CHROME=true
      - CONNECTION_TIMEOUT=300000
      - MAX_CONCURRENT_SESSIONS=10
      - CHROME_REFRESH_TIME=600000
      - DEFAULT_BLOCK_ADS=true
      - DEFAULT_STEALTH=true

4. Start the Docker containers

  1. On your ubuntu server, run the following command to start the Docker containers:

docker-compose up -d

5. Add a website to monitor

  1. Open a web browser and navigate to http://localhost:5000

changedetection.io

  1. Paste the URL of the website you want to monitor in the URL field

new website

  1. Click Edit > Watch

  2. Under time between checks, select your preferred interval

time between checks

  1. Under the Request tab, set Fetching method to Playwright Chromium/Javascript via 'ws://playwright-chrome:3000/?stealth=1&--disable-web-security=true' and Wait seconds before extracting text to a number higher than 15

  2. Under the Notifications tab, enter the Telegram chat id and token you copied earlier as the following format:

tgram://<token>:<chat-id>
  1. Click Save

You should now receive a notification when the website changes.

Conclusion

In this article, you learned how to monitor a website for changes. This is useful if you want to be notified when a website changes, for example, when a new blog post is published, a black friday sale starts, or an item is back in stock.

Resources

Buy Me a Coffee at ko-fi.com

Website created by Yassine Fathi © 2023

Built with Next.js & Hosted in my Homelab