🏎️ @nberlette/f1

Scraping build photos of the Formula 1 track in Las Vegas

This is an autonomous image scraper developed using TypeScript, Deno, and GitHub Actions. It was purpose-built to document the historic Formula 1 track construction in Las Vegas, Nevada, slated to host the inaugural Heineken Silver Grand Prix on November 18th. The images will be stitched together to form timelapse videos of the track’s lifecycle.

📸 Latest • 🎬 Timelapse • 🗓️ Previous Images • ℹ️ Project Details • 🌟 Star It!

AI-generated artwork of a Formula 1 car racing down the Las Vegas Strip

Since June 3rd, 2023 this project has collected nearly 20,000 images, all courtesy of a live-streaming construction camera provided by the track’s developers. This project is not affiliated with Formula 1.

Track Details and Statistics

Estimated Top Speed	Circuit Length	Corners	Straights	DRS Zones
212 mph • 342 km/h	3.8 miles • 6.12 km	17	3	2

Latest Snapshot

Timelapse Preview

Note: this video was created with images from 2023-08-15 - 2023-10-12

About

The first scrape happened on June 3rd, 2023. As of October 18th it has surpassed 18,500 commits, equivalent to over 1.2GB of image data. Photos are stored in the ./assets folder of this repository, and also persisted to a Deno KV database backed by FoundationDB.

The origin of the scraped images is an real-time photo feed, sourced directly from the official Formula 1 website.

⚠️ This project is for educational purposes and is not affiliated with Formula 1.

📖 Click here for an in-depth explanation of the scrape process

Tools Used

Deno v1.37.2
- Rust-based JS runtime, sandboxed, with great TS/TSX support.
- Provides the tools for network and file system operations.
TypeScript 5.2.2
- Superset of JavaScript featuring advanced static typechecking.
- Better type safety means more readable and maintainable code.
GitHub Actions
- Provides free macOS virtual machines powering the scraper.
- Responsible for scheduled execution of the scraper workflow
- Temporarily stores the image artifacts
Deno KV (currently in beta)
- Provides us with global data persistence and caching
ffmpeg (timelapse feature is unstable)
- Leveraged to automatically generate timelapse videos

AI-generated F1 art created with SDXL 1.0 and the prompt "Formula 1 cars on the Las Vegas Strip"

How it Works

The majority of the work happens in main.ts, despite it only being 3 lines of code. It is responsible for invoking the scraper located in src/scrape.ts, and is ran every 10 minutes by a GitHub Action defined by the workflow in main.yml.

Assets and Data

Images are named after their capture time as a JPEG file in UTC. For example, an image captured at 2023-07-09T04:28:57 would be saved as ./assets/2023-07-09/04_28_57.jpg. The latest image is always saved as ./assets/latest.jpg for easy access.

Scrape Process, Step-by-Step

GitHub Actions runs the scrape workflow every ~10 minutes, depending on traffic.
The runner checks out the repo, installs Deno, and prepares to scrape.
deno task scrape is executed, which runs the main.ts file.
main.ts imports scrape() from src/scrape.ts, which contains the read and write functions.
🔍 READ: read() is called with IMAGE_URL.
- Internally, the Fetch API is used to download the image.
  If the request fails, it will be retried up to ATTEMPTS times, with a short pause between each successive attempt.
- If all attempts are exhausted without success, the run will terminate.
- Otherwise, a new instance of the Image class is returned.
💾 WRITE: write() is called with the new Image. Before writing the image, it runs through some checks:
1. The Image.hash is checked against the hash “table” in Deno KV.
  - If an entry exists, the image is stale and won’t make it any further.
  - If Deno KV is unavailable, the image data is checked against latest.jpg via a timing-safe equality comparison, avoiding exposure to timing-based attacks.
  - The job starts over at step 5 and repeats until a new image is found, or the maximum ATTEMPTS are all used. If nothing is found by now, the job fails.
2. If the scraper has made it this far, then we have a fresh image and need to store it.
  - Image.write() persists the image to Deno KV.
    
    The key is generated by the Image API, using the image timestamp.
  - The image timestamp is indexed with its unique SHA-256 hash in Deno KV.
    
    This prevents later scrapes from duplicating this image. It also means
    if you try to instantiate a new Image from an old hash, it will always
    return the original image and its original timestamp.
  - Image.writeFile() saves it to the local file system.
    
    The filename is generated by the Image API, using the image timestamp.
  - Image.writeFile() also saves it to ./assets/latest.jpg,
3. The setOutput helper pipes the image metadata to the GitHub Actions runner, to be used in the commit step.
4. The scrape is now complete and the runner proceeds to the final steps.
The photo is stored as a workflow artifact for 90 days.
All changes are committed + pushed to the repository.
🏁 The job finishes successfully and the runner is terminated. Hooray!

Previous Snapshots

October 17th	October 15th	October 13th	October 11th

October 9th	October 7th	October 5th	October 3rd

October 2nd	September 28th	September 24th	September 20th

September 16th	September 12th	September 8th	September 4th