Scraping Interest Rates: A Step-by-Step Guide in Python and PHP

In this blog post, we’ll explore how to scrape current interest rates from the Federal Reserve’s H.15 page using Python and PHP. This tutorial will guide you through creating scripts that fetch and display the interest rate value from the top-right cell of the table on the page.

Python Script (Locally Scrape)

To get started, ensure you have requests and BeautifulSoup libraries installed:

pip install requests beautifulsoup4

Here’s the Python script:

import requests
from bs4 import BeautifulSoup

def get_top_right_interest_rate():
    url = "https://www.federalreserve.gov/releases/h15/"
    response = requests.get(url)

    if response.status_code == 200:
        soup = BeautifulSoup(response.content, 'html.parser')
        table = soup.find('table', {'class': 'statistics'})
        
        if table:
            rows = table.find_all('tr')
            if len(rows) > 1:
                first_data_row = rows[1]
                cells = first_data_row.find_all('td')
                if cells:
                    top_right_value = cells[-1].get_text(strip=True)
                    print(f"Interest Rate: {top_right_value}")
                else:
                    print("Error: No cells found in the first data row.")
            else:
                print("Error: Not enough rows in the table.")
        else:
            print("Error: Table with interest rates not found.")
    else:
        print(f"Error: Unable to fetch data (Status code: {response.status_code})")

# Example usage
get_top_right_interest_rate()

PHP Script (Remote Scrape for Web Services)

Ensure your PHP environment has cURL and DOM extensions enabled. Here’s the PHP script:

<?php

function getTopRightInterestRate() {
    $url = "https://www.federalreserve.gov/releases/h15/";

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    $response = curl_exec($ch);

    if ($response === false) {
        echo "Error: Unable to fetch data (cURL Error: " . curl_error($ch) . ")";
        return;
    }

    curl_close($ch);

    $dom = new DOMDocument();
    @$dom->loadHTML($response);

    $tables = $dom->getElementsByTagName('table');
    $table = null;

    foreach ($tables as $t) {
        if (strpos($t->getAttribute('class'), 'statistics') !== false) {
            $table = $t;
            break;
        }
    }

    if ($table) {
        $rows = $table->getElementsByTagName('tr');
        if ($rows->length > 1) {
            $firstDataRow = $rows->item(1);
            $cells = $firstDataRow->getElementsByTagName('td');
            if ($cells->length > 0) {
                $topRightValue = $cells->item($cells->length - 1)->nodeValue;
                echo "Interest Rate: " . trim($topRightValue);
            } else {
                echo "Error: No cells found in the first data row.";
            }
        } else {
            echo "Error: Not enough rows in the table.";
        }
    } else {
        echo "Error: Table with interest rates not found.";
    }
}

getTopRightInterestRate();

?>

Benefits of Using Python vs. PHP

Python:

1. Ease of Use: Python’s requests and BeautifulSoup libraries make it straightforward to fetch and parse HTML.
2. Readability: Python’s syntax is clear and easy to read, making it a great choice for beginners.
3. Community Support: Python has a vast community and extensive documentation, providing plenty of resources for troubleshooting and learning.

PHP:

1. Server-Side Execution: PHP is ideal for web-based applications and can be easily integrated into web servers for dynamic content generation.
2. Availability: PHP is widely available on web hosting services, making it convenient for deploying web scraping scripts on live websites.
3. Integration: PHP can be seamlessly integrated with HTML and other web technologies, making it a good choice for full-stack web development.

Both Python and PHP are powerful tools for web scraping. The choice between them depends on your specific needs, existing tech stack, and personal preference. Happy coding!