A comprehensive web scraper for extracting Summer 2025 course information from all Washington State Community Colleges using the CTC Link system.
- Scrapes Summer 2025 courses from all WA Community Colleges
- Two versions: basic (requests + BeautifulSoup) and enhanced (Selenium support)
- Handles JavaScript-heavy pages with Selenium
- Exports data to JSON and CSV formats
- Comprehensive logging and error handling
- Respectful rate limiting
ctclink_scraper.py- Basic scraper using requests and BeautifulSoupctclink_scraper_enhanced.py- Enhanced scraper with Selenium supportrequirements.txt- Python dependenciesREADME.md- This file
- Install Python 3.7+ if not already installed
- Install required packages:
pip install -r requirements.txt
- For Selenium support, install ChromeDriver:
- Download from https://chromedriver.chromium.org/
- Add to PATH or place in project directory
python ctclink_scraper.pypython ctclink_scraper_enhanced.pyThe enhanced scraper will prompt you to:
- Choose whether to use Selenium (recommended: yes)
- Choose whether to run in headless mode (recommended: yes)
wa_cc_summer2025_courses.json- Complete course data in JSON formatwa_cc_summer2025_courses.csv- Course data in CSV format for spreadsheet analysiswa_cc_summer2025_courses_enhanced.json- Enhanced scraper JSON outputwa_cc_summer2025_courses_enhanced.csv- Enhanced scraper CSV output
The scraper includes the following Washington State Community Colleges:
- WA230: Bellevue College
- WA240: Edmonds College
- WA245: Everett Community College
- WA250: Green River College
- WA255: Highline College
- WA260: Lake Washington Institute of Technology
- WA265: North Seattle College
- WA270: Olympic College
- WA275: Peninsula College
- WA280: Pierce College District
- WA285: Renton Technical College
- WA290: Seattle Central College
- WA300: Shoreline Community College
- WA305: Skagit Valley College
- WA310: South Puget Sound Community College
- WA315: South Seattle College
- WA320: Spokane Community College
- WA325: Spokane Falls Community College
- WA330: Tacoma Community College
- WA335: Walla Walla Community College
- WA340: Wenatchee Valley College
- WA345: Whatcom Community College
- WA350: Yakima Valley College
You can modify the institution codes in the wa_cc_codes dictionary within either scraper file to add, remove, or update institutions.
-
No courses found:
- Summer 2025 term may not be available yet
- Website structure may have changed
- Check network connectivity
-
Selenium errors:
- Ensure ChromeDriver is installed and in PATH
- Try running in non-headless mode for debugging
- Check Chrome browser version compatibility
-
Access denied:
- The website may have anti-bot protection
- Try increasing delays between requests
- Use different User-Agent strings
- Run the enhanced scraper in non-headless mode to see browser interactions
- Check log output for specific error messages
- Manually visit the URL to verify it's accessible
- Ensure compliance with the website's terms of service
- Use reasonable delays between requests
- Consider contacting the institution for official data access
- This tool is for educational and research purposes
Feel free to submit issues or pull requests to improve the scraper's functionality or add support for additional institutions.
This scraper is provided as-is. The functionality may break if the CTC Link website structure changes. Always verify the accuracy of scraped data against official sources.