The Sourcerer's Apprentice

The adventures of David Heinemann in IT & software development

Announcing QdbScraper

| Comments

The aptly titled ‘QdbScraper’ is a Perl script that can scrape and save all the quotes from a given QdbS-powered quote database as individual text files. At the moment, it is only designed to scrape QdbS’ default template, but support for others may be added in the future.

QdbScraper can be downloaded directly or found on GitHub. It is published under the New BSD license and should run on any platform.

Usage
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
QdbScraper.pl [-d] [-f] [-p] [-s] [-w] URL

  -d        Manually set the directory that quotes are saved into.
            (default: automatic)
  -f        Create a fortune cookie file from extracted quotes (boolean)
  -w        Manually set the wait delay (in seconds) between each page
            load. (default: 2)
  -s        Manually set the starting page. (default: 1)
  -p        Manually set the total number of pages. (default: automatic)
  URL       The URL of the QDB to be scraped.  See URL Instructions for
            more information.

URL Instructions:
  Always omit file names in the URL.
    use:        http://www.foobar.com/qdb
    instead of: http://www.foobar.com/qdb/index.php

  Always omit trailing slashes in the URL.
    use:        http://www.foobar.com/qdb
    instead of: http://www.foobar.com/qdb/

Comments