Release notes on Leo Geoffrion [Skidmore College] web archiving project
Created for the Union - Skidmore - Hamilton - Amherst web archiving project
Sponsored by the Center for Educational Technology.

[Feb. 2, 2003]

I'm starting to think about creating some of the code to explore the ideas in my paper.  Ideally, I'd prefer to do it in a form that is readily shared across our schools

This leads to some technology questions for our group...

1.  What Operating system (e.g. Windows2000, Solaris2, ...) is on your main campus web server?

2.  Which web server software (e.g. Apache, IIS, ...) runs your main web server?

3.  What databases (e.g. Oracle, mysql, Postgres, ...) are available to your main server?

4.  What scripting languages (e.g. Perl, PHP, ColdFusion, ...)  are available for linking databases and web pages?

I'd like to see whether we share a common software core.

[Feb. 13, 2003]

As I reported previously, I have started to write some of the code to watch sites for changed information.

You can preview some of the preliminary pages at   http://ldg.skidmore.edu/archives

You'll find two forms linked off of there:

•   define a new site to be watched.

•   edit an existing site.

Please feel free to test them out.  I'd be particularly interested in determining whether they include the appropriate fields for your organizations.  For example, would you need other information in order for these to work at your school?

These are crude pages with no surrounding decoration and only partial debugging.

You will probably not be successful in actually defining a new site since you need to know much specific information on the file locations of the underlying pages.

Both forms manipulate an underlying mysql database that contains the following tables:

•      site                 basic information about each site

•      pages            the list of watched pages associated with each site

•      pagedata       the data used to detect if a page has changed

•      status_dict     a dictionary for the status codes

•      log                  event log to track activity

There is a third invisible component that you don't see on that page. It's the script that scans all files defined by the above forms.  If any file has changed, it logs the information into the database.

I next need to write the scripts to take follow-up actions when the changes are noted.

All of the pages are written in Perl for a Unix-like environment with mysql as the underlying database.  If you eventually want the scripts for your campus, you'll need to have your IT people install the CGI, DBI, and DBD extensions to Perl.

•     CGI is the cgi extension (often part of the standard Perl installation)

•     DBI is the common database interface for Perl

•     DBD is the Perl database driver for MySQL

All are available for download and installation from cpan.perl.org

The DBI/DBD installation is moderately messy for some systems.  It took several tries to get them properly installed on my Linux test box, and I'm still working with our IT people to get it working cleanly on the main campus server (Solaris).

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Leo D. Geoffrion, Ph.D.                    email:  ldg@skidmore.edu
Webmaster  
Strategic Communications               phone:  518 580-5735
Skidmore College                              Fax:  518 580-5748
815 No. Broadway
Saratoga Springs, NY  12866
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

  Return to the Archiving Home Page
Sponsored by Union College and the Center for Educational Technology