[Bio] / Sprout / ERDBLoader.pl Repository:
ViewVC logotype

View of /Sprout/ERDBLoader.pl

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.1 - (download) (as text) (annotate)
Tue Sep 30 15:23:55 2008 UTC (11 years, 3 months ago) by parrello
Branch: MAIN
New loader technology.

#!/usr/bin/perl -w

# Copyright (c) 2003-2006 University of Chicago and Fellowship
# for Interpretations of Genomes. All Rights Reserved.
# This file is part of the SEED Toolkit.
# The SEED Toolkit is free software. You can redistribute
# it and/or modify it under the terms of the SEED Toolkit
# Public License.
# You should have received a copy of the SEED Toolkit Public License
# along with this program; if not write to the University of Chicago
# at info@ci.uchicago.edu or the Fellowship for Interpretation of
# Genomes at veronika@thefig.info or download a copy from
# http://www.theseed.org/LICENSE.TXT.

use strict;
use Tracer;
use ERDB;
use ERDBLoadGroup;
use ERDBGenerate;
use Stats;

=head1 ERDBLoader Script

    ERDBLoader [options] <database> <group1> <group2> ...

ERDB Database Load Finisher

=head2 Introduction

This script finishes the database load process begun by [[ERDBGeneratorPl]].

[[ERDBGeneratorPl]] divides the source data into sections, and generates a
partial load file for each section of each table. To finish the load process, we
need to combine the partial files into single files and load the resulting
single files into the database tables.

Like [[ERDBGeneratorPl]], this script acts on load groups-- sets of related
tables that are loaded at the same time. For each table in a named group that
does not exist in the database, the script first attempts to find a completed
data file. If one does not exist, it attempts to create one by collating section
files. Once the collated section file is finished, it is loaded into the

=head2 Positional Parameters

=over 4

=item database

Name of the ERDB database. This should be the class name for the subclass used
to access the database.


=head2 Command-Line Options

=over 4

=item trace

Specifies the tracing level. The higher the tracing level, the more messages
will appear in the trace log. Use E to specify emergency tracing.

=item replace

Normally, if a table exists in the database, it will not be loaded. If this
option is specified, however, existing tables will be dropped and recreated.

=item resume

If specified, then the group list must contain a single group. The specified
group and all groups after it in the group list will be processed.

=item user

Name suffix to be used for log files. If omitted, the PID is used.

=item sql

If specified, turns on tracing of SQL activity.

=item background

Save the standard and error output to files. The files will be created
in the FIG temporary directory and will be named C<err>I<User>C<.log> and
C<out>I<User>C<.log>, respectively, where I<User> is the value of the
B<user> option above.

=item help

Display this command's parameters and options.

=item warn

Create an event in the RSS feed when an error occurs.

=item phone

Phone number to message when the script is complete.



# Get the command-line options and parameters.
my ($options, @parameters) = StandardSetup([qw(ERDBLoadGroup ERDB Stats) ],
                                              trace => ["", "tracing level"],
                                              replace => ["", "if specified, existing tables will be overwritten"],
                                              resume => ["", "if specified, the specified group and all groups that normally come after it will be processed"],
                                              phone => ["", "phone number (international format) to call when load finishes"]
                                           "<database> <group1> <group2> ...",
# Set a variable to contain return type information.
my $rtype;
# Insure we catch errors.
eval {
    # Get the parameters.
    my ($database, @groups) = @parameters;
    # Connect to the database.
    my $erdb = ERDB::GetDatabase($database);
    # Fix the group list.
    my @realGroups = ERDBLoadGroup::ComputeGroups($erdb, $options, \@groups);
    # Get the source object and load directory for this database.
    my $source = $erdb->GetSourceObject();
    my $directory = $erdb->LoadDirectory();
    # Get the list of sections.
    my @sectionList = $erdb->SectionList($source);
    # Create a statistics object to track our progress.
    my $stats = Stats->new();
    # Get the hash of group names to table names.
    my $groupHash = ERDBLoadGroup::GetGroupHash($erdb);
    # Loop through the groups.
    for my $group (@realGroups) {
        # Get the list of tables for this group.
        my $tableList = $groupHash->{$group};
if ($@) {
    Trace("Script failed with error: $@") if T(0);
    $rtype = "error";
} else {
    Trace("Script complete.") if T(2);
    $rtype = "no error";
if ($options->{phone}) {
    my $msgID = Tracer::SendSMS($options->{phone}, "ERDBLoader terminated with $rtype.");
    if ($msgID) {
        Trace("Phone message sent with ID $msgID.") if T(2);
    } else {
        Trace("Phone message not sent.") if T(2);


MCS Webmaster
ViewVC Help
Powered by ViewVC 1.0.3