[Bio] / FigKernelPackages / raelib.pm Repository:
ViewVC logotype

View of /FigKernelPackages/raelib.pm

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.27 - (download) (as text) (annotate)
Mon Oct 3 20:05:34 2005 UTC (14 years, 2 months ago) by redwards
Branch: MAIN
CVS Tags: caBIG-dataload-0, caBIG-00-00-00
Changes since 1.26: +49 -2 lines
Initial release of cookies code, including myseed

# -*- perl -*-

=pod

=head1 RAE Library

 Some routines and things that Rob uses. Please feel free to use at will and incorporate into
 your own code or move them into FIG.pm or elsewhere.

=cut

package raelib;
use strict;
use Bio::SeqIO;
use Bio::Seq;
use Bio::SeqFeature::Generic;
use FIG;
my $fig=new FIG;

=head2 new

Just instantiate the object and return $self

=cut

sub new {
 my ($class)=@_;
 my $self={};
 return bless $self, $class;
}
   



=head2 features_on_contig

 Returns a reference to an array containing all the features on a contig in a genome.
 
 use: 

 my $arrayref=$rae->features_on_contig($genome, $contig);

 or
 
 foreach my $peg (@{$rae->features_on_contig($genome, $contig)}) {
  ... blah blah ...
 }

 returns undef if contig is not a part of genome or there is nothing to return, otherwise returns a list of pegs
 
 v. experimental and guaranteed not to work!

=cut

sub features_on_contig {
 my ($self, $genome, $contig)=@_;
 # were this in FIG.pm you'd use this line:
 #my $rdbH = $self->db_handle;

 my $rdbH = $fig->db_handle;
 my $relational_db_response=$rdbH->SQL('SELECT id FROM features WHERE  (genome = \'' . $genome . '\' AND  location ~* \'' . $contig . '\')');
 # this is complicated. A reference to an array of references to arrays, and we only want the first element. 
 # simplify.
 my @results;
 foreach my $res (@$relational_db_response) {push @results, $res->[0]}
 return \@results;
}







=head2 pirsfcorrespondence

Generate the pirsf->fig id correspondence. This is only done once and the correspondence file is written. This is so that we can easily go back and forth.

The correspondence has PIR ID \t FIG ID\n, and is probably based on ftp://ftp.pir.georgetown.edu/pir_databases/pirsf/data/pirsfinfo.dat

This method takes three arguments:
   from    : pirsfinfo.dat file
   to      : file to write information to
   verbose : report on progress 

Note that if the from filename ends in .gz it assumed to be a gzipped file and will be opened accordingly.

Returns the number of lines in the pirsinfo file that were read.

=cut

sub pirsfcorrespondence { 
 my ($self, $from, $to, $verbose)=@_;
 unless (-e $from) {
  print STDERR "File $from does not exist as called in $0\n";
  return 0;
 }
 if ($from =~ /\.gz$/) {
  open(IN, "|gunzip -c $from") || die "Can't open $from using a gunzip pipe";
 }
 else {
  open (IN, $from) || die "Can't open $from";
 }
 open (OUT, ">$to") || die "Can't write to $to";
 my $linecount;
 while (<IN>) {
  $linecount++;
  if ($verbose && !($linecount % 10000))  {print STDERR "Parsed $linecount lines\n"}
  if (/^>/) {print OUT; next}
  chomp;
  foreach my $peg ($self->swiss_pir_ids($_)) {
   print OUT $_, "\t", $peg, "\n";
  }
 }
 close IN;
 close OUT;
 return $linecount;
}

=head2 uniprotcorrespondence

Generate a correspondence table between uniprot knowledge base IDs and FIG ID's.

The uniprot KB file is in the form:  UniProtKB_Primary_Accession | UniProtKB_ID | Section | Protein Name

 This method takes three arguments:
   from    : uniprotKB file
   to      : file to write information to
   verbose : report on progress 

Note that if the from filename ends in .gz it assumed to be a gzipped file and will be opened accordingly.

 Returns the number of lines in the uniprotkb file that were read.

=cut

sub uniprotcorrespondence {
 my ($self, $from, $to, $verbose)=@_;
 unless (-e $from) {
  print STDERR "File $from does not exist as called in $0\n";
  return 0;
 }
 if ($from =~ /\.gz$/) {
  open(IN, "|gunzip -c $from") || die "Can't open $from using a gunzip pipe";
 }
 else {
  open (IN, $from) || die "Can't open $from";
 }
 open (OUT, ">$to") || die "Can't write to $to";
 my $linecount;
 while (<IN>) {
  chomp;
  $linecount++;
  if ($verbose && !($linecount % 10000))  {print STDERR "Parsed $linecount lines\n"}
  my @line=split /\s+\|\s+/;
  my $added;
  foreach my $peg ($self->swiss_pir_ids($line[0])) {
   print OUT "$_ | $peg\n";
   $added=1;
  }
  unless ($added) {print OUT "$_\n"}
 }
 close IN;
 close OUT;
 return $linecount;
}

=head2 prositecorrespondence

Generate a correspondence table between prosite and seed using sp id's and seed ids.

The SwissProt prosite file is from ftp://ca.expasy.org/databases/prosite/release_with_updates/prosite.dat and is in horrible swiss prot format, so we'll parse out those things that we need and put them in the file

The output file will have the following columns:

prosite family accession number, prosite family name, family type, swiss-prot protein id, fig protein id.

The family type is one of rule, pattern, or matrix. Right now (Prosite Release 19.2 of 24-May-2005) there are 4 rules, 1322 patterns, and 521 matrices.

 This method takes three arguments:
   from    : prosite file
   to      : file to write information to
   verbose : report on progress 

Note that if the from filename ends in .gz it assumed to be a gzipped file and will be opened accordingly.

 Returns the number of lines in the prosite file that were read.

=cut

sub prositecorrespondence {
 my ($self, $from, $to, $verbose)=@_;
 unless (-e $from) {
  print STDERR "File $from does not exist as called in $0\n";
  return 0;
 }
 if ($from =~ /\.gz$/) {
  open(IN, "|gunzip -c $from") || die "Can't open $from using a gunzip pipe";
 }
 else {
  open (IN, $from) || die "Can't open $from";
 }
 open (OUT, ">$to") || die "Can't write to $to";
 my $linecount;
 my ($famac, $famname, $famtype)=('','',''); 
 while (<IN>) {
  chomp;
  $linecount++;
  if ($verbose && !($linecount % 10000))  {print STDERR "Parsed $linecount lines\n"}
  if (m#//#) {($famac, $famname, $famtype)=('','',''); next}
  elsif (m/^ID\s*(.*?);\s*(\S+)/) {($famname, $famtype)=($1, $2); next}
  elsif (m/^AC\s*(\S+)/) {$famac=$1; $famac =~ s/\;\s*$//; next}
  next unless (m/^DR/); # ignore all the other crap in the prosite file for now. Note we might, at some point, want to grab all that, but that is for another time.
  #
  # this is the format of the DR lines:
  # DR   P11460, FATB_VIBAN , T; P40409, FEUA_BACSU , T; P37580, FHUD_BACSU , T;
  s/^DR\s*//;
  foreach my $piece (split /\s*\;\s*/, $_) {
   my ($acc, $nam, $unk)=split /\s*\,\s*/, $piece;
   foreach my $fig ($self->swiss_pir_ids($acc)) {
    print OUT join "\t", ($famac, $famname, $famtype, $acc, $fig), "\n";
   }
  }
 }
}

=head2 swiss_pir_ids()

SwissProt/PIR have lots of ID's that we want to get, usually in this order - uni --> tr --> sp. This routine will map swissprot/pir ids to fig id's, and return an array of FIG id's that match the ID.

=cut

sub swiss_pir_ids {
 my ($self, $id)=@_;
 return () unless ($id);
 $id =~ s/^\s+//; $id =~ s/\s+$//; # trim off the whitespace
 
 my @return=($fig->by_alias("uni|$id"));
 return @return if ($return[0]);
 
 @return=($fig->by_alias("tr|$id"));
 return @return if ($return[0]);

 @return=($fig->by_alias("sp|$id"));
 return @return if ($return[0]);
 
 return ();
}

=head2 ss_by_id

 Generate a list of subsystems that a peg occurs in. This is a ; separated list.
 This is a wrapper that removes roles and ignores essential things

=cut

sub ss_by_id { 
 my ($self, $peg)=@_;
 my $ssout;
 foreach my $ss (sort $fig->subsystems_for_peg($peg)) 
 {
  next if ($$ss[0] =~ /essential/i); # Ignore the Essential B-subtilis subsystems
  $ssout.=$$ss[0]."; ";
 }
 $ssout =~ s/; $//;
 return $ssout;
}

=head2 ss_by_homol

 Generate a list of subsystems that homologs of a peg occur in. This is a ; separated list.
 This is also a wrapper around sims and ss, but makes everything unified

=cut

sub ss_by_homol {
 my ($self, $peg)=@_;
 return unless ($peg);
 my ($maxN, $maxP)=(50, 1e-20);

 # find the sims
 my @sims=$fig->sims($peg, $maxN, $maxP, 'fig');

 # we are only going to keep the best hit for each peg
 # in a subsystem
 my $best_ss_score; my $best_ss_id;
 foreach my $sim (@sims)
 {
  my $simpeg=$$sim[1];
  my $simscore=$$sim[10];
  my @subsys=$fig->subsystems_for_peg($simpeg);
  foreach my $ss (@subsys)
  {
   if (! defined $best_ss_score->{$$ss[0]}) {$best_ss_score->{$$ss[0]}=$simscore; $best_ss_id->{$$ss[0]}=$simpeg}
   elsif ($best_ss_score->{$$ss[0]} > $simscore)
   {
    $best_ss_score->{$$ss[0]}=$simscore;
    $best_ss_id->{$$ss[0]}=$simpeg;
   }
  }
 }

 my $ssoutput=join "", (map {"$_ (".$best_ss_id->{$_}."), "} keys %$best_ss_id);

 $ssoutput =~ s/, $//;
 return $ssoutput;
}

=head2 tagvalue

 This will just check for tag value pairs and return either an array of values or a single ; separated list (if called as a scalar)
 
 e.g. $values=raelib->tagvalue($peg, "PIRSF"); print join "\n", @$values;
 
 Returns an empty array if no tag/value appropriate.

 Just because I use this a lot I don't want to waste rewriting it. 

=cut

sub tagvalue {
 my ($self, $peg, $tag)=@_;
 my @return;
 my @attr=$fig->feature_attributes($peg);
 foreach my $attr (@attr) { 
  my ($gotpeg, $gottag, $val, $link)=@$attr;
  push @return, $val if ($gottag eq $tag);
 }
 return wantarray ? @return : join "; ", @return;
}

=head2 locations_on_contig

Return the locations of a sequence on a contig.

This will look for exact matches to a sequence on a contig, and return a reference to an array that has all the locations.

my $locations=$raelib->locations_on_contig($genome, $contig, 'GATC', undef);
foreach my $bp (@$locations) { ... do something ... }

first argument  : genome number
second argument : contig name
third argument  : sequence to look for
fourth argument : beginning position to start looking from (can be undef)
fifth argument  : end position to stop looking from (can be undef)
sixth argument : check reverse complement (0 or undef will check forward, 1 or true will check rc)

Note, the position is calculated before the sequence is rc'd

=cut

sub locations_on_contig {
 my ($self, $genome, $contig, $sequence, $from, $to, $check_reverse)=@_;
 my $return=[];
 
 # get the dna sequence of the contig, and make sure it is uppercase
 my $contig_ln=$fig->contig_ln($genome, $contig);
 return $return unless ($contig_ln);
 unless ($from) {$from=1}
 unless ($to) {$to=$contig_ln}
 if ($from > $to) {($from, $to)=($to, $from)}
 my $dna_seq=$fig->dna_seq($genome, $contig."_".$from."_".$to);
 $dna_seq=uc($dna_seq);

 # if we want to check the rc, we actually rc the query
 $sequence=$fig->reverse_comp($sequence) if ($check_reverse);
 $sequence=uc($sequence);

 # now find all the matches
 my $posn=index($dna_seq, $sequence, 0);
 while ($posn > -1) {
  push @$return, $posn;
  $posn=index($dna_seq, $sequence, $posn+1);
 }
 return $return;
}


=head2 scrolling_org_list

This is the list from index.cgi, that I call often. It has one minor modification: the value returned is solely the organisms id and does not contain genus_species information. I abstracted this here: 1, so I could call it often, and 2, so I could edit it once.

use like this push @$html, $raelib->scrolling_org_list($cgi, $multiple, $default);

multiple selections will only be set if $multiple is true 

default will set a default to override (maybe) korgs

=cut

sub scrolling_org_list {
 my ($self, $cgi, $multiple, $default)=@_;
 unless ($multiple) {$multiple=0}
 
 my @display = ( 'All', 'Archaea', 'Bacteria', 'Eucarya', 'Viruses', 'Environmental samples' );

 #
 #  Canonical names must match the keywords used in the DBMS.  They are
 #  defined in compute_genome_counts.pl
 #
 my %canonical = (
        'All'                   =>  undef,
        'Archaea'               => 'Archaea',
        'Bacteria'              => 'Bacteria',
        'Eucarya'               => 'Eukaryota',
        'Viruses'               => 'Virus',
        'Environmental samples' => 'Environmental Sample'
     );

 my $req_dom = $cgi->param( 'domain' ) || 'All';
 my @domains = $cgi->radio_group( -name     => 'domain',
                                     -default  => $req_dom,
                                     -override => 1,
                                     -values   => [ @display ]
                                );

 my $n_domain = 0;
 my %dom_num = map { ( $_, $n_domain++ ) } @display;
 my $req_dom_num = $dom_num{ $req_dom } || 0;

 #
 #  Viruses and Environmental samples must have completeness = All (that is
 #  how they are in the database).  Otherwise, default is Only "complete".
 #
 my $req_comp = ( $req_dom_num > $dom_num{ 'Eucarya' } ) ? 'All'
              : $cgi->param( 'complete' ) || 'Only "complete"';
 my @complete = $cgi->radio_group( -name     => 'complete',
                                   -default  => $req_comp,
                                   -override => 1,
                                    -values   => [ 'All', 'Only "complete"' ]
                       );
 #
 #  Use $fig->genomes( complete, restricted, domain ) to get org list:
 #
 my $complete = ( $req_comp =~ /^all$/i ) ? undef : "complete";
 
 my $orgs; my $label;
 @$orgs =  $fig->genomes( $complete, undef, $canonical{ $req_dom } );
 
 foreach (@$orgs) {
   my $gs = $fig->genus_species($_);
   my $gc=scalar $fig->all_contigs($_);
   $label->{$_} = "$gs ($_) [$gc contigs]";
  }

 @$orgs = sort {$label->{$a} cmp $label->{$b}} @$orgs;

 my $n_genomes = @$orgs;

 return (         "<TABLE>\n",
                  "   <TR>\n",
                  "      <TD>",
	          $cgi->scrolling_list( -name     => 'korgs',
                                        -values   => $orgs,
					-labels   => $label,
                                        -size     => 10,
					-multiple => $multiple,
					-default  => $default,
                                      ), $cgi->br,
                  "$n_genomes genomes shown ",
                  $cgi->submit( 'Update List' ), $cgi->reset, $cgi->br,
                  "      </TD>",
                  "      <TD>",
                  join( "<br>", "<b>Domain(s) to show:</b>", @domains), "<br>\n",
                  join( "<br>", "<b>Completeness?</b>", @complete), "\n",
                  "</TD>",
                  "   </TR>\n",
                  "</TABLE>\n",
        );
}


=head2 scrolling_subsys_list

Create a scrolling list of all subsystems. Just like scrolling_org_list, this will make the list and allow you to select multiples.

use like this 

push @$html, $raelib->scrolling_subsys_list($cgi, $multiple);

=cut

sub scrolling_subsys_list {
 my ($self, $cgi, $multiple)=@_;
 $multiple=0 unless (defined $multiple);
 my @ss=sort {uc($a) cmp uc($b)} $fig->all_subsystems();
 my $label;
 # generate labels for the list
 foreach my $s (@ss) {my $k=$s; $k =~ s/\_/ /g; $k =~ s/  / /g; $k =~ s/\s+$//; $label->{$s}=$k}
 return $cgi->scrolling_list(
  -name    => 'subsystems',
  -values  => \@ss,
  -labels  => $label,
  -size    => 10,
  -multiple=> $multiple,
 );
}

=head2 subsys_names_for_display

Return a list of subsystem names for display. This will take a list as an argument and return a nice clean list for display.

$raelib->subsys_names_for_display(@ss);
or
$raelib->subsys_names_for_display($fig->all_subsystems());

=cut

sub subsys_names_for_display {
 my ($self, @ss)=@_;
 foreach (@ss) {s/\_/ /g; 1 while (s/  / /g); s/\s+$//}
 return @ss;
}

=head2 GenBank

 This object will take a genome number and return a Bio::Seq::RichSeq object that has the whole genome
 in GenBank format. This should be a nice way of getting some data out, but will probably be quite slow 
 at building the object.

 Note that you need to call this with the genome name and the contig. This will then go through that contig.

 Something like this should work

 foreach my $contig ($fig->all_contigs($genome)) {
  my $seqobj=FIGRob->GenBank($genome, $contig);
  # process the contig
 }
 
=cut

sub GenBank {
 my ($self, $genome, $contig)=@_;
 my $gs=$fig->genus_species($genome);
 return unless ($gs);
 unless ($contig) {
  print STDERR "You didn't provide a contig for $gs. I think that was a mistake. Sorry\n";
  return;
 }
 my $len=$fig->contig_ln($genome, $contig);
 unless ($len) {
  print STDERR "$contig from $gs doesn't appear to have a length. Is it right?\n";
  return;
 }


 # first find all the pegs ...
 my $features; # all the features in the genome
 my $allpegs; # all the pegs
 my $translation; # all the protein sequences
 foreach my $peg ($fig->pegs_of($genome)) {
  my @location=$fig->feature_location($peg);
  my $func=$fig->function_of($peg);
  foreach my $loc (@location) {
   $loc =~ /^(.*)\_(\d+)\_(\d+)$/;
   my ($cg, $start, $stop)=($1, $2, $3);
   next unless ($cg eq $contig); 
   # save this information for later
   $features->{'peg'}->{$loc}=$func;
   $allpegs->{'peg'}->{$loc}=$peg;
   $translation->{'peg'}->{$loc}=$fig->get_translation($peg);
  }
 }
 # ... and all the RNAs
 foreach my $peg ($fig->rnas_of($genome)) {
  my @location=$fig->feature_location($peg);
  my $func=$fig->function_of($peg);
  foreach my $loc (@location) {
   $loc =~ /^(.*)\_(\d+)\_(\d+)$/;
   my ($cg, $start, $stop)=($1, $2, $3);
   next unless ($cg eq $contig);
   # save this information for later
   $features->{'rna'}->{$loc}=$func;
   $allpegs->{'rna'}->{$loc}=$peg;
  }
 }


 # now get all the contigs out
 my $seq=$fig->dna_seq($genome, $contig."_1_".$len);
 my $description = "Contig $contig from " . $fig->genus_species($genome);
 my $sobj=Bio::Seq->new(
          -seq              =>  $seq, 
	  -id               =>  $contig, 
	  -desc             =>  $description, 
	  -accession_number =>  $genome
	  );
 foreach my $prot (keys %{$features->{'peg'}}) {
   $prot =~ /^(.*)\_(\d+)\_(\d+)$/;
   my ($cg, $start, $stop)=($1, $2, $3);
   my $strand=1;
   if ($stop < $start) {
    ($stop, $start)=($start, $stop);
    $strand=-1;
 }
  
 my $feat=Bio::SeqFeature::Generic->new(
        -start         =>  $start,
        -end           =>  $stop,
        -strand        =>  $strand,
        -primary       =>  'CDS',
	-display_name  =>  $allpegs->{'peg'}->{$prot},
	-source_tag    =>  'the SEED',
        -tag           =>  
                       {
                       db_xref     =>   $allpegs->{'peg'}->{$prot},
		       note        =>  'Generated by the Fellowship for the Interpretation of Genomes',
                       function    =>  $features->{'peg'}->{$prot},
		       translation =>  $translation->{'peg'}->{$prot}
		      }
       );
 
   $sobj->add_SeqFeature($feat);
 }
 
 foreach my $prot (keys %{$features->{'rna'}}) {
   $prot =~ /^(.*)\_(\d+)\_(\d+)$/;
   my ($cg, $start, $stop)=($1, $2, $3);
   my $strand=1;
   if ($stop < $start) {
    ($stop, $start)=($start, $stop);
    $strand=-1;
   }
  
   my $feat=Bio::SeqFeature::Generic->new(
        -start         =>  $start,
        -end           =>  $stop,
        -strand        =>  $strand,
        -primary       =>  'RNA',
        -source_tag    =>  'the SEED',
        -display_name  =>  $allpegs->{'rna'}->{$prot},
        -tag           =>  
                      {
		       db_xref     =>   $allpegs->{'rna'}->{$prot},
                       note        =>  'Generated by the Fellowship for the Interpretation of Genomes',
                       function    =>  $features->{'rna'}->{$prot},
		      }
       );
 
  $sobj->add_SeqFeature($feat);
 }
 return $sobj;
}
 
=head2 best_hit

 Returns the FIG id of the single best hit to a peg

 eg

 my $bh=$fr->best_hit($peg);
 print 'function is ', scalar $fig->function_of($bh);

=cut 

sub best_hit {
 my ($self, $peg)=@_;
 return unless ($peg);
 
 my ($maxN, $maxP)=(1, 1e-5);
 my @sims=$fig->sims($peg, $maxN, $maxP, 'fig');
 return ${$sims[0]}[1];
}


=head1 read_fasta

Read a fasta format file and return a reference to a hash with the data. The key is the ID and the value is the sequence. If you supply the optional keep comments then the comments (anything after the first white space are returned as a sepaarte hash).

Usage:
my $fasta=$raelib->read_fasta($file);
my ($fasta, $comments)=$raelib->read_fasta($file, 1);

=cut

sub read_fasta {
 my ($self, $file, $keepcomments)=@_;
 open (IN, $file) || die "Can't open $file";
 my %f; my $t; my $s; my %c;
 while (<IN>) {
  chomp;
  if (/^>/) {
   if ($s) {
    $f{$t}=$s;
    undef $s;
   }
   s/^>(\S+)\s*//;
   $t=$1;
   $c{$t}=$_ if ($_);
  }
  else {$s .= $_}
 }
 $f{$t}=$s;
 if ($keepcomments) {return (\%f, \%c)} 
 else {return \%f}
}

=head1 rc

Reverse complement. It's too easy.

=cut

sub rc {
 my ($self, $seq)=@_;
 $seq=~tr/GATCgatc/CTAGctag/;
 $seq = reverse $seq;
 return $seq;
}


=head2 cookies

Handle cookies. This method will get and set the value of the FIG cookie. Cookies are name/value pairs that are stored on the users computer. We then retrieve them using this method. The cookies are passed in as a reference to a hash, and the method returns a tuple of the cookie that can be passed to the browser and a reference to a hash with the data.

If you do not pass any arguments the whole cookie will be returned.

Use as follows:

($cookie, $data) = raelib->cookie($cgi, \%data); 

You do not need to pass in any data, in that case you will just get the cookie back

Underneath, I create a single cookie called FIG which stores all the information. The names and value pairs are stored using = to join name to value and ; to concatenate. This way we can create a single cookie with all the data. I am using the FIG::clean_attribute_key method to remove unwanted characters from the name/value pairs, so don't use them.

Note that for the moment I have put this routine here since it needs to maintain the state of the cookie (i.e. it needs to know what $self is). It should really be in HTML.pm but that is not, as far as I can tell, maintaining states?

=cut

sub cookie {
 my ($self, $cgi, $input)=@_;
 return unless ($cgi);
 $self->{'cookie'}=$cgi->cookie(-name=>"FIG") unless ($self->{'cookie'});
 
 # first, create a hash from the existing cookie data
 my $cookie;
 map {
  my ($kname, $kvalue)=split /\=/, $_;
  $cookie->{$kname}=$kvalue;
 } split /\;/, $self->{'cookie'};

 if ($input) 
 {
  # add the values that were passed in
  map {$cookie->{FIG->clean_attribute_key($_)}=$input->{$_}} keys %$input;
  # put everything back together and set the cookie
  my $newcookie=join ";", map {$_ . "=" . $cookie->{$_}} keys %$cookie;
  $self->{'cookie'}=$cgi->cookie(-name=>"FIG", -value=>$newcookie, -expires=>'+1y');
 }
 
 return ($self->{'cookie'}, $cookie);
}




1;


MCS Webmaster
ViewVC Help
Powered by ViewVC 1.0.3