A Walk-through of RiboPy API¶
Contents:
Introduction¶
Ribosome Profiling is a sequencing based method to study protein synthesis transcriptome-wide. Actively translating mRNAs are engaged with ribosomes and protein synthesis rates can be approximated by the number of ribosomes that are translating a given mRNA. Ribosome profiling employs an RNase digestion step to recover fragments of RNA protected by ribosomes which are called Ribosome Protected Footprints (RPFs).
Ribosome profiling data analyses involve several quantifications for each transcript. Specifically, the lengths of the RPFs provide valuable biological information (see, for example, Lareau et al. and Wu et al.). To facilitate ribosome profiling data analyses as a function of RPF length in a highly efficient manner, we implemented a new data format, called ribo. Files in ribo format are called .ribo files.
RiboPy package is an Python interface for .ribo files. The package offers a suite of reading functions for .ribo files, and provides plotting functions that are most often employed in ribosome profiling analyses. Using RiboPy, one can import .ribo files into the Python environment, read ribosome profiling data into pandas data frames and generate essential plots in a few lines of Python code.
This document is structured into several sections. First, we give an overview of the Ribo File format and define transcript regions. Second, we provide instructions and requirements for the installation of RiboPy. Third, we describe how to import a .ribo file to the Python environment and demonstrate essential ribosome profiling data analyses in three sections:
In the last two sections, we describe some advanced features including renaming transcripts using aliases, accessing .ribo file attributes, and getting region boundaries that define transcript regions. In the last section, we explain the three optional types of data, which may exist in a .ribo file.
- Metadata: A .ribo file may contain metadata for each experiment and for the .ribo file itself.
- Coverage: A .ribo file can keep the nucleotide level transcriptome coverage.
- RNA-Seq: A ribosome profiling experiment can be paired with an RNA-Seq experiment to study ribosome occupancy together with transcript abundance.
.ribo File Format ¶
.ribo files are built on an HDF5 architecture and has a predefined internal structure (Figure 1). For a more detailed explanation of the ribo format, we refer to the readthedocs page of ribo.
While many features are required in .ribo files, quantification from paired RNA-Seq data and nucleotide-level coverage are optional.
Transcript Regions¶
The main protein coding region of a transcript is called the coding sequence (CDS). Its boundaries are called transcription start / stop sites. The region consisting of the nucleotides, between the 5’ end of the transcript and the start site, not translated to protein, is called 5’ untranslated region (5’UTR). Similarly, the region having the nucleotides between the stop site and the 3’ end of the transcript, is called 3’ untranslated region (3’UTR). To avoid strings and variable names starting with a number and containing an apostrophe character, we use the names UTR5 and UTR3 instead of 5’UTR and 3’UTR respectively.
Installation ¶
RiboPy requires Python version 3.6 or higher.
Availability¶
The source code of RiboPy package is available in a public Github repository.
Pip¶
RiboPy can be install via pip:
pip install ribopyConda¶
It is recommended to install RiboPy in a separate conda environment. For this, install conda first by following the instructions here.
The following commands will download an environment file, called enviroenment.yaml, and install RiboPy inside a conda environmen named ribo.
wget https://github.com/ribosomeprofiling/riboflow/blob/master/environment.yaml
conda env create -f environment.ymlFrom the Source Code¶
pip install git+https://github.com/ribosomeprofiling/ribopy.git# Some formatting forthe rest of the notebook
from IPython.core.display import HTML
HTML("""
<style>
.output_png {
display: table-cell;
text-align: center;
vertical-align: middle;
}
</style>
""")
Getting Started¶
First, we download a sample ribo file.
! wget https://github.com/ribosomeprofiling/ribo_manuscript_supplemental/raw/master/sidrauski_et_al/ribo/without_coverage/all.ribo
Next, we import the ribopy package
import ribopy
%matplotlib inline
Generating a Ribo Object ¶
To interact with a .ribo file, it is necessary to create a ribo object that provides a direct handle to the file and displays its various attributes. As an example, we processed publicly available ribosome profiling data (GEO accession number GSE65778) from HEK293 cells using RiboFlow pipeline to generate the .ribo file in this document. More precisely, we picked two ribosome profiling experiments coming from untreated HEK293 cells (accession numbers: GSM1606107 and GSM1606108) and two RNA-Seq experiments coming from the same line of untreated cells (accession numbers: GSM1606099 and GSM1606100). Each ribo experiment can be paired with a single RNA-Seq experiment. So we arbitrarily paired the ribosome profiling experiment GSM1606107 with RNA-Seq experiment GSM1606099 and GSM1606108 with GSM1606100, when making the example .ribo file in this document.
# Import the Ribo Object from the RiboPy package
from ribopy import Ribo
ribo_path = "all.ribo"
ribo_object = Ribo(ribo_path, alias = ribopy.api.alias.apris_human_alias)
Once the ribo object is created, we can inquire about the contents of the .ribo file by calling the print_info method.
ribo_object.print_info()
The above output provides information about the individual experiments that are contained in the given ribo object. In addition, this output displays some of the parameters, that were used in generating the .ribo file, such as left span, right span and metagene radius.
For a detailed explanation of the contents of this output, we refer to the online documentation of the ribo format.
In what follows, we demonstrate a typical exploration of ribosome profiling data. We start with the length distribution of RPFs.
Length Distribution ¶
Several experimental decisions including the choice of RNase can have a significant impact on the RPF length distribution. In addition, this information is generally informative about the quality of the ribosome profiling data.
We use the method plot_lengthdist to generate the distribution of the reads mapping to a specific region. This method has also a boolean argument called normalize. When normalize is False, the y-axis displays the total number of reads mapping to the specified region. When fraction is True, the y-axis displays the quotient of the same number as above divided by the total number of reads reported in the experiment.
The following code plots the coding region mapping RPF length distribution.
ribo_object.plot_lengthdist(region_type = "CDS",
normalize = True,
experiments = ["GSM1606107", "GSM1606108"])
We can also plot the absolute number of reads instead of the fraction of total reads by changing the argument normalize = False.
ribo_object.plot_lengthdist(region_type = "CDS",
normalize = False,
experiments = ["GSM1606107", "GSM1606108"])
We can extract the numerical values used to produce the above plot using the method get_length_dist as follows. The parameters of this method will be explained in more detail later in this document.
ribo_object.get_length_dist(region_name = "CDS",
experiments = ["GSM1606107", "GSM1606108"])
Metagene Analysis¶
A common quality control step in ribosome profiling analyses is the inspection of the pileup of sequencing reads with respect to the start and stop site of annotated coding regions. Given that ribosomes are predominantly translating annotated coding regions, these plots are informative about the enrichment at the boundaries of coding regions and also provide information regarding the periodicity of aligned sequencing reads. This type of plot is called a metagene plot as the reads are aggregated around translation start and stop sites across all transcripts.
The parameter “metagene radius” is the number of nucleotides surrounding the start/stop site and hence defines the region of analysis. For each position, read counts are aggregated across transcripts. This cumulative read coverage (y-axis) is plotted as a function of the position relative to the start/stop site (x-axis).
metagene_radius attribute of the ribo object gives us the metagene radius.
ribo_object.metagene_radius
We can plot the ribosome occupancy around the start or stop sites using plot_metagene. The following code produces the metagene plot at the start site for the experiments GSM1606107 and GSM1606108. The values on the y-axis are the raw read counts.
ribo_object.plot_metagene(site_type = "start",
experiments = ["GSM1606107", "GSM1606108"],
range_lower = 15,
range_upper = 40)
To better compare these experiments, we can normalize the coverage by setting normalize = True. In the following example, in addition to these parameters, we set site_type = "stop" to see the stop site coverage.
ribo_object.plot_metagene(site_type = "stop",
experiments = ["GSM1606107", "GSM1606108"],
normalize = True,
range_lower = 15,
range_upper = 40)
The method get_metagene can be used to read the metagene data into a pandas data frame. When using sum_lengths = True and sum_references = True, the user will obtain the most concise data corresponding to the sum of reads surrounding the start / stop site across all read lengths and transcripts. As a result, one row of values will be reported for each experiment.
metagene_start = \
ribo_object.get_metagene(site_type = "start",
range_lower = 15,
range_upper = 40,
sum_lengths = True,
sum_references = True
)
# In the column indeces, 0 corresponds to the start site
# The entire dataframe is too wide. So we display
# the 5 nucleotides to the right and the left of the
# start site.
metagene_start[range(-5,6)]
To maintain the counts at each individual read length summed across transcripts, use sum_lengths = True and sum_references = False. If the metagene data of a single read length, say 30, is needed, set range_lower and range_lower to 30.
metagene_start_trans = \
ribo_object.get_metagene(site_type = "start",
range_lower = 30,
range_upper = 30,
sum_lengths = True,
sum_references = False,
alias = True
)
metagene_start_trans[range(-5,6)]
If we wish to preserve the read counts for individual transcript sum across a range of read lengths, we set sum_references = False and sum_lengths = True. While it is possible to set sum_references = False and sum_lengths = False, run times might be slower for this option, and running the method with these options requires a substantial amount of memory.
metagene_start_length = \
ribo_object.get_metagene(site_type = "start",
range_lower = 15,
range_upper = 40,
sum_lengths = False,
sum_references = True,
alias = True
)
# The entire dataframe gives metagene data for both experiments
# from the relative positions -50 to 50.
# Here we restrict our attention to one experiment: GSM1606107
# and the positions -5 to 5 by slicing the data frame.
metagene_start_length.loc["GSM1606107"][range(-5,6)]
A Note on Aggregating the Output¶
In the above function calls, we used two parameters sum_lengths and sum_references which are used, in general, by reading and plotting functions in RiboPy. These parameters determine how data is aggregated.
More precisely:
- sum_lengths:
- If
True, the counts will be summed up across the given read length interval. Otherwise, ifsum_lengthsisFalse, the counts at each individual read length will be included separately in the output.
- If
- sum_references:
- If
True, the counts will be summed up across all the transcripts. Otherwise, ifsum_referencesisFalse, the counts at each individual transcript will be reported separately.
- If
Region Counts¶
Another important aspect of ribosome profiling data is the number of reads mapping to the different regions of the transcripts, namely, 5’UTR, CDS and 3’UTR. A large number of reads mapping to UTR5 or UTR3 regions might indicate a poor quality ribosome profiling data since ribosomes occupy CDS. Furthermore, the distribution of reads across these regions can be associated with the RNase choice in the experiment. For example in Miettinen and Bjorklund, it was shown that ribosome profiling experiments are dependent on digestion conditions.
Before going into the Python methods, we briefly explain how region counts are computed, introduce our naming convention and define the regions used in ribo format.
For each read mapped to the transcriptome, we take the first nucleotide on the 5’ end of the read and determine the corresponding region. After doing this for all reads, the accumulated values give us the region counts.
As mentioned earlier, a messenger RNA transcript is partitioned into three regions: 5’UTR, CDS and 3’UTR. For technical reasons, we rename 5’UTR as UTR5 and 3’UTR as UTR3.
It is well-known that ribosomes pause, or move slower, around start and stop sites. As a result, we observe peaks around start and stop sites in metagene plots. This behavior of ribosome makes it harder to perform certain analyses such as coverage, translation efficiency, periodicity and uniformity analysis with accuracy. To tackle this problem, we introduce two additional regions called UTR5 junction and UTR3 junction, and modify the definition of the regions UTR5, CDS and UTR3 as shown in Figure 2. This way, we keep regions around start and stop sites separate when doing such analyses.
More precisely, first, we fix two integers: left span (l) and right span (r) and define the junction regions as follows.
UTR5 junction: This region consists of l nucleotides to the left of the start site , and r nucleotides to the right of the start site.
UTR3 junction: This region consists of l nucleotides to the left of the stop site , and r nucleotides to the right of the stop site.
Using these junction regions, we re-define the conventional regions as follows.
UTR5: This region is the set of nucleotides between the 5' end of the transcript and the UTR5 junction.
CDS: This region is the set of nucleotides between the UTR5 junction and UTR3 junction.
UTR3: This region is the set of nucleotides between the UTR3 junction and the 3' end of the transcript.
Similar to the get_metagene method, the get_region_counts method has the sum_references and sum_lengths parameters. As previously mentioned, sum_references specifies whether or not to sum across the transcripts, and sum_lengths specifies whether or not to sum across the read lengths.
The following code will plot the number of sequencing reads whose 5’ ends map to the UTR5, CDS, and UTR3 as a stacked bar plot. To facilitate comparison between experiments, the percentage of the regions counts are plotted and the percentage of reads mapping to CDS are printed on the plot.
ribo_object.plot_region_counts(experiments = ["GSM1606107", "GSM1606108"],
range_lower = 15,
range_upper = 40,
horizontal = True)
To get the region counts, for example for the coding sequence, summed across both lengths and transcripts, set region_name = "CDS", sum_lengths = True and sum_references = True.
ribo_object.get_region_counts(experiments = ["GSM1606107", "GSM1606108"],
region_name = "CDS",
range_lower = 15,
range_upper = 40,
sum_lengths = True,
sum_references = True)
When presented with the option of preserving the region counts at each individual read length, it may be preferable to present the transcript names as their shortened aliases.
To get the data only summed across the read lengths, set sum_lengths = True and sum_references = False. Note that the alias = TRUE in this case, and instead of original.ribo, we are using alias.ribo.
ribo_object.get_region_counts(experiments = ["GSM1606107", "GSM1606108"],
region_name = "CDS",
range_lower = 15,
range_upper = 40,
sum_lengths = True,
sum_references = False,
alias = True)
To get the data in its full form, preserving the information each individual read length and transcript, set sum_lengths = False and sum_references = False.
ribo_object.get_region_counts(experiments = ["GSM1606107", "GSM1606108"],
region_name = "CDS",
range_lower = 15,
range_upper = 40,
sum_lengths = False,
sum_references = False,
alias = True)
Advanced Features ¶
Alias ¶
In the beginning, when we created the ribo object, we used an optional parameter
alias = ribopy.api.alias.apris_human_alias.
ribo_object = Ribo(ribo_path, alias = ribopy.api.alias.apris_human_alias)
The transcriptome reference, that we used to generate this .ribo file, is from GENCODE. In particular, transcript names in this reference are long for the sake of completeness. For example, they include Ensemble transcript and gene ids in addition to standard gene symbols (HGNC).
ribo_object.transcript_names[:2]
A user might want to work with simplified ids for convenience. To provide this functionality, an optional parameter called alias can be provided when initializing a ribo object as shown above.
The default renaming function extracts the fifth entry in the transcript name, separated by “|”.
def apris_human_alias(x):
return x.split("|")[4]
apris_human_alias("ENST00000335137.4|ENSG00000186092.6|OTTHUMG00000001094.4|-|OR4F5-201|OR4F5|1054|protein_coding|")
The renaming can be changed by defining an appropriate method for alias. For example, we can use encode gene id together with the transcript name.
def defalternative_human_alias(x):
x_pieces = x.split("|")
return x_pieces[1] + "_" + x_pieces[4]
ribo_object = ribo_object = Ribo(ribo_path, alias = defalternative_human_alias)
ribo_object.get_region_counts(experiments = ["GSM1606107", "GSM1606108"],
region_name = "CDS",
range_lower = 15,
range_upper = 40,
sum_lengths = True,
sum_references = False,
alias = True)
Object Attributes ¶
If we want a returned list of previously printed attributes, of a ribo object, then the attribute info will return all of the attributes found in the root of the .ribo file as well as information on each of the experiments. For a detailed explanation, see the readthedocs page of RiboPy.
Note that this simply returns a named list of many of the ribo object contents. The returned list organizes the information into three separate values, has.metadata, attributes, and experiment.info. This is a more efficient alternative to simply referencing the values in the ribo object since the method reads directly from the .ribo file handle and not the downstream declared ribo object.
ribo_object.info
Optional Data ¶
Length distribution, metagene coverage and region counts are essential to ribosome profiling data analysis and these data exist in every .ribo file. However, for certain types of analysis, additional data might be required. For example, periodicity and uniformity analyses require the knowledge of number of reads at each nucleotide position, aka coverage data. Another analysis, called translation efficiency, can be done when transcript abundance information is present. For these types of analyses, .ribo files offer two types of optional data: coverage data and RNA-Seq data.
It might be helpful to have data explaining how ribosome profiling data is collected, prepared and processed. For this, .ribo files has an additional field, called metadata, to store such data for each experiment and for the .ribo file itself.
Optional data don’t necessarily exist in every .ribo file. Their existence can be checked as follows.
ribo_object.print_info()
In the above output, we see that both of the experiments have all optional data as the values in the columns ‘Coverage’, ‘RNA-Seq’ and ‘Metadata’ are "*". An absence of "*" would indicate indicate the absence of the corresponding data.
Metadata ¶
A .ribo file can contain metadata for each individual experiment as well as the ribo file itself. If we want to see the metadata of a given experiment, then we can use the get_metadata method and specify the experiment of interest.
To view the metadata of the .ribo file, we use the get_metadata method without any arguments.
ribo_object.get_metadata()
To retrieve metadata from one of the experiments, we provide the parameter experiment. Note that metadata is of the type dictionary.
ribo_object.get_metadata(experiment = "GSM1606107")
Coverage ¶
For all quantifications, we first map the sequencing reads to the transcriptome and use the 5’ most nucleotide of each mapped read. Coverage data is the total number of reads whose 5’ends map to each nucleotide position in the transcriptome.
Within a .ribo file, the coverage data, if exists, is typically the largest data set in terms of storage, and it accounts for a substantial portion of a .ribo file’s size, when present. The get_coverage function returns the coverage information for one specific transcript at a time.
Since coverage data is an optional field of .ribo files, it is helpful to keep track of the experiment names with coverage data. Once the list is obtained, the experiments of interest can easily be chosen and extracted.
In the example below, we read the entire coverage data into a dictionary. Then we access the coverage data of the transcript labeled as 'ENSG00000197568.13_HHLA3-202'. The coverage data coming from lengths, from 28 to 32 are summed up in the resulting array.
coverage_data = ribo_object.get_coverage(experiment = "GSM1606107",
alias = True,
range_lower = 28,
range_upper = 32)
coverage_data["ENSG00000197568.13_HHLA3-202"]
RNA-Seq ¶
Most ribosome profiling experiments generate matched RNA-Seq data to enable analyses of translation efficiency. We provide the ability to store RNA-Seq quantification in .ribo files to facilitate these analyses. We store RNA-seq quantifications in a manner that parallel the region counts for the ribosome profiling experiment. Specifically, the RNA-Seq data sets contain information on the relative abundance of each transcript at each of the following transcript regions.
- 5’ Untranslated Region (UTR5)
- 5’ Untranslated Region Junction (UTR5_junction)
- Coding Sequence (CDS)
- 3’ Untranslated Region Junction (UTR3_junction)
- 3’ Untranslated Region (UTR3)
# Take RNA-Seq data only for the experiment GSM1606107
rnaseq_data = ribo_object.get_rnaseq().loc["GSM1606107"]
# Let's rename the transcript names to their shorter form manually
rnaseq_data.index = \
list(map(apris_human_alias,
ribo_object.transcript_names.tolist() ) )
rnaseq_data
Using the resulting data frame, one get the CDS counts for the experiment GSM1606108.
rnaseq_data["CDS"]