Summary

scPred is a general method to predict cell types based on variance structure decomposition. It selects the most cell type-informative principal components from a dataset and trains a prediction model for each cell type. The principal training axes are projected onto the test dataset to obtain the PCs scores for the test dataset and the trained model(s) is/are used to classify single cells.

For more details see our pre-print on bioRxiv:

scPred: Single cell prediction using singular value decomposition and machine learning classification

Application of scPred

First, we load the scPred package and tidyverse.

library("scPred")
library("tidyverse")

We will work with single cell data of pluripotent, blood, skin and neural cells sequenced at low coverage. For more details about the study, see Low-coverage single-cell mRNA sequencing reveals cellular heterogeneity and activated signaling pathways in developing cerebral cortex.

The count matrix and metadata may be obtained here from Hemberg’s lab.

Read the gene expression data (SingleCellExperiment object), calculate CPM values and extract metadata.

download.file("https://scrnaseq-public-datasets.s3.amazonaws.com/scater-objects/pollen.rds", destfile = "~/Downloads/pollen.rds")
require(SingleCellExperiment)
pollen <- readRDS("~/Downloads/pollen.rds")
pollen_counts <- normcounts(pollen)
pollen_cpm  <- apply(pollen_counts, 2, function(x) (x/sum(x))*1000000)
pollen_metadata <- as.data.frame(colData(pollen))

Let’s explore the cell type information

table(pollen_metadata$cell_type2)
## 
##       blood      dermal      neural pluripotent 
##         113          99          65          24

A total of 301 cells are included in the dataset.

For demostration purposes, we split a gene expression matrix into two groups (train and test datasets) based cell type information using the createDataPartition() function from the caret package (already loaded with scPred).

The train partition will be used to train a prediction models for each cell type and finally, the models will be tested using the test partition.

set.seed(1234)
i <- createDataPartition(pollen_metadata$cell_type2, p = 0.70, list = FALSE)
train_data <- t(pollen_cpm[, i])
test_data <- t(pollen_cpm[, -i])

train_info <- pollen_metadata[i, , drop = FALSE]
test_info <- pollen_metadata[-i, , drop = FALSE]

Training step

Eigendecomposition

The first part of the scPred algorithm consists on decomposing the gene expresion matrix of the training dataset to obtained a low dimensional space that can describe most of the variance of the dataset. The eigenDecompose function calculates the first n principal components and log-transforms the input gene expression values to stabilize the variance. It returns an scPred object.

set.seed(1234)
scp <- eigenDecompose(train_data, n = 10)

Then, we assign the metadata containing the cell type information. Row names in the metadata dataframe must match the row names from the eigendecompsed gene expression matrix.

scPred::metadata(scp) <- train_info

Feature selection

Next, we select the principal components that explain the class identity of each cell type using the getFeatureSpace function. This function applies a Wilcoxcon rank sum test to determine the informative principal components according to a categorical variable variable. In this case, we want to predict the cell types in the cell_type2 columns from the metadata. Run ?getFeatureSpace for more details.

scp <- getFeatureSpace(scp, pVar = "cell_type2")
## DONE!

The features slot contains the principal components that explain the class identity.

  • pValue contains the associated p-value for each principal component obtained using the Wilcoxon Rank sum test
  • pValueAdj is the adjusted p-value depending omn the correction criterion applied. By defauls a false discovery rate corrections is performed
  • expVar contrains the explained variance by each principal component
  • cumExpVar contains the cumulative variance explained

All prrincipal components for each cell type are ranked by p-value.

scp@features
## $blood
##    PC       pValue    pValueAdj    expVar cumExpVar
## 1 PC2 6.186072e-26 6.186072e-25 20.291116  20.29112
## 2 PC5 3.594918e-10 1.797459e-09  8.573597  28.86471
## 3 PC3 2.822222e-08 9.407408e-08 11.433759  40.29847
## 4 PC4 2.866624e-07 7.166560e-07 10.080527  50.37900
## 
## $dermal
##    PC       pValue    pValueAdj   expVar cumExpVar
## 1 PC2 1.378227e-29 1.378227e-28 20.29112  20.29112
## 2 PC1 1.300270e-09 6.501350e-09 26.21813  46.50925
## 3 PC4 7.311207e-08 2.437069e-07 10.08053  56.58977
## 4 PC6 2.872213e-05 7.180532e-05  7.26231  63.85208
## 5 PC3 2.034155e-04 4.068310e-04 11.43376  75.28584
## 
## $neural
##    PC       pValue    pValueAdj   expVar cumExpVar
## 1 PC1 9.513036e-21 9.513036e-20 26.21813  26.21813
## 2 PC4 1.103454e-15 5.517270e-15 10.08053  36.29866
## 3 PC3 7.019078e-05 2.339693e-04 11.43376  47.73242
## 
## $pluripotent
##     PC       pValue    pValueAdj    expVar cumExpVar
## 1  PC6 8.590373e-12 8.590373e-11  7.262310   7.26231
## 2  PC4 2.420729e-10 1.210364e-09 10.080527  17.34284
## 3  PC5 1.797335e-08 5.991115e-08  8.573597  25.91643
## 4  PC9 1.169709e-07 2.924273e-07  3.512692  29.42913
## 5  PC8 9.169523e-05 1.833905e-04  4.313980  33.74311
## 6 PC10 4.675783e-03 7.792971e-03  3.418549  37.16165
## 7  PC3 1.065838e-02 1.522625e-02 11.433759  48.59541
## 8  PC2 1.376060e-02 1.720075e-02 20.291116  68.88653

We can plot the principal components grouped by the prediction variable using the plotEigen() function

plotEigen(scp, group = "cell_type2")

Model training

We can now train prediction models for blood, dermal, neural, and pluripotent cell types.

scp <- trainModel(scp, seed = 66)

If we print the scPred object we can look at a summary of the slots contained in it.

  • Expression data: shows the number of cells, genes, and principal components computed.
  • Metadata information: Show the columns in the metadata slot. If columns are factor objects, they can be used as response veriables to train a prediction model
  • Prediction: Shows the prediction variable as indicated using the getFeatureSpace() function
  • Informative PCs per class: shows the number of discriminant principal components for each class (e.g. cell type)
  • Training: Shows the description of the classification model used for training. For each class, performance metrics such as AUROC,accuracym or kappa are shown

The four models showed a specificity of 1 and a sensitivity of 0.99 to 1.

scp
## 'scPred' object
## - Expression data
##       Cells =  213
##       Genes =  21413
##       PCs =  10
## - Metadata information
##       cell_type1, cell_type2
## - Prediction
##       Variable = cell_type2
## - Informative PCs per class
##       blood = 4
##       dermal = 5
##       neural = 3
##       pluripotent = 8
## - Training
##       Model: Support Vector Machines with Radial Basis Function Kernel
##       Class -> blood
##       AUROC = 1, Sensitivity = 0.988, Specificity = 1
##       Class -> dermal
##       AUROC = 1, Sensitivity = 1, Specificity = 1
##       Class -> neural
##       AUROC = 1, Sensitivity = 1, Specificity = 1
##       Class -> pluripotent
##       AUROC = 1, Sensitivity = 1, Specificity = 1

We can plot the distribution of probabilities to see the performance of the predictions for each cell class using

The getTrainResults() function extracts the predictions results obtained from the resampling step for training the prediction model.

res <- getTrainResults(scp)

We can plot the calculated probabilities for each cell type versus our cell labels:

mapply(function(x,y){dplyr::rename(x, probability = !! enquo(y))}, res, names(res), SIMPLIFY = FALSE) %>% 
  Reduce(rbind, .) -> train_probabilities

train_probabilities %>% 
    select(object, obs, probability, "other") %>% 
    ggplot() +
    aes(probability, fill = obs) +
    geom_histogram(bins = 30, color = "black") +
    geom_vline(xintercept = 0.9, color = "red") +
  facet_wrap(~object) +
    theme_bw()

In the previous figure we can observe that a threshold of 0.9 classifies all dermal, neural and pluripotent cells correctly and almost all blood cells too. Each panel represents a prediction model and the colors the known true classes. All other cells are cells except the positive class (for example, for the blood prediction model all other cells are either dermal, neural, or pluripotent)

Prediction step

Once the models have been trained they can be applied to predict cell types in other dataset, for this demonstration we’ll use the test partition/ scPredict() projects the training principal axes onto the test dataset and predicts the cell identity using the trained models. By default, scPredict() uses a threshold of 0.9 to classify the cells into categories.

predictions <- scPredict(scp, newData = test_data, threshold = 0.9)

scPredict() returns a dataframe with the probabilities of each cell to belong to any of the cell classes. The predClass columns is set using the provided threshold.

predictions
##                    blood      dermal      neural pluripotent   predClass
## Hi_2338_1    0.006655574 0.998660453 0.007440833 0.005008939      dermal
## Hi_2338_2    0.005973183 0.997339365 0.007017599 0.004994127      dermal
## Hi_2338_4    0.011011438 0.993270822 0.009921552 0.004863395      dermal
## Hi_2338_5    0.017761575 0.995914310 0.006822524 0.003790678      dermal
## Hi_2338_17   0.024224216 0.902759744 0.014851823 0.005115090      dermal
## Hi_2339_7    0.993408005 0.012042407 0.008148723 0.005297782       blood
## Hi_2339_8    0.997380247 0.007232218 0.005281070 0.005405074       blood
## Hi_2339_9    0.964734320 0.006105625 0.013411268 0.005270282       blood
## Hi_2339_11   0.993005444 0.007104105 0.004966572 0.005266879       blood
## Hi_2339_14   0.965739837 0.038056046 0.006520396 0.005291451       blood
## Hi_K562_2    0.995894429 0.005652630 0.002859333 0.003934111       blood
## Hi_K562_4    0.989691640 0.007281384 0.009687081 0.004711595       blood
## Hi_K562_10   0.988013653 0.006033125 0.002608131 0.004933672       blood
## Hi_BJ_1      0.010147845 0.995365243 0.005814412 0.003934786      dermal
## Hi_BJ_2      0.007979329 0.990617637 0.009042959 0.005142810      dermal
## Hi_BJ_6      0.009356208 0.995618949 0.008309160 0.005081485      dermal
## Hi_BJ_8      0.010222853 0.993650077 0.006426775 0.005213459      dermal
## Hi_BJ_10     0.007288164 0.996559180 0.005883454 0.004426737      dermal
## Hi_BJ_11     0.007089321 0.992122295 0.007084840 0.005234914      dermal
## Hi_BJ_12     0.007945532 0.995737015 0.005083199 0.004576097      dermal
## Hi_BJ_17     0.008009722 0.994678439 0.007570398 0.005267832      dermal
## Hi_BJ_19     0.007158134 0.989414384 0.008616647 0.005318494      dermal
## Hi_BJ_30     0.008399897 0.996345448 0.006304303 0.004977812      dermal
## Hi_BJ_34     0.009390083 0.996917264 0.007515548 0.003792042      dermal
## Hi_K562_19   0.994617229 0.006784354 0.009251010 0.004032288       blood
## Hi_K562_20   0.953711538 0.008983164 0.004315662 0.004139242       blood
## Hi_K562_22   0.994391053 0.005913928 0.004323663 0.004350294       blood
## Hi_K562_23   0.996412632 0.005749871 0.005622456 0.004688627       blood
## Hi_K562_25   0.994925606 0.005923280 0.004797062 0.004737113       blood
## Hi_K562_31   0.996464299 0.005393526 0.003702113 0.004653214       blood
## Hi_K562_34   0.995227159 0.006150064 0.005739733 0.005181328       blood
## Hi_K562_36   0.996773580 0.005926799 0.006433327 0.004129075       blood
## Hi_K562_38   0.995700707 0.005717990 0.003582286 0.004625931       blood
## Hi_K562_40   0.995912496 0.005283276 0.002926409 0.004131742       blood
## Hi_K562_41   0.996614506 0.005355295 0.004310329 0.004487269       blood
## Hi_HL60_2    0.996008108 0.003203810 0.005795405 0.005782189       blood
## Hi_HL60_4    0.986202938 0.002948087 0.016482828 0.004618899       blood
## Hi_HL60_6    0.997596728 0.001202118 0.007585141 0.004881700       blood
## Hi_HL60_7    0.982888385 0.006903896 0.009560793 0.005387137       blood
## Hi_HL60_14   0.997717195 0.001131019 0.005303001 0.005319515       blood
## Hi_HL60_15   0.995420209 0.007555687 0.008033794 0.005274234       blood
## Hi_HL60_23   0.994897770 0.002550216 0.006079878 0.005537577       blood
## Hi_HL60_33   0.996792166 0.004219183 0.006055872 0.005153275       blood
## Hi_HL60_36   0.995829430 0.003149186 0.005590083 0.005020308       blood
## Hi_HL60_37   0.996538603 0.006027821 0.008660022 0.005188812       blood
## Hi_HL60_43   0.996048487 0.003662060 0.005691881 0.004947632       blood
## Hi_HL60_48   0.996612696 0.004402476 0.007093712 0.005819998       blood
## Hi_HL60_52   0.995951072 0.004375453 0.007077423 0.004929166       blood
## Hi_HL60_54   0.997391250 0.003718601 0.006168916 0.005145478       blood
## Hi_iPS_1     0.006773016 0.006315371 0.005107275 0.956059574 pluripotent
## Hi_iPS_4     0.005459630 0.005991352 0.006274916 0.938796808 pluripotent
## Hi_iPS_7     0.007263328 0.006166505 0.003852328 0.968530750 pluripotent
## Hi_iPS_8     0.006315538 0.004690963 0.087484043 0.919965537 pluripotent
## Hi_iPS_10    0.005696906 0.006815225 0.002949809 0.962061727 pluripotent
## Hi_iPS_16    0.005472227 0.006513774 0.004396123 0.957432101 pluripotent
## Hi_iPS_23    0.006540182 0.005926688 0.005573124 0.937263083 pluripotent
## Hi_Kera_2    0.005790139 0.990108122 0.007238154 0.004516955      dermal
## Hi_Kera_6    0.007670905 0.986628692 0.006459826 0.005081744      dermal
## Hi_Kera_7    0.003594498 0.996075392 0.006550450 0.003735582      dermal
## Hi_Kera_8    0.006915099 0.996701772 0.007179775 0.003989066      dermal
## Hi_Kera_9    0.010128375 0.979702056 0.005616018 0.004241997      dermal
## Hi_Kera_10   0.002340482 0.996582641 0.006275293 0.004080765      dermal
## Hi_Kera_11   0.004360678 0.992117012 0.006753084 0.003618307      dermal
## Hi_Kera_13   0.006507960 0.997093650 0.006786817 0.003808418      dermal
## Hi_Kera_14   0.013371787 0.984571905 0.006841817 0.004315667      dermal
## Hi_Kera_16   0.012030558 0.970493319 0.005210746 0.004420132      dermal
## Hi_Kera_25   0.004569195 0.994097757 0.006362785 0.003744316      dermal
## Hi_Kera_39   0.004015330 0.990008026 0.006202351 0.003942243      dermal
## Hi_Kera_40   0.077702287 0.758051254 0.006991119 0.004444644  unassigned
## Hi_GW21.2_2  0.005387813 0.005735801 0.997628545 0.004467724      neural
## Hi_GW21.2_7  0.006159599 0.005077643 0.999277999 0.005481231      neural
## Hi_GW21.2_8  0.007246919 0.005188322 0.999270411 0.006086956      neural
## Hi_GW21.2_10 0.006596429 0.004754111 0.999664466 0.006451112      neural
## Hi_GW21.2_13 0.007135911 0.005351180 0.998825375 0.006651760      neural
## Hi_GW21.2_14 0.036515341 0.009006564 0.985959169 0.006464610      neural
## Hi_GW21_3    0.006739270 0.003618348 0.999669848 0.004820364      neural
## Hi_GW21_5    0.010231166 0.013654829 0.998429909 0.003903507      neural
## Hi_NPC_3     0.004529544 0.005334441 0.998956256 0.004840877      neural
## Hi_NPC_10    0.004695227 0.004655696 0.998736001 0.004358722      neural
## Hi_NPC_13    0.005328657 0.004962464 0.993770313 0.006186888      neural
## Hi_GW16_1    0.009173163 0.005942934 0.998591466 0.004733551      neural
## Hi_GW16_2    0.012745847 0.006665138 0.996897389 0.004281073      neural
## Hi_GW16_8    0.007429963 0.004603408 0.994664971 0.005579737      neural
## Hi_GW16_9    0.009558793 0.007997607 0.997377036 0.003998296      neural
## Hi_GW16_20   0.005432284 0.004486833 0.999827641 0.003163685      neural
## Hi_GW16_23   0.024155920 0.004874453 0.999093780 0.004073372      neural
## Hi_GW16_24   0.008210821 0.003561112 0.999554156 0.004285569      neural
## Hi_GW16_26   0.008582737 0.007208059 0.998088158 0.003999380      neural

We can plot the probabilities and compare our predictions to the true cell types

predictions %>% 
  mutate(true = test_info$cell_type2) %>% 
  gather(key = "model", value = "probability", 1:4) %>% 
  ggplot() +
  aes(probability, fill = true) +
  geom_histogram(color = "black") +
  geom_vline(xintercept = 0.9, color = "red") +
  facet_wrap(~model) +
  theme_bw() +
  scale_fill_viridis_d()
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

In the previous plot, each panel represents the prediction model for each cell type with the output distribution probabilities. The colors in all plot represents the “true”. We can observe that all blood, neural and pluripotent cells were correctly classified using a threshold of 0.9. Only one dermal cell was labeled as unassigned as it was below the threshold. This cell has a probability of 0.75 of being dermal.

Reproducibility

options(width = 70)
devtools::session_info(include_base = TRUE)
## Warning: 'DESCRIPTION' file has an 'Encoding' field and re-encoding is
## not possible
## Session info --------------------------------------------------------
##  setting  value                       
##  version  R version 3.5.0 (2018-04-23)
##  system   x86_64, darwin15.6.0        
##  ui       X11                         
##  language (EN)                        
##  collate  en_AU.UTF-8                 
##  tz       Australia/Brisbane          
##  date     2018-07-24
## Packages ------------------------------------------------------------
##  package              * version    date      
##  abind                  1.4-5      2016-07-21
##  assertthat             0.2.0      2017-04-11
##  backports              1.1.2      2017-12-13
##  base                 * 3.5.0      2018-04-24
##  bindr                  0.1.1      2018-03-13
##  bindrcpp             * 0.2.2      2018-03-29
##  Biobase              * 2.40.0     2018-05-01
##  BiocGenerics         * 0.26.0     2018-05-01
##  BiocParallel         * 1.14.1     2018-05-06
##  bitops                 1.0-6      2013-08-17
##  broom                  0.4.5      2018-07-03
##  caret                * 6.0-80     2018-05-26
##  cellranger             1.1.0      2016-07-27
##  class                  7.3-14     2015-08-30
##  cli                    1.0.0      2017-11-05
##  codetools              0.2-15     2016-10-05
##  colorspace             1.3-2      2016-12-14
##  compiler               3.5.0      2018-04-24
##  crayon                 1.3.4      2017-09-16
##  CVST                   0.2-2      2018-05-26
##  datasets             * 3.5.0      2018-04-24
##  ddalpha                1.3.4      2018-06-23
##  DelayedArray         * 0.6.1      2018-06-15
##  DEoptimR               1.0-8      2016-11-19
##  devtools               1.13.6     2018-06-27
##  digest                 0.6.15     2018-01-28
##  dimRed                 0.1.0      2017-05-04
##  dplyr                * 0.7.6      2018-06-29
##  DRR                    0.0.3      2018-01-06
##  evaluate               0.10.1     2017-06-24
##  forcats              * 0.3.0      2018-02-19
##  foreach                1.4.4      2017-12-12
##  foreign                0.8-70     2017-11-28
##  GenomeInfoDb         * 1.16.0     2018-05-01
##  GenomeInfoDbData       1.1.0      2018-05-28
##  GenomicRanges        * 1.32.3     2018-05-16
##  geometry               0.3-6      2015-09-09
##  ggExtra              * 0.8        2018-04-04
##  ggplot2              * 3.0.0      2018-07-03
##  glue                   1.2.0      2017-10-29
##  gower                  0.1.2      2017-02-23
##  graphics             * 3.5.0      2018-04-24
##  grDevices            * 3.5.0      2018-04-24
##  grid                   3.5.0      2018-04-24
##  gtable                 0.2.0      2016-02-26
##  haven                  1.1.2      2018-06-27
##  hms                    0.4.2      2018-03-10
##  htmltools              0.3.6      2017-04-28
##  httpuv                 1.4.4.2    2018-07-02
##  httr                   1.3.1      2017-08-20
##  ipred                  0.9-6      2017-03-01
##  IRanges              * 2.14.10    2018-05-16
##  irlba                * 2.3.2      2018-01-11
##  iterators              1.0.9      2017-12-12
##  jsonlite               1.5        2017-06-01
##  kernlab              * 0.9-26     2018-04-30
##  knitr                  1.20       2018-02-20
##  labeling               0.3        2014-08-23
##  later                  0.7.3      2018-06-08
##  lattice              * 0.20-35    2017-03-25
##  lava                   1.6.2      2018-07-02
##  lazyeval               0.2.1      2017-10-29
##  lubridate              1.7.4      2018-04-11
##  magic                  1.5-8      2018-01-26
##  magrittr             * 1.5        2014-11-22
##  MASS                   7.3-50     2018-04-30
##  Matrix               * 1.2-14     2018-04-13
##  matrixStats          * 0.53.1     2018-02-11
##  memoise                1.1.0      2017-04-21
##  methods              * 3.5.0      2018-04-24
##  mime                   0.5        2016-07-07
##  miniUI                 0.1.1.1    2018-05-18
##  mnormt                 1.5-5      2016-10-15
##  ModelMetrics           1.1.0      2016-08-26
##  modelr                 0.1.2      2018-05-11
##  munsell                0.5.0      2018-06-12
##  nlme                   3.1-137    2018-04-07
##  nnet                   7.3-12     2016-02-02
##  parallel             * 3.5.0      2018-04-24
##  pillar                 1.2.3      2018-05-25
##  pkgconfig              2.0.1      2017-03-21
##  pls                    2.6-0      2016-12-18
##  plyr                   1.8.4      2016-06-08
##  pROC                 * 1.12.1     2018-05-06
##  prodlim                2018.04.18 2018-04-18
##  promises               1.0.1      2018-04-13
##  psych                  1.8.4      2018-05-06
##  purrr                * 0.2.5      2018-05-29
##  R6                     2.2.2      2017-06-17
##  RColorBrewer           1.1-2      2014-12-07
##  Rcpp                   0.12.17    2018-05-18
##  RcppRoll               0.3.0      2018-06-05
##  RCurl                  1.95-4.10  2018-01-04
##  readr                * 1.1.1      2017-05-16
##  readxl                 1.1.0      2018-04-20
##  recipes                0.1.3      2018-06-16
##  reshape2               1.4.3      2017-12-11
##  rlang                  0.2.1      2018-05-30
##  rmarkdown              1.10       2018-06-11
##  robustbase             0.93-1     2018-06-23
##  rpart                  4.1-13     2018-02-23
##  rprojroot              1.3-2      2018-01-03
##  rstudioapi             0.7        2017-09-07
##  rvest                  0.3.2      2016-06-17
##  S4Vectors            * 0.18.3     2018-06-08
##  scales                 0.5.0      2017-08-24
##  scPred               * 0.0.0.9000 2018-07-24
##  sfsmisc                1.1-2      2018-03-05
##  shiny                  1.1.0      2018-05-17
##  SingleCellExperiment * 1.2.0      2018-05-01
##  splines                3.5.0      2018-04-24
##  stats                * 3.5.0      2018-04-24
##  stats4               * 3.5.0      2018-04-24
##  stringi                1.2.3      2018-06-12
##  stringr              * 1.3.1      2018-05-10
##  SummarizedExperiment * 1.10.1     2018-05-11
##  survival               2.42-4     2018-06-30
##  tibble               * 1.4.2      2018-01-22
##  tidyr                * 0.8.1      2018-05-18
##  tidyselect             0.2.4      2018-02-26
##  tidyverse            * 1.2.1      2017-11-14
##  timeDate               3043.102   2018-02-21
##  tools                  3.5.0      2018-04-24
##  utils                * 3.5.0      2018-04-24
##  viridisLite            0.3.0      2018-02-01
##  withr                  2.1.2      2018-03-15
##  xml2                   1.2.0      2018-01-24
##  xtable                 1.8-2      2016-02-05
##  XVector                0.20.0     2018-05-01
##  yaml                   2.1.19     2018-05-01
##  zlibbioc               1.26.0     2018-05-01
##  source                                          
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  local                                           
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  Bioconductor                                    
##  Bioconductor                                    
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  local                                           
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  local                                           
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.1)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  Bioconductor                                    
##  Bioconductor                                    
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  local                                           
##  local                                           
##  local                                           
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  local                                           
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  local                                           
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  CRAN (R 3.5.0)                                  
##  local (IMB-Computational-Genomics-Lab/scPred@NA)
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  local                                           
##  local                                           
##  local                                           
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  local                                           
##  local                                           
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  CRAN (R 3.5.0)                                  
##  Bioconductor                                    
##  CRAN (R 3.5.0)                                  
##  Bioconductor