Unexpected Topology in Gubbins Newick File: An Analysis of Branch Arrangement with Respect to SNPs

I generated a dummy file and utilized Gubbins to eliminate recombination. Here are the observations I made:

• SNPs are identified based on the bases in the first sequence.
  
  
• Gaps or Ns do not contribute to SNPs
  
  
• The initial sequence is designated as the reference sequence, and any modifications made to it can result in notable changes in the output. These changes may range from slight to significant depending on the similarity between the sequences.

Furthermore, I detected an issue with the newick file in a distinct dataset when employing Gubbins. Specifically, the branches that lack SNPs should have been positioned adjacently, but were instead organized differently, resulting in a distinct topology.

 For instance, in the given example, all the samples without SNPs should have been grouped together, but Gubbins tree (newick) failed to do so.

To achieve this, I utilized the postGubbins.filtered_polymorphic_sites.fasta file and produced a tree using fasttree, resulting in the accurate output illustrated below:

Hope that helps!

Resolved - scenic+ pip install error

  pip install -e .

Obtaining file:///home/prakki/sw/scenicplus

  Preparing metadata (setup.py) ... error

  ERROR: Command errored out with exit status 1:

   command: /home/prakki/sw/pyenv/versions/3.6.4/bin/python3.6 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/prakki/sw/scenicplus/setup.py'"'"'; __file__='"'"'/home/prakki/sw/scenicplus/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-rn24et6_

       cwd: /home/prakki/sw/scenicplus/

  Complete output (22 lines):

  Traceback (most recent call last):

    File "<string>", line 1, in <module>

    File "/home/prakki/sw/scenicplus/setup.py", line 48, in <module>

      "Operating System :: OS Independent",

    File "/home/prakki/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup

      return distutils.core.setup(**attrs)

    File "/home/prakki/sw/pyenv/versions/3.6.4/lib/python3.6/distutils/core.py", line 108, in setup

      _setup_distribution = dist = klass(attrs)

    File "/home/prakki/.local/lib/python3.6/site-packages/setuptools/dist.py", line 444, in __init__

      k: v for k, v in attrs.items()

    File "/home/prakki/sw/pyenv/versions/3.6.4/lib/python3.6/distutils/dist.py", line 281, in __init__

      self.finalize_options()

    File "/home/prakki/.local/lib/python3.6/site-packages/setuptools/dist.py", line 732, in finalize_options

      ep.load()(self, ep.name, value)

    File "/home/prakki/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2434, in load

      return self.resolve()

    File "/home/prakki/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2440, in resolve

      module = __import__(self.module_name, fromlist=['__name__'], level=0)

    File "/home/prakki/sw/scenicplus/.eggs/setuptools_scm-7.1.0-py3.6.egg/setuptools_scm/__init__.py", line 5

      from __future__ import annotations

      ^

  SyntaxError: future feature annotations is not defined

  ----------------------------------------

WARNING: Discarding file:///home/prakki/sw/scenicplus. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Solution:

For some reason, although I installed python3.8 version for scenic+, still it reported error. And surprisingly, my python version was found to be 3.6

$ python --version

Python 3.6.4

but, I had python3.8 version installed in the scenicplus environment.

$ which python3.8

/home/prakki/anaconda3/envs/scenicplus/bin/python3.8

Assuming I am using older version of pip, I checked the above bin had a latest pip version.

and running the below command finally worked in the installation of scenic

/home/prakki/anaconda3/envs/scenicplus/bin/pip install .

devtools installation error

 installing *source* package ‘textshaping’ ...

** package ‘textshaping’ successfully unpacked and MD5 sums checked

** using staged installation

Package harfbuzz was not found in the pkg-config search path.

Perhaps you should add the directory containing `harfbuzz.pc'

to the PKG_CONFIG_PATH environment variable

No package 'harfbuzz' found

Package fribidi was not found in the pkg-config search path.

Perhaps you should add the directory containing `fribidi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'fribidi' found

Using PKG_CFLAGS=

Using PKG_LIBS=-lfreetype -lharfbuzz -lfribidi -lpng

--------------------------- [ANTICONF] --------------------------------

Configuration failed to find the harfbuzz freetype2 fribidi library. Try installing:

 * deb: libharfbuzz-dev libfribidi-dev (Debian, Ubuntu, etc)

 * rpm: harfbuzz-devel fribidi-devel (Fedora, EPEL)

 * csw: libharfbuzz_dev libfribidi_dev (Solaris)

 * brew: harfbuzz fribidi (OSX)

If harfbuzz freetype2 fribidi is already installed, check that 'pkg-config' is in your

PATH and PKG_CONFIG_PATH contains a harfbuzz freetype2 fribidi.pc file. If pkg-config

is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:

R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'

-------------------------- [ERROR MESSAGE] ---------------------------

<stdin>:1:10: fatal error: hb-ft.h: No such file or directory

compilation terminated.

--------------------------------------------------------------------

ERROR: configuration failed for package ‘textshaping’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/textshaping’

Warning in install.packages :

  installation of package ‘textshaping’ had non-zero exit status

ERROR: dependency ‘textshaping’ is not available for package ‘ragg’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/ragg’

Warning in install.packages :

  installation of package ‘ragg’ had non-zero exit status

ERROR: dependency ‘ragg’ is not available for package ‘pkgdown’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/pkgdown’

Warning in install.packages :

  installation of package ‘pkgdown’ had non-zero exit status

ERROR: dependency ‘pkgdown’ is not available for package ‘devtools’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/devtools’

Warning in install.packages :

  installation of package ‘devtools’ had non-zero exit status

The downloaded source packages are in

‘/tmp/RtmpbqMThv/downloaded_packages’

* using staged installation

Package libtiff-4 was not found in the pkg-config search path.

Perhaps you should add the directory containing `libtiff-4.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libtiff-4' found

Package libtiff-4 was not found in the pkg-config search path.

Perhaps you should add the directory containing `libtiff-4.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libtiff-4' found

Using PKG_CFLAGS=

Using PKG_LIBS=-lfreetype -lpng16 -ltiff -lz -ljpeg -lbz2

-----------------------------[ ANTICONF ]-------------------------------

Configuration failed to find one of freetype2 libpng libtiff-4. Try installing:

 * deb: libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev (Debian, Ubuntu, etc)

 * rpm: freetype-devel libpng-devel libtiff-devel libjpeg-turbo-devel (Fedora, CentOS, RHEL)

 * csw: libfreetype_dev libpng16_dev libtiff_dev libjpeg_dev (Solaris)

If freetype2 libpng libtiff-4 is already installed, check that 'pkg-config' is in your

PATH and PKG_CONFIG_PATH contains a freetype2 libpng libtiff-4.pc file. If pkg-config

is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:

R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'

-------------------------- [ERROR MESSAGE] ---------------------------

<stdin>:1:10: fatal error: ft2build.h: No such file or directory

compilation terminated.

--------------------------------------------------------------------

ERROR: configuration failed for package ‘ragg’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/ragg’

Warning in install.packages :

  installation of package ‘ragg’ had non-zero exit status

ERROR: dependency ‘ragg’ is not available for package ‘pkgdown’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/pkgdown’

Warning in install.packages :

  installation of package ‘pkgdown’ had non-zero exit status

ERROR: dependency ‘pkgdown’ is not available for package ‘devtools’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/devtools’

Warning in install.packages :

  installation of package ‘devtools’ had non-zero exit status

The downloaded source packages are in

‘/tmp/RtmpmOvwEJ/downloaded_packages’

Needed to run these packages on Linux terminal:

sudo apt install build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev

sudo apt-get -y build-dep libcurl4-gnutls-dev

sudo apt-get -y install libcurl4-gnutls-dev

sudo apt-get install libcurl4-openssl-dev libssl-dev

sudo apt install libharfbuzz-dev libfribidi-dev

sudo apt-get install libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev

Resolved- Error in scale.default: length of 'scale' must equal the number of columns of 'x'

I encountered the following error when running prcomp function in my shiny app:

Error in scale.default: length of 'scale' must equal the number of columns of 'x'

My command was:

prcomp(Annotated_Gene_PA_df_t, scale = pca_scale)

pca_scale was a radio button with code:

radioButtons("pca_scale", label = "Scale", inline = TRUE, choices = c("TRUE", "FALSE"), selected = "FALSE"),

When giving input as choices from the GUI, passing TRUE/FALSE will result in an error because they are passed in as strings meaning that they are passed with quotes "TRUE"/"FALSE" as in the above case. 

Whereas we need to pass TRUE or FALSE as boolean values - without any quotes.

The below command takes user input as true/false as 1/0 and then the next command converts it into a logical boolean value.

So I changed the above line in ui.R like this:

radioButtons("pca_scale", label = "Scale", inline = TRUE, choices = c("TRUE"=1, "FALSE"=0), selected = "FALSE")

and in the server.R, I changed input$pca_scale to :

as.logical(as.numeric(input$pca_scale))

Resolved - UnsatisfiableError: The following specifications were found to be incompatible with each other

I encountered an error creating a conda envrionment when following single cell best practices tutorial available at:

https://www.sc-best-practices.org/introduction/raw_data_processing.html#a-real-world-example  

Below is the issue:

conda create -n af_xmpl -y -c bioconda python=3.9 salmon alevin-fry pyroe

Collecting package metadata (current_repodata.json): done

Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.

Collecting package metadata (repodata.json): done

Solving environment: | 

Found conflicts! Looking for incompatible packages.

This can take several minutes.  Press CTRL-C to abort.

failed                                                                                                                                                      

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Below commands did not work!

conda update --all --yes
conda config --set channel_priority flexible
conda config --set channel_priority false

conda create -n af_xmpl 

conda activate af_xmpl
conda install -c bioconda python=3.9 salmon alevin-fry pyroe

The following command worked when I added -c conda-forge

conda create -n af_xmpl 
conda activate af_xmpl

conda install -c conda-forge -c bioconda python=3.9 salmon alevin-fry pyroe

Resolved - dist function - NAs introduced by coercion

# Get euclidean dist

euclideanDist <- dist(simMatrix, method = "euclidean")

Warning message:

In dist(simMatrix, method = "euclidean") : NAs introduced by coercion

This is because the first column is factor. So, convert the 1st column into rownames which will take care of this error.

simMatrix <- simMatrix %>% tibble::column_to_rownames("Column1Header")

  # Get euclidean dist

  euclideanDist <- dist(simMatrix, method = "euclidean")

Now, no more error since all the columns in the matrix have only numerical values.

Installing Seurat Package in Rstudio

While installing  seurat, I had to install several dependencies in linux system. Keeping a log of them.

install.packages('Seurat')

Had an issue with curl:

ERROR: configuration failed for package ‘curl’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/curl’

Warning in install.packages :

  installation of package ‘curl’ had non-zero exit status

sudo apt-get install libcurl4-openssl-dev

configure: error: geos-config not found or not executable.

ERROR: configuration failed for package ‘rgeos’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/rgeos’

 sudo apt install libgeos-dev

 install.packages('Seurat')

ERROR: configuration failed for package ‘nloptr’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/nloptr’

Warning in install.packages :

  installation of package ‘nloptr’ had non-zero exit status

sudo apt-get install libnlopt-dev

ERROR: configuration failed for package ‘xml2’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/xml2’

Warning in install.packages :

  installation of package ‘xml2’ had non-zero exit status

sudo apt-get install libxml2-dev 

install.packages("xml2")

ERROR: configuration failed for package ‘hdf5r’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/hdf5r’

sudo apt-get install libhdf5-dev

---------------------------

The below are not necessary for Seurat but I needed to run for other packages

ERROR: configuration failed for package ‘systemfonts’

* removing ‘/home/ramadatta/R/x86_64-pc-linux-gnu-library/4.1/systemfonts’

sudo apt -y install libfontconfig1-dev

Bash - partially resolved - cannot create temp file for here-document: no space left on device

Deleted some of the files from /var/log first

Then docker was not used for sometime, so deleted all the dockers images which reclaimed some space!!

1 2 3 4 5

cd /var/log du -sh * sudo du -ahx / | sort -rh | head -20 sudo docker image prune --all

Useful conda commands list

List the conda environments

conda info --envs

Create a conda environment

conda create --name PlasClass

Activate a conda environment

conda activate PlasClass

Install a package in conda environment

conda install -c bioconda plasclass

Install multiple packages of specific version while creating a conda environment

conda create -n PlasClass python=3.7 scikit-learn=0.21.3 plasclass

Install specific version of package in a conda environment

conda install spades=3.9.0

Search for the package versions available to install in a conda environment

conda search spades

Remove an environment

conda env remove -n PlasClass

Export 

conda env export > singlecell_env.yml

install using yml file

working with mamba  fast installations

useful tips

Difference between PCA and PCoA

 PCA and PCoA are really similar. In fact, PCA is just a type of PCoA that uses euclidean distances! So we could say:

Type of Ordination:
- MDS
- CCA
- PCoA
  - PCoA of Jaccard distances
  - PCoA of Bray-Curtis dissimilarities
  - PCoA of Euclidian distances (this is also called PCA)
  - PCoA of UniFrac distances
source: https://forum.qiime2.org/t/pca-vs-pcoa-which-is-the-appropriate-one-for-microbiome-data/5974/2