# Comparative Phylogenomics Pipeline Toolkit

An end-to-end Python workflow for comparative protein family analysis, from proteome quality filtering and annotation to phylogenetic inference, rogue taxa pruning, and GeneRax reconciliation.

## What This Repository Does

This repository provides CLI scripts to run a multi-step bioinformatics workflow:

1. BUSCO-based proteome quality filtering.
2. BLAST masking and SwissProt-assisted annotation.
3. Species-wise sequence reduction and orthogroup generation.
4. InterProScan, CLANS clustering, and annotation refinement.
5. MAFFT + trimAl alignments and IQ-TREE (UFB/TBE) phylogenies.
6. Rogue taxa detection/pruning.
7. GeneRax input preparation and reconciliation.
8. Tree/header unmasking and reconciliation event post-processing.

## Core Pipeline Scripts

The main workflow is implemented with these scripts:

- `folder_setup.py`
- `run_busco_pipeline.py`
- `busco_fasta_combiner.py`
- `blast1_pipeline.py`
- `swissprot_annotation.py`
- `sortNcdhit.py`
- `broccoli_pipeline.py` ##manual analysis is imp. for next run
- `interproscan_pipeline.py`
- `receptorAnalysis.py`
- `clans_runner.py` ##manual analysis is imp. for next run
- `filter_CLANS_sequences.py`
- `blast2_pipeline.py`
- `assignFannotationA2.py`
- `alignA2.py`
- `iqtree_ufb_pipeline.py`
- `iqtree_tbe_pipeline.py`
- `rogueTaxaRemover.py`
- `generaxMappingFileBuilder.py` #for TBE mapping
- `generaxUFB_MappingFileBuilder.py`# for UFB mapping
- `generax_reconciliation.py`

## Post-Processing and Utility Scripts

- `1_species_classification.py`: NCBI-based species classification helper.
- `1_unmask_fasta_headers.py`: Replace masked FASTA headers using JSON mapping.
- `2_unmasking_tree_leaves.py`: Replace masked tree leaves with full labels/classification.
- `3_unmask_reconciliation_xml.py`: Unmask RecPhyloXML leaves and export rename report.
- `31_update_species_to_Fullclassification_xml.py`: Replace species labels in RecPhyloXML.
- `4_fix_newick_tree.py`: Fix problematic Newick formatting for downstream tools.

## Requirements

### Python

- Python 3.9+
- Packages from `requirements.txt`:
  - `biopython`
  - `tqdm`
  - `pandas`
  - `pyyaml`
  - `regex`

Install:

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

### External Tools

Most pipeline steps call external command-line tools. Configure their paths in `config.yaml`.

- BUSCO
- BLAST+ (`makeblastdb`, `blastp`, `psiblast`, `blastdbcmd`)
- CD-HIT
- Broccoli
- FastTree
- InterProScan
- PHOBIUS
- MAFFT
- trimAl
- IQ-TREE2
- RogueNaRok (`rnr-prune`, optional LSI tools)
- GeneRax (+ `mpirun`)
- CLANS (command wrapper)

## Configuration

The scripts use `config.yaml` via `ConfigLoader.py`. Before running the workflow, update:

- `analysis_path`
- `proteom_path`
- `input_query_proteins`
- `swissprot_db`
- `annotation2_sps`
- executable/tool paths
- thread counts (`num_thr`)

## Example Run Order

Adjust paths and parameters for your dataset:

```bash
python folder_setup.py --base_dir /path/to/project_root
python run_busco_pipeline.py
python busco_fasta_combiner.py --busco_completeness_score 85
python blast1_pipeline.py
python swissprot_annotation.py
python sortNcdhit.py
python broccoli_pipeline.py
python interproscan_pipeline.py --param_file /path/to/clans_pathways_params.yaml
python receptorAnalysis.py --cdhit_threshold 0.99
python clans_runner.py --start_exp 20 --end_exp 35 --step 3
python filter_CLANS_sequences.py --remove_ids /path/to/clans2_ids_to_remove.json
python blast2_pipeline.py --annotation_species_dir /path/to/annotation2_species
python assignFannotationA2.py
python alignA2.py
python iqtree_ufb_pipeline.py
python iqtree_tbe_pipeline.py
python rogueTaxaRemover.py
python generaxMappingFileBuilder.py
python generax_reconciliation.py --input_dir /path/to/analysis/tbe_reconciliation_inputFiles
```

## Reconciliation Event Processing

Typical post-reconciliation helpers:

```bash
python 3_unmask_reconciliation_xml.py -i input.xml -m masked.json -s species_map.json -o unmasked.xml
python test_extract_reconciliation_events.py -i unmasked.xml -o events.csv -m species_map.json
python test_generax_speciesEventCounts.py -i speciesEventCounts.csv -m species_map.json -o speciesEventCounts_classified.csv
```