KIARVA (backend)

The Karolinska Institutet Adaptive Immune Receptor Gene Variant Atlas (KIARVA) is a web-based atlas of germline immunoglobulin (IG) heavy chain (IGH) alleles.

The KIARVA backend serves as an API to the KIARVA frontend.

Table of Contents

• Background
• Cite this portal
• Contributing
• How to get help
• Credits
• Development
  • Architecture Overview
  • Running a local copy of the API and contributing to the codebase

Background

Variation in the immunoglobulin (IG) locus arises from both structural and allelic diversity. The Karolinska Institutet Adaptive Immune Receptor Gene Variant Atlas (KIARVA) catalogs germline-encoded IG heavy chain (IGH) alleles identified in 2,486 individuals from the 1000 Genomes Project (1KGP), comprising 561 IGHV, 51 IGHD, and 12 IGHJ variants. The data was generated using the ImmuneDiscover pipeline (Corcoran et al., BioRxiv) and includes allele frequencies across 5 superpopulations and 25 subpopulations, many reported here for the first time. KIARVA provides downloadable FASTA files and validation scores via IgSNPer, with sequence data available with or without flanking regions to support antibody sequence assignment and gene association studies.

Cite this portal

If you use KIARVA in research, please cite the portal using
RRID: SCR_026682, including the version number and access date
(https://kiarva.scilifelab.se).

If you reuse the code, please also cite this repository using the metadata provided in CITATION.cff (see the “Cite this repository” button).

Website content and data are available for non-commercial reuse under CC BY-NC 4.0. The source code in this repository is released under the MIT License.

Contributing

We welcome contributions, small and large, to our codebase, dashboards and documentation. They will be published after review and approval by the KIARVA team. Fork, open a pull request, or contact us to discuss ideas! Information on the technical details of the portal can be found in the development section.

How to get help

If you have any questions regarding the scientific content or data associated with KIARVA, please get in touch by emailing us at Martin Corcoran and Gunilla Karlsson Hedestam. If you have any questions regarding any of the code associated with KIARVA, please get in touch by emailing us at precisionmedicine@scilifelab.se.

Credits

The scientific content, background and data is provided by Gunilla Karlsson Hedestam's research group at Karolinska Institutet. The service was built and is maintained by a team at Karolinska Institutet and SciLifeLab Data Centre. We are grateful to many collaborators for their contributions.

Development

Architecture Overview

The backend of the KIARVA web app is written in Python with the Flask framework. It handles KIARVA data served at endpoints meant to be consumed by the KIARVA frontend. KIARVA is a read-only app, serving research data comprised of germline immunoglobulin (IG) heavy chain (IGH) alleles generated by the ImmuneDiscover pipeline. The data is provided to us in the form of .tsv files, which we read and load to a database on startup.

Core Frameworks and Libraries

• Language: Python 3.13
• Framework: Flask 3.1
• Core libraries: Flask-SQLAlchemy 3.1, Flask-Migrate 4.1

Repository Overview

KIARVA Flask app
├── app.py
├── config.py
├── constants.py
├── data
│   ├── compressed
│   │   ├── tsv_files-prepub.zip
│   │   └── tsv_files-prod.zip
├── db.py
├── docker
│   ├── Dockerfile
│   └── entrypoint.sh
├── loaders
│   ├── load_tsv_to_db.py
│   └── plot_loader.py
├── mafft/
├── migrations/
├── models
│   └── immunediscoverdata.py
├── pytest.ini
├── README.md
├── repositories
│   ├── allele.py
│   ├── aminoacid.py
│   ├── igsnper.py
│   ├── plot_options.py
│   └── population.py
├── requirements.txt
├── requirements-dev.txt
├── resources
│   └── immunediscoverdata.py
├── schemas.py
├── security.py
├── services
│   ├── alignment.py
│   ├── fasta_generation.py
│   ├── frequencies.py
│   └── sequence_search.py
├── tests
│   ├── conftest.py
│   ├── generate_mock_data.py
│   ├── mock_data/
│   ├── test_routes.py
│   └── test_translation.py
└── utils
    ├── fasta_processing.py
    ├── progress_bar.py
    └── translation.py

Top-Level Files

• app.py – Main entry point of the Flask application.
• config.py – Configuration settings for the app (e.g. database paths, debug flags).
• constants.py – Global constants used throughout the app.
• db.py – Initializes and manages the database connection.
• schemas.py – Defines input/output validation schemas (e.g. for API endpoints).
• security.py – Authentication and access control logic, if used.
• pytest.ini – Configuration for Pytest.
• README.md – Project documentation.
• requirements.txt – Version-locked runtime Python dependencies (what the container installs).
• requirements-dev.txt – Runtime dependencies plus test tooling, for local development and CI.
• requirements-top.txt – The direct dependencies, unlocked, for reference.

Containerization

• docker/Dockerfile – Builds the Docker image for the app.
• docker/entrypoint.sh – Startup script for the Docker container (e.g. runs migrations, launches server).

Domain-Specific Code

• models/ - Defines the SQLAlchemy models representing database tables (e.g. immunediscoverdata.py).

• repositories/ - Responsible for querying the database. Each module encapsulates access to one domain (e.g. alleles, populations, plots).

• services/ - Contains application logic that operates on data — such as sequence alignment, frequency calculations, or generating FASTA files. Services are independent of request/response handling.

• utils/ - Generic helper functions (e.g. FASTA parsing, printing progress bars) that don’t belong to a specific domain.

• loaders/ - Scripts for loading or preprocessing data into the application or database — for example from TSV files or precomputing plots.

• resources/ - Contains the Flask endpoints serving the data.

Testing

Uses Pytest with Flask and db fixtures.

Under tests/ -

• test_routes.py – Tests for API endpoints.
• test_translation.py – Tests for the genetic code translation in utils/translation.py.
• conftest.py – Shared test fixtures (e.g. test client, mock DB).
• generate_mock_data.py – Generates mock inputs for testing. The dataset is kept minimal and deliberate: it uses real gene/allele naming and one row per edge case, each documented in the file. Run python -m tests.generate_mock_data after editing it to rewrite both mock_data/in/ and mock_data/compressed/.
• mock_data/ – Static mock input files used by tests, in the same layout as data/: the tests unpack mock_data/compressed/ into mock_data/in/ the same way production unpacks the real data, so the .zip is the file that is actually loaded.

Data & Dependencies

• data/compressed/ - Contains input data used by the application. The raw data consists of .tsv formated data provided by the Gunilla Hedestam research group. The data is compressed to decrease size. The file tsv_files-prod.zip contains the currently public data that is exposed in our production environment. The file tsv_files-prepub.zip contains data that is pre-publication and should not be available publically yet. Because of this, tsv_files-prepub.zip is encrypted. The reason it appears in the repository is so that we can expose it in a separate password protected environment for internal use of Gunilla Hedestam's group before publication.

• mafft/ - Holds the MAFFT binary or source files used for sequence alignment. This is typically used internally by services/alignment.py.

• migrations/ - Generated by Alembic (Flask-Migrate) for tracking database schema changes.

Running a local copy of the API and contributing to the codebase

Fork and clone the repository

Important: You must work through a fork of the repository. Do not commit directly to the main repository.

1. Fork the repository: Click the "Fork" button at the top right of the main repository page to create your own copy.

2. Clone your fork: Clone your forked repository to your local machine:

   git clone https://github.com/YOUR_USERNAME/kiarva-backend.git

3. Add the upstream remote: Add the original repository as an upstream remote to keep your fork updated:

   git remote add upstream https://github.com/ScilifelabDataCentre/kiarva-backend.git

Please note that we require all commits to be verified, so you must sign your commits. For information on how to set this up, see the GitHub documentation here.

Fetch changes at any time from this remote:

git pull upstream dev

Local Development Setup

Follow these steps to get the app running locally:

Clone the Repository

git clone https://github.com/ScilifelabDataCentre/kiarva-backend
cd kiarva-backend

Create a Virtual Environment

Use Python’s built-in venv to manage dependencies:

python3 -m venv .venv
source .venv/bin/activate  # On Windows use: .venv\Scripts\activate 

Install Dependencies

Once the virtual environment is activated, install all required packages:

pip install -r requirements-dev.txt

This installs the runtime dependencies from requirements.txt plus the packages needed to run the test suite. The container image installs only requirements.txt, so test tooling is not shipped to production.

Create a .env File for Environment Variables

In the project root, create a .env file with the following content:

API_KEY="test123"

This API key is required for all requests to the app. You will need to include it in the headers when testing or making requests:

X-api-key: test123

You can change the key as you wish, but it must match the one in .env.

Apply Database Migrations

Set up the local database using Flask-Migrate:

SKIP_DATA_LOAD=1 flask db upgrade

This has to happen before the app will start: without the table, create_app() raises rather than serving an empty database. SKIP_DATA_LOAD=1 is what makes the migration itself exempt, and it is needed because the flask CLI builds the app through that same factory to run a migration — at which point the tables it is about to create do not exist yet. docker/entrypoint.sh sets it on the same command, for the same reason.

Install MAFFT (Required for Sequence Alignment)

The app uses the external tool MAFFT for gene sequence alignment. To install it:

• Download MAFFT for your system. (MAFFT - a multiple sequence alignment program )
• Follow the installation instructions.
• Ensure it is accessible from the command line (i.e. mafft command should work in your terminal).

Note: The mafft/ folder in this repo contains a .tar.gz archive used only in containerized deployments (e.g. Kubernetes).

Run the App

To run the app in development mode with debugging:

flask run --debug

Optional: Run Tests

To verify your setup and test API routes:

pytest 

When calling endpoints in your tests or tools like Postman or curl, remember to include the required Authorization header:

curl http://localhost:5000/some-endpoint -H "X-api-key: test123"

Docker

You can also use the ghcr image to build and run a container.

Create a branch and develop

To create a new branch and start developing in it:

git branch my_branch
git checkout my_branch

After doing this you can make any changes you want. You must then either add all changed files or specific changed files to your commit:

git add -A

or

git add my_changed_file

You can then commit and push to your branch:

(NOTE: DO NOT FORGET TO SIGN YOUR COMMITS, by policy only signed commits can be merged into the main branches.)

git commit -S -m "My commit"
git push origin my_branch

The code is now in your branch on your fork, but it does not get merged into the main repository without being reviewed as a pull request.

Make a pull request

Once you're finished with your edits and they are committed and pushed to your branch, it's time to open a pull request from your fork to the main repository.

You can find full documentation on the GitHub help website. In short:

• Visit your fork: https://github.com/YOUR_USERNAME/kiarva-backend
• Find the branch my_branch that you created and pushed to
• Click the button that reads "New Pull Request"
• GitHub will automatically detect that you want to create a pull request to the upstream repository
• Add/change title as well as a description of what you've done
• Add reviewers from the organization to review your pull request
• Click Create Pull Request

Once created, a member of our team will review your changes and reach out to you if they have any questions or requests for changes.

Once approved, they will be merged and deployed in the next release.