Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .DS_Store
Binary file not shown.
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ authors:
family-names: Yi
affiliation: Renaissance Computing Institute, University of North Carolina
orcid: https://orcid.org/0000-0002-5699-1259
repository-code: 'https://github.com/TranslatorSRI/Babel'
repository-code: 'https://github.com/NCATSTranslator/Babel'
abstract: >-
Babel creates cliques of equivalent identifiers across
many biomedical vocabularies.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ other APIs intended primarily for Translator users.

You can find out more about NodeNorm at its
[Swagger interface](https://nodenormalization-sri.renci.org/docs) or
[in this Jupyter Notebook](https://github.com/NCATSTranslator/NodeNormalization/blob/master/documentation/NodeNormalization.ipynb).
[in this Jupyter Notebook](https://github.com/NCATSTranslator/NodeNormalization/blob/main/documentation/NodeNormalization.ipynb).

### What is the Name Resolver (NameRes)?

Expand All @@ -125,7 +125,7 @@ autocomplete text fields, it has also been used for named-entity linkage.

You can find out more about NameRes at its
[Swagger interface](https://name-resolution-sri.renci.org/docs) or
[in this Jupyter Notebook](https://github.com/NCATSTranslator/NameResolution/blob/master/documentation/NameResolution.ipynb).
[in this Jupyter Notebook](https://github.com/NCATSTranslator/NameResolution/blob/main/documentation/NameResolution.ipynb).

## Understanding Babel outputs

Expand Down
7 changes: 4 additions & 3 deletions docs/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,12 @@ equivalence cliques.

Each line of a compendium file is a JSON object representing one clique. A clique includes:

- `identifiers` — list of all equivalent CURIEs, in preferred-prefix order
- `identifiers` — list of all equivalent CURIEs, in preferred-prefix order. Each entry carries its
own label (`l`), descriptions (`d`, collected from UberGraph and sorted shortest first) and taxa
(`t`). There is no clique-level `descriptions` field
- `ic` — information content score (from UberGraph)
- `taxa` — associated taxa (for genes, proteins, etc.)
- `taxa` — associated taxa (for genes, proteins, etc.); the union of the per-identifier `t` values
- `preferred_name` — the preferred human-readable label for the clique
- `descriptions` — descriptions collected from UberGraph
- `type` — Biolink semantic type

The first identifier in `identifiers` is the preferred identifier for the clique. See
Expand Down
2 changes: 1 addition & 1 deletion docs/Babel.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"id": "13384de2-1c99-4aa6-8792-de8ba393589e",
"metadata": {},
"source": [
"This Jupyter Notebook shows you what it looks like when you [run Babel](https://github.com/NCATSTranslator/Babel/blob/master/docs/Downloads.md)."
"This Jupyter Notebook shows you what it looks like when you [run Babel](https://github.com/NCATSTranslator/Babel/blob/main/docs/Downloads.md)."
]
},
{
Expand Down
3 changes: 3 additions & 0 deletions docs/Conflation.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ on, NodeNorm will:
identifier is the first identifier in the clique, and which consists of all the identifiers from
all the cliques included in that conflation.

For how to turn each conflation on when querying, see the
[NodeNorm API documentation](https://github.com/NCATSTranslator/NodeNormalization/blob/main/documentation/API.md).

## How are types handled for conflated cliques?

Babel does not assign a type to any conflations. When NodeNorm is called with a particular
Expand Down
9 changes: 5 additions & 4 deletions docs/DataFormats.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ This entry consists of the following fields:
| identifiers[0].l | G6PC1 | A label for this identifier. This will almost always be from the source of the CURIE (in this case, the label is from the NCBI Gene database). |
| identifiers[0].d | (blank in this example, but usually 1-3 sentences) | A description of this identifier or concept from this source. |
| identifiers[0].t | ["NCBITaxon:9606"] | A list of taxa that this concept is found in as NCBITaxon CURIEs. NCBITaxon:9606 refers to the species _Homo sapiens_. |
| preferred_name | G6PC1 | The preferred name for this clique. This is not currently used by NodeNorm, but will be in the future. |
| preferred_name | G6PC1 | The preferred name for this clique. NodeNorm returns this as the `label` of the normalized clique. Note that it is not necessarily the label of the clique leader — see below. |
| taxa | ["NCBITaxon:9606"] | A list of taxa that this concept is found in as NCBITaxon CURIEs. This is combined from all the individual taxa from each identifier. |
| type | biolink:Gene | The Biolink type of this concept. Must be a class from the [Biolink model](https://biolink.github.io/biolink-model/) with a `biolink:` prefix. |

Expand Down Expand Up @@ -140,8 +140,10 @@ This entry consists of the following fields:
| taxon_specific | true or false | True if this concept is associated with one or more specific taxa; false if it is not taxon-specific. |
| types | ["Gene", "GeneOrGeneProduct", ...] | A list of Biolink types (without the `biolink:` prefix) for this concept. This is arranged in the same order provided by the Biolink Model Toolkit, starting with the narrowest concept, expanding to the broadest, followed by mixins. |

Note that the synonym files are generated with DrugChemical conflation turned on, but GeneProtein
conflation turned off.
Note that the per-type synonym files are generated with DrugChemical conflation turned on, but
GeneProtein conflation turned off. A separate `synonyms/GeneProteinConflated.txt.gz` is also
produced, containing the same concepts with GeneProtein conflation applied, so that NameRes can be
loaded with that conflation turned on.

## Conflation files

Expand Down Expand Up @@ -350,7 +352,6 @@ If the edge you expected is absent, the link was never generated upstream (a com
RxCUI-typing problem); if it is present but the CURIEs still aren't conflated, the pair was dropped
by a conflation filter — look up the reason in the paired per-run exclusion report
`babel_outputs/reports/drugchemical/excluded_pairs.tsv.gz` (its `reason` column names the filter).
`docs/debugging/Conflation.md` walks through this two-report diagnosis flow end to end.

`Identifier.parquet` — one row per identifier extracted into an `ids/` file:

Expand Down
5 changes: 3 additions & 2 deletions docs/Deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,14 @@ to users who aren't system administrators for these tools:
2. Store the Babel outputs alongside other Babel releases on Hatteras.
3. Deploy a new NodeNorm instance
1. Split the Babel outputs into smaller files to improve load times and put them on a public web
server.
server. The split sizes and the reason for splitting are documented in
[NodeNorm's loader documentation](https://github.com/NCATSTranslator/NodeNormalization/blob/main/documentation/Loader.md).
2. Update the Translator-devops repo with the URL to these Babel output files.
3. Create a [Redis R3 External] instance to store identifiers.
4. Run the [NodeNorm loader] to load the Babel outputs into a Redis instance.
5. Create a [NodeNorm web server] to share the data in a Redis instance.
4. Deploy a new NameRes instance (either
[locally](https://github.com/NCATSTranslator/NameResolution/blob/master/documentation/Deployment.md)
[locally](https://github.com/NCATSTranslator/NameResolution/blob/main/documentation/Deployment.md)
or
[on Kubernetes](https://github.com/helxplatform/translator-devops/tree/ed25b5f5bfe2383ade8457da97341c90500f5291/helm/name-lookup))
1. Create an empty Apache Solr instance.
Expand Down
12 changes: 8 additions & 4 deletions docs/Understanding.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,13 +69,17 @@ conflated.

## Where do the clique descriptions come from?

Currently, all descriptions for NodeNorm concepts come from
Currently, all descriptions come from
[UberGraph](https://github.com/INCATools/ubergraph/). You will note that
descriptions are collected for every identifier within a clique, and then the
description associated with the most preferred identifier is provided for the
preferred identifier. Descriptions are not included in NameRes, but the
`description` flag can be used to include any descriptions when returning
cliques from NodeNorm.
preferred identifier.

Descriptions are written to the [compendium files](./DataFormats.md#compendia-files)
only: the [synonym files](./DataFormats.md#synonym-files) have no description field.
Whether a frontend exposes them therefore depends on which files it is built from --
NodeNorm reads the compendia and can return them via its `description` flag, while
anything built from the synonym files has nothing to return.

## What are "information content" values?

Expand Down
Loading
Loading