fix: NLTK data download during Docker builds

[peeeteeer]

Description

This PR fixes the Docker image build failure caused by NLTK 3.10.0 rejecting the temporary download path for punkt_tab.

The Dockerfiles already downloaded NLTK data at build time to avoid runtime downloads by unstructured, but NLTK_DATA was only set after the download command. With newer NLTK path validation, /app/nltk_data must be registered before the downloader writes package files there.

Changes

• Set NLTK_DATA=/app/nltk_data before running nltk.downloader.
• Add --exit-on-error so Docker builds fail cleanly instead of prompting interactively.
• Apply the same fix to Dockerfile and Dockerfile.lite.
• Keep the current resolved NLTK version; no dependency rollback or direct download fallback.

Validation

• Verified with a clean build of Dockerfile.lite:

docker build --no-cache -f Dockerfile.lite -t rag-api-lite-nltk-download-fix .

The build completed successfully and downloaded punkt_tab, averaged_perceptron_tagger, and averaged_perceptron_tagger_eng.

Peter Rothlaender ginkgo.rothlaender@external.daimlertruck.com on behalf of Daimler Truck AG.
Provider & Legal Notice | Daimler Truck
Copyright (c) {2026} Daimler Truck AG

[danny-avila]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5eb1891f3a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Make NLTK download failures fail the image build

When any NLTK package cannot be fetched or resolved, --exit-on-error does not make python -m nltk.downloader return a non-zero status; I checked the NLTK CLI source and this option only sets halt_on_error, then the main loop breaks on rv == False without calling sys.exit, so Docker will still continue and can publish an image missing the required data. Please wrap this in a Python one-liner that exits based on nltk.download(...) results or otherwise force a non-zero status; the same command is present in Dockerfile.lite.

Useful? React with 👍 / 👎.