Skip to content

Upgrade CouchDB and prometheus export to newer versions; adjust aux files #1641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vkuznet wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Upgrade CouchDB and prometheus export to newer versions; adjust aux files #1641

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
decba04
Upgrade CouchDB and prometheus export to newer versions; adjust aux f…
vkuznet Aug 20, 2025
12696d2
Adjust Dockerfile to avoid two issues: E: Package 'python' has no ins…
vkuznet Sep 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions docker/couchdb/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,20 @@
FROM couchdb:3.2.2
FROM cmssw/filebeat:latest as filebeat
FROM couchdb:3.4.1
MAINTAINER Valentin Kuznetsov vkuznet@gmail.com

RUN apt-get update && apt-get install -y vim less sudo wget unzip python pip
RUN pip install CMSCouchapp
RUN apt-get update && apt-get install -y vim less sudo wget unzip python3 pip
# NOTE:
# RUN pip install CMSCouchapp
# when we install CMSCouchapp on Debian system it fails with PEP 668 error
# which prevents installing external (Python packages from pip) on Debian system
# there are two ways to resolve it:
# - option A: install with (potential) break of system packages
# RUN pip install --break-system-packages CMSCouchapp
# - option B: install in virtual environment
RUN apt install -y python3.11-venv
Copy link
Copy Markdown
Contributor

@amaltaro amaltaro Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why python3.11? Is it the base python version that gets installed with apt-get?

BTW, do we still have to install python? Or does it come with the CouchDB vanilla image?

RUN python3 -m venv /data/venv
ENV PATH="/data/venv/bin:$PATH"
RUN pip install --no-cache-dir CMSCouchapp

ENV WDIR=/data
ENV USER=_couchdb
Expand All @@ -24,9 +36,10 @@ RUN mkdir -p $WDIR
WORKDIR ${WDIR}

# get binary build of couchdb exporter
RUN curl -ksLO https://github.com/gesellix/couchdb-prometheus-exporter/releases/download/v27.0.0/couchdb-prometheus-exporter_27.0.0_Linux_x86_64.tar.gz
RUN tar xfz couchdb-prometheus-exporter_27.0.0_Linux_x86_64.tar.gz
RUN rm couchdb-prometheus-exporter_27.0.0_Linux_x86_64.tar.gz LICENSE README*.md
ARG VER=30.14.0
Copy link
Copy Markdown
Contributor

@amaltaro amaltaro Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest moving this argument variable to the top of the file and rename it to a more meaningful name (EXPORTER_VER ?).

RUN curl -ksLO https://github.com/gesellix/couchdb-prometheus-exporter/releases/download/v${VER}/couchdb-prometheus-exporter_${VER}_Linux_x86_64.tar.gz
RUN tar xfz couchdb-prometheus-exporter_${VER}_Linux_x86_64.tar.gz
RUN rm couchdb-prometheus-exporter_${VER}_Linux_x86_64.tar.gz LICENSE README*.md
ENV PATH="${WDIR}:${PATH}"

# add necessary scripts
Expand All @@ -47,7 +60,12 @@ ADD manage /data/srv/current/config/couchdb/manage

ENV PATH="/opt/couchdb/bin:/usr/local/bin/:${PATH}"

# copy exporters
COPY --from=filebeat /usr/share/filebeat /usr/share/filebeat
COPY --from=filebeat /usr/bin/filebeat /usr/bin/filebeat

# setup final environment
USER $USER
WORKDIR $WDIR
ENV PATH="/data/venv/bin:$PATH"
CMD ["./run.sh"]
3 changes: 3 additions & 0 deletions docker/couchdb/manage
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ COLOR_OK="\\033[0;32m"
COLOR_WARN="\\033[0;31m"
COLOR_NORMAL="\\033[0;39m"
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/kerberos/bin:/usr/local/bin:/opt/couchdb/bin
if [ -f /data/venv/bin/couchapp ]; then
PATH=/data/venv/bin:$PATH
fi

# Does the credential file exist and is it readable?
if [ ! -r $COUCH_CREDS ]; then
Expand Down
13 changes: 8 additions & 5 deletions docker/couchdb/monitor.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

# start couchdb exporter
COUCH_CONFIG=/data/srv/current/auth/couchdb/couchdb_config.ini
export COUCHDB_NODE_NAME="couchdb@$(hostname)"
Copy link
Copy Markdown
Contributor

@amaltaro amaltaro Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you removing this line? Do you have to rebase this branch?

# test if file is not zero size
if [ -s "${COUCH_CONFIG}" ]; then
sudo cp /etc/secrets/$fname /data/srv/current/auth/$srv/$fname
sudo chown $USER.$USER /data/srv/current/auth/$srv/$fname
nohup couchdb-prometheus-exporter -telemetry.address=":9984" -logtostderr=true \
--config=$COUCH_CONFIG -databases.views=false --databases=_all_dbs 2>&1 1>& couchdb_exporter.log < /dev/null &
#sudo cp /etc/secrets/$fname /data/srv/current/auth/$srv/$fname
#sudo chown $USER.$USER /data/srv/current/auth/$srv/$fname
Copy link
Copy Markdown
Contributor

@amaltaro amaltaro Aug 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why secrets are removed? I think it is needed to be able to query CouchDB and collect the metrics.

echo "INFO: starting couchdb prometheus exporter with /data/couchdb_exporter.log"
nohup /data/couchdb-prometheus-exporter -telemetry.address=":9984" \
--config=$COUCH_CONFIG -databases.views=false 2>&1 1>& /data/couchdb_exporter.log < /dev/null &
else
echo "ERROR: couchdb_config.ini file is empty and prometheus exporter cannot be started!"
fi
Expand All @@ -17,7 +17,10 @@ fi
if [ -f /etc/secrets/filebeat.yaml ] && [ -f /usr/bin/filebeat ]; then
ldir=/tmp/filebeat
mkdir -p $ldir/data
echo "INFO: start filebeat with $ldir/log"
nohup /usr/bin/filebeat \
-c /etc/secrets/filebeat.yaml \
--path.data $ldir/data --path.logs $ldir -e 2>&1 1>& $ldir/log < /dev/null &
else
echo "ERROR: filebeat process is not started as /etc/secrets/filebeat.yaml does not exist"
fi