Skip to content

[New Integration] Initial commit of EfficientIP Intergration#18505

Open
jasperklaren wants to merge 7 commits intoelastic:mainfrom
jasperklaren:main
Open

[New Integration] Initial commit of EfficientIP Intergration#18505
jasperklaren wants to merge 7 commits intoelastic:mainfrom
jasperklaren:main

Conversation

@jasperklaren
Copy link
Copy Markdown

Proposed commit message

Title: Add EfficientIP DDI integration for DNS and DHCP log ingestion

WHAT:

This adds a new community Elastic integration for EfficientIP DDI (DNS, DHCP, IPAM)
appliances. The integration collects syslog messages over UDP and normalizes them
into ECS-compliant documents via a multi-stage ingest pipeline architecture:

A default router pipeline (default.yml) parses the syslog envelope using grok
(priority, timestamp, service name, PID), classifies messages by service name
(dhcpd/dhcpv6 → DHCP, named → DNS, httpd → AUDIT), and delegates to
the appropriate sub-pipeline. It also handles timezone-aware timestamp parsing,
GeoIP/ASN enrichment on client.ip, IP type conversion with error handling,
related.ip/related.hosts population, and recursive null/empty value cleanup
via a Painless script.

The DHCP sub-pipeline (pipeline_dhcp.yml) uses message-type-specific grok
processors gated by ctx.message.contains(...) checks to parse the full DHCP
lifecycle: DISCOVER, OFFER, REQUEST, ACK, NAK, RELEASE, EXPIRE, INFORM,
DECLINE, LEASEQUERY, and REFUSED events, plus DHCPv6 messages (Encapsulated
Solicit, Advertise NA, Relay-forward). Each message type has multiple grok
patterns ordered from most-specific to least-specific to handle variations in
EfficientIP's log format (optional client hostname, relay info, UID, transaction
ID, lease duration fields). MAC addresses are normalized (separator replacement
and uppercasing). All extracted IPs (client, relay, router, interface, link/peer
address) are converted to IP type and appended to related.ip.

The DNS sub-pipeline (pipeline_dns.yml) parses query logs and responses using
grok with custom pattern definitions (CLIENT, VIEW). A Painless script parses
multi-record DNS answer data (TTL, type, data tuples) into structured
dns.answers arrays. The registered_domain processor extracts the registered
domain from dns.question.name. Server IPs are enriched into related.ip and
query names into related.hosts.

Fields are mapped under efficient_ip.log.* with sub-groups for DHCP
(client_hostname, lease, relay, interface, trans_id, uid, etc.) and DNS
(category, answers, RPZ fields). ECS fields (client., server., dns.,
network.
, observer., process., event.*) are populated directly by the
pipelines.

The integration uses a single UDP input stream with configurable listen address,
port, timezone offset, and optional processors. Pipeline tests cover 6 DHCP and
7 DNS sample log lines with expected output validation.

WHY:

EfficientIP is a widely deployed DDI solution with no existing Elastic
integration. Organizations running EfficientIP lack a supported, turnkey way to
ingest and normalize their DNS/DHCP event logs into the Elastic Stack for
monitoring, threat detection, and compliance use cases (DNS query analysis, DHCP
lease tracking, network anomaly detection). This integration fills that gap as a
community contribution, with IPAM/AUDIT parsing planned for future iterations.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

image

@jasperklaren jasperklaren requested a review from a team as a code owner April 17, 2026 13:55
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

Reviewers

Buildkite won't run for external contributors automatically; you need to add a comment:

  • /test : will kick off a build in Buildkite.

NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details.

Comment thread packages/efficient_ip/build/packages/efficient_ip/0.0.1/manifest.yml Outdated
Comment thread packages/efficient_ip/data_stream/log/manifest.yml
Comment thread packages/efficient_ip/data_stream/log/agent/stream/udp.yml.hbs
Comment thread packages/efficient_ip/data_stream/log/fields/fields.yml
…tream/log/elasticsearch/ingest_pipeline/pipeline_dhcp.yml

Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>
…cluding manifest, sample events, documentation, and images.
@andrewkroh andrewkroh added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. New Integration Issue or pull request for creating a new integration package. labels Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. New Integration Issue or pull request for creating a new integration package.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants