[BUG] Spark 4.0.0 v2 Implementation Header and Data Address Options

[kaylee-misuraca]

Am I using the newest version of the library?

• I have made sure that I'm using the latest version of the library.

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

1. The v2 implementation doesn't seem to recognize the header option. If it is included or left off, the result is the same and the header is read as the first row of data.
2. The v2 implementation doesn't read past the start cell provided.

Expected Behavior

1. I expect the header to be recognized when the option is passed as header = True
2. I expect the starting cell given to the dataAddress option to return that cell and all rows below and columns to the right

Steps To Reproduce

Read in downloaded file:

issue_example.xlsx

spark.read.format("excel").option("header", "true").load([downloaded file from above])

Attempt to start reading at second row:

spark.read.format("excel").option("header", "true").option("dataAddress", "'Sheet1'!A2").load([downloaded file from above])

Environment

- Spark version: 4.0.0
- Spark-Excel version: 4.0.0_0.31.2
- OS: Azure Databricks Ubuntu 24.04.2 LTS
- Cluster environment: DBR 17.0

Anything else?

The v1 implementation seems to be working as expected.

[azwanzuharimi]

Having the same issue

[dabruehl]

I see the same issue.

df = (
    spark.read.format("excel")
    .option("inferSchema", "True")
    .option("header", "True")
    .option("dataAddress", "'data'!A2")
    .load("test.xlsx")
)

df.show()

V2 implementation throws an error:
[EXCEL_DATA_SOURCE_NOT_ENABLED] Excel data source is not enabled in this cluster. SQLSTATE: 56038

Environment:
Databricks DBR 17.3 LTS
Spark Excel dev.mauch:spark-excel_2.13:4.0.0_0.31.2

[nightscape]

I'm not a 100% sure, but it seems that Databricks registers its own excel datasource which overrides this one:
See #896
Can you set environment variables in the notebook?
If so, I could imagine allowing to override the name of the data source here with an env var EXCEL_DATA_SOURCE_NAME:
https://github.com/nightscape/spark-excel/blob/main/src/main/scala/dev/mauch/spark/excel/v2/ExcelDataSource.scala#L47
Not sure if this would work though.

[dabruehl]

This is the reason why it did not work for DBR 17 for some time:

https://learn.microsoft.com/en-gb/azure/databricks/query/formats/excel

However, I created a Databricks support ticket. They fixed the issue with December 2025 maintenance update. Since the update, following setup works for me:

• Databricks DBR 17.3 LTS
• Spark Excel dev.mauch:spark-excel_2.13:4.0.0_0.31.2

[nightscape]

I still don't understand fully. Is Databricks using this plug-in or did they roll their own?