Error generated in Synapse when I query dataverse data in Spark Notebook

Robert O'Neill 6 Reputation points

Hey there,

I am having an issue querying the data from the Synapse Link for Dataverse in a Spark Notebook within Synapse.

I am able to run a SQL query against the data (which appears in Synapse as a Lake Database) and it returns data. See below
👁 203162-image.png

However when I run a query in Spark Notebook I get the following error:

AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
Traceback (most recent call last):

File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/session.py", line 723, in sql
return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped)

File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/py4j/java_gateway.py", line 1304, in call
return_value = get_return_value(

File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 117, in deco
raise converted from None

pyspark.sql.utils.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException

See Screenshot~:
👁 203080-image.png

The Synapse workspace has Owner and Storage Blob Data Contributor access on the storage account.

Anyone have any ideas? I'm really stuck with this one.

  1. HimanshuSinha 19,637 Reputation points • Microsoft Employee • Moderator

    Hello @Robert O'Neill ,
    Thanks for the question and using MS Q&A platform.
    As we understand the ask here is how to get around the error while running the notebook scripts ., please do let us know if its not accurate.

    Try adding yourself to “Storage Blob Data Contributor”.. as the session run on your context you should have access to the underlaying storage..
    Please do let me if you have any queries.
    Thanks
    Himanshu

    • Please don't forget to click on 👁 130616-image.png
      or upvote 👁 130671-image.png
      button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
      • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
  2. Robert O'Neill 6 Reputation points

    Thanks for the reply Himanshu.

    That's correct I am trying to query the data in Spark Notebook.

    I already have Storage Blob Data Contributor access.

  3. Broomfield, Darrien 0 Reputation points

    Did you ever solve this? Having this exact same issue in 2025 still

Sign in to comment
Sign in to answer

Your answer