Skip to content

[java-bigquery] BigQuery: METASTORE format not supported.  #12159

Closed as not planned

Description

@vejeta

Recently, while using the library, a new error have appeared

j.l.IllegalArgumentException: Format METASTORE is not supported
	at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)

That error happens when trying to access a bigquery table that doesn't belong to one of the expected formats

TableDefinition.java#L174)
      default:        throw new IllegalArgumentException("Format " + tablePb.getType() + " is not supported");

Is this something that it is expected to be supported soon?

Environment details

  1. Specify the API at the beginning of the title: BigQuery
  2. OS type and version: Irrelevant
  3. Java version: All java versions
  4. version(s): all versions

Steps to reproduce

  1. Run the following code to get the list of bigquery tables and their metadata.

Code example

    private val client: BigQuery = BigQueryOptions.newBuilder.setProjectId(projectId).build().getService
    
    client.listDatasets(projectId).iterateAll().asScala.map(_.getDatasetId)

Stack trace

j.l.IllegalArgumentException: Format METASTORE is not supported
	at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
	at c.g.c.b.TableInfo$BuilderImpl.<init>(TableInfo.java:253)
	at c.g.c.bigquery.Table.fromPb(Table.java:690)
	at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:966)
	at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:963)
	at c.g.c.c.Iterators$6.transform(Iterators.java:829)
	at c.g.c.c.TransformedIterator.next(TransformedIterator.java:52)
	at c.g.c.PageImpl$PageIterator.computeNext(PageImpl.java:72)
	at c.g.c.c.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
	at c.g.c.c.AbstractIterator.hasNext(AbstractIterator.java:140)
	at s.c.c.JavaCollectionWrappers$JIteratorWrapper.hasNext(JavaCollectionWrappers.scala:46)
	at s.c.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:99)
	at s.c.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
	at s.c.c.JavaCollectionWrappers$JIterableWrapper.map(JavaCollectionWrappers.scala:83)

External references such as API reference guides

This seems to be related to the new Iceberg formats
https://medium.com/@cesar.cordoba/iceberg-format-a-short-introduction-6cca39e48ac
https://cloud.google.com/bigquery/docs/iceberg-tables

Any additional information below

Checking the information from the command line over that dataset

 bq ls --format=json --project_id="project_id" "dataset"

We can verify several tables have the "METASTORE" format.

{"kind":"bigquery#table","id":"project_id:dataset.mytable","tableReference":{"projectId":"project_id","datasetId":"dataset","tableId":"mytable"},"type":"METASTORE","creationTime":"1711215011000"},

Checking that table:

bq show --format=json "project_id:dataset.mytable"
"type":"METASTORE","externalDataConfiguration":{"sourceUris":["s3://edited.metadata.json"],"sourceFormat":"ICEBERG"

Activity

  1. added
    priority: p3Desirable enhancement or fix. May not be included in next release.
    on Oct 22, 2025
  2. jinseopkim0 commented on Oct 22, 2025

    @jinseopkim0
    Contributor

    If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

  3. changed the title [-]BigQuery: METASTORE format not supported. [/-] [+][java-bigquery] BigQuery: METASTORE format not supported. [/+] on Mar 23, 2026
  4. jjmew commented on Nov 18, 2026

    @jjmew

    Closing this issue due to inactivity (no activity in over two years). If it's still relevant, please feel free to reopen it or comment with any additional information, such as the library version and steps to reproduce. Thank you!

  5. vejeta commented on Nov 19, 2026

    @vejeta
    Author

    Still happening on 2.77.0 (and 2.43.0). My original snippet was wrong, sorry: listDatasets never touches tables. It's listTables, getTable and Dataset.list:

    java.lang.IllegalArgumentException: Format METASTORE is not supported
        at com.google.cloud.bigquery.TableDefinition.fromPb(TableDefinition.java:174)
    

    BigLake Iceberg tables come back as "type": "METASTORE" and fromPb throws on any type it doesn't know. Since that happens inside the page iterator, one Iceberg table breaks listTables().iterateAll() for the whole dataset. The Python client handles it fine.

    I sent a fix with tests in #15076: adds Type.METASTORE (parsed as ExternalTableDefinition) and falls back instead of throwing on unknown types. Hope that approach is ok, but feel free to close it if you'd rather fix it differently or not at all. Could you reopen this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: bigqueryIssues related to the BigQuery API.priority: p3Desirable enhancement or fix. May not be included in next release.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions