Skip to content

Expose catalog labels on Table - #4321

Open
yayaha wants to merge 5 commits into
apache:mainfrom
yayaha:iceberg-python-labels-2-table
Open

yayaha wants to merge 5 commits into
apache:mainfrom
yayaha:iceberg-python-labels-2-table

Conversation

@yayaha

@yayaha yayaha commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Rationale for this change

This PR is part 2 of 2. It depends on #4320. The first two commits are from #4320.

This PR adds Table.labels, the same as SupportsLabels in apache/iceberg#18046:

  • The REST catalog sets Table.labels in load_table, create_table, and register_table. rename_table gets labels through load_table.
  • Tables from other catalogs, staged tables from create_table_transaction, and tables built directly have empty labels.
  • Labels are not table state, the same as in Java. Table.__eq__ ignores them. refresh() and commits keep the labels from the first load.
  • View has no labels attribute. Java does not expose view labels either.

Difference from Java: a pickled table keeps its labels. Java does not serialize them.

table = catalog.load_table("db.events")
table.labels.object_labels  # {"owner": "team-a"}
for field in table.labels.fields:
    print(table.schema().find_column_name(field.field_id), field.labels)

Are these changes tested?

Yes. test_table_labels_from_rest_responses tests that load_table, create_table, and register_table set Table.labels. It also tests that a pickled table keeps its labels, __eq__ ignores labels, and refresh() keeps the first labels. test_labels_default_to_empty tests a table built without labels. I also loaded a tagged table from a Unity Catalog REST endpoint. Table.labels matched the labels JSON in the response.

Full unit suite: 4525 passed, 5 failed. The 5 failures are the same as in #4320.

Are there any user-facing changes?

Yes. Table has a new labels attribute and a new optional labels keyword argument. It is the last argument, so existing calls do not change. Please add the changelog label.

AI disclosure: This pull request and its description were written by Claude.

The REST spec adds an optional `labels` field to LoadTableResult and
LoadViewResult for catalog-provided enrichment such as ownership or
classification. PyIceberg dropped the field. Add `Labels` and
`FieldLabel` models and parse `labels` on `TableResponse` and
`ViewResponse`, matching the Java read path (apache/iceberg#18045).
@laskoviymishka
laskoviymishka self-requested a review November 9, 2026 21:58

@laskoviymishka laskoviymishka left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @yayaha.

With this, tables loaded from the REST catalog carry the catalog's labels, and the semantics follow Java's BaseTable: labels stay out of equality and survive refresh(). What's missing is a pickle test, since that's where we intentionally differ from Java.

I wrote the Java and Go sides of labels, so I'd like either @Fokko or @kevinjqliu to sign off on the new python side of Table API before this goes in.

request_headers=TEST_HEADERS,
)
catalog = RestCatalog("rest", uri=TEST_URI, token=TEST_TOKEN)
table = catalog.load_table(("fokko", "table"))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this test also cover create_table and register_table? They hand labels to the same _response_to_table (create, register), but only load is exercised. Mocking the three responses and checking labels in a for loop here would do it, per AGENTS.md's preference for a loop over parametrize.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f1d11c0. The test now mocks load, create, and register, and it checks labels on all three tables. I renamed it to test_table_labels_from_rest_responses.

request_headers=TEST_HEADERS,
)
table.refresh()
assert table.labels == expected

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we pin the pickle behavior here? The description calls it the one difference from Java, where the field is private final transient, and no test under tests/ pickles a Table today.

Suggested change
assert table.labels == expected
assert table.labels == expected
assert pickle.loads(pickle.dumps(table)).labels == expected

This also needs import pickle at the top of the file.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f1d11c0, with your suggestion and import pickle.

A string, list, number, or bool `labels` value failed the whole load.
Java `LabelsParser` reads keys only under `hasNonNull`, so it returns
empty labels for any non-object value. Do the same.
Labels parsed from the load response were not reachable from the
loaded table. Add `Table.labels`. The REST catalog sets it on load,
create, and register. Other catalogs and staged tables
(`create_table_transaction`) get empty labels. Labels are not table
state: they are not part of equality, and refresh and commits keep
them. This matches the Java `SupportsLabels` behavior
(apache/iceberg#18046).
Create and register hand labels to the same `_response_to_table` as
load, but only load was tested. Check all three. Also pin that a
pickled table keeps its labels, the one intended difference from Java,
where `BaseTable.labels` is transient.
@yayaha
yayaha force-pushed the iceberg-python-labels-2-table branch from d211434 to f1d11c0 Compare November 10, 2026 17:14

@Fokko Fokko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! 🙌 The labels model is clean — the null/non-object handling matches Java's LabelsParser behavior, and the test coverage for create/register/pickle paths is exactly right. Thanks @yayaha!

@Fokko
Fokko added this pull request to the merge queue Nov 10, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 10, 2026

@rambleraptor rambleraptor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Late to the party on this one, but this looks great! Excited to get labels support out there.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants