Repository navigation
Conversation
The REST spec adds an optional `labels` field to LoadTableResult and LoadViewResult for catalog-provided enrichment such as ownership or classification. PyIceberg dropped the field. Add `Labels` and `FieldLabel` models and parse `labels` on `TableResponse` and `ViewResponse`, matching the Java read path (apache/iceberg#18045).
laskoviymishka
left a comment
There was a problem hiding this comment.
Thanks @yayaha.
With this, tables loaded from the REST catalog carry the catalog's labels, and the semantics follow Java's BaseTable: labels stay out of equality and survive refresh(). What's missing is a pickle test, since that's where we intentionally differ from Java.
I wrote the Java and Go sides of labels, so I'd like either @Fokko or @kevinjqliu to sign off on the new python side of Table API before this goes in.
| request_headers=TEST_HEADERS, | ||
| ) | ||
| catalog = RestCatalog("rest", uri=TEST_URI, token=TEST_TOKEN) | ||
| table = catalog.load_table(("fokko", "table")) |
There was a problem hiding this comment.
There was a problem hiding this comment.
Done in f1d11c0. The test now mocks load, create, and register, and it checks labels on all three tables. I renamed it to test_table_labels_from_rest_responses.
| request_headers=TEST_HEADERS, | ||
| ) | ||
| table.refresh() | ||
| assert table.labels == expected |
There was a problem hiding this comment.
Could we pin the pickle behavior here? The description calls it the one difference from Java, where the field is private final transient, and no test under tests/ pickles a Table today.
| assert table.labels == expected | |
| assert table.labels == expected | |
| assert pickle.loads(pickle.dumps(table)).labels == expected |
This also needs import pickle at the top of the file.
There was a problem hiding this comment.
Done in f1d11c0, with your suggestion and import pickle.
A string, list, number, or bool `labels` value failed the whole load. Java `LabelsParser` reads keys only under `hasNonNull`, so it returns empty labels for any non-object value. Do the same.
Labels parsed from the load response were not reachable from the loaded table. Add `Table.labels`. The REST catalog sets it on load, create, and register. Other catalogs and staged tables (`create_table_transaction`) get empty labels. Labels are not table state: they are not part of equality, and refresh and commits keep them. This matches the Java `SupportsLabels` behavior (apache/iceberg#18046).
Create and register hand labels to the same `_response_to_table` as load, but only load was tested. Check all three. Also pin that a pickled table keeps its labels, the one intended difference from Java, where `BaseTable.labels` is transient.
d211434 to
f1d11c0
Compare
rambleraptor
left a comment
There was a problem hiding this comment.
Late to the party on this one, but this looks great! Excited to get labels support out there.
Rationale for this change
This PR is part 2 of 2. It depends on #4320. The first two commits are from #4320.
This PR adds
Table.labels, the same asSupportsLabelsin apache/iceberg#18046:Table.labelsinload_table,create_table, andregister_table.rename_tablegets labels throughload_table.create_table_transaction, and tables built directly have empty labels.Table.__eq__ignores them.refresh()and commits keep the labels from the first load.Viewhas nolabelsattribute. Java does not expose view labels either.Difference from Java: a pickled table keeps its labels. Java does not serialize them.
Are these changes tested?
Yes.
test_table_labels_from_rest_responsestests thatload_table,create_table, andregister_tablesetTable.labels. It also tests that a pickled table keeps its labels,__eq__ignores labels, andrefresh()keeps the first labels.test_labels_default_to_emptytests a table built without labels. I also loaded a tagged table from a Unity Catalog REST endpoint.Table.labelsmatched thelabelsJSON in the response.Full unit suite: 4525 passed, 5 failed. The 5 failures are the same as in #4320.
Are there any user-facing changes?
Yes.
Tablehas a newlabelsattribute and a new optionallabelskeyword argument. It is the last argument, so existing calls do not change. Please add thechangeloglabel.AI disclosure: This pull request and its description were written by Claude.