Datasets:
Bump framework version to 2.3.0
Browse filesPin ud-hf-parquet-tools to v1.3.0 and document the Parquet schema changes, official Hugging Face namespace cleanup, and dataset card updates.
Refs: https://huggingface.co/datasets/universal-dependencies/universal_dependencies/discussions/2
Refs: https://huggingface.co/datasets/universal-dependencies/universal_dependencies/discussions/3
- ADDING_NEW_UD_VERSION.md +1 -1
- CHANGELOG.md +3 -0
- MIGRATION.md +5 -6
- RELEASE.md +1 -0
- pyproject.toml +2 -2
- tools/README.md +1 -1
ADDING_NEW_UD_VERSION.md
CHANGED
|
@@ -432,7 +432,7 @@ need to rebuild all existing UD branches (`2.7` ... `2.17`).
|
|
| 432 |
|
| 433 |
```bash
|
| 434 |
# Repository root
|
| 435 |
-
export FRAMEWORK_VER=v2.
|
| 436 |
|
| 437 |
for UD_VER in 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18; do
|
| 438 |
git switch "${UD_VER}"
|
|
|
|
| 432 |
|
| 433 |
```bash
|
| 434 |
# Repository root
|
| 435 |
+
export FRAMEWORK_VER=v2.3.0
|
| 436 |
|
| 437 |
for UD_VER in 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18; do
|
| 438 |
git switch "${UD_VER}"
|
CHANGELOG.md
CHANGED
|
@@ -7,12 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
| 7 |
|
| 8 |
## [Unreleased]
|
| 9 |
|
|
|
|
|
|
|
| 10 |
### Changed
|
| 11 |
|
| 12 |
- **Parquet Schema**
|
| 13 |
- Regular-token `upos` values are now stored as strings instead of HuggingFace `ClassLabel` indices.
|
| 14 |
- Regular-token `head` values are now stored as 16-bit integers instead of strings.
|
| 15 |
- Empty-node `head` values remain strings because CoNLL-U empty nodes may use `_`.
|
|
|
|
| 16 |
|
| 17 |
- **Dataset Card**
|
| 18 |
- Updated examples and links for the official Hugging Face namespace: `universal-dependencies/universal_dependencies`.
|
|
|
|
| 7 |
|
| 8 |
## [Unreleased]
|
| 9 |
|
| 10 |
+
## [2.3.0] - 2026-08-18
|
| 11 |
+
|
| 12 |
### Changed
|
| 13 |
|
| 14 |
- **Parquet Schema**
|
| 15 |
- Regular-token `upos` values are now stored as strings instead of HuggingFace `ClassLabel` indices.
|
| 16 |
- Regular-token `head` values are now stored as 16-bit integers instead of strings.
|
| 17 |
- Empty-node `head` values remain strings because CoNLL-U empty nodes may use `_`.
|
| 18 |
+
- Pinned `ud-hf-parquet-tools` to `v1.3.0`.
|
| 19 |
|
| 20 |
- **Dataset Card**
|
| 21 |
- Updated examples and links for the official Hugging Face namespace: `universal-dependencies/universal_dependencies`.
|
MIGRATION.md
CHANGED
|
@@ -4,10 +4,10 @@ This repository uses Parquet-only HuggingFace dataset branches for Universal
|
|
| 4 |
Dependencies releases. Current examples use the official dataset location:
|
| 5 |
`universal-dependencies/universal_dependencies`.
|
| 6 |
|
| 7 |
-
##
|
| 8 |
|
| 9 |
-
|
| 10 |
-
|
| 11 |
|
| 12 |
- `upos` is now stored as a list of strings instead of HuggingFace `ClassLabel`
|
| 13 |
indices.
|
|
@@ -56,9 +56,8 @@ write_conllu(ds, "output.conllu")
|
|
| 56 |
|
| 57 |
### Compatibility Notes
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
updated schema.
|
| 62 |
|
| 63 |
## Archived v1.x to v2.0 Notes
|
| 64 |
|
|
|
|
| 4 |
Dependencies releases. Current examples use the official dataset location:
|
| 5 |
`universal-dependencies/universal_dependencies`.
|
| 6 |
|
| 7 |
+
## v2.2.x to v2.3.0
|
| 8 |
|
| 9 |
+
Loader/framework v2.3.0 updates the generated Parquet schema for regular CoNLL-U
|
| 10 |
+
token fields:
|
| 11 |
|
| 12 |
- `upos` is now stored as a list of strings instead of HuggingFace `ClassLabel`
|
| 13 |
indices.
|
|
|
|
| 56 |
|
| 57 |
### Compatibility Notes
|
| 58 |
|
| 59 |
+
`ud-hf-parquet-tools>=1.3.0` can still reconstruct older Parquet outputs where
|
| 60 |
+
`upos` was stored as a `ClassLabel`. New outputs should use the v2.3.0 schema.
|
|
|
|
| 61 |
|
| 62 |
## Archived v1.x to v2.0 Notes
|
| 63 |
|
RELEASE.md
CHANGED
|
@@ -467,6 +467,7 @@ git push origin v2.1.1
|
|
| 467 |
|
| 468 |
| Version | Date | Type | Highlights |
|
| 469 |
|---------|------|------|------------|
|
|
|
|
| 470 |
| v2.2.1 | 2026-05 | Patch | UD 2.18 release support, annotated-tag release docs, dataset card paper URL update |
|
| 471 |
| v2.2.0 | 2026-03 | Minor | Parquet stale-artifact guard controls, `--check-extra`, `--prune-extra` |
|
| 472 |
| v2.1.0 | 2026-02 | Minor | CLARIN snapshot pipeline, flag-driven `--ud-ver` workflow, validation blocked-treebank ignored count |
|
|
|
|
| 467 |
|
| 468 |
| Version | Date | Type | Highlights |
|
| 469 |
|---------|------|------|------------|
|
| 470 |
+
| v2.3.0 | 2026-08 | Minor | Parquet `upos` string and `head` integer schema, official HF namespace cleanup, dataset card citation/history updates |
|
| 471 |
| v2.2.1 | 2026-05 | Patch | UD 2.18 release support, annotated-tag release docs, dataset card paper URL update |
|
| 472 |
| v2.2.0 | 2026-03 | Minor | Parquet stale-artifact guard controls, `--check-extra`, `--prune-extra` |
|
| 473 |
| v2.1.0 | 2026-02 | Minor | CLARIN snapshot pipeline, flag-driven `--ud-ver` workflow, validation blocked-treebank ignored count |
|
pyproject.toml
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
[project]
|
| 2 |
name = "universal-dependencies"
|
| 3 |
-
version = "2.
|
| 4 |
description = "UD Dependencies Data Set"
|
| 5 |
readme = "README.md"
|
| 6 |
requires-python = ">=3.12"
|
|
@@ -67,4 +67,4 @@ url = "https://test.pypi.org/simple/"
|
|
| 67 |
explicit = true
|
| 68 |
|
| 69 |
[tool.uv.sources]
|
| 70 |
-
ud-hf-parquet-tools = { git = "https://github.com/bot-zen/ud-hf-parquet-tools", tag = "v1.
|
|
|
|
| 1 |
[project]
|
| 2 |
name = "universal-dependencies"
|
| 3 |
+
version = "2.3.0"
|
| 4 |
description = "UD Dependencies Data Set"
|
| 5 |
readme = "README.md"
|
| 6 |
requires-python = ">=3.12"
|
|
|
|
| 67 |
explicit = true
|
| 68 |
|
| 69 |
[tool.uv.sources]
|
| 70 |
+
ud-hf-parquet-tools = { git = "https://github.com/bot-zen/ud-hf-parquet-tools", tag = "v1.3.0" }
|
tools/README.md
CHANGED
|
@@ -43,7 +43,7 @@ you need to regenerate artifacts in all UD version branches.
|
|
| 43 |
|
| 44 |
```bash
|
| 45 |
# Run from repository root
|
| 46 |
-
export FRAMEWORK_VER=v2.
|
| 47 |
|
| 48 |
for UD_VER in 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18; do
|
| 49 |
git switch "${UD_VER}"
|
|
|
|
| 43 |
|
| 44 |
```bash
|
| 45 |
# Run from repository root
|
| 46 |
+
export FRAMEWORK_VER=v2.3.0
|
| 47 |
|
| 48 |
for UD_VER in 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18; do
|
| 49 |
git switch "${UD_VER}"
|