Upcoming Migration to Python 3.11 for openEO UDFs
This announcement is primarily relevant for users running custom UDFs using openEO. The openEO Terrascope backend will be migrating from Python 3.8 to Python 3.11 over the next couple of months.
Timeline
| Phase | What’s happening | Dates (CET) |
|---|---|---|
| 1. Migration (non-UDF & synchronous jobs) | We switch non-UDF batch jobs and synchronous jobs to Python 3.11. Python 3.8 remains available via job options. | Oct 6 |
| 2. Default Update | All new jobs default to Python 3.11. Python 3.8 remains available via options. | Oct 28 |
| 3. Python 3.8 Deprecation | The Python 3.8 image is frozen and will receive no further feature updates or support. | Nov 28 |
1. Migration Phase (Oct 6)
We will switch all batch jobs that do not contain UDFs and synchronous requests to use Python 3.11. No action is required on your part for this step, but please report any issues resulting from the change on the forum. If you still need Python 3.8, you can explicitly select it via your batch job options.
2. Default Update (Oct 28)
Python 3.11 becomes the default runtime for all openEO jobs. If you still need Python 3.8, you can explicitly select it via your batch job options.
3. Python 3.8 Deprecation (Nov 28)
Finally, the Python 3.8 image will be frozen, which means it will receive no new features or updates. Please finish any remaining migrations before then.
We encourage you to start testing today and share feedback. If you run into any issues, post them here so we can address them promptly. Thank you for helping us ensure a smooth transition!
Python3.8 vs Python3.11 UDF dependencies
| Library | Old Version | New Version |
|---|---|---|
| geopandas | 0.13.2 | 1.0.1 |
| numpy | 1.22.4 | 2.3.3 |
| pandas | 1.5.3 | 2.3.0 |
| rasterio | 1.2.10 | 1.3.11 |
| scipy | 1.8.1 | 1.16.0 |
| shapely | 2.0.7 | 2.1.1 |
| xarray | 0.16.2 | 2024.7.0 |
Please note: starting 28/10, all UDF batch jobs will run with python3.11 by default. You can already test this new environment by setting the following job option:
'image-name': 'python311'
If you notice any problems during or after the release, then you can always revert back to the python3.8 version with this job option:
'image-name': 'python38'
The python3.8 image will remain available until the end of this year. More details on configuring job options can be found here:
https://documentation.dataspace.copernicus.eu/APIs/openEO/job_config.html
