Many of you out there may be wondering ‘when are we going to release QGIS 3.0?’. Last year (2015) we started investigating when and how we would release QGIS 3.0. We promised (see Anita Graser’s post about this) that we would convey clearly to our users and developers our plans well before making the QGIS 3.0 release. In this post I will try to lay out some of the considerations for a QGIS 3.0 release and at the end of this post there is an opportunity for you to present your ideas.
Why 3.0?
Typically (when following semantic versioning) a major release is reserved for times when you break the API of your software. Breaking API is not a trivial decision for the QGIS project since we have hundreds of thousands of users out there who depend on QGIS to ‘just work’, and many developers who need to maintain third party software written on top of the QGIS API.
From time to time breaking the API is necessary to accommodate updating the architecture with improved approaches, new base libraries and fixes to sub-optimal decisions made in the past.
What are the implications of breaking the API?
One of the reasons we are hesitant about releasing an API breaking QGIS 3.0 release is that it will have a huge impact, potentially breaking the hundreds of plugins in the plugin repository. Plugins would no longer compatible with new API and plugin authors would be required to manually review their plugins to identify and update the places in their plugins where the code is no longer compatible with the new API.
The breadth of the updates required depends largely on
- how many backwards incompatible changes we make to the API
- how many places plugin authors have used parts of the API which have changed
I will talk more about how we can mitigate API breaking changes further on in this article.
What will be the key changes for 3.0?
There are four key areas that we are looking to change in 3.0:
- Updating Qt4 to Qt5: This is the basic set of libraries on which QGIS is built and provides a high level, platform independent abstraction layer for building a graphical user application. Qt also provides libraries for carrying out disk i/o, networking operations, and graphics drawing operations (key functionality for QGIS). Qt4 (on which QGIS is currently based) is now not actively being developed by the Qt library maintainers and we are expecting to experience problems in the near future building Qt4 for some platforms (e.g. OS X) or having readily accessible binaries (e.g. Debian Testing and the upcoming release of Debian “Stretch”). Making QGIS work with Qt5 has already been worked on (chiefly by Matthias Kuhn) who together with Marco Bernasocchi produce the Android “QField” port of QGIS which is based completely on Qt5. There are however some outstanding limitations in the newer Qt5 that impact on QGIS – in particular with the embedded web browser widgets (used chiefly in the QGIS composer but also a few other places in QGIS).
- Updating PyQt4 to PyQt5: These are the python language bindings for Qt which the QGIS python API relies on. When we shift to the Qt5 C++ library, we also want to shift to the updated PyQt5 python library so that we can benefit from the new Qt5 API within the python environment too.
- Updating Python 2.7 to Python 3: Currently we bundle in Python 2.7 in our windows installers and require 2.7 on other platforms where we do not co-bundle Python with QGIS. Python 3 is the latest version of python and is recommended by the Python project. Python 2 is slightly incompatible with Python 3 (in much the same way as QGIS 2 -> QGIS 3 will be incompatible). The python developers have made Python 3 largely backwards compatible to Python 2, but the compatibility in the opposite direction is not as good.
- Improving the QGIS API itself: One of the issues with maintaining API compatibility between releases is that you have to live with your design choices for a long time. In QGIS we try our best not to break the API within a minor release series – not always with success as the more hard core developers will attest to. Releasing an API incompatible version of QGIS for 3.0 will give us an opportunity to ‘clean house’ by fixing things in the API that we are unhappy with. You can see a provisional list of proposed API changes for 3.0 by looking at the 3.0 API issues list.
Mitigating 3.0 API breakages
As I mentioned the 3.0 release will break API from the 2.x release of QGIS and there is the potential that many existing plugins, applications and other code that rely on the current API will be broken. So what can we do to mitigate the changes? Matthias Kuhn, Jürgen Fischer, Nyall Dawson, Martin Dobias and other core developers have been looking at ways to mitigate the number of API breaking changes whilst still advancing the QGIS codebase to be based on the next generation of libraries and its own internal API. During our last QGIS Project Steering Committee meeting we ran through various possibilities. Matthias Kuhn kindly joined the meeting to help clarify our options going forward which I have tried to summarise in the table below:
| QGIS 2.14 LTR | QGIS 2.16 ??? | QGIS 3.0 | |
| Release date | End Feb | 4 months after 2.14 | 8 month cycle? |
| Notes | Update python code of core QGIS to be Python 3 compatible and PyQt5 compatible (partial implementation for key functionality e.g. console, python core plugins etc.) | ||
| Qt4 | Yes
Deprecated in Debian Stretch (due in a year) (webkit removed) |
Yes | No |
| Qt5 | No
Misses QWebView – new replacement not on all platforms. Also misses QPainter Engine. |
Yes | Yes |
| PyQt4 | Yes | Yes | No |
| PyQt5 | No | Yes | Yes |
| Python 2 | Yes | Yes | No |
| Python 3 | No | Yes | Yes |
| API Cleanup | No | No | Yes |
| Wrappers PyQt5 -> PyQt4 Provide ~90% backwards compatibility |
No | Yes | Yes |
| Mainstream Binary | Qt4 Based | Qt4 Based | Qt5 Based |
| Funding priority | Python wrappers |
There are two key things to note about Matthias’ proposal:
- In the first phase, work would be done in the QGIS 2.x series to complete support for Qt5, PyQt5, Python 3.0 whilst still supporting Qt4, PyQt4 and Python 2.7. This implies that all changes made in the first phase would be backwards compatible with previous QGIS 2.x releases. Python wrappers will be introduced so that the old PyQt4 API can still mostly be used when compiling against Qt5, PyQt5, Python 3.0. When using QGIS compiled against Qt4, PyQt4 and Python 2.7 there would be no compatibility breakage.
- In the second phase, we would work to produce QGIS 3.0 which introduces a number of API breaking changes, completely removing Python 2.7, Qt4 and PyQt4 support. The python wrappers produced in the first phase would be kept and relied on to ensure that a large proportion of python code (plugins, scripts etc.) developed for QGIS 2.x releases continue to work in QGIS 3.x releases. In this phase we would also introduce the QGIS API changes which may break some plugins. To address this we will provide a migration guide to try to ease the process for those moving from code depending on the QGIS 2.x releases to the QGIS 3.x releases.
Caveat emptor
There are a couple of ‘gotchas’ that we should raise at this point as the above makes the migration to QGIS 3.0 sound fairly painless.
- The first thing we should emphasise is that while the approach laid out above tries to minimise the amount of work python script and plugins writers have to do, this will not be a 100% effort free solution for python coders using QGIS. There will very likely be cases where code needs to be adjusted and in all cases at the very least it will probably need to be reviewed in order to ensure that it still functions properly.
- There is no formal funding set aside to pay for developers to spend their time working on the migration process. Because of this it is going to be incredibly hard to give accurate timelines as to how long each part of the process will take. We need to take this uncertainty into account in our planning. Of course we welcome donations to help make this happen.
- There may be developers and institutions out there funding new features for the QGIS 2.x series and this may affect your work. You should include in your project plans and budgets some allocation to cope with the migration to the QGIS 3.x platform.
- If we do the work in the ‘master branch’ there may be a protracted time during which our master branch is unstable and in flux due to ongoing updates towards QGIS 3.0.
- If we do the work in a ‘3.0 branch’, we run the risk that the 3.0 development may drag on longer unless there is a devoted group of developers working on it and getting it ready to merge to master.
Proposals
In the light of all the above information, we propose one of two courses of action:
Proposal 1:
Do an interim release of 2.16 and then commence work on 3.0 in master with an 8 month development window. Work on 3.0 related stuff could already begin in 2.16 (see python3/pytq5), only incompatible changes have to be postponed to post 2.16.
Advantage: Main focus of work would be in master branch. Work scheduled for the near future can be released in expected timelines. Plugins will continue to work with master. People can start to write and test their code in a portable manner.
Disadvantage: Difficult to determine timelines as we don’t have funding
Proposal 2:
Create a long running 3.0 branch for the port to Qt5, Python 3.0 and PyQt5 and call for developers to get their 3.0 work in there. Continue with 2.x releases with the usual frequency until 3.0 is ready.
Advantage: We can release it ‘when it’s ready’. If there is no funding for 3.0 work subsequent releases are not jeopardised.
Disadvantage: Duplication of effort as work in master coming in needs to be ported over to the 3.0 branch.
Alternative proposals
Do you have an alternative proposal? We would like to get all the proposals on the table so that we (the PSC in consultation with core developers) can make the best judgement of how to approach the nitty-gritty process of managing the QGIS 3.0 development process. If you wish to submit a proposal, please send it to me ([email protected]) with the subject line ‘QGIS 3.0 Proposal‘. Please keep your proposal very short and succinct as we just need the high level concepts.
Here is the proposal submitted by Matthias Kuhn that you can use as a reference of how we might like a proposal to look:
QGIS 2.16 Release as usual in 4 months
-> PyQt5 Support
-> Python 3 Support
-> Wrapper library for PyQt4/PyQt5
-> Maybe a helper transition script that does 80% of the rewrite
-> All old plugins still work
-> Some python code is updated (console, plugin manager, processing) to
have some guidelines and experience how to update python code
-> For future debian, mac osx… versions there’s a qt5 version around
(with almost no plugins working)
During the same time: make some noise that QGIS 3 is coming and we need
everybody to put some money and dev time aside for it and that it’s
going to be amazing.
—————-
After that: 8 months break for 3.0 (maybe some betas after 4 months and
every month after)
Back to normal, everybody happy – except the lazy plugin devs who didn’t
update – 🙂
QGIS.org blog 