Fix file uploads on localckan

[Zharktas]

Fixes #211

cgi.FieldStorage has been dropped from ckan since python 3 migration and its going away completely in python 3.13, this just replaces it with werkzeug FileStorage which ckan uses in the core as well.

[Zharktas]

@wardi are we really wanting to run tests on old ckan versions or should we just update them as well ?

[wardi]

We can drop tests on old ckans for sure

[JVickery-TBS]

If using werkzeug now, do we need to add that into the requirements in this repo?

@wardi I was unable to find whatever fix we may have put in our fork of CKAN Core to fix this issue. Not sure if this was ever an issue with CKAN 2.9 or not? I will keep looking a bit more

[wardi]

@JVickery-TBS Not sure about requirements.

If you're using LocalCKAN then you need to have ckan in the same venv so we could assume werkzeug is there, and is the correct version for the ckan you're running.

If you're only using ckanapi for RemoteCKAN we don't need it.

If we did add a werkzeug requirement it would have to be an optional dependency: pip install ckanapi[localckan] or similar.

[JVickery-TBS]

@wardi ahh okay makes sense. How does the RemoteCKAN send files? Just as a normal http POST?

AHA! I found it, and my original thought was right about the bool method of the cgi.FieldStorage being super broken and always returning False even if the object has data.

https://github.com/ckan/ckan/blob/2.9/ckan/lib/uploader.py#L263

bool(upload_field_storage)

if upload_field_storage is a cgi.FieldStorage object, this will always fail. https://bugs.python.org/issue19097 (python/cpython#63296)

Looks like the CloudStorage's plugin code does this:

if isinstance(upload_field_storage, (ALLOWED_UPLOAD_TYPES)):
            if len(upload_field_storage.filename):
            ......

So it is not doing bool on the cgi.FieldStorage. Not sure if we should do something like that upstream on the default CKAN uploaders?

[Zharktas]

Remote CKAN uses HTTP api and that works even now, you just have to deal with api tokens which is not ideal if you are adding files to a ckan running locally. I'll update the workflows to run on supported ckan versions.

[Zharktas]

Done. Cloudstorage and similar do checks on cgi.FieldStorage and werkzeug FileStorage as they try to support old CKAN versions, thats not really required any more as those versions aren't supported by the tech team either.