Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No dynamic encoding for adapted params #158

Open
osvill opened this issue Mar 13, 2023 · 0 comments
Open

No dynamic encoding for adapted params #158

osvill opened this issue Mar 13, 2023 · 0 comments

Comments

@osvill
Copy link

osvill commented Mar 13, 2023

All adapted query parameters in copy_to.py line 45 have the default encoding of latin1. This comes from the psycopg2.extensions.adapt function.

This behavior leads to an implicit error if you try something like MyModel.objects.filter(name="björn").to_csv(export_path) and your database's encoding is set to UTF8, because of the ö in björn.
The generated SQL looks like: COPY (SELECT "mymodel"."id", "mymodel"."name", "mymodel"."num", "mymodel"."dt", "mymodel"."parent_id" FROM "mymodel" WHERE "mymodel"."name" = 'bj�rn ') TO STDOUT DELIMITER ',' CSV HEADER and there will be no match with name='bj�rn' for björn.

So my suggestion is to add following code to copy_to.py line 49:

48    with connections[self.using].cursor() as c:
49        # set client encoding to adapted params    # new
50        client_encoding = c.connection.encoding    # new
51        for p in adapted_params:    # new
52            p.encoding = client_encoding if client_encoding else p.encoding    # new

This will generate the right SQL:
COPY (SELECT "mymodel"."id", "mymodel"."name", "mymodel"."num", "mymodel"."dt", "mymodel"."parent_id" FROM "mymodel" WHERE "mymodel"."name" = 'björn') TO STDOUT DELIMITER ',' CSV HEADER

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant