Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drop indices/constraints default should be False #117

Open
jesusch opened this issue Mar 26, 2020 · 3 comments
Open

drop indices/constraints default should be False #117

jesusch opened this issue Mar 26, 2020 · 3 comments
Labels

Comments

@jesusch
Copy link

jesusch commented Mar 26, 2020

while using this lib I really shot our database in the head bc those are default to True
we are bulk inserting small pieces (40krows) into a ~400mio table every minute
dropping indices and recreating them all and all over again might not be the best default

@palewire
Copy link
Owner

This is an interesting question. I'm not familiar enough with the use cases of our user base to know what people would prefer here. The use case that led me to create the library is one where the entire database is flushed with each run, so I wasn't thinking about this.

@geoffrey-eisenbarth
Copy link
Contributor

geoffrey-eisenbarth commented Mar 5, 2023

My particular usecase is similar to @jesusch, especially if "upsert" is merged in. Removing and re-applying large indexes/constraints can really be a slowdown, where I'm under the impression that the intention of setting their defaults to True is to speed up things.

I think I'm +1 for "opt in" and setting defaults to False.

@geoffrey-eisenbarth
Copy link
Contributor

@palewire In regards to PR #156 , a command like ON CONFLICT FROM CONSTRAINT "constraint_name" DO UPDATE SET ... will fail if the (model-level) constraint is removed (said PR also implements "model-level" indexes and constraints via the Meta object).

I feel like this is another sign that maybe "opt in" for drop_constraints and drop_indexes might be best.

If you feel strongly against this, then I would need to add some code in between the from_csv method (which receives the drop_constraints and drop_indexes arguments) and the instantiation of CopyMapping (which is where on_conflict is defined).

Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants