feat: Flex - Removed required field false positive for stop_id + added Foreign key violation for location_groups_id #1834
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…dation on location_group_id.
📝 Acceptance Test Report📋 Summary❌ The rule acceptance test has failed for commit 76011f5 📊 Notices ComparisonNew Errors (18 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Errors (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
New Warnings (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Warnings (1 out of 1575 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
🛡️ Corruption Check0 out of 1575 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
📝 Acceptance Test Report📋 Summary❌ The rule acceptance test has failed for commit 247f277 📊 Notices ComparisonNew Errors (18 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Errors (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
New Warnings (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Warnings (1 out of 1575 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
🛡️ Corruption Check0 out of 1575 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
davidgamez
reviewed
Sep 10, 2024
| @@ -0,0 +1,52 @@ | |||
| /* | |||
| * Copyright 2020 Google LLC | |||
There was a problem hiding this comment.
[picky]:
Suggested change
| * Copyright 2020 Google LLC | |
| * Copyright 2024 MobilityData |
davidgamez
reviewed
Sep 10, 2024
Comment on lines
76
to
80
| @ConditionallyRequired | ||
| GtfsTime startPickupDropOffWindow(); | ||
|
|
||
| @ConditionallyRequired | ||
| GtfsTime endPickupDropOffWindow(); |
There was a problem hiding this comment.
Is there any validator implementing the conditional required logic for these fields? If not, please remove the annotation and bring it back when the logic is implemented.
|
About the faliing check
|
📝 Acceptance Test Report📋 Summary❌ The rule acceptance test has failed for commit 9551526 📊 Notices ComparisonNew Errors (18 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Errors (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
New Warnings (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Warnings (1 out of 1575 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
🛡️ Corruption Check0 out of 1575 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
davidgamez
approved these changes
Sep 10, 2024
|
@jcpitre These changes look to be expected and positive! Learnings from this:
|
Do you mean this: GtfsGeojsonFeatureUniqueLocationIdValidator.java |
The stop_times in this datataset has nearly 42 million records. I added 5 new fields in the schema for that file. So at least for memory it's quite a chunk more, and it takes more time to load. The new validator has to go through these 42 million records. Let me correct that: it went from 322 to 893 seconds. It is indeed surprising that it almost tripled |
📝 Acceptance Test Report📋 Summary❌ The rule acceptance test has failed for commit 77fa357 📊 Notices ComparisonNew Errors (18 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Errors (22 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
New Warnings (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Warnings (1 out of 1575 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
🛡️ Corruption Check0 out of 1575 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
jcpitre
added
the
do not merge
|
This contribution does not follow the conventions set by the Google Java style guide. Please run the following command line at the root of the project to fix formatting errors: |
📝 Acceptance Test Report📋 Summary❌ The rule acceptance test has failed for commit f1239d4 📊 Notices ComparisonNew Errors (18 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Errors (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
New Warnings (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Warnings (0 out of 1575 datasets, ~0%) ✅No changes were detected due to the code change. 🛡️ Corruption Check0 out of 1575 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
📝 Acceptance Test Report📋 Summary✅ The rule acceptance has passed for commit d339411 📊 Notices ComparisonNew Errors (0 out of 1 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Errors (0 out of 1 datasets, ~0%) ✅No changes were detected due to the code change. New Warnings (0 out of 1 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Warnings (0 out of 1 datasets, ~0%) ✅No changes were detected due to the code change. 🛡️ Corruption Check0 out of 1 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
📝 Acceptance Test Report📋 Summary❌ The rule acceptance test has failed for commit 6618a15 📊 Notices ComparisonNew Errors (18 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Errors (20 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
New Warnings (21 out of 1575 datasets, ~1%) ❌Details of new errors due to code change, which is above the provided threshold of 1%.
Dropped Warnings (0 out of 1575 datasets, ~0%) ✅No changes were detected due to the code change. 🛡️ Corruption Check0 out of 1575 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
|
With the new 12G heap the acceptance tests seem to run properly. |
jcpitre
removed
the
do not merge
emmambd
approved these changes
Sep 13, 2024
jcpitre
deleted the
1749-flex-add-location-group-schema-and-update-stop-time-schema
branch
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1749
Summary:
Also verified that feature
Zone-based Demand Responsive Transitis present in the list of features for a dataset with Flex. We could not test this in #1776 until the @required was removed from stopId.Expected behavior:
Most of missing required fields related to stopId should disappear from reports.
Some other errors or warning that were masked by the missing required field notice should appear. Some of these will have to be corrected other issues. e.g. #1755
Please make sure these boxes are checked before submitting your pull request - thanks!
gradle testto make sure you didn't break anything