Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JP-3721: Simplify ModelContainer #8831

Draft
wants to merge 89 commits into
base: main
Choose a base branch
from
Draft

JP-3721: Simplify ModelContainer #8831

wants to merge 89 commits into from
+155 −307

Commits on Jul 29, 2024

  1. add ModelLibrary

    @braingram @emolter
    braingram authored and emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    5a82f06 View commit details
    Browse the repository at this point in the history

  2. update tweakreg to use ModelLibrary

    @braingram @emolter
    braingram authored and emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    98fb07f View commit details
    Browse the repository at this point in the history

  3. remove minimize_memory option for skymatch

    @braingram @emolter
    braingram authored and emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    395298d View commit details
    Browse the repository at this point in the history

  4. update skymatch to use ModelLibrary

    @braingram @emolter
    braingram authored and emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    4b2ce13 View commit details
    Browse the repository at this point in the history

  5. temporary ModelContainer to and from ModelLibrary converter, assign_m… 

    …twcs to library
    @emolter
    emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    018e3ce View commit details
    Browse the repository at this point in the history

  6. replaced container with library in resample

    @emolter
    emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    74b6607 View commit details
    Browse the repository at this point in the history

  7. Revert "temporary ModelContainer to and from ModelLibrary converter, … 

    …assign_mtwcs to library"

This reverts commit 018e3ce.
    @emolter
    emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    9233dd2 View commit details
    Browse the repository at this point in the history

  8. put in nogroupid try except statements

    @emolter
    emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    bd6207b View commit details
    Browse the repository at this point in the history

  9. ModelLibrary for resample imaging, ModelContainer for resample spec

    @emolter
    emolter committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    5c3cfc3 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Merge remote-tracking branch 'braingram/outlier_detection_steps' into… 

    … JP-3690
    @emolter
    emolter committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    c1615dd View commit details
    Browse the repository at this point in the history

  2. fixing problems from merge of outlier detection changes

    @emolter
    emolter committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    b4a7ce1 View commit details
    Browse the repository at this point in the history

  3. outlier detection to ModelLibrary for imaging modes

    @emolter
    emolter committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    36c2b40 View commit details
    Browse the repository at this point in the history

  4. small fix to resample_spec

    @emolter
    emolter committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    c78841a View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'upstream/master' into JP-3690

    @emolter
    emolter committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    9cdf2b7 View commit details
    Browse the repository at this point in the history

  6. convert to ModelLibrary and back again in resample_spec_step to avoid… 

    … duplicated utilities
    @emolter
    emolter committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    3103f82 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. bugfixes for outlier detection unit tests

    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    2f85218 View commit details
    Browse the repository at this point in the history

  2. expose on_disk for all steps, some cleanup to passing libraries betwe… 

    …en steps
    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    570dc26 View commit details
    Browse the repository at this point in the history

  3. using map_function where applicable, more unit test bug fixes

    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    ad9902d View commit details
    Browse the repository at this point in the history

  4. fix typo in pyproject.toml

    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    3d4fa0d View commit details
    Browse the repository at this point in the history

  5. bump version of stpipe

    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    3db69d3 View commit details
    Browse the repository at this point in the history

  6. mark ModelLibrary as not part of stdatamodels

    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    5ca774e View commit details
    Browse the repository at this point in the history

  7. added changelog entry

    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    d694178 View commit details
    Browse the repository at this point in the history

  8. first draft of docs changes

    @emolter
    emolter committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    6066f2b View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. fixing regtest failures for spec3 pipeline, adding library to mtwcs

    @emolter
    emolter committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    48b1050 View commit details
    Browse the repository at this point in the history

  2. integrate assign_mtwcs changes with spec3 pipeline

    @emolter
    emolter committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    99ef2f6 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. debug coron3 pipeline

    @emolter
    emolter committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    55262c0 View commit details
    Browse the repository at this point in the history

  2. emptying data arrays in input to model_blender inside resample

    @emolter
    emolter committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    f3ae92e View commit details
    Browse the repository at this point in the history

  3. decreasing memory usage of outlier step using profiler

    @emolter
    emolter committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    e27acc2 View commit details
    Browse the repository at this point in the history

  4. revert refactor that introduced a bug in resample

    @emolter
    emolter committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    d6c1642 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. update skymatch input spec

    @emolter
    emolter committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    189b375 View commit details
    Browse the repository at this point in the history

  2. bugfix for failed asdf load of recursive wcs transform

    @emolter
    emolter committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    a2dd947 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. Merge branch 'master' into JP-3690

    @emolter
    emolter authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    4307f22 View commit details
    Browse the repository at this point in the history

  2. small changes from memory profiling and review

    @emolter
    emolter committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    1ca5c9b View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. bugfix for failed to load area extension and other metadata

    @emolter
    emolter committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    6f08b61 View commit details
    Browse the repository at this point in the history

  2. handle asn_table and asn_pool metadata properly

    @emolter
    emolter committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    b124a8b View commit details
    Browse the repository at this point in the history

  3. make ind_asn_type case-insensitive

    @emolter
    emolter committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    62bacbd View commit details
    Browse the repository at this point in the history

  4. fix output filenames from outlier_detection

    @emolter
    emolter committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    30dcb3f View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. bugfix for HDRTAB association info in i2d files

    @emolter
    emolter committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    5e9e672 View commit details
    Browse the repository at this point in the history

  2. re-add python 3.13 pin that was accidentally clobbered

    @emolter
    emolter committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    f87809e View commit details
    Browse the repository at this point in the history

  3. reverting accidental clobber of stcal pin dependency

    @emolter
    emolter committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    ca62baf View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. updates after reviews by myself and by @braingram

    @emolter
    emolter committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    d47ef4b View commit details
    Browse the repository at this point in the history

  2. ruff style check

    @emolter
    emolter committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    26e5436 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. call img.area

    @emolter
    emolter committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    8f902b7 View commit details
    Browse the repository at this point in the history

  2. bugfixes for remove s2d files and for mtimage regtest

    @emolter
    emolter committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    33d66c9 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Merge branch 'master' into JP-3690

    @emolter
    emolter committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    b357edc View commit details
    Browse the repository at this point in the history

  2. attempted fix for resample and source_catalog result filenames

    @emolter
    emolter committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    f35725a View commit details
    Browse the repository at this point in the history

  3. remove setting of asn pool and table name in resample

    @emolter
    emolter committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    4ed0892 View commit details
    Browse the repository at this point in the history

  4. pushing bad things to remote to diagnose regtest

    @emolter
    emolter committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    79f91c6 View commit details
    Browse the repository at this point in the history

  5. yet another attempt to fix filename issue

    @emolter
    emolter committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    3fd25fc View commit details
    Browse the repository at this point in the history

  6. attempt to propagate fix also into source_catalog

    @emolter
    emolter committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    28fa217 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. bugfix for updating table and pool name in library._assign_member_to_… 

    …model
    @emolter
    emolter committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    1863ef1 View commit details
    Browse the repository at this point in the history

  2. fixes based on @braingram review

    @emolter
    emolter committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    cb5594e View commit details
    Browse the repository at this point in the history

  3. fix ruff style check and remove unnecessary comment

    @emolter
    emolter committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    099580c View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. fix unit test and revert changes to file naming

    @emolter
    emolter committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    6ff67d3 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. new attempted fix of output filenames

    @emolter
    emolter committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    6cad768 View commit details
    Browse the repository at this point in the history

  2. removed one more manual change to output file naming

    @emolter
    emolter committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    91b1b19 View commit details
    Browse the repository at this point in the history

  3. changed has_groups conditional to reflect master branch

    @emolter
    emolter committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    bf069ed View commit details
    Browse the repository at this point in the history

  4. attempted fix single regtest failure for miri image3 crf files

    @emolter
    emolter committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    5a44905 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. Merge branch master into JP-3690

    @emolter
    emolter committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    e828ebc View commit details
    Browse the repository at this point in the history

  2. fixes per @melanieclarke comments

    @emolter
    emolter committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    5272ac9 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. first draft of minimal container

    @emolter
    emolter committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    a4d4dda View commit details
    Browse the repository at this point in the history

  2. added basic context management

    @emolter
    emolter committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    b176520 View commit details
    Browse the repository at this point in the history

  3. merge JP-3690

    @emolter
    emolter committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    cb59904 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. merge master into JP-3690

    @emolter
    emolter committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    2afe670 View commit details
    Browse the repository at this point in the history

  2. fixed failure to raise NoGroupID for in-memory models with None for o… 

    …bservation parameters
    @emolter
    emolter committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    04e2fa5 View commit details
    Browse the repository at this point in the history

  3. start to modify pipelines to account for container changes

    @emolter
    emolter committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    fff437c View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. Merge branch 'JP-3690' into JP-3721

    @emolter
    emolter committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    f8c0596 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into JP-3690

    @emolter
    emolter committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    36ab34d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'JP-3690' into JP-3721

    @emolter
    emolter committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    22fb9c9 View commit details
    Browse the repository at this point in the history

  4. fix file naming with sourcemodelcontainer

    @emolter
    emolter committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    53635c1 View commit details
    Browse the repository at this point in the history

  5. removed schema url from container

    @emolter
    emolter committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    225c21e View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Merge branch 'master' into JP-3721

    @emolter
    emolter committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    1b1e50b View commit details
    Browse the repository at this point in the history

  2. removed some instances of container.meta

    @emolter
    emolter committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    71a2a3b View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Merge branch 'master' into JP-3721

    @emolter
    emolter committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    a115695 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. Merge branch 'master' into JP-3721

    @emolter
    emolter committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    fca4449 View commit details
    Browse the repository at this point in the history

  2. small fixes to metadata setting and tests

    @emolter
    emolter committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    d876ad5 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Merge remote-tracking branch 'upstream/master' into JP-3721

    @emolter
    emolter committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    7121335 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/master' into JP-3721

    @emolter
    emolter committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    b439bb7 View commit details
    Browse the repository at this point in the history

  3. in the middle of fixing filename issues

    @emolter
    emolter committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    f6ed4b8 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Merge branch 'master' of https://github.com/spacetelescope/jwst into … 

    …JP-3721
    @emolter
    emolter committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    a5d529e View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. Merge branch 'main' of https://github.com/spacetelescope/jwst into JP… 

    …-3721
    @emolter
    emolter committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    cec0ba6 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. Merge branch 'main' of https://github.com/spacetelescope/jwst into JP… 

    …-3721
    @emolter
    emolter committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    7c4d949 View commit details
    Browse the repository at this point in the history

  2. fix stpipe unit tests

    @emolter
    emolter committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    8b0be05 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. draft fix for JP-2038

    @emolter
    emolter committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    a4f22f3 View commit details
    Browse the repository at this point in the history

  2. fix master background unit and regression test failures

    @emolter
    emolter committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    ecf4b0f View commit details
    Browse the repository at this point in the history

  3. fix regtest errors in niriss wfss stemming from extract_1d

    @emolter
    emolter committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    52c9fb8 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of https://github.com/spacetelescope/jwst into JP… 

    …-3721
    @emolter
    emolter committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    80ae0f3 View commit details
    Browse the repository at this point in the history

  5. fix badpix_selfcal regtest failure

    @emolter
    emolter committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    3e5deda View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' of https://github.com/spacetelescope/jwst into JP… 

    …-3721
    @emolter
    emolter committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    a893d5e View commit details
    Browse the repository at this point in the history