Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with replacing variable value in pygrib library #227

Open
terence0o opened this issue Dec 25, 2023 · 3 comments
Open

Problem with replacing variable value in pygrib library #227

terence0o opened this issue Dec 25, 2023 · 3 comments

Comments

@terence0o
Copy link

Hello,
I have been working on a project that involves processing weather data using the Pygrib library in Python. During my implementation, I encountered an issue with replacing a new variable value within a GRIB file using pygrib.
I am using the Pygrib library to read FNL data files, and attempting to replace a group of values of the same size into the corresponding positions in the file.I have successfully accomplished the replacement, but I have noticed that the values I replace with have some slight variations. Here is my program:

out_var = f[grid_index]
out_var.values = new_data
print(out_var.values)

then I discover out_var.values is not equal new_data

I checked the data type, shape, and reviewed the documentation and various online resources, but I am having difficulties understanding the correct approach to replace the variable value.

@jswhit
Copy link
Owner

jswhit commented Dec 29, 2023

are you accounting for the fact that GRIB uses lossy compression (so the data files you write into the file will not be exactly the same as the values you read back)?

@terence0o
Copy link
Author

are you accounting for the fact that GRIB uses lossy compression (so the data files you write into the file will not be exactly the same as the values you read back)?

I really appreciate your response. Yes, after replacing certain variables (RH) with partial levels, the data is not exactly consistent with the original. Does this mean that lossy compression is an inherent characteristic of GRIB2, and unavoidable?

@stephane-21
Copy link

Yes the grib2 format is a lossy compression format, I believe that internally it will store your data on a smaller number of bits. Something to investigate is if the number of bits storage size can be modified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants