-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dagster 1.7.5 - 'Value is not JSON serializable' returning metadata #21773
Comments
Thinking this might be caused by #21576 |
Hi @kubiani , happy to try some debugging if you can maybe advise what to look for? |
@the4thamigo-uk For my case I'm actually thinking this may have been a misuse of the For example I was doing something along the lines of: metadata = {
"success": {
"count": MetadataValue.int(success_total()),
"size": MetadataValue.float(success_size())
},
"errors": {
"count": MetadataValue.int(error_total()),
"size": MetadataValue.float(error_size())
}
} However, removing the nesting seems to fix things, and allows the UI to plot the value. Would be nice if it recognized nested values but this seems like the intended use. metadata = {
"success_count": MetadataValue.int(success_total()),
"success_size": MetadataValue.float(success_size()),
"error_count": MetadataValue.int(error_total()),
"error_size": MetadataValue.float(error_size())
}
# or can still do nested, but values won't get plotted in UI
# /> maps both `success` and `errors` to MetadataValue.json
metadata = {
"success": {
"count": success_total(),
"size": success_size()
},
"errors": {
"count": error_total(),
"size": error_size()
}
} |
@kubiani So are you saying this is not a bug? |
@the4thamigo-uk Generally, no I don't think so. At least not in my case of using it in nested metadata values. However, YMMV Perhaps at the least there could be an improvement to more gracefully unwrap/warn if the MetadataValue is accidently used in a nested context. |
@kubiani looks like this may have been fixed in 1.7.8. At least Im not seeing the issue anymore. |
Dagster version
dagster, version 1.7.5
What's the issue?
I have an asset that obtains the metadata for an upstream asset (via
DagsterInstance.get_event_records()
), and attaches this upstream metadata as a subkey in its ownOutput
metadata.When I obtain the upstream metadata via
AssetMaterialization.metadata
, it has the form :I pass this metadata on in the following way, and in dagster 1.7.4 it used to work :
But in dagster 1.7.5 I get :
It seems that the regression is due to code in
Output
because it looks like the metadata I retrieve from the upstreamAssetMaterialization
has not changed in form between 1.7.4 and 1.7.5 .What did you expect to happen?
It used to work in 1.7.4 but not in 1.7.5
How to reproduce?
No response
Deployment type
None
Deployment details
No response
Additional information
No response
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
The text was updated successfully, but these errors were encountered: