How to define default values for config of graph_asset?

[ivan-tsarev-bioage]

Hi everybody!
We are now exploring how to switch from job-and-graph orchestration to assets.
And right now I'm struggling a lot figuring out a painless way to make logic inside assets re-usable with configuration.

So our old-style code usually looks like this:

class OldStyleConfig(Config):
    non_default_value: str
    defaulted_value: str = "default"


@op
def old_style_op(context: OpExecutionContext, config: OldStyleConfig):
    context.log.info(config.defaulted_value)
    context.log.info(config.non_default_value)
    return 1


@graph
def old_style_graph():
    return old_style_op()


old_style_job = old_style_graph.to_job(
    config={
        "ops": {
            "old_style_op": {
                "config": {
                    "non_default_value": "bar",
                    "defaulted_value": "buz"
                }
            }
        }
    }
)

And that what I see in UI:

In such a manner we got:

• set default config for every job in the code on the job level, which allows us to reuse graph in multiple places with just configuration
• easy access to change default config for the job on-the-go in the UI, without touching code
• it's also possible to throw some sugary config_mapping-like function to generate config dict from simpler set of args
  Overall, that looks like exactly what we want from configuring reusable logic.

So now I'm trying to reproduce this behaviour with asset somehow.
The less resistance path is using graph_asset, since we already have tons of graphs everywhere and we want to reuse it for sure.
My graph_asset looks like:

@graph_asset(
    config={
        "old_style_graph": {
            "ops": {
                "old_style_op": {
                    "config": {"non_default_value": "bar", "defaulted_value": "buz"}
                }
            }
        }
    }
)
def new_graph_asset():
    return old_style_graph()

Unfortunately, in UI it looks quite disappointing:

We lost all virtually all information on what the config is in the code and option to edit it on the go from UI.

Trying to use config_mapping like this:

@config_mapping(
    config_schema={
        "defaulted_value": Field(str, default_value="default"),
        "non_default_value": str,
    }
)
def graph_asset_config_mapping(val):
    return {
        "old_style_graph": {
            "ops": {
                "old_style_op": {
                    "config": {
                        "non_default_value": val["non_default_value"],
                        "defaulted_value": val["defaulted_value"]}
                }
            }
        }
    }


@graph_asset(
    config=graph_asset_config_mapping
)
def new_graph_asset():
    return old_style_graph()

In the UI I got following picture:

• UI can generate draft config for us but
• It drops params with default values from the config, only showing it in the right side-column
• It loose completely ability to define default value at the asset/job level which makes it quite useless.

Apparently, plain software-based asset share the same unpleasant feature, from what I understand - we can define default value for parameter on Config object-level, but not when actually using and re-using it in assets, making Assets either share default value or setting them in UI only.

We probably can subclass Config object for every asset redefining default values, but it sounds a bit too complicated to do to be honest.

[mudravrik]

I happened to make yet another approach to making asset a proper reusable piece of logic and solved the above described issue with on of the most weird and cryptic snippet I've ever written:

class BaseFooConfig(Config):
    foo: str


def patch_config_with_defaults(value: BaseFooConfig):
    class ModifiedFooConfig(BaseFooConfig):
        foo: str = value.foo
    return ModifiedFooConfig


@asset
def foo_asset(config: patch_config_with_defaults(BaseFooConfig(foo="buz"))):
    pass

It essentially allows my less-experienced colleagues to copy-paste asset definition function and tune it in-place to create another asset with very same logic but different params.
It still feels way more vulnerable comparing to regular jobs, but with proper encapsulation of logic in the asset_fn we can live with that.

[mudravrik]

Spent i bit more time with pydantic docs and created this thing for somewhat generalised default value injection based on created Config instance:

def inject_default_from_instance(instance_w_defaults: Config):
    model_w_defaults = create_model(
        __model_name=instance_w_defaults.__repr_name__() + "WithDefaults",
        __base__=type(instance_w_defaults),
        **{
            k: (
                instance_w_defaults.model_fields[k].annotation,
                FieldInfo.merge_field_infos(
                    instance_w_defaults.model_fields[k],
                    default=getattr(models_w_defaults, k)
                )
            )
            for k in instance_w_defaults.model_fields.keys()
        }
    )
    return model_w_defaults
    
class BaseFooConfig(Config):
  foo: str 

new_config = inject_default_from_instance(BaseFooConfig(foo="bar"))

new_config()
# BaseFooConfigWithDefaults(foo='bar')

In-asset usage is pretty much the same:

@asset
def foo_asset(config: inject_default_from_instance(BaseFooConfig(foo="buz"))):
    pass

and following result in the Launchpad: