New API design proposal of graphic-walker

[ObservedObserver]

Background

The current API design of graphic-walker is challenging to use. Initially, graphic-walker employed a simpler specification structure. However, as the complexity of graphic-walker increased, a new, albeit informal specification emerged, resembling more of an internal state.

The original specification supported a reactive API. It allowed for passing through a prop in graphic-walker, and the internal state would adapt accordingly. This method is how RATH interacts with graphic-walker.

However, as the internal state of graphic-walker grew more intricate, the specification became harder to align with this state due to their differing visualization design spaces.

Currently, using graphic-walker requires heavy reliance on the imperative usage of the internal store, instead of a declarative API. Hence, there's a need to redesign the API of graphic-walker.

Objective

1. Enable graphic-walker to be used as a controlled component.
2. Adhere to the principle of progressive design: Developers should be able to use the tool knowing only the essential API, without being burdened by excessive context.

Basic Example

Uncontrolled component:

import { GraphicWalker } from 'graphic-walker'
<GraphicWalker />

Controlled component for data:

import { GraphicWalker } from 'graphic-walker'
<GraphicWalker
  data={[...]}
  meta={[...]}
/>

Controlled component for visualization state:

import { GraphicWalker } from 'graphic-walker'
<GraphicWalker
  data={[...]}
  meta={[...]}
  spec={spec}
  onSpecChange={(newSpec, propPath) => {}}
/>

API Reference

Current Interface:

export interface IGWProps {
    dataSource?: IRow[];
    rawFields?: IMutField[]; // Consider renaming for clarity
    spec?: Specification;
    hideDataSourceConfig?: boolean; // Name doesn't accurately reflect its purpose
    i18nLang?: string;
    i18nResources?: { [lang: string]: Record<string, string | any> };
    keepAlive?: boolean | string;
    fieldKeyGuard?: boolean; // Discuss the default value
    /** @default "vega" */
    themeKey?: IThemeKey;
    themeConfig?: GWGlobalConfig; // It can be a top-level config of Vega, encompassing more than just themes
    dark?: IDarkMode;
    storeRef?: React.MutableRefObject<IGlobalStore | null>; // We aim to reduce reliance on storeRef with other props, but it remains essential.
    computation?: IComputationFunction;
    toolbar?: {
        extra?: ToolbarItemProps[];
        exclude?: string[];
    };
    geographicData?: IGeographicData & {
        key: string;
    }; // The design conflicts with geoList
    enhanceAPI?: {
        header?: Record<string, string>;
        features?: {
            askviz?: string | boolean;
        }
    };
    computationTimeout?: number;
    onError?: (err: Error) => void;
    geoList?: IGeoDataItem[];
    channelScales?: IChannelScales; // Ambiguous purpose
}

Prop	Refactor Needed?	Remarks
dataSource	No	-
rawFields	Yes	Consider renaming for clarity
spec	No	-
hideDataSourceConfig	Yes	Name doesn't reflect its purpose
i18nLang	No	-
i18nResources	No	-
keepAlive	No	-
fieldKeyGuard	Yes	Discuss default value
themeKey	No	-
themeConfig	Yes	Can be a top-level config for more than themes
dark	No	-
storeRef	Yes	Reduce reliance; remains essential
computation	No	-
toolbar	No	-
geographicData	Yes	Conflicts with geoList
enhanceAPI	No	-
computationTimeout	No	-
onError	No	-
geoList	Yes	Design conflicts
channelScales	Yes	Ambiguous purpose

Actions

• Update the spec to the latest DSL state.

[ObservedObserver]

Discuss&Opinion

Why directly using internal state to replace specification is a bad idea?

The internal state contains lots of information that is not coding-friendly for developers. Some of the information in the DSL is actually generated by a program and not controlled by humans.

For example, setting the fid (field ID), dragId should not be the work of programmers.

So I think we still need two types or levels of specification, one is for the internal state, one is a skeleton version of internal state, which is more easy to write for programmers.

Internal state

[
    {
        "config": {
            "defaultAggregated": true,
            "geoms": [
                "auto"
            ],
            "coordSystem": "generic",
            "limit": -1
        },
        "encodings": {
            "dimensions": [
                {
                    "dragId": "gw_AyzM",
                    "fid": "gender",
                    "name": "gender",
                    "basename": "gender",
                    "semanticType": "nominal",
                    "analyticType": "dimension"
                },
                {
                    "dragId": "gw_STqa",
                    "fid": "race/ethnicity",
                    "name": "race/ethnicity",
                    "basename": "race/ethnicity",
                    "semanticType": "nominal",
                    "analyticType": "dimension"
                },
                {
                    "dragId": "gw_VcXV",
                    "fid": "parental level of education",
                    "name": "parental level of education",
                    "basename": "parental level of education",
                    "semanticType": "nominal",
                    "analyticType": "dimension"
                },
                {
                    "dragId": "gw__smW",
                    "fid": "lunch",
                    "name": "lunch",
                    "basename": "lunch",
                    "semanticType": "nominal",
                    "analyticType": "dimension"
                },
                {
                    "dragId": "gw_S5-r",
                    "fid": "test preparation course",
                    "name": "test preparation course",
                    "basename": "test preparation course",
                    "semanticType": "nominal",
                    "analyticType": "dimension"
                },
                {
                    "dragId": "gw_mea_key_fid",
                    "fid": "gw_mea_key_fid",
                    "name": "Measure names",
                    "analyticType": "dimension",
                    "semanticType": "nominal"
                }
            ],
            "measures": [
                {
                    "dragId": "gw_wRod",
                    "fid": "math score",
                    "name": "math score",
                    "basename": "math score",
                    "analyticType": "measure",
                    "semanticType": "quantitative",
                    "aggName": "sum"
                },
                {
                    "dragId": "gw_kpYc",
                    "fid": "reading score",
                    "name": "reading score",
                    "basename": "reading score",
                    "analyticType": "measure",
                    "semanticType": "quantitative",
                    "aggName": "sum"
                },
                {
                    "dragId": "gw_Ld6M",
                    "fid": "writing score",
                    "name": "writing score",
                    "basename": "writing score",
                    "analyticType": "measure",
                    "semanticType": "quantitative",
                    "aggName": "sum"
                },
                {
                    "dragId": "gw_mea_val_fid",
                    "fid": "gw_mea_val_fid",
                    "name": "Measure values",
                    "analyticType": "measure",
                    "semanticType": "quantitative",
                    "aggName": "sum"
                },
                {
                    "dragId": "gw_count_fid",
                    "fid": "gw_count_fid",
                    "name": "Row count",
                    "analyticType": "measure",
                    "semanticType": "quantitative",
                    "aggName": "sum",
                    "computed": true,
                    "expression": {
                        "op": "one",
                        "params": [],
                        "as": "gw_count_fid"
                    }
                }
            ],
            "rows": [
                {
                    "dragId": "gw_SJh_",
                    "fid": "reading score",
                    "name": "reading score",
                    "basename": "reading score",
                    "analyticType": "measure",
                    "semanticType": "quantitative",
                    "aggName": "mean"
                }
            ],
            "columns": [
                {
                    "dragId": "gw_S6tt",
                    "fid": "parental level of education",
                    "name": "parental level of education",
                    "basename": "parental level of education",
                    "semanticType": "nominal",
                    "analyticType": "dimension"
                }
            ],
            "color": [
                {
                    "dragId": "gw_6Qqj",
                    "fid": "gender",
                    "name": "gender",
                    "basename": "gender",
                    "semanticType": "nominal",
                    "analyticType": "dimension"
                }
            ],
            "opacity": [],
            "size": [],
            "shape": [],
            "radius": [],
            "theta": [],
            "longitude": [],
            "latitude": [],
            "geoId": [],
            "details": [],
            "filters": [],
            "text": []
        },
        "layout": {
            "showActions": false,
            "showTableSummary": false,
            "stack": "stack",
            "interactiveScale": false,
            "zeroScale": true,
            "size": {
                "mode": "auto",
                "width": 320,
                "height": 200
            },
            "format": {},
            "geoKey": "name",
            "resolve": {
                "x": false,
                "y": false,
                "color": false,
                "opacity": false,
                "shape": false,
                "size": false
            }
        },
        "visId": "gw_hytE",
        "name": "Chart 1"
    }
]

What we expected

[
    {
        "encodings": {
            "dimensions": [
                "gender",
                "race/ethnicity",
                "parental level of education",
                "lunch",
                "test preparation course"
            ],
            "measures": [
                "math score",
                "reading score",
                "writing score"
            ],
            "rows": [
                {
                    "name": "reading score",
                    "aggName": "mean"
                }
            ],
            "columns": [
                "parental level of education"
            ],
            "color": [
                "gender"
            ]
        },
        "name": "Chart 1"
    }
]

[ObservedObserver]

Case study

previous API

import { Pencil2Icon } from "@radix-ui/react-icons";
import { Button } from "@/components/ui/button";
import { GraphicWalker } from "@kanaries/graphic-walker";
import { useEffect, useRef, useState } from "react";
import { IGlobalStore } from "@kanaries/graphic-walker/dist/store";
import Modal from "@/components/ui/modal";

interface ChartEditorProps {
    spec: any;
    rawFields: any;
    dataSource: any;
    computationService: any;
    themeConfig?: any;
    onChangeSpec?: (any) => void;
}

export default function ChartEditor(props: ChartEditorProps) {
    const { spec, rawFields, dataSource, computationService, themeConfig, onChangeSpec } = props;
    const chart1 = useRef<IGlobalStore | null>(null);
    const [editing, setEditing] = useState(false);

    useEffect(() => {
        if (chart1.current) {
            // chart1.current?.vizStore.initState();
            chart1.current.vizStore.importStoInfo({
                dataSources: [
                    {
                        id: "dataSource-0",
                        data: [{}],
                    },
                ],
                datasets: [
                    {
                        id: "dataset-0",
                        name: "DataSet",
                        rawFields,
                        dsId: "dataSource-0",
                    },
                ],
                specList: spec,
            });
        }
    }, [spec, rawFields, dataSource]);
    return (
        <span>
            <Button
                className="float-right"
                variant="outline"
                size="icon"
                onClick={() => {
                    setEditing(true);
                }}
            >
                <Pencil2Icon className="h-4 w-4" />
            </Button>
            <Modal
                show={editing}
                onClose={() => {
                    setEditing(false);
                    const spec = chart1.current?.vizStore.exportViewSpec();
                    console.log(spec);
                    spec && onChangeSpec?.(spec);
                }}
            >
                <div className="w-full h-full">
                    <GraphicWalker
                        themeConfig={themeConfig}
                        hideDataSourceConfig
                        dataSource={dataSource}
                        rawFields={rawFields}
                        storeRef={chart1}
                        fieldKeyGuard={false}
                        computation={computationService}
                    />
                </div>
            </Modal>
        </span>
    );
}

what we expected in new API

import { Pencil2Icon } from "@radix-ui/react-icons";
import { Button } from "@/components/ui/button";
import { GraphicWalker } from "@kanaries/graphic-walker";
import { useEffect, useState } from "react";
import Modal from "@/components/ui/modal";

interface ChartEditorProps {
    spec: any;
    rawFields: any;
    dataSource: any;
    computationService: any;
    themeConfig?: any;
    onChangeSpec?: (newSpec: any) => void;
}

export default function ChartEditor(props: ChartEditorProps) {
    const { spec, rawFields, dataSource, computationService, themeConfig, onChangeSpec } = props;
    const [editing, setEditing] = useState(false);
    const [currentSpec, setCurrentSpec] = useState(spec);

    const handleSpecChange = (newSpec: any) => {
        setCurrentSpec(newSpec);
    };

    return (
        <span>
            <Button
                className="float-right"
                variant="outline"
                size="icon"
                onClick={() => setEditing(true)}
            >
                <Pencil2Icon className="h-4 w-4" />
            </Button>
            <Modal
                show={editing}
                onClose={() => {
                    setEditing(false);
                    console.log(currentSpec);
                    onChangeSpec?.(currentSpec);
                }}
            >
                <div className="w-full h-full">
                    <GraphicWalker
                        themeConfig={themeConfig}
                        hideDataSourceConfig
                        data={dataSource}
                        meta={rawFields}
                        spec={currentSpec}
                        onSpecChange={handleSpecChange}
                        computation={computationService}
                    />
                </div>
            </Modal>
        </span>
    );
}