ENG-564: Node-based dataset editor with drill-down, CRUD, metadata editing, and YAML panel#7687
ENG-564: Node-based dataset editor with drill-down, CRUD, metadata editing, and YAML panel#7687
Conversation
Swap the Monaco YAML editor for an interactive React Flow graph where each collection and field is a node. Clicking a node opens a drawer to edit its metadata. - Add DatasetNodeEditor with D3 hierarchy layout (LR tree) - Add custom node components (root, collection, field) matching the taxonomy tree visual style with hover context - Add DatasetTreeHoverContext for parent/child/inactive highlighting - Add custom bezier edges that change color on hover - Add DatasetNodeDetailPanel drawer for editing metadata - Enable "Edit dataset" button for SaaS connectors - Remove test results/logs sections for full-canvas layout
…diting to dataset node editor - Add drill-down: overview shows root→collections, clicking a collection shows its fields - Add breadcrumb navigation bar with back button when drilled into a collection - Add/delete collections (via "+" on root node) and fields (via "+" on field nodes for nesting) - Add AddNodeModal with uniqueness validation for naming new nodes - Add DatasetEditorActionsContext to provide CRUD callbacks to node components - Expand detail panel with collapsible fides_meta sections for both collections and fields - Add delete button with confirmation dialog in detail panel - Fix node overlap by constraining container width and increasing layout spacing
The node-based dataset editor should only be used for SaaS connections. DB connectors retain the original Monaco YAML editor with reachability checks.
…down mode - Add-field modal now includes description, data categories, and full fides_meta fields (matching the edit panel) so metadata can be set at creation time. - Remove "+" button from dataset root node (overview mode). - Show "+" on collection node only when it is the drill-down root.
DRY up duplicated code between AddNodeModal and DatasetNodeDetailPanel: - FieldMetadataFormItems component for the 8 fides_meta form fields - buildFieldMeta helper for converting form values to fides_meta - DATA_TYPE_OPTIONS and REDACT_OPTIONS constants
Use useTaxonomies() to populate the data categories Select with suggestions from the existing taxonomy, while still allowing free-form tags input. Extracted as shared DataCategoryTagSelect component.
Only re-fit the viewport when the graph structure changes (node count or focused collection), not when field metadata is edited.
When a collection or field is added, the new node gets a 1.5s yellow highlight animation that fades out, making it easy to spot in the graph.
All 6 base types support the [] array suffix per parse_data_type_string in the backend. Add all 12 variants to the dropdown.
- Replace centered Modal with right-side Drawer matching the edit panel style, with a "Create" button in the drawer header. - Include array data type variants (e.g. string[], object[]) in the data type dropdown.
Adds a toggleable YAML code editor at the bottom of the node graph with two-way sync: graph changes update the YAML and YAML edits update the graph (debounced 500ms with error display).
- Hoist layout options to module constant to prevent useMemo busting - Stabilize onMouseEnter with functional state update (removes dep) - Precompute ancestor/descendant sets per hover instead of per-node - Export ProtectedFieldsInfo from useDatasetGraph, remove duplicate - Remove unused DatasetNodeData type export - Remove no-op onMount from Monaco editor - Add aria-label to add field/nested field buttons - Show add buttons on focus-within (keyboard accessibility)
- Derive selectedNodeData from rawNodes via useMemo instead of storing a stale snapshot in state. Now always reflects the latest dataset. - Debounce detail panel onValuesChange (300ms) to avoid triggering full dataset/graph recomputes on every keystroke. - Guard against null/non-array fields and collections from incomplete YAML edits (e.g. bare "-" list items, partial typing).
…e editor forms These advanced fields are better edited directly in YAML.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
Greptile SummaryThis PR replaces the YAML-only dataset editor with a node-based visual editor ( Key observations:
Confidence Score: 3/5
Important Files Changed
Last reviewed commit: "Remove return_all_el..." |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| <div style={{ display: "flex", alignItems: "center", gap: 8 }}> | ||
| <Typography.Text strong>{nodeData?.label}</Typography.Text> | ||
| <Tag |
There was a problem hiding this comment.
Flex over div with inline flex styles
The Drawer title uses a <div> with display: "flex" inline styles. Per the project's semantic HTML guideline, Ant Design's Flex component should be used instead of <div> elements for flex containers.
| <div style={{ display: "flex", alignItems: "center", gap: 8 }}> | |
| <Typography.Text strong>{nodeData?.label}</Typography.Text> | |
| <Tag | |
| <Flex align="center" gap={8}> | |
| <Typography.Text strong>{nodeData?.label}</Typography.Text> |
The same pattern appears in the closing tag — change </div> to </Flex> and add Flex to the fidesui import.
Rule Used: Avoid using div elements when possible. Use sema... (source)
Learnt From
ethyca/fides#6763
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| <div | ||
| ref={reactFlowRef} | ||
| className="size-full" | ||
| style={{ display: "flex", flexDirection: "column" }} | ||
| > |
There was a problem hiding this comment.
Flex instead of div for flex container
This outer wrapper uses display: "flex" and flexDirection: "column" via inline styles on a <div>. The project guideline prefers using Ant Design's Flex component over bare <div> elements for flex containers.
| <div | |
| ref={reactFlowRef} | |
| className="size-full" | |
| style={{ display: "flex", flexDirection: "column" }} | |
| > | |
| <Flex | |
| ref={reactFlowRef} | |
| className="size-full" | |
| vertical | |
| > |
The same applies to the YAML panel container at line 598 (<div style={{ ..., display: "flex", flexDirection: "column" }}>) and the canvas wrapper at line 562 (<div style={{ flex: "1 1 auto", ... }}>). All three div flex containers can be replaced with Flex (setting vertical for column direction and using flex, style for sizing props). This pattern also appears in DatasetEditorSection.tsx at the isSaas branch wrapper (line ~283).
Rule Used: Avoid using div elements when possible. Use sema... (source)
Learnt From
ethyca/fides#6763
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| const buildAncestryMaps = (edges: Edge[]) => { | ||
| // parentOf: child → parent | ||
| const parentOf = new Map<string, string>(); | ||
| edges.forEach((e) => { |
There was a problem hiding this comment.
e is a 1-character variable name. The project convention requires full, descriptive names for variables. Rename to edge for clarity. The same applies on line 59 inside getDescendants.
| edges.forEach((e) => { | |
| edges.forEach((edge) => { |
Line 59 should also be updated:
edges.forEach((edge) => {
if (!children.has(edge.source)) {
children.set(edge.source, []);
}
children.get(edge.source)!.push(edge.target);
});
Rule Used: Use full names for variables, not 1 to 2 character... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| useEffect(() => { | ||
| if (nodeCount > 0) { | ||
| setTimeout(() => { | ||
| reactFlowInstance.fitView({ padding: 0.2 }); | ||
| }, 150); | ||
| } | ||
| }, [nodeCount, focusedCollection, reactFlowInstance]); |
There was a problem hiding this comment.
setTimeout cleanup in useEffect
The setTimeout call for fitView is not cleaned up in the effect's return function. If the component unmounts or the effect re-runs within the 150 ms window, the stale timer will fire and call reactFlowInstance.fitView against a potentially stale closure. While fitView doesn't trigger React state updates (so there's no unmounted-component warning), it's best practice to always clear timers in useEffect cleanup.
| useEffect(() => { | |
| if (nodeCount > 0) { | |
| setTimeout(() => { | |
| reactFlowInstance.fitView({ padding: 0.2 }); | |
| }, 150); | |
| } | |
| }, [nodeCount, focusedCollection, reactFlowInstance]); | |
| useEffect(() => { | |
| if (nodeCount > 0) { | |
| const timerId = setTimeout(() => { | |
| reactFlowInstance.fitView({ padding: 0.2 }); | |
| }, 150); | |
| return () => clearTimeout(timerId); | |
| } | |
| return undefined; | |
| }, [nodeCount, focusedCollection, reactFlowInstance]); |
1 participant
Ticket ENG-564
Description Of Changes
Replace the YAML-only dataset editor with an interactive node-based visual editor for SaaS connections using
@xyflow/react. DB connections retain the existing YAML editor. The node editor supports drill-down navigation, CRUD operations, full fides_meta metadata editing, and a collapsible two-way YAML editor panel.https://www.loom.com/share/8d3bb2ce183b446c9a53d6b57550c8d4
Code Changes
DatasetNodeEditor.tsx- Main ReactFlow component with drill-down state, breadcrumb navigation, add/delete/update handlers, highlight animation for new nodes, collapsible YAML editor panel with two-way sync (graph↔YAML)DatasetNodeDetailPanel.tsx- Drawer for editing node metadata: description, data_categories, collapsible fides_meta sections for collections (after, erase_after, skip_processing) and fields (data_type, identity, primary_key, read_only, redact). Debounced updates (300ms). Delete with confirmation.DatasetEditorSection.tsx- Conditional rendering: SaaS → DatasetNodeEditor, DB → Monaco YAML editor. Save handler branches accordingly.AddNodeModal.tsx- Right-side Drawer for naming new collections/fields with uniqueness and format validation. In field mode, includes description, data categories, and collapsible fides_meta form.FieldMetadataFormItems.tsx- Shared form components and helpers extracted to eliminate duplication between AddNodeModal and DatasetNodeDetailPanel. Includes DataCategoryTagSelect with taxonomy suggestions.context/DatasetEditorActionsContext.tsx- Context providing add/delete callbacks to node componentscontext/DatasetTreeHoverContext.tsx- Hover highlighting with precomputed ancestor/descendant sets for performanceuseDatasetGraph.ts- Converts Dataset to ReactFlow nodes/edges. Two views: overview (root→collections) and drill-down (collection→fields with nested support). Guards against malformed YAML data.useDatasetNodeLayout.ts- Dagre LR layout with configurable spacingnodes/DatasetRootNode.tsx- Dataset root nodenodes/DatasetCollectionNode.tsx- Collection node with "+" button (drill-down only), field count badgenodes/DatasetFieldNode.tsx- Field node with "+" button for nested fields, data category count, protected badgenodes/DatasetNodeHandle.tsx- Reusable LR handle componentnodes/DatasetNode.module.scss- Node styling, hover states, highlight animation, accessible focus-within for add buttonsedges/DatasetTreeEdge.tsx- Custom BezierEdge with hover-aware stylingtest-datasets.tsx- Adjusted layout for full-height canvasConnectorParametersForm.tsx- Show "Edit dataset" button for SaaS connectionsSteps to Confirm
/systems/configure/{id}/test-datasetsfor a SaaS connectionPre-Merge Checklist
CHANGELOG.mdupdated