Enterprise data science · o9 Digital Brain

DSML
Studio.

Extract. Transform. Load, and trust it.

A data scientist’s day is an ETL loop: find governed data, shape it, test a model, ship it, watch it run. I designed the workspace where that loop stays connected instead of scattering across six separate tools.

System design

Carried to every stageProjectEnvironmentModel

01DiscoverSelected dimensions + measures
02ExploreProfiled dataset + access scope
03BuildVersioned code + parameters
04ValidateRegistered model candidate
05OperateVersion + environment + schedule
06ObserveRequestId → logs → recovery

Fig. 01 One model’s journey through six stages. Each stage passes something forward, and project, environment and model ride along the whole way.

Problem: Specialist tools.
Governed data.No shared thread between them.
My role: Product Designer0→1 system design, six connected workspaces
Worked with: Data scientists + engineeringObservation sessions, architecture reviews and the implementation specification
Status: A design hypothesisNot measured in use yet. Validation is the next step.

Scroll to unpack

Extract · 01 / The problem

Every view worked on its own.
Nobody’s work did.

Six tools.
One model.
No shared thread.

Data scientists worked in specialist tools while planners worked inside the governed enterprise model. Between them sat copied identifiers, brittle scripts, hidden assumptions, and delayed feedback.

How I framed it: how might we connect flexible data-science work to governed enterprise planning, without losing context at each handoff?

Six handoffs, two operating modelsDSML Studio: the context travels with the work

A solid line: context carried from stage to stage.A reconstruction of the operating model, informed by observation, not a recorded trace

Extract · 02 / The evidence

The spec described services.
Watching data scientists showed me the work.

I ran the same extract, transform, load discipline on the research itself: five inputs, one synthesis, three concrete decisions.

ExtractInputs I could access

TransformHow I synthesised it

Core taskMove a model from governed data to production

Repeated riskContext disappears between tools, roles, and handoffs

UX responseCarry context, preview commitments, trace every run

LoadWhat changed in the design

01Intent-based navigationFind, build, operationalise, monitor

02Preview before actionShow data, parameters, impact first

03Request-level traceabilityModel, version, environment, request ID

What this research is

A day-in-the-life walkthrough and fly-on-the-wall sessions with data scientists, then the implementation specification, the platform’s existing patterns, mature specialist tools, and architecture reviews with engineering.

What it isn’t

A formal study. The sessions were early in the project and the notes weren’t kept, so there are no quotes or session counts here; what they taught me is recalled and labelled that way. It set a design hypothesis, not a measured usability result.

Extract · 03 / Who it’s for

Four roles,
one shared model.

The same model passes through data science, engineering, planning, and operations. Each role needs different evidence before they can move the work forward.

Extract · 04 / What already worked

Data scientists already had tools they trusted.
Rebuilding them would throw that away.

The specification embeds three open tools rather than replacing them. I studied what each already did well, then designed only what the shell around it had to add.

Notebooks

JupyterLab

Kept as itself: Cells, kernels and the notebook habits people bring with them.
The shell adds: The module you already chose, your theme, and Save as plugin into the governed library.

Experiments

MLflow

Kept as itself: Run tracking and side-by-side comparison.
The shell adds: Runs grouped by the project’s module, and one decision on top: Register candidate

Observability

Kibana & Spark UI

Kept as itself: Logs, dashboards and job history, as operators know them.
The shell adds: Sign-in and role-scoped access, and the failed run’s RequestId to start from.

Borrow the tool. Own the shell. Module, environment, role and scope stay stable around whatever is embedded.

From the implementation specification and pattern research · the pairing is design judgement

Transform · 05 / Strategy & principles

The riskiest questions were below the screen.
I answered those before drawing any.

The specification described six services. Read as one person’s task, they became three moves, repeated. Then I settled the open questions most likely to fail quietly.

ExtractFind governed data

Model Explorer and Data Explorer pull from the live planning model or cloud files, never a silent copy.

TransformClassify, model, package

EDA, notebooks, and plugins turn a raw frame into a versioned, reusable artifact.

LoadDeploy, trace, learn

Deployments and Observability put a result into production and keep it traceable when it fails.

Left open in the specificationHow the design answers it

Is a separate “create plugin” flow needed?

No. A notebook saves as a plugin: one artifact in two states, not two products.

What does “context aware” mean for the logs?

The request’s identifiers travel into observability, so a failure is traced from its own run, not searched for.

Who can see a file, and at what level?

Five access scopes, from private to tenant-wide, recorded in the file’s own metadata.

Questions paraphrased from the implementation specification · answers are design judgement

Five principles, set before the screens

01Context travels with the work.Never retyped between stages. The request ↓
02Evidence before commitment.Profile before a model; preview before a run. Decision 02 ↓
03One screen, one decision.Edit and execute never share a state. Decision 01 ↓
04Borrow the tool, own the shell.Familiar inside, stable around it. Decision 04 ↓
05Every run leaves a trail.Identifiers follow the run automatically. Decision 03 ↓

Hold the screens that follow to these five.

Transform · 06 / Entry points

Nobody starts on a home page.
Every door carries you in with context.

A data scientist arrives mid-task: from another workspace, a finished run, or a plugin’s code. Each door lands on the right screen with the context already set.

A
Enabled for the organisation
The workspace is switched on per organisation and shown to the people whose role needs it, so everyone else’s navigation stays unchanged.
B
From another workspace
The modules are already loaded and the one you were in is still selected.
C
From a finished analysis
The notification opens the exploratory analysis with that run’s results.
D
From a plugin’s code
Code is read-only in the plugin view. Edit opens it in a notebook, ready to change.

Every door lands inside one shell

DSML Studio

Arrives withRoleModuleEnvironmentModel

Entry rules from the implementation specification, drawn as a diagram · no internal screens shown

Transform · 07 / Architecture

Organised by service, it was a menu.
Organised by intent, it’s a path.

First, where it sits in the platform. Then the navigation, organised by four goals: find data, build, operationalise and monitor.

Already in the platformPlanning workspacesWhere planners act on the plan

A governed writeback into the measures planners already use

This projectDSML Studio

Discover
Explore
Build
Validate
Operate
Observe

Embeds, as themselvesJupyterLabMLflowKibana & Spark UI

A governed query or a cloud file, never a silent copy

Already in the platformGoverned planning model + cloud filesThe data both sides trust

Fig. 07 Where it sits. DSML Studio is the new layer between the governed data and the plan it writes back to. The tools it embeds keep their own interfaces.

Data foundation

Find + qualify

Model ExplorerData ExplorerMetadata

Build

Create + package

NotebooksPluginsParameters

Operationalize

Compare + release

ExperimentsDeploymentsRegistry

Monitor

Trace + recover

HardwareLogsRun history

Groups follow the product’s navigation; layers are named as in the implementation specification.

6workspaces connectedone lifecycle

12execution parametersoptional, typed, reusable

3trace identifiersPlugin · Batch Job · Request

5access levelsprivate to tenant-wide

Designed scope, not impact. These count what the design covers. None of them is a measured result.

Transform · 08 / Key decisions

Four decisions that made the workflow
safer and clearer.

Each adds a deliberate check before a consequential action, or removes repeated work where context can travel on its own.

Decision / 01

Separate authoring from execution

Decision 01 · ModeSystem state

EDITConfigCodeSave state

EXECUTEScopeParametersRun state

Why

Editing changes the reusable artifact; executing creates a run with scope and parameters. Mixing them would make save state, permissions, and failure recovery harder to understand.

The trade-off: one more explicit switch, far less ambiguity.

Mode clarity

Error prevention

Decision / 02

Make preview a gate, not a courtesy

Decision 02 · PreviewSystem state

Plugin configScopeParameters

QUERY
PREVIEWExecute unlocks

Why

The final query is composed from plugin config, scope, and parameters. Preview turns that hidden composition into evidence you can inspect before Execute becomes available.

The trade-off: a step before execution.

Safe exploration

Commitment threshold

Decision / 03

Preserve trace context automatically

Decision 03 · TraceIdentifiers · sample

PluginId12346

BatchJobId63ee…4a6

RequestId82a1…0c7

Why

PluginId, BatchJobId, and RequestId travel from execution into observability. The user should never have to copy an opaque identifier between tools.

The trade-off: it needs service-level handshake contracts.

Visibility

Recoverability

Decision / 04

Embed specialist tools without losing the shell

Decision 04 · ShellContext layers

o9 GLOBAL CONTEXT

JupyterLabMLflowObservability

Role · environment · model · scope

Why

JupyterLab, MLflow, and observability patterns stay recognisable, while the o9 shell keeps module, environment, role, and planning context stable.

The trade-off: integration complexity around theme and navigation.

Familiarity

Contextual integrity

Designing with data

Comparison only works when metrics answer a decision.

The experiment view avoids a “dashboard of everything.” It prioritises run identity, model family, accuracy, duration, owner, version, and status: the minimum set needed to register a candidate with confidence.

Sample runsFrom the rebuild’s experiment table · not production performance

demand_forecast_w27 · first four runsRun records · sample

RMSE, lower is better → · R² ↑

Candidate Other run

Load · 09 / Follow the request

One model, from data to production.
Its context never retyped.

Finally, everything above in one run. Watch it end to end, then step through each stage: the screen, the question it answers, the context it receives, and what it passes forward.

Prototype rebuild · filmSample data · 1:03 · silent

DSML Studio · one request, end to end

Film One request, recorded in the Carbon rebuild: the governed query, a data profile, a notebook saved as a plugin, the preview gate, a run comparison, a promotion, and a failure traced to its request. The captions name the six stages.

Then take it one stage at a time.

Six stages · one model’s context

Stage questions and handoffs are from the design’s journey; the global context bar (project, environment, model) is on every screen of the rebuild. Captures are from the Carbon rebuild, and all data is sample data.

Load · 10 / Outcome & reflection

What changed, and what still
needs validation.

The six workspaces now form one understandable workflow. The next step is measuring whether teams complete cross-role tasks faster and recover from failures with less support.

What success should be measured by

Hypotheses · not yet measured

01DiscoverTime to a usable governed datasetNot measured

02BuildContext switches per model iterationNot measured

03ValidateTime to select and register a candidateNot measured

04OperateMedian time from failure to causeNot measured

What I can say

The designs and the Carbon rebuild show six workspaces joined by carried context, a preview gate before execution, and request-level traceability into observability. The implementation specification defines the services underneath.

What I don’t claim

A launch, adoption, or any measured change in task time or failure recovery. Quotes or counts from the early observation sessions, whose notes weren’t kept. The run metrics shown are sample data.

Component handshakes, identifiers, preview gates, and service boundaries made the experience coherent in ways a polished screen alone could not.

Design below the UI layer.

Next, I’d measure time to a first governed dataset, failed-run recovery, confidence when registering a model, and what gets lost at cross-role handoffs.

Longitudinal task evidence.

Surface model lineage, feature contribution, and confidence where a planner consumes the writeback, not only inside DSML Studio.

Explainability at the planning moment.

Six stages.
One model.
One thread, end to end.

Complex systems become usable when every handoff carries context.

DSMLStudio.

Every view worked on its own.Nobody’s work did.

The spec described services.Watching data scientists showed me the work.

Four roles,one shared model.

Data scientists already had tools they trusted.Rebuilding them would throw that away.