GitHub user eugenegujing edited a discussion: Proposal: Making the workspace
agent useful to a new user by seeing run errors, reading the docs, knowing the
datasets
The agent can build and run workflows, but it cannot see the errors from a run
the user started, cannot read Texera's own documentation, and does not know
which datasets the user has. I want to propose three small, independent
changes, each tracked as its own issue and PR. The first two are ready to
start, and the third is a sketch I would like feedback on first. This follows
up on #9246.
```mermaid
flowchart TB
subgraph browser [Browser]
canvas["Workflow canvas<br/>run state, errors, row counts"]
chat[Agent chat]
end
subgraph service [Agent service]
ctx[Context sent to the model]
tools[Agent tools]
end
docs[("docs/<br/>bundled in the image")]
files[("file-service<br/>the user's datasets")]
canvas == "1. report on the user's last run" ==> chat
chat -- "prompt" --> ctx
ctx -- "LLM" --> tools
tools == "2. searchDocs, readDoc" ==> docs
tools -. "3. listDatasets, listDatasetFiles (later)" .-> files
```
The thick arrows are new in changes 1 and 2. The dotted arrow is the sketch in
change 3. Everything else exists today.
## 1. Let the agent see the errors from the user's own run
Today the agent only sees an error when it ran the operator itself through its
`executeOperator` tool. When the user clicks Run and the workflow fails, the
agent has no idea. So "why did this fail?" is the one question a new user most
wants to ask it, and the one it cannot answer. The system prompt already tells
the model how to fix a failing operator in place. What is missing is the input.
The change is on the frontend side. When the user sends a message, the frontend
attaches a report on the last run the user started from that browser: the fatal
errors, the last error-level console message of each operator (long stack
traces truncated), and each operator's state and input and output row counts,
as suggested in the comments below. The report is attached once the run has
finished, or while it is still running if an operator has already logged an
error. A run the agent started itself is not reported. The agent service then
shows the report under the matching operator in the dataflow section of the
context, the same way compilation errors already appear there. Two short prompt
rules ask the model to explain an error in plain language, name the operator,
suggest the fix and ask before editing anything, and to read the row counts
when the user asks why a result is empty.
```mermaid
sequenceDiagram
actor U as User
participant B as Browser
participant CU as Computing unit
participant A as Agent service
participant M as LLM
U->>B: Click Run
B->>CU: Execute the workflow
CU-->>B: Run state, row counts, errors (websocket)
Note over B: Today this stays in the browser
U->>B: "Why did this fail?"
rect rgba(46, 160, 67, 0.15)
B->>A: Prompt + report on the user's last run
A->>M: Context with the error and row counts under each operator
end
M-->>A: Explanation and a suggested fix
A-->>B: Answer
B-->>U: Answer
```
Why not read the errors from the backend? Fatal errors are not persisted
anywhere. They live in the computing unit's in-memory state and reach the
browser over the websocket. The frontend already holds exactly this data, and
the AI Fix tab from #8637 reads it. Attaching it to the prompt needs no backend
or database change. If fatal errors get persisted later, the source can switch
to a REST call without changing the agent side.
This touches the frontend agent and execution services, the agent-service types
and context serialization, two prompt rules, and tests. It edits the same file
as #9244, so it will land after that.
## 2. Give the agent the documentation
The agent has no way to look anything up in Texera's own documentation, so a
question about how Texera works is answered from the model's general knowledge.
Meanwhile, most of the questions @zyratlo's researcher asked were "how do I do
X in the UI", and the answers exist in `docs/` and on texera.apache.org.
The change is to bundle `docs/` into the agent-service image at build time and
build an in-memory keyword index over the pages on startup, split by section.
Two read-only tools go on top of it. `searchDocs(query)` returns the best
matching sections with a link to the published page, and `readDoc(path,
section?)` returns the text. What the tools return is kept in its own section
of the model's context for the rest of the task, with a size limit. One new
prompt rule says: for questions about how Texera works, search the docs, read
the page, answer with a link, and do not edit the workflow.
```mermaid
flowchart LR
subgraph build [Build time]
md["docs/*.md"] -- "COPY docs" --> image[agent-service image]
end
subgraph startup [Startup]
index["In-memory keyword index<br/>one entry per section"]
end
subgraph question [A user question]
q["How do I upload a dataset?"] --> search[searchDocs]
search --> read[readDoc]
read --> answer["Answer with a link<br/>to the published page"]
end
image --> index
index --> search
index --> read
```
I want to keep this small. No embeddings, no model download, no new dependency.
A plain keyword index over about 500 KB of markdown is enough. Docs are bundled
rather than fetched at runtime so the answers match the deployed version and
work offline. When #9031 regenerates the operator reference pages, the index
picks them up with no change.
One side effect is useful on its own. Running a set of new-user questions
against the index also shows which questions have no page at all, and that list
can go to #8773.
This is two new files (the index and the tools), one new section in the
context, one new prompt rule, one `COPY docs` line in the Dockerfile, a
`TEXERA_DOCS_DIR` override for local runs, and tests. #4932 is related, since
llms.txt could be generated from the same index later.
## 3. A sketch for letting the agent see the user's datasets
The agent does not know which datasets the user has or what a dataset file path
looks like, so it guesses paths and gets them wrong (#4588).
The idea is two read-only tools backed by file-service endpoints that already
exist, `listDatasets()` and `listDatasetFiles(dataset, version?)`. The second
returns each file's full path in the form the file scan operators accept, so
the agent can fill it in directly. The tools call file-service with the user's
own delegated token, so the user's access rules are enforced where they already
are. This is discovery only. Reading a dataset from inside a Python UDF is the
mount feature (#6866, #6896, #6897) and the questions raised in #9048.
There is prior work here. @aicam prototyped dataset tools (list, get, versions,
and more) as part of a standalone MCP server on a fork branch in July,
following the discussion in #5610, where the conclusion was that Texera's own
actions should stay inline tools. The shape above is the inline version of that
prototype's read-only subset. #5918 raised how the agent should call backend
endpoints without duplicating the REST contract by hand, and #9395 now proposes
one set of endpoints for every kind of resource. I would rather build these
tools on that than add another hand-written client.
I am not proposing to start this yet. The agent service does not bind an agent
to its owner (#5268) and is getting request authentication in #9043. A tool
that lists a user's datasets should land after those, so I would rather agree
on the shape now and open the issue once #9043 is in.
## What I am asking
- Any suggestions on the three changes, or other ideas for what the agent
should be able to see or do for a new user?
- For 2, bundling docs into the image versus fetching from the website at
runtime. I prefer bundling for the reasons above.
- For 3, any objection to the shape?
I will open one issue per change and link the PRs here.
GitHub link: https://github.com/apache/texera/discussions/9377
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]