GitHub user eugenegujing created a discussion: Proposal: Making the workspace agent useful to a new user by seeing run errors, reading the docs, knowing the datasets
The agent can build and run workflows, but it cannot see the errors from a run the user started, cannot read Texera's own documentation, and does not know which datasets the user has. I want to propose three small, independent changes, each tracked as its own issue and PR. The first two are ready to start, and the third is a sketch I would like feedback on first. This follows up on #9246. ## 1. Let the agent see the errors from the user's own run Today the agent only sees an error when it ran the operator itself through its `executeOperator` tool. When the user clicks Run and the workflow fails, the agent has no idea. So "why did this fail?" is the one question a new user most wants to ask it, and the one it cannot answer. The system prompt already tells the model how to fix a failing operator in place. What is missing is the input. The change is on the frontend side. When the user sends a message, the frontend attaches the outcome of the latest run, which means the fatal errors and the last error-level console message of each operator, with stack traces truncated. The agent service then shows them under the matching operator in the dataflow section of the context, the same way compilation errors already appear there. A short prompt rule asks the model to explain the error in plain language, name the operator, suggest the fix, and offer to apply it before editing anything. Why not read the errors from the backend? Fatal errors are not persisted anywhere. They live in the computing unit's in-memory state and reach the browser over the websocket. The frontend already holds exactly this data, and the AI Fix tab from #8637 reads it. Attaching it to the prompt needs no backend or database change. If fatal errors get persisted later, the source can switch to a REST call without changing the agent side. This touches the frontend agent service, the agent-service types and context serialization, one prompt rule, and tests. It edits the same file as #9244, so it will land after that. ## 2. Give the agent the documentation The system prompt tells the model to read documentation first to understand exact definitions, but there is no documentation tool. The model is told to do something it cannot do. Meanwhile, most of the questions @zyratlo's researcher asked were "how do I do X in the UI", and the answers exist in `docs/` and on texera.apache.org. The change is to bundle `docs/` into the agent-service image at build time and build an in-memory keyword index over the pages on startup, split by section. Two read-only tools go on top of it. `searchDocs(query)` returns the best matching sections with a link to the published page, and `readDoc(path, section?)` returns the text. The prompt rule above becomes concrete. For questions about how Texera works, search the docs, read the page, answer with a link, and do not edit the workflow. I want to keep this small. No embeddings, no model download, no new dependency. A plain keyword index over about 500 KB of markdown is enough. Docs are bundled rather than fetched at runtime so the answers match the deployed version and work offline. When #9031 regenerates the operator reference pages, the index picks them up with no change. One side effect is useful on its own. Running a set of new-user questions against the index also shows which questions have no page at all, and that list can go to #8773. This is one new tool file, one prompt change, one `COPY docs` line in the Dockerfile, a `TEXERA_DOCS_DIR` override for local runs, and tests. #4932 is related, since llms.txt could be generated from the same index later. ## 3. A sketch for letting the agent see the user's datasets The agent does not know which datasets the user has or what a dataset file path looks like, so it guesses paths and gets them wrong (#4588). The idea is two read-only tools backed by file-service endpoints that already exist, `listDatasets()` and `listDatasetFiles(dataset, version?)`. The second returns each file's full path in the form the file scan operators accept, so the agent can fill it in directly. The tools call file-service with the user's own delegated token, so the user's access rules are enforced where they already are. This is discovery only. Reading a dataset from inside a Python UDF is the mount feature (#6866, #6896, #6897) and the open questions in #9048. There is prior work here. @aicam prototyped dataset tools (list, get, versions, and more) as part of a standalone MCP server on a fork branch in July, following the discussion in #5610, where the conclusion was that Texera's own actions should stay inline tools. The shape above is the inline version of that prototype's read-only subset. #5918 raises how the agent should call backend endpoints without duplicating the REST contract by hand, and I would rather follow whatever comes out of that than add another hand-written client. I am not proposing to start this yet. The agent service does not bind an agent to its owner (#5268) and is getting request authentication in #9043. A tool that lists a user's datasets should land after those, so I would rather agree on the shape now and open the issue once #9043 is in. ## What I am asking - Any suggestions on the three changes, or other ideas for what the agent should be able to see or do for a new user? - For 2, bundling docs into the image versus fetching from the website at runtime. I prefer bundling for the reasons above. - For 3, any objection to the shape, or to waiting on #9043? I will open one issue per change and link the PRs here. GitHub link: https://github.com/apache/texera/discussions/9377 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
