GitHub user kz930 edited a discussion: Where should the helper functions go in 
an exported Python script?

When a workflow is exported to Python, the script defines every helper function 
it calls before the workflow code starts. These helpers handle things like 
number parsing, type casts and null handling. For a small workflow the helpers 
take up most of the file. With the export series in #8325 applied, a CSV source 
followed by one Filter exports as 93 lines. 72 of them are seven helper 
functions, and the workflow itself is 16 lines at the bottom.

I would like to agree on a layout before going further. These are the options I 
see.

**A. Keep everything in one script (current).** The script is self-contained 
and runs with nothing extra. The cost is length, and the workflow logic is hard 
to find.

The export modal for a CSV, Filter, Sort, Limit and Bar Chart workflow. The 
first screen is all helper code.

<img width="1316" height="901" alt="option-a-inline-export-modal" 
src="https://github.com/user-attachments/assets/46cacba3-5ffc-4cc2-8158-071fb1d2ed90";
 />

**B. Download a zip with a helper module.** The zip holds `script.py` and 
`texera_support.py`. The script starts with `from texera_support import ...` 
and only names the helpers it uses. The module comes from the same Texera that 
wrote the script, so the two always match, and nothing needs to be installed. 
The cost is a second file, so a single copy and paste of the script no longer 
runs. There is a draft prototype in #9240. Below is the same workflow with it, 
and the button left of copy downloads the zip.

<img width="1440" height="757" alt="texera-support-1-export-modal" 
src="https://github.com/user-attachments/assets/261cdcb3-7d0e-415b-9426-5106e755c41c";
 />

**C. Publish a PyPI package.** The script starts with `pip install 
texera-support` and imports from it. This gives the shortest script. It needs 
an Apache release to publish from, a PPMC vote, and a decision on the package 
name. Each script would also need the package version that matches the Texera 
that exported it.

B and C are compatible. A script written for B keeps the same import line if 
the package is published later, and only the zip step goes away.

#9240 stays a draft until the export series in #8325 has merged. The outcome 
here decides whether it moves forward or gets closed.

I'd appreciate hearing everyone's thoughts on these options. Any feedback or 
suggestions are welcome. Thank you!

GitHub link: https://github.com/apache/texera/discussions/9368

----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]

Reply via email to