CarlosJuncher03 opened a new issue, #8203:
URL: https://github.com/apache/hop/issues/8203
### Apache Hop version?
2.20
### Java version?
21
### Operating system
Linux
### What happened?
When creating a **Test Pipeline** from an existing pipeline, the Apache Hop
interface starts experiencing freezes.
The issue seems to occur mainly when the Test Pipeline attempts to resolve
the **output metadata of transforms that use variables to determine their
output**.
In some cases, the interface becomes unresponsive or experiences significant
slowdown while attempting to resolve the metadata.
## Observed behavior
The original pipeline works normally.
However, when creating or using a **Test Pipeline** based on that pipeline,
Hop apparently attempts to resolve the metadata across all transforms. When a
transform is unable to provide the expected metadata, this process seems to
enter a state that causes the UI to freeze or become blocked.
Example of a pipeline that freezes when creating a Test Pipeline:
<?xml version="1.0" encoding="UTF-8"?>
<pipeline-transforms>
<transforms>
<transform>
<name>Iniciar</name>
<type>RowGenerator</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<never_ending>N</never_ending>
<interval_in_ms>5000</interval_in_ms>
<row_time_field>now</row_time_field>
<last_time_field>FiveSecondsAgo</last_time_field>
<limit>1</limit>
<fields></fields>
<attributes></attributes>
<GUI>
<xloc>94</xloc>
<yloc>98</yloc>
</GUI>
</transform>
<transform>
<name>Fonte Sql</name>
<type>TableInput</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<sql/>
<limit>0</limit>
<execute_each_row>N</execute_each_row>
<variables_active>Y</variables_active>
<connection>${conexao_origem}</connection>
<lookup/>
<sql_from_file/>
<use_named_parameters>N</use_named_parameters>
<specify_fields>N</specify_fields>
<validate_specified_fields>N</validate_specified_fields>
<fields></fields>
<attributes></attributes>
<GUI>
<xloc>372</xloc>
<yloc>98</yloc>
</GUI>
</transform>
<transform>
<name>Salva S3</name>
<type>ParquetFileOutput</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>${numero_threads}</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<filename_base/>
<filename_ext/>
<filename_include_date>N</filename_include_date>
<filename_include_time>N</filename_include_time>
<filename_include_datetime>Y</filename_include_datetime>
<filename_datetime_format>yyyyMMdd-HHmmss</filename_datetime_format>
<filename_include_copy>Y</filename_include_copy>
<filename_include_split>Y</filename_include_split>
<filename_split_size>1000000</filename_split_size>
<filename_create_parent_folders>Y</filename_create_parent_folders>
<filename_compression_before_extension>Y</filename_compression_before_extension>
<compression_codec>UNCOMPRESSED</compression_codec>
<version>2.0</version>
<row_group_size>268435456</row_group_size>
<data_page_size>8192</data_page_size>
<dictionary_page_size>1048576</dictionary_page_size>
<fields></fields>
<attributes></attributes>
<GUI>
<xloc>510</xloc>
<yloc>98</yloc>
</GUI>
</transform>
<transform>
<name>Log Inicial</name>
<type>WriteToLog</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<displayHeader>Y</displayHeader>
<limitRows>N</limitRows>
<limitRowsNumber>0</limitRowsNumber>
<logmessage>Iniciando carga raw
Schema: ${nome}
SQL origem: ${caminho_arquivo_raw_sql}
Destino S3: ${caminho_s3}
</logmessage>
<loglevel>Basic</loglevel>
<fields></fields>
<attributes></attributes>
<GUI>
<xloc>232</xloc>
<yloc>98</yloc>
</GUI>
</transform>
<transform>
<name>Metricas Fluxo</name>
<type>TransformsMetrics</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<transforms>
<transform>
<name>Fonte Sql</name>
<copyNr/>
<transformRequired>N</transformRequired>
</transform>
</transforms>
<transformnamefield/>
<transformidfield/>
<transformlinesinputfield/>
<transformlinesoutputfield/>
<transformlinesreadfield/>
<transformlinesupdatedfield/>
<transformlineswrittenfield>QUANTIDADE_LINHAS</transformlineswrittenfield>
<transformlineserrorsfield/>
<transformsecondsfield/>
<attributes></attributes>
<GUI>
<xloc>650</xloc>
<yloc>98</yloc>
</GUI>
</transform>
<transform>
<name>Log Final</name>
<type>WriteToLog</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<displayHeader>Y</displayHeader>
<limitRows>N</limitRows>
<limitRowsNumber>0</limitRowsNumber>
<logmessage>Carga raw finalizada
Schema: ${nome}
Destino S3: ${caminho_s3}
Quantidade de linhas gravadas:
</logmessage>
<loglevel>Basic</loglevel>
<fields>
<field>
<name>QUANTIDADE_LINHAS</name>
</field>
</fields>
<attributes></attributes>
<GUI>
<xloc>1066</xloc>
<yloc>98</yloc>
</GUI>
</transform>
<transform>
<name>Ultima Linha</name>
<type>DetectLastRow</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<resultfieldname>ultima_linha</resultfieldname>
<attributes></attributes>
<GUI>
<xloc>788</xloc>
<yloc>98</yloc>
</GUI>
</transform>
<transform>
<name>Java Filtro</name>
<type>JavaFilter</type>
<description/>
<distribute>Y</distribute>
<custom_distribution/>
<copies>1</copies>
<partitioning>
<method>none</method>
<schema_name/>
</partitioning>
<condition>ultima_linha = true</condition>
<send_true_to>Log Final</send_true_to>
<send_false_to/>
<attributes></attributes>
<GUI>
<xloc>928</xloc>
<yloc>98</yloc>
</GUI>
</transform>
</transforms>
<order>
<hop>
<from>Iniciar</from>
<to>Log Inicial</to>
<enabled>Y</enabled>
</hop>
<hop>
<from>Fonte Sql</from>
<to>Salva S3</to>
<enabled>Y</enabled>
</hop>
<hop>
<from>Salva S3</from>
<to>Metricas Fluxo</to>
<enabled>Y</enabled>
</hop>
<hop>
<from>Log Inicial</from>
<to>Fonte Sql</to>
<enabled>Y</enabled>
</hop>
<hop>
<from>Metricas Fluxo</from>
<to>Ultima Linha</to>
<enabled>Y</enabled>
</hop>
<hop>
<from>Ultima Linha</from>
<to>Java Filtro</to>
<enabled>Y</enabled>
</hop>
<hop>
<from>Java Filtro</from>
<to>Log Final</to>
<enabled>Y</enabled>
</hop>
</order>
<notepads>
</notepads>
<transform_error_handling>
</transform_error_handling>
</pipeline-transforms>
### Issue Priority
Priority: 3
### Issue Component
Component: Hop Gui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
