Hey Everyone
In Spark, suppose i have the following df.
```
df = spark.createDataFrame([['A', 'A06', 'B', 'B02', '202412'], ['A',
'A04', 'B', 'B03', '202501'], ['B', 'B01', 'C', 'C02', '202411'], ['B',
'B03', 'A', 'A06', '202502']], 'entity_code: string, entity_rollup: string,
target_entity_code:
h-talebzadeh-ph-d-5205b2/>
>
>
>
>
>
> On Sun, 9 Mar 2025 at 14:12, Dhruv Singla wrote:
>
>> Hi Everyone
>>
>> Hope you are doing well
>>
>> I have the following dataframe.
>>
>> df = spark.createDataFrame(
>> [
>>
er (nullable = true)
> |-- AB_amnt: long (nullable = true)
> |-- AA_amnt: long (nullable = true)
> |-- AC_amnt: long (nullable = true)
> |-- load_date: date (nullable = true)
>
> ++---+---+---+--+
> |code|AB_amnt|AA_amnt|AC_amnt|load_date |
> ++---
Hi Everyone
Hope you are doing well
I have the following dataframe.
df = spark.createDataFrame(
[
[1, 'AB', 12, '2022-01-01']
, [1, 'AA', 22, '2022-01-10']
, [1, 'AC', 11, '2022-01-11']
, [2, 'AB', 22, '2022-02-01']
, [2, 'AA', 28, '2022-02-10']
Hey Everyone
Hope you are doing well
I was looking through the datetime format¹ in the docs and think that the
docs are a little incorrect while describing the formatting for year
format. It is written as "... If the count of letters is less than four
(but not two), then the sign is only output f
Hey Everyone
Hope you are doing well
I am trying to compare structs with structs using the IN clause. Here is
what I found.
The following query comparing structs gives error
SELECT struct(1, 2) IN (
SELECT struct(c1, c2)
FROM (VALUES (1, 2), (3, 4)) AS t(c1, c2)
);
Error - Cannot resolve "(name
The spark version 3.5.1
On Fri, Jul 26, 2024 at 6:54 PM Dhruv Singla wrote:
> Hey Everyone
>
> Hope you are doing well
>
> I am trying to compare structs with structs using the IN clause. Here is
> what I found.
> The following query comparing structs gives error
>
Hey Team
I've posted a question of StackOverflow. The link is -
https://stackoverflow.com/questions/78644118/understanding-exchange-in-spark-ui
I haven't got any responses yet. If possible could you please look into it?
If you need me to write the question in the mailing list, I can do that as
we
You can reproduce the behavior in ordinary Scala code if you keep reduce in
an object outside the main method. Hope it might help
On Mon, Apr 17, 2023 at 10:22 PM Dhruv Singla wrote:
> Hi Team
>I was trying to run spark using `sbt console` on the terminal. I am
> able to
Hi Team
I was trying to run spark using `sbt console` on the terminal. I am
able to build the project successfully using build.sbt and the following
piece of code runs fine on IntelliJ. The only issue I am facing while
running the same on terminal is that the Executor keeps running and is no
10 matches
Mail list logo
